Lennart Regebro: Everybody is wrong! (About language popularity)

PyPL has gotten a lot of attention lately as index of programming language popularity. It criticizes TIOBE, the most popular popularity index, and it has some valid criticism, mainly that it is a lagging index, it looks, amongst other things, at how many pages about the language that Google finds when you search. PyPL instead uses Google Trends to look at how many searches for a language. That does make sense. And while TIOBE searches for “<language> programming” TIOBE believes this underestimates some languages, as you don’t need to type “programming” as a qualification for PHP.

The difference in results are not just significant, but stunning. In PyPL, Objective-C doesn’t even chart, while in TIOBE it’s the third most popular language. How can this be? One of these indexes must be wildly wrong about Objective-C. And unfortunately, it’s PyPL. Because PyPL falls in the same trap it tries to avoid. Instead of underestimating PHP by looking at “PHP programming” when “PHP” will do, it completely underestimates Objective-C by searching for “Objective-C tutorial” when that’s simply not a search people uses. Objective-C has seen a massive rise in popularity that’s to Apples iOS. Nobody cares that it’s Objective-C. What they search for is “iPhone tutorial” or rather “iPhone programming tutorial” or similar.

But those search terms are too wide, because you can use other languages as well.

The solution? Probably Google hits or searches shouldn’t be used at all, or at least only be a small part of the total score. PyPL also claims that “An analysis of language tag followers on StackOverflow, or of the visits of the Wikipedia language pages result in a similar ranking of objective-c.” That’s complete bogus. If you look at how many visits the Objective-C page on Wikipedia has, or how many new questions per day there are on Stackoverflow, then Objective-C ranks as about half as popular as Python. That’s a big difference from TIOBE who ranks it as twice as popular than Python. But it’s an even bigger difference than PyPL where Python is 7 times as popular as Objective-C.

The only real conclusion to this is of course that ranking languages is hard.