Netsight Developers: Open Source Content Management Systems - A Tale of Two Reports

Last week two very different reports into Open Source Content Management Systems were published:

• 2010 Comparing Open Source Content Management Systems: WordPress, Joomla, Drupal and Plone by Idealware
• 2010 Open Source CMS Market Share Report by Water & Stone

The two reports look at very different aspects of Open Source Content Management Systems, and are intended to be used in very different ways.

Idealware

The report from Idealware looks at Joomla!, Drupal, Plone and Wordpress and goes into quite significant details on each of them. There is a fully open and disclosed methodology, with details on the funding and contributors to the report. It is 81 pages long -- although a number of pages are adverts for various companies, and a directory of consultants is included. Idealware primarily work with non-profit organisations and the report starts of clarifying what a CMS is and why an organisation would need one. It gives a good rundown of the process of implementing a CMS and gives good sound advice on the work required in terms of tailoring any kind of CMS to your organisation. And goes into detail on each system in 13 key areas:

One of the very clear things that comes from the report is really how good all these systems have become, yet they all have strengths (and weaknesses) in different areas. As is often said, there is no one 'better' CMS only 'better for my purpose' and of course that is dependant on your intended uses.

I hope Idealware don't mind me revealing the punchline here, but this graphic really illustrates this variation, just look at the dark blue circles which represent 'excellent' scores in that area. As you can see Plone and Wordpress are almost complete opposites.

Water & Stone

The Water & Stone report looks predominantly at the brand awareness and market share of 20 Open Source Content Management Systems. Whilst it covers the same four CMSs as the Idealware report it also covers a number of others, some of which I've hardly heard of.

I have to say, I'm not very comfortable with this report. Whilst it does openly say it is looking at the market share of the systems, it does so without any consideration for the use and suitability of those systems to those tasks. For example, is it really any surprise that Wordpress shows up as the highest ranked site by Alexa? Wordpress is a blogging tool with a hosted blogging service, more people setup blogs than, say, setup Alfresco for Enterprise Content Management. Does that make Alfresco a less 'good' tool for the job? No of course not, it is just a different tool for a different job.

Similarly they compare number of mentions of the systems on Twitter. What does this really tell us? I follow what goes on in the Plone world on Twitter and there are more than enough tweets each day for me to keep up with. Drupal looks like it has around 50 times the mentions. That doesn't really help me though if I am interested in Drupal, I'd probably just drown in the volume.

Does it help me choose a CMS? No, not at all. That would be like saying I should plough my field with a Honda Civic rather than a Massey Ferguson just because they are both vehicles and Honda Civic is mentioned most on Twitter. Conversely it doesn't matter how many books are published on Massey Ferguson if one of your requirements is to fit a child seat to your prospective vehicle. As an aside, did you know that Lamborghini used to make tractors?

One aspect that Water & Stone didn't touch on if they are measuring the health and size of the communities for these projects is the number of conferences, developer sprints and training days they have around the world and the number and diversity of attendees.

So whilst this report does offer some interesting insights into various brand metrics of Open Source Content Management Systems, I don't think it offers much value to those looking to choose or evaluate a CMS for their needs.