Starzel.de: Porting Plone to Python 3

Since I wrote the proposal to port Plone to Python 3, so much has happened that a status update is needed.

Let's step back a little: The first steps towards Python 3 were taken during the sprint at the Plone conference in Barcelona. The epic PLIP to update to Zope 4 was merged and we started porting individual packages to Python 3 without being able to run any tests. With the help of sixer and python-modernize we tried to get the most obvious import and syntax issues out of the way. That work was continued at the Alpine City Sprint in Innsbruck and in the end, we treated more than 150 packages like this. Some of this work is documented in the Plone tracker.

Along the way I wrote a best-practice guide to port Plone packages this way.

As the PLIP states, there are several approaches to porting Plone to Python 3:

1. Migrate and test packages that have no dependency to CMFPlone and test them in Python 2 and 3.
2. Prepare packages for Python 3 without being able to test them in Python 3.
3. Start up Plone on Python 3 and fix whatever breaks. When start-up works, create a Plone Site and again, fix whatever breaks.
4. Port plone.testing and plone.app.testing to Python 3 and start running tests. Fix what breaks during the setup of the layers.
5. Run the tests with Python 3 and fix all broken tests.

At the sprint in Innbruck I started with #3 and kept going after the sprint until I was able to create an instance. At the Plone Tagung in Berlin I was able to demo the creation of a site but nothing was rendered yet.

After that, I kept going and I was finally able to create a site and manage content, which is what a CMS is about. It looked a bit raw but I was able to add and edit some content - yay!

Work continued at an unsteady pace, and with important contributions from Michael Howitz, Alessandro Pisa and David Glick, things started to get better, the theme started working. Slowly broken features bekame the exception, not the rule.

Last week at the Zope 4 Welcome Sprint in Halle we removed all feature blockers that we had found so far. Plone with Python 3 looks and feels like it used to in Python 2. During the sprint there was also a lot of progress on:

• the wsgi setup
• logging and tracebacks when using wsgi
• porting plone.testing
• a new theme for the ZMI
• beta releases for many packages.

There was also some progress on the difficult issue of database migrations. It seems like zodbupdate is the best tool to do that but there is probably a lot of work ahead.

There is a Jenkins job and as of today, it is running (not passing) tests for all packages (except Archetypes) with Python 3.

At the moment we run 6594 tests from 115 packages with 257 failures and 315 errors - not bad when you keep in mind that we still focus on features, not on tests. Tests for a couple of packages are green, including plone.api which is great since plone.api uses most of our stack one way or another. Jenkins runs every three hours and here you can see the progress over time.

Here is a screenshot from today that shows Plone doing some pretty non-trivial things: Editing a recurring Event and also Richtext with Images and Links:

Next steps:

• Create a demo site running on Python 3 that people can use to find broken features. This will happen at the Plonator Sprint in Munich.
• Fix all tests that look like they fail because of a broken feature.
• Fix all remaining tests to pass in Python 2 and Python 3. Since there are a gazillion doctests that will take some time.
• Port plone.app.robotframework to python 3 and fix robottests.
• Experiment with porting a ZODB with Plone to Python 3.

If you want to participate you can simply look at the failing tests and start fixing them. You can also try to fix one of the open issues.
The setup of a coredev environment with Python 3 is really simple and documented.