Plone 4.3 is coming! With it come some changes to Plone's Unified Installer.
We try not to make any significant changes in the way Plone's installer's work for new micro version (+0.0.1) updates to Plone. However, the coming release of a new minor version update (4.3) offers a good opportunity to shake things up a bit. Here's a review of what's new. Your testing, feedback and bug reports will help improve it.
The target audience for this longish blog post is system integrators who already use the Unified Installer and would like an explanation of what's changed and why. New users really shouldn't care :)
Encouraging Use of System Packages
The first Unified Installer, created by Kamal Gill back during the Plone 2 series, shipped with a Dutch cheese shop full of source packages: Python, libz, readline, libjpeg and a lot more. We needed all that because many of the *nix system platforms of that era were missing these packages or their header files, or they had buggy or old versions (Apple, I'm looking at you). Also — it was a bit embarrassing to say — Plone wanted an old version of Python that wasn't on recent Linux distributions. So, we included all that with the installer and figured out how to build them on lots of different machines.
That situation has changed a lot. The current generation of Linux and BSD distributions are doing a great job with their package managers. Not only are all of Plone's dependencies easy to install, but it's very easy to keep them up-to-date with security patches. Having Plone's installer include and build all of these packages is no longer going to help you keep up-to-date; instead, it's going to trap you with particular versions of several major libraries. We still, though, have to have an installer that will be able to deal with old platforms and those (Apple, I'm looking at you again) which seem to have a perverse desire to keep major libraries out-of-date.
So, what's an installer to do?
Beginning with the 4.3 installer, we're following a strategy of encouraging use of system packages/ports whenever possible. So, instead of simply building a Python 2.7, we'll see if the system python2.7 will work. If it does, great; if it doesn't we ask the user to specify a Python-2.7 they'd like to use or have them use a --build-python flag to tell the installer to go ahead and try to build its own.
Here's the logic for picking/building a Python:
If the installer decides to use a found or '--with-python' specified python binary, it uses virtualenv to create a virtualized copy of the Python with no inherited site packages. So, the system or specified Python is untouched, and the copy we use is hopefully not contaminated by incompatible or bogus site packages.
libxml2/libxslt
In general, if Plone needs a major library that's usually available via system packages or ports, the installer will check for the library and it's header files and make use of it. If it's missing, the installer tells you to use your package or port manager to install it. We have to handle libxml2 and libxslt a bit differently, though. The new Diazo theming engine needs these libraries to be very current — so current that a system that's even a year or two old may not have adequate versions available via its package manager.
The installer checks for the availability of libxml2 and libxslt developer packages and checks their versions. If they aren't available, or aren't current enough, the installer will tell you what's needed. It will also offer to build them for you by downloading the current libraries and building them statically for use by the Python lxml package (which is Plone's required Python package for connecting to libxml2 and libxslt). This won't happen automatically, though. You'll have to set the '--static-lxml' flag on the install.sh command line. Also, you'll need to have an Internet connection available to download the library sources, as they're no longer included with the installer.
Other Libraries
Most other libraries required by Plone, like libssl and libz, are much less of a problem — as they're always (by my experience) available and up-to-date-enough when installed by platform package and port managers. We do include a couple of library sources, though, to deal with the oddities of OS X: libjpeg and readline. libjpeg isn't included with OS X, and the readline library Apple includes isn't really readline. These problems are easy to solve with MacPorts, but we don't want to ask most OS X users to install MacPorts and learn to use the port system just to get those two libraries.
Root (sudo) Installs
If you run the installer via sudo, you'll get some security enhancements due to the ability of the superuser to create system users and groups and set directory and file ownership and privileges.
The installer now creates two users and a group, which contains both users:
- plone_buildout
- This user will own most of the install, including the buildout cache and all the Plone source and byte-code (.pyc/.pyo) files.
- plone_daemon
- This user will own the instance's var directory and everything under it. The long-lived processes (ZEO server, client, stand-alone ZOPE) will run with this identity.
- plone_group
- This system group will have both plone_buildout and plone_daemon as members. The instances var directory and its contents will mostly be owned and writable by this group. This strategy allows the plone_buildout user to be able to create var subdirectories and files that the plone_daemon user will be able to use.
Running Buildout
Ideally, you'll never run buildout as root. Instead, run buildout with a command like:
sudo -u plone_buildout bin/buildout
In fact, if you try to run buildout with superuser privileges, a buildout extension that's specified in the default buildout.cfg will try to stop you. The error raised:
User attempt to give system ownership to Internet
Running ZEO & ZOPE
Likewise, you'll want to start the long-lived ZEO and ZOPE server processes under the plone_daemon user:
sudo -u plone_daemon bin/plonectl start
or, skipping plonectl, with a command like:
sudo -u plone_daemon bin/client1 console
What's gained?
The new strategy is an alternative to the common mechanism of creating a plone user, then installing, running and maintaining everything under that user's ownership.
The big gain is that the long-lived, Internet-connected, client and server processes will have fewer privileges. They should only be able to write data files, logs and pid files — and should not be able to write into the program directories.
Don't like plone_daemon/plone_buildout?
Specify your own user and/or group names as command-line options.
Buildout Config Files Refactored
If you take a look at the new buildout.cfg file created by the installer, it's cleaner than ever. The goal is to only have in this file what most users are likely to change. Most of the complex parts are in a base.cfg file that sets the basics for common components.
The refactoring of the buildout configuration files should make it very easy to have the installer work with a custom market buildout, for example an .edu installer aimed at department/school uses. We can even include those kits with the main installer and use a new --template= option to pick them instead of the default.
Directory flexibility
You may now specify locations for the var and backup directories. So, if you'd like a more traditional *nix install, you can install to /usr/local/..., put your var somewhere like /var/opt/... and your backup directories on a remote mount.