This week, a group of 15 people is gathering in lovely Barcelona to work, experiment and play with new concepts for our favorite CMS, Plone.
And we’re not the only ones, a few countries further north a group of people are doing another sprint, in Berlin! But I’ll let the Berlin people do their own reporting.
We divided up in three teams, delving into the front-end, restapi and back-end (more or less), with lots of interaction going on between the teams.
Since my strengths aren’t exactly diving into the deep inward workings of Plone and underlying technologies, I had the nice job to try to summarize where the work is going for now, in terms that hopefully anybody with a keen interest in the community can understand.
Do note that in describing some of the work in terms that even I can understand, I may have grossly oversimplified or misrepresented some things. So, if your reaction is “huh?”, it’s probably wise to think “oh, polyester has gotten it all wrong” and ask accordingly, before putting on your angry hat and take to da interwebz….
REST api
Probably the easiest to describe, as it is already quite well documented and used in production, is plone.restapi. It has clear goals and design ideas, and saw a lot of progress already this week.
Endpoints (that’s functionality to all folks new to the jargon: things you can get information from, or create content, or change workflow state or permissions) are being added as we speak. And some of the the tougher issues with CORS and other security-relevant topics are being tackled.
One recurring theme on this is that one of the key strengths of Plone, real traversal, is not something that others do very well, or even reckon with. As an example, the current version of OpenAPI (formerly known as Swagger), a formal description language for API’s which would be great to use as a documenting and discovery tool, wants to know all endpoints in advance. That’s all very fine and dandy if your site basically looks like index.php?incomprehensiblemumbojumbo, but ours are better behaved. In short, we need to parametrize the path in the URI, and do no know in advance how they will look or how many there will be. And that’s a problem for the tooling right now, although they are aware of it.
On the plus side, the way Plone does real traversal makes it a really, really good fit for a proper REST interface. So, that will be the way we’re moving forward.
Status: A beta version of plone.rest (the underlying HTTP verbs) should come out this week, and the actual plone.restapi will see a proper alpha release with many new endpoints and improved documentation.
Polyester Crystal Ball Assessment: expect this to be useful already in the very near future, and absolutely essential in the slightly longer run. The documentation-fu runs strongly in this one, too.
PloneClient
At the same time, a very talented bunch of people is hacking away at a prototype for a full, modern client based purely on said REST api.
To my slight disappointment, the Great JS Framework Wars were not decided by pillow-fight, but by actual reasoned discussion. And that means that this client is being written in Angular2. Of course, remember, anybody can always implement a client in their preferred framework-du-jour, but for this prototype this decision stands.
so, after a few days of intense discussions, frantic typing and good food, what is already working as ng2 components?
- breadcrumbs
- navigation
- administrative interface (aka ‘the toolbar’)
- search (yep, it works, and fast)
- dynamic components ( ‘tiles, also behaving with portlet functionality, rendered as Angular components’ to lay-people like me)
- there will still be a concept of left, right, footer and header slots,
- Mosaic will be implemented, but for now stick to having it on the ‘content’ slot only, and not deal with inheritance.
- and it all looks comfortingly familiar, with Barceloneta-style theming
Of course there is still a long list of work in progress:
- routing (well, our brand, with full traversal) *sucks* in Angular2 (at least in RC1, there is hope…), as they do not fully grasp the power of this idea. See the recurring theme here? But Eric Bréhault has a patch to make it work.
- json-based form generation. Feed it the same json-schema based schemes as delivered by plone.restapi, and you can have your edit and add forms generated. There’s still questions of validation: some can be done client-side, but some only server-side. But it’s a very promising route (eh, no pun intented)
- adaptability: you will be able to override any HTML or CSS template used in PloneClient. Call it “jbot for Angular”, if you will.
- Through The Web Theming editor.
- Javascript for overrides of what comes with Angular, or for unrelated add-ons, can be added using Webpack. Yep, new technology, but hey it works!
- and there’s work done to get a registry to declare dynamic components. I’d almost call it ‘zcml for Angular’. That’s probably not the scientific term, but helps me understand.
all in all, a pretty impressive list, and already enough to give you an understanding that such a one-page client is not only possible, but could be pretty cool, and fun to use both for end users and integrators.
Polyester Crystal Ball Assessment: well, color me impressed. It will take work, and we may run into some issues. But such a client seems eminently possible, and I’m pretty much convinced that the needs and wants of ‘casual’ integrators, themers and tinkerers will be met. I say: bring it on!
PloneServer
The team whose work was hardest to grasp for me was the group working on future & experimental backends. The ideas behind it are clear enough:
- create a headless CMS backend
- with modular services for users/groups, searching and indexing that may be better filled with best-of-breed tools (hi there, elasticsearch and friends)
- of course working on Python 3
- able to use async technologies, websockets and the like
yet the devil is in the details, and un-tangling the many tangled layers of Zope, CMF and more that now form ourmille-feuille of a stack is no easy task.
At the same time it is also crucial that we as a community start experimenting, and testing some of it in real world scenarios.
So, experimenting there was. And do note: none of this is final, we’re definitely in the “let’s try some crazy stuff and see how far we get “ stage here…
Basically we took the top layer (dexterity) and the bottom layer (ZTK) and ripped out everything in between, and see how far that would get us before adding layers back.
Some results so far:
- dexterity works with it, as do most of the dependencies for it. All on Python 3.5. Work continues to get more dependencies going.
- you can already register a contenttype, create it, view it, and get the json representation of it, sharing format with what plone.restapi gives you.
- the server uses asyncio. In very unscientific terms, “aiohttp is the new medusa”
- a new plone.server could replace Zope2. It would use the actively maintained Zope Toolkit, which includes zopesecurity, the component architecture of Zope3, DublinCore, schemas and low-level security.
- workflow is still an open question, although there are candidates that should be looked at
- the registry packages have already been ported to Python3
- it would be possible to have a compatible plone.api working on Plone5, a future Plone6 (with a cleaned-up Zope) and this further-in-the-future vision, let’s call it Plone7 for now.
the surprising part of this (and remember: this was meant to be experimental) is that quite a bit of the porting to Python3 can already go back into mainstream Plone.
And of course that it shows there is an active interest, and developing ideas, in growing the next generation of Plone. All while keeping up with our proud heritage of allowing far-reaching customization and modularity. So if you need to plug in another search system or another source of user/group data, that will be possible.
Polyester Crystal Ball Assessment:expect this to be at least two years away for any simple mortal like me. Also expect lots of changes and lessons learnt. But do be very excited about the experiments going on, which will grow into real-life solutions being vetted with real-world sites, and getting us a clear path into the future. Oh, and part of the Python3 porting is usable far earlier than I had expected.
I never promised you a rose garden…
As you would expect, all groups also encountered complex problems.
Security is always hard. In a world with discoverable API’s, and back-ends having to deal with potentially malicious (or just sloppily programmed, which might lead to the same results) front-ends this becomes ever more important, on every level. Remember: every new web technology leads to a new security acronym. Want websockets? Learn about CSWSH…
Luckily this is very much in everybody’s mind here, and security is a core key ingredient and not an afterthought on what people are developing.
Traversal still remains as powerful an idea as when Jim Fulton created Zope in 1998. It’s how the web should work. Yet many of the new tools and techniques that we are now incorporating do not deal with it very well. On the other hand, it’s what sets us apart, and we may have to adapt the tools to do the right thing.
Untrusted Code (a.k.a. Restricted Python) is both pretty hard to deal with, from a security standpoint, yet also a reality for any kind of serious universal CMS with sophisticated users. The balance here is a fine one.
The JavaScriptecosystem still remains very much in flux.
If you ever thought “buildout” could be a bit of a diva, just wait until you meet the Nuclear Powered Mushroom. It throws tantrums like a three-year-old on a sugar rush.
And there have been 13 days since the last grunt-gulp-bower-webpack-rollup replacement…
FAQ / Panic Attack
I’m sure a lot of you will have many questions. As did I.
So I asked them ;-)
- OMG THEY KILLED
KENNYThrough The Web
eh, no they didn’t, I’m glad to report. The ability for integrators and site admins to customize just about every aspect is firmly on the radar, and can be maintained even using things like Angular. - YOU GUYS DECIDE EVERYTHING BEHIND MY BACK
eh, no, not really, All of this is firmly in line with the roadmap and ‘2020′ discussions that are broad community discussions. Part of that is experimenting, to base decisions on actual learned experiences and not guesswork. - I WANT MY OLD WAYS BACK
hmmm. Yep, there you have a point. No, the <blink> tag is not coming back. And neither is the old, old way of doing javascript.
It’s the web, and it’s 2016.
What we do need to do, is make sure that we do a better job of documenting the new ways of doing things that will come as we move forward. But yes, learning new techniques is part of the job description…
wait… there’s more!
The sprint is not even over. And there will be many more sprints and discussions, both online and in real life. But let me say that this was one good way of giving all those future discussions a much more solid foundation of prototypes, experiments and performance statistics.
Time to get involved!
This particular experience was brought to you by the following people:
Ramon Navarro Bosch, Víctor Fernández de Alba, Timo Stollenwerk, Eric Bréhault, Asko Soukka, Nathan Van Gheem, Sam Schwartz, Lukas Graf, Thomas Buchberger, Eric Steele, Berta Capdevila Cano, Rob Gietema, Aleix Llusa Serrá, Albert Casado and Paul Roeland, with liberal helpings of sunshine, culture, cuisine and assorted friends & family who where visiting Barcelona…