Davide Moro: Pyramid starter seed template powered by Yeoman (part 1)

Book of the month I'm reading this summer: Pylons/Pyramid (http://docs.pylonsproject.org/en/latest).


Pyramid (http://www.pylonsproject.org) is a minimal Python-based web development framework that let you "start small and finish big".

It stole a lot of (good) ideas and concepts from other mature Python web frameworks and it is build with the pluggable and extensible concepts in mind. Read: no need to fork applications.

Furthermore Pyramid is database and template engine agnostic: you are free.

From the very beginning Pyramid allows you to become productive quickly. So why not start with something of useful?

Pyramid + Yeoman

The goal of this experiment is integrate yeoman with Pyramid (or other frameworks like NodeJs/Express with AngularJS or Plone as already did), preserving the yeoman's workflow.

UPDATE 20140926: here you can see a Plone + AngularJS + Yeoman article (collective.angularstarter)

In this article I'll talk about what are the benefits you get integrating your Pyramid app with Yeoman, in future posts I'll discuss how they work under the hood with additional technical details omitted here (each used component deserves an entire blog post).

Yeoman

You might wonder why? Because of the importance of tooling. Since it is very important build an effective developer tooling ecosystem, I want to integrate the simple starter demo app with commonly used tools to help you stay productive. So this simple application prototype it is just an experiment that should help you to integrate with modern web development tools provided by the yeoman workflow stack (http://yeoman.io).

Choosing the right tools is very important for the best develop experience and I cannot work anymore without Yeoman, especially when coding with Javascript.

Grunt

Yeoman it is internally based on three important components (nodejs powered):

• yo, scaffolding tool like pcreate, paster or zopeskel. It is widely adopted by a large and trasversal community
• grunt, system used for build, preview and test your software. Gulp is another popular option
• bower, used for dependency management, so that you no longer have to manually download and manage your scripts

Bower

So with the yeoman's tools you can just code, avoid annoying repetitive tasks and don't worry about:

• javascript testing setup
• javascript code linting
• javascript/css minification and merging
• image minification
• html minification
• switch to CDN versions of you vendor plugins in production mode
• auto-reload browser
• much much more

So let's see together what happened to our pyramid starter demo template created with pcreate -t starter integrated with a yeoman's generator-webapp project.

The result will be a Pyramid starter seed project integrated with modern non Python-based web development tools.

Goals

Management of third party assets

You no longer have to manually download and manage your scripts with the Bower package manager.

From http://bower.io:

"""Bower works by fetching and installing packages from all over, taking care of hunting, finding, downloading, and saving the stuff you’re looking for."""

So just type something like: bower install angular-translate --save and you'll get the rigth resource with pinning support.

Tasks automation

Automation, automation, automation.

From http://gruntjs.com:

"""Why use a task runner? In one word: automation. The less work you have to do when performing repetitive tasks like minification, compilation, unit testing, linting, etc, the easier your job becomes. After you've configured it, a task runner can do most of that mundane work for you—and your team—with basically zero effort."""

Examples:

• grunt serve
• grunt test
• grunt build
• grunt YOUR TASK 
• etc

Jslint

No more deploy Javascript code with bad indentation, syntax errors or bad code practices.

All syntax errors or bad practise will be found.

Image minification

The build process will detect and minify automatically all your asset images.

Uncss task

Modern (and heavy) UI frameworks like Twitter Bootstrap provide an excellent solution for prototyping your initial project, but most of the times you are using a very minimal subset of their functionalities.

This inspiring Addy Osmani's blog post helps you to remove unused css in your pages with a grunt task named grunt-uncss (https://github.com/addyosmani/grunt-uncss):

• http://addyosmani.com/blog/removing-unused-css

The original not-minified bootstrap.css weights in at 120 kB before removing unused rule.

Css concat and minification

You can split your css code into different files and then the build process will concat and minify them creating a unique app.css file. This way you write modular and better readable css files, reducing the number of browser requests.

The theme.css file is quite small but in real projects you can save more. In this case:

The configured build pipeline is concat, uncss and cssmin. 122.85 kB (original bootstrap.css) -> 4.64 kB (uncss) -> 3.45 kB (minification)

Automatic CDN-ification

It is handy using unminified versions of third party javascript libraries during development and switch to CDN versions in production mode with well known benefits for your website.

Don't worry: the cdnify task will take care about this boring issue. Automatically.

You save a boring manual and error-prone configuration.

Composable bootstrap.js version

The Pyramid starter project is based on Twitter Bootstrap.

Twitter Bootstrap

Depending on your project you can load the whole Twitter Bootstrap Javascript code at once or including individual plugins.

As you can see the Javascript component of Twitter Bootstrap is very modular: http://getbootstrap.com/javascript. So if you don't use a particular feature, just don't include it.

This way in development mode you will have all individual plugins splitted in different files, in production it will served a unique concatenated and minified Javascript file built automatically.

So if you just need alert.js and dropdown.js you can get a 2.79 kB plugins.js:

The concatenation of alert.js and dropdown.js produces a 7.06 kB, that weight in at 2.79 kB after minification instead of the 8.9 kB (gzipped) bootstrap-min.js corresponding to not gzipped 27.2 kB.

Html (template) minification

Since the ZPT/Chameleon templating language is an extension of HTML with xml syntax,

Brower are able to display unrendered ZPT/Chameleon templates

theorically it can play well with html minificators.

I know, template minification can lead to potential unexpected problems due to minification issues on template files... but this is my personal playground, so let me play please!

So... why not switch to a pre-compiled minified template of your ZPT/Chameleon files when you are in "production mode"?

Obviously during development you will use the original template files.

The interesting side of this approach is that there is no overhead at response time, since the minification task runs just one time before deploying your application. It might be an option if you want just your html minified and you cannot feasibly add to your site or project additional optimization tools at web server level.

Anyway I have tried this mad experiment and... if you don't use too aggressive minification params, it seems to work fine with good results. Try it at your own risk or just disable it. Here you can the effects on the generated index.html used in production:

Template minified (7.62 kB -> 4.16 kB)

Result: a lighter Pyramid

Same results but a lighter Pyramid app:

Let's see how it behave the standard Pyramid starter project:

Standard Pyramid starter project (production.ini)

And the Pyramid starter seed:

Pyramid starter seed (production.ini)

As you can see the seed version is ~38 Kb smaller and more performant.

Useful links

• uncss - http://addyosmani.com/blog/removing-unused-css/
• alternative installation of nodejs - http://greenfinity.hu/en/blog/installing-javascript-from-zc-buildout
• performance optimization techniques - http://www.smashingmagazine.com/2014/09/08/improving-smashing-magazine-performance-case-study
• @CSSconfeu CSS performance tooling slides (updated 20140919) - https://speakerdeck.com/addyosmani/css-performance-tooling 

That's all?

No, you can do more, for example:

• reduce the  number or requests (for example you can merge vendor.css and app.css)
• create and keep updated css sprites with grunt (https://github.com/Ensighten/grunt-spritesmith)
• manage and upload all your assets to professional services like Amazon AWS (for example you can serve up all your images, styles and scripts from a S3 bucket + CloudFront). This way Pyramid will be able to handle more requests. Pyramid let you put static media on a separate webserver during production with static_url() in conjunction with add_static_view(), without having to change your templates code
• generate static gzipped assets with Grunt and let your webserver serve them
  
• install and configure dedicated performance modules at webserver level (Apache's mod_pagespeed)

Let me know what you think about that, please. Hope soon I will manage to write the second part of this blog post explaining how I did it. In the mean time you can:

• have a look at the pyramid_starter_seed repository on Github https://github.com/davidemoro/pyramid_starter_seed
• follow me on Twitterhttps://twitter.com/davidemoro
• keep in touch with me on Google+ (https://plus.google.com/+davidemoro) or Linkedin (https://www.linkedin.com/in/davidemoro82)

Links

• pyramid_starter_seed (github repo): https://github.com/davidemoro/pyramid_starter_seed
• part 1 (benefits of yeoman fully integrated with a web framework like Pyramid, this blob post): http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeomam-part-1.html
• part 2 (how to install pyramid_starter_seed ant its prerequisites) - http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeoman-part-2.html
• part 3 (how to achieve these results and manage things with grunt, personalize pyramid_starter_seed and create your own starter template) - http://davidemoro.blogspot.com/2014/09/pyramid-starter-seed-yeoman-part-3.html