Using lesscss with Pyramid and Fanstatic

We all know twitter bootstrap. It became really popular in the past few months. We saw it all over the internet with same header and same colors.

In this post I'll show you how to customize twitter bootstrap with pyramid and Fanstatic. In other words, how to use lesscss (the language used by bootstrap) with pyramid.

lesscss allow to write dynamic css stylesheets with a specific language. You can find a lot of examples and documentation on their website.

In this example we will use:

  • pyramid_fanstatic: A Fanstatic integration as a pyramid tween. It also provide a pyramid scaffold and an helper to reload the development server when a static resource change.
  • js.lesscss: A Fanstatic package to embed lesscss as a python dependencie. This package was created by Stéphane Klein and I've update it to use lessc to compile .less files

Ok. Let's go!

Create a virtualenv and install a bunch of packages:

$ virtualenv myenv
$ cd myenv
$ source bin/activate
$ pip install pyramid_fanstatic js.lescss

Now that we have a virtualenv with all dependencies we need to create a pyramid application:

$ pcreate -s starter -s pyramid_fanstatic lesscss_example
$ cd lesscss_example

A README_FANSTATIC.txt file will show you how to finalize your installation. We need that because the pyramid_fanstatic scaffold does not redifine files like setup.py and __init__.py so you can use it with all scaffolds included in pyramid and other projects. Here is the file content:

To finalize your installation you'll need to follow those steps.

Add those line the app:main section of your development.ini:

[app:main]

fanstatic.bottom = true
fanstatic.debug = true

Add some requirements to your setup.py:

requires = ['pyramid', 'pyramid_debugtoolbar',
            'pyramid_fanstatic',
            # if you want to use lesscss
            #'js.lesscss'
            ]

Also add those entry points to the same file bellow the paste.app_factory:

# Fanstatic resource library
[fanstatic.libraries]
lesscss_example = lesscss_example.resources:library

# A console script to serve the application and monitor static resources
[console_scripts]
pserve-fanstatic = lesscss_example.resources:pserve

You also need to add pyramid_fanstatic tween to your application. Add the following to your __init__.py file:

config.include('pyramid_fanstatic')

Run python setup.py develop to get the pserve-fanstatic script available.

That's it. Once all those steps are done we can have a look at the resources.py created by pyramid_fanstatic scaffold. Here is a modified version to activate the main.less resource:

from fanstatic import Library
from js.lesscss import LessResource

library = Library('lesscss_example', 'resources')

less_resource = LessResource(library, 'main.less')


def pserve():
    """A script aware of static resource"""
    import pyramid.scripts.pserve
    import pyramid_fanstatic
    import os

    dirname = os.path.dirname(__file__)
    dirname = os.path.join(dirname, 'resources')
    pyramid.scripts.pserve.add_file_callback(
                pyramid_fanstatic.file_callback(dirname))
    pyramid.scripts.pserve.main()

The trick is to use a LessResource instead of the standard Fanstatic Resource. This class compile the .less file at init time. The pserve() function is the target of our pserve-fanstatic entry point. It run pyramid's pserve script after adding the Fanstatic resources to the list of files watched by the monitor used to reload the application when a file changed. Don't forget the --reload option when running this script else the monitor is not initialized.

Modify the main.less file in the resources directory:

$ cat lesscss_example/resources/main.less                                                                    ✹
// main lesscss style sheet for lesscss_example

@color: #ccc;
@border: thin solid black;

blockquote {background-color: @color; border: @border;}

Ok, we are ready to build our first view. Just add this to your __init__.py:

from lesscss_example.resources import less_resource


def home(request):
    """a simple view to test our resource"""
    less_resource.need()
    return {}

Fanstatic use a thread local registry to store the resource used by a request so the only thing needed is to use the resource method .need() and this resource will be injected in your html.

Let's register the view and it's associated route:

config.add_route('home', '/')
config.add_view(home, route_name='home', renderer='templates/mytemplate.pt')

mytemplate.pt is a very basic template with an empty <head />. Fanstatic will inject resources tags for you. You remember that ?

Easy, right ? Check the result with pserve development.ini. At this time nothing should work. Why ? Because we want to use lessc and we dont have it.

I'll not cover node.js / less installation here. You can find a lot of tutorial on other places. You can also find a buildout.cfg to get lessc in your buildout at the bottom of this post. Notice than lesscss only work with node 0.4.12 for me at the moment.

pserve-Fanstatic will try to find lessc binary in the following directories:

  • $PWD/bin
  • $HOME/bin
  • /usr/local/bin
  • /usr/bin

Once you got a lessc binary, run pserve-fanstatic:

$ pserve-fanstatic --reload development.ini

This will run the app in "compile" mode. This mean that the .less is compiled when you are starting the server but the browser will use the .less file. When you are ok with the result, restart the server with the standard pserve script. Your browser will use a compiled version of the .less file (.less.min.css).

You can have a look at the minified resource:

$ cat lesscss_example/resources/main.less.min.css
blockquote{background-color:#cccccc;border:thin solid #000000;}

That's nice. How about twitter bootstrap ? Let's try it. Checkout bootstrap in your resources/ directory:

$ git clone https://github.com/twitter/bootstrap.git lesscss_example/resources/bootstrap

And register bootstrap.less in the Fanstatic registry. Add this to your resources.py file:

bootstrap = LessResource(library, 'bootstrap/lib/bootstrap.less')

Register a view to test the new resource:

from lesscss_example.resources import bootstrap


def home2(request):
    """a simple view who use bootstrap"""
    bootstrap.need()
    return {}


config.add_route('bootstrap', '/bootstrap')
config.add_view(home2, route_name='bootstrap',
                      renderer='templates/mytemplate.pt')

And thats it. Run pserve-fanstatic and go to /bootstrap. Play with the variables.less and have fun. You'll get a bootstrap.less.min.css usable in production.

As a bonus I'll show how to use buildout to make all that stuff easier. This will install node.js and lessc so it can be used by pserve-fanstatic.

[buildout]
# run eggs, pserve and node parts
parts = eggs node
# set our project as a develop egg
develop = .
# dont check for new releases
newest = false
# prefer stable release (only buildout can do that!)
prefer-final = true

[eggs]
# the eggs part will install all dependencies and setup our application in
# development mode
recipe = z3c.recipe.scripts
eggs =
    pyramid
    lesscss_example

[node]
# install node.js + lessc
recipe = gp.recipe.node
# you can use an existing node binary
#binary = /usr/local/bin/lessc
npms = less
scripts = lessc

That's all... All this code is available as a github repository.

buildout vs pip. Why I choose buildout

25/35/2009 python buildout pip

pip and buildout are two way to deal with python eggs. Both have the same functionality eg. Install python eggs in a isolated environment.

I've learn buildout first because I came from the zope world and it's the standard way to install zope for a while now. But when pip appears I give it a try.

At the first look pip seems interesting because it allow to install a bundle of packages with fixed versions. But the main problem is that pip use virtualenv to isolate packages so each time you need a new environment you need a new virtualenv and fetching all packages again. This can take a lot of time if you are using lxm or python library with C code. And more and more if you have a lot of projects.

Instead, you can share eggs between buildout's directories. You just need to tell where they are. Add this to your ~/.buildout/default.cfg:

[buildout]
eggs-directory=/home/gawel/eggs

That's all. When buildout need a egg he'll try to find it in this directory before fetching. If no version is found the egg is fetched and installed in this directory. Of course, you can have more than one version per package. You can tell buildout which version to use (see bellow).

I'm a Pylons fan so I already have all packages needed in my ~/eggs directory to install a new pylons environment. Let's create a new project in an isolated environment:

gawel:~/tmp% date
Mar 25 aoû 2009 21:49:11 CEST
gawel:~/tmp% mkdir pylons
gawel:~/tmp% cd pylons
gawel:~/tmp/pylons% vi buildout.cfg
gawel:~/tmp/pylons% buildout
Creating directory '/Users/gawel/tmp/pylons/bin'.
Creating directory '/Users/gawel/tmp/pylons/parts'.
Creating directory '/Users/gawel/tmp/pylons/develop-eggs'.
Installing eggs.
Generated script '/Users/gawel/tmp/pylons/bin/paster'.
Generated script '/Users/gawel/tmp/pylons/bin/sphinx-build'.
Generated script '/Users/gawel/tmp/pylons/bin/sphinx-quickstart'.
Generated script '/Users/gawel/tmp/pylons/bin/sphinx-autogen'.
gawel:~/tmp/pylons% ./bin/paster create -t pylons myproject
Selected and implied templates:
  Pylons#pylons  Pylons application template

Variables:
  egg:      myproject
  package:  myproject
  project:  myproject
Enter template_engine (mako/genshi/jinja2/etc: Template language) ['mako']:
(...)
  Copying templates/default_project/test.ini_tmpl to ./myproject/test.ini
Running /Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python setup.py egg_info
gawel:~/tmp/pylons% date
Mar 25 aoû 2009 21:50:24 CEST

This take 1mn13s. Now try with pip. I'll not ;)

That's the main reason why I choose buildout.

buildout also make unit testing easyer. Just put this config file in your package root:

[buildout]
newest = false
parts = eggs
develop = .

[eggs]
recipe = zc.recipe.egg
eggs =
  YourPackageName
  nose

Run buildout. And you'll be able to run ./bin/nosetests in an isolated environment with your package installed in develop mode (develop = .). I have one in all my projects if you need some examples.

Another reason is that buildout can be extended easily. One feature that exist in pip but not in buildout is the ability to fetch eggs from VCS's urls. This is not a builtin feature in buildout. But I've created a buildout extension for that (gp.vcsdevelop). And you know what ? This extension use pip !! ;) By the way, there is no plugin system in pip AFAIK.

Now the last reason. I wonder how pip's users upgrade an existing project. Do they need to install another environment ? With buildout i'ts easy. There is an extension to list all packages versions used by a buildout project. You just need to use the generated file as a buildout's version.cfg and tell buildout to use it.

[buildout]
versions = versions.cfg
...

Then update this file on your production server. Run bin/buildout again. That's it. Your project is up to date and use the correct versions just because buildout create is own sys.path with required eggs.

gawel:~/tmp/pylons% cat bin/paster
#!/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python

import sys
sys.path[0:0] = [
  '/Users/gawel/eggs/Pylons-0.9.7-py2.6.egg',
  '/Users/gawel/eggs/PasteScript-1.7.3-py2.6.egg',
  '/Users/gawel/eggs/setuptools-0.6c9-py2.6.egg',
  (...)
  '/Users/gawel/eggs/WebHelpers-0.6.4-py2.6.egg',
  '/Users/gawel/eggs/Routes-1.10.3-py2.6.egg',
  ]

import paste.script.command

if __name__ == '__main__':
    paste.script.command.run()

For all those reason, I will not use pip for now.

I know that buildout is more complicated than pip. But it's also more powerful. So, are you planning to learn how buildout works ? If so, I've wrote a How To for Pylons. You can also find it on pylonshq. I think this can help you to learn buildout even if you don't plan to use Pylons (but you should too ;).

collective.releaser rocks !

02/12/2009 python buildout

Tarek have made some great works on collective.releaser and collective.dist the last months. There is currently a refactor branch.

The goal of collective.releaser is to normalize the release process of a buildout based project.

I've worked on it the last week to implement the latest needed steps to allow an easy way to release a buildout project.

So here is how it works. collective.releaser came with a paster template to generate a buildout skel. You got a dev.cfg for development with a develop option with all your develop eggs in it and a prod.cfg for production without develop and all versions fixed in the versions section.

I've modified the project_release so you just need to run:

$ bin/project_release --version=1.0

This will release eggs founded in the dev.cfg's develop option on a pypi server and fix the versions in buildout.cfg. If no version is founded for an egg in the versions section of the prod.cfg then the egg is released at the version passed to the command line. If a version is founded then the egg is released at this version. Here is an example:

The dev.cfg:

[buildout]
parts = eggs

develop =
  ../collective.releaser
  ../my.project

[eggs]
recipe = zc.recipe.egg
eggs =
  collective.releaser
  my.project

The prod.cfg:

[buildout]
parts = eggs
versions = versions

[versions]
collective.releaser=0.6.1

[eggs]
recipe = zc.recipe.egg
eggs =
  collective.releaser
  my.project

collective.releaser package is already at 0.6.1 so no release is needed. my.project as no versions specified so a new 1.0 release is generated and uploaded on your pypi server.

The generated buildout.cfg will look like this:

[buildout]
extends = prod.cfg
versions = versions

[versions]
collective.releaser=0.6.1
my.project=1.0

After all eggs are released, project_release will generate a tarball with the tagged version of the project and all eggs founded in the develop option.

Then to upgrade your project you just need to untar the archive on your production server. Launch bin/buildout. And that's it.

This will probably make life easier for developers ! Hope we can make a new self generated release soon :)

Fetching packages from various vcs in your buildout

28/44/2009 python buildout

I've recently released a buildout extension to checkout packages in a buildout from various vcs.

Here is a buildout config used to install this blog's skin package. This checkout deliverance from svn and pyquery from his bitbucket repository:

[buildout]
newest=false
parts = eggs
index=http://pypi.zopyx.com/
develop=
    .
extensions=gp.vcsdevelop
vcs-extend-develop=
    svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=deliverance
    hg+ssh://bb/olauzanne/pyquery/@tip#egg=pyquery

[eggs]
recipe=zc.recipe.egg
eggs=
    GawelOrgSkin

This extension is less than 100 lines of code. All checkout stuff is handle by pip which support many vcs.

Combine zc.buildout and pip benefits

07/08/2008 python buildout pip lxml

I've just released a new zc.buildout recipe which allow to install packages with pip.

I am getting really excited about it because it has many advantages taken from both components.

  1. The recipe adds a virtualenv in the parts/ directory of your buildout then use this binary to generate executable python scripts. So you have a clean sandbox.
  2. The recipe is based on zc.recipe.egg#scripts so you can share your eggs between buildouts as usual.
  3. Of course, you can install some .pybundle files.
  4. You can build package from svn with the editables option.
  1. Each line found in the install option is the last part of a pip command. This allow you to build eggs with dependencies. For example to install lxml in a pure sandbox without libxml2 and libxslt installed you need Cython installed and this command line python setup.py install --static-deps to install lxml. This is easy with the recipe. Here is a sample configuration file for this case:
[buildout]
# the cache dir is used by buildout & pip
download-cache = download
parts = eggs

[eggs]
recipe = gp.recipe.pip

# eggs installed by pip (also add the Deliverance bundle)
install =
    Cython
    --install-option=--static-deps lxml==2.2alpha1
    http://deliverance.openplans.org/dist/Deliverance-snapshot-latest.pybundle

# eggs installed by zc.recipe.egg
eggs =
    Paste
    pyquery

That's all !! This works perfectly on my Mac OSX and should works on most system.

In fact zc.recipe.egg will work in most cases but when you need a clean sandbox and some extra options, gp.recipe.pip is a good alternative.