Web programming with Python

I want to redevelop a little application I made some years ago with Zope2.
But I want it to be smallest/simplest/smartest/pythonic possible. Hence multi-megabytes frameworks like Django or Turbogears aren't desired.

So I started to investigate...

After half a day looking for packages in my Debian Etch and reading documentation, I started to study more thoroughly many python+web-related software:

Template engine

  • Jinja (0.9) OK
  • templayer NOTOK
  • mako NOTOK
  • simpletal NOTOK
  • cheetah

I like Jinja because it's both small and simple to use and powerful all at the same time (remember "small is beautiful" ;-). Though maybe not the fastest one but a bit of Psyco will help for sure. The doc says it looks like Django a lot, which makes me wonder because it's a lot smaller. The filters system (inspired by Django) is amazing... I already plan on writing a "widget framework" through a type-specific view/subview system using a "view" filter.

Less appealing is the fact that the version I looked at in Debian is quite old, and since then Jinja evolved into something much bigger and somewhat more complex (in 1.2 and 2.x versions) despite not bringing much to the templates themselves.

Templayer is more about embedding direct Python inside pages, which I dislike. At least, Jinja is "sandboxed" with a restricted but sufficient expression language.

Same for Mako: despite it looks like Jinja (it actually inspired Jinja partially), and looking exhaustive, it is somewhat complex and embeds too much direct Python in the template for my own taste.

SimpleTAL templates are XML files with special "tal:" attributes on key elements. I want something more generic than XML-only.

Now that I think about this subject, I think it won't be very difficult (or big) to write an engine like jinja, since even simpler constructs would be enough. Basically, you need (assuming HTML or XML adapted syntax):

  • placeholders for data or text fragments: 
    ${object.attr.val}
  • expression language: the filter composition model seems to be smart and fine: 
    ${val|esc}
    ${val|format "%08X"}
    ${val|subst "x", "y"}
Though the single-side '|' call separator prevents more complex multi-arg function composition/call. 
  Even the "," comma is ambiguous: should v|f1 x,y|f2 be computed as (v|f1 x,y)|f2 or v|f1 x,(y|f2).
  • conditionals: 
    <?if cond1?>
    <?else if cond2?>
    <?else?>
    <?endif?>
  • peudo-variables/functions: 
    ${scope:function}
  • loops: 
    <?for var1[,var2...] in iter1[,iter2...] ?>
    ${var1}
    ${for:first/last/inner/only/iternum/odd/even/cycle/...}
    <?elsefor?>
    <?endfor?>
  • template composition: either inclusion with optional parameters (my preference) or inheritance; Inheritance seems more complicated and less natural because the syntax to override block definitions in derived templates is heavy and doesn't mix as natural HTML/XML in the page.
    Direct inclusion, no arguments: 
    <?include "template"?>
More elaborate sub-template call with arguments (ok, not quite lightweight yet): 
    <?call "template"[ with arg1="val1", arg2="val2"]?><?block "B1"?>...<?endblock?>...<?endcall?>

Templates could easily be compiled to byte-code on the fly with the compiler in the standard library, then eventually accelerated with Psyco (since it's essentially string handling).

Actually, by the time I think about it, I started writing a version of this. Here is ninja.py.

URL mapping to action/view/controllers

  • Routes (1.5.2)
  • CherryPy (2.2.x) NOTOK

Routes has a clean separation of mapping model between URLs and code: it parses specified URL parts into controller, action and arguments to be passed to functions/methods; Then it maps controller and action names to objects and methods through a registry. I find it's easier to work with, and more powerful; It was designed to implement RESTful easily, which is nice and seems effective.

CherryPy uses "exposition" of global or member functions to show them directly as URLs (with names from walking along the object tree). Query arguments (GET or POST) are converted to call arguments, which is quite pratical -- though I'm not sure how file uploads are handled. Looks somewhat difficult to get clean URLs since it makes the internal code structure appear near-directly outside. It's also oriented towards static/predefined URLs.

WSGI-hosting HTTP server

  • Flup (0.2126) NOTOK
  • wsgid (0.7, not in Debian) HALFOK
  • CherryPy (2.2.x / 3.1.x) NOTOK by itself, OK for bare server in wsgid
  • wsgiref, the standard implementation in Python 2.5 and later

Flup is more comprehensive than the Debian package description says. It has

  • server gateways for many protocols (including AJP that is more usual to Tomcat and Java application servers)
  • session, compression, error management
  • basic (but practical) request routing to functions

But actually Flup only has gateways, it is missing a plain HTTP server and hence doesn't meet my needs. Sad, it was quite near and doesn't miss a lot to achieve it :-/

CherryPy embeds a threaded WSGI server, URL mapping and other auth/session/cookie/etc tools. May look interesting at first, but the provided components aren't WSGI-based, so you end up installing a lot of things with it, of which very few are actually used, because you're going to install and use WSGI variants of those infrastructure components. Also, the thread-based model is interesting to share database connections, but not so to achieve high performance / parallelism (remember the Python "GIL" -- Global Interpreter Lock). For a web server I prefer the process/fork model, or a mixed one where I could limit the number of threads per process.

After some first tries with wsgid I was disappointed because it didn't work. A bit of debugging and corrections have shown some parts of it are full of bugs, so I abandonned it at first. At a 2nd look I saw it embeds the core HTTP server from CherryPy, so I tried to use it more directly and it was a success. Here is some sample code which works (with v0.7): 

    from wsgid.servers.cherrypy import WSGIServer
    from wsgid.main import get_config as get_wsgid_config
    import sys
    HTTP_OK = "200 OK"
    TYPE_TEXT = ("Content-type", "text/plain")
    def hello(env, start):
        start(HTTP_OK, [TYPE_TEXT])
        yield "Hello, world !\n"
    config = get_wsgid_config(sys.argv)
    server = WSGIServer(config, hello)
    try:
        server.start()
    except KeyboardInterrupt:
        print "Interrupt!"
        server.stop()