WebAlchemy vs StaticGenerator

When two months ago I published WebAlchemy code, several people asked me how to make it working with nginx or lighttpd. Yesterday new tool similar to WebAlchemy StaticGenerator was published. Now it seems right time to explain how to use WebAlchemy with nginx and other servers that don't have .htaccess files, highlight some important differences between WebAlchemy and StaticGenerator and say thanks to Django for the signal framework that makes it all possible.

I designed WebAlchemy to support publishing of resources of any content type with any URL scheme and ability to be used in RESTful environment when GET-requests to same url are served as static requests, but POST-, PUT- and other requests are served by Django. It means that for example request to /feed/comments/ that also served by WebAlchemy on this site should return application/rss+xml content type, but request to normal page like /about/ should return text/html; charset=utf-8 content type. It's not possible to detect content type from url itself, so web-server should use resolved filename extension and we end with situation when we can't have single index file for directories and sometimes need to use index.html, sometimes index.xml, sometimes index.pdf, etc.

Other issue and important difference between WebAlchemy and StaticGenerator is additional load that each tool creates. StaticGenerator generates static versions of changed pages every time when master model is changed and doesn't take into account transaction, so same path may be transformed even several times during a transaction. WebAlchemy don't create additional load on web-server, it don't generates static version especially, but uses generated in result of real user request content (all future requests of this url are served as static).

Both WebAlchemy and StaticGenerator use post_save Django signals but WebAlchemy process all signals at once when request is completed and so it's able to remove duplicates (when changes in several models affects same path) and can't create problems for monitored models. However StaticGenerator generates static versions of resources for each signal, and if during generating of the resource an error will occur, saving of original model that fired the signal will be failed also.

So if you don't want to use .htaccess files, to resolve above problems you need to address different content types issue. For example for all non-html content it's possible to use URLs with file extension (/feed.rss2 instead of /feed/). It solves the problem, but limits you a bit in choosing of URL format.

When content types issues are resolved, you can use WebAlchemy with any web-server. You just need to use your trivial publisher that don't deal with .htaccess files. For example, you can add to publishers.py form WebAlchemy following TrivialPublisher or wrote your own.

class TrivialPublisher(object):
    """ Save/delete only page content and don't deal with .htaccess.
    This publisher can be used with any web-server that can check 
    presense of static file and send published version of the resource 
    if static file exists.
    The server configuration may look like (used nginx syntax):
    http {
      upstream django {
        server localhost:8000;
      }
      server {
        server_name  example.com;
        root   /var/www/;

        location / {
          if (-f $request_filename/index.html) {
            rewrite (.*) $1/index.html break;
          }
          if (-f $request_filename/index.xml) {
            rewrite (.*) $1/index.xml break;
          }
          if (!-f $request_filename) {
            proxy_pass http://django;
            break;
          }
        }
      }
    }
    """

    def __init__(self, root):
        self.root = root

    def resolve_path(self, path):
        """Return cached path to be used in mod_rewrite for given path.
        >>> p = TrivialPublisher('/var/www/')
        >>> p.resolve_path('/')
        '/var/www/index.html'
        >>> p.resolve_path('/dir/')
        '/var/www/dir/index.html'
        >>> p.resolve_path('/dir/noext')
        '/var/www/dir/noext'
        >>> p.resolve_path('/dir/file.css')
        '/var/www/dir/file.css'
        """
        if path[-1] == '/':
            path += "index.html"
        return os.path.join(self.root, path[1:])

    def save_file(self, path, response):
        fpath = self.resolve_path(path)
        if not os.path.exists(os.path.dirname(fpath)):
            os.makedirs(os.path.dirname(fpath))
        write_file(fpath, response.content)

    def delete_file(self, path):
        fpath = self.resolve_path(path)
        remove_file(fpath)

Nothing in original WebAlchemy core code was changed so you can use it. Link to source code and instructions for WebAlchemy configuration you can find at original blog entry WebAlchemy accelerates Django in 100 times.