Running Django on Windows (with performance tests)

Django is a trendy Python-based framework for web applications. Its ever-growing popularity owes to availability of quick development tools, inbuilt administrative interface and high operation speed. Unfortunately, until now there was no easy tried-and-true way to deploy and run Django-apps on IIS web-server.

Introducing Helicon Zoo – a repository of popular web-frameworks for Microsoft IIS. It bases on Microsoft Web Platform Installer (WebPI) technology to deploy apps. WebPI also handles different dependencies and takes care of installation of necessary components, such as Python, Django, various database drivers and modules. And, of course, Helicon Zoo itself which sticks it all to MS IIS 7.

How to use

Web development process implies use of two relatively independent environments – development and production. Helicon Zoo may be used in production as well as on developer’s machine, or in both places. In either case the sequence of actions might be:

To start, you need to download Web Platform Installer from Microsoft website (http://www.microsoft.com/web/downloads/platform.aspx) and install it. WebPI already includes wide range of frameworks and applications for IIS like PHP, ASP.NET, WordPress, Drupal, phpBB. To launch Helicon Zoo add new feed to WebPI:

• Run WebPI and click Options
  
• In «Display additional scenarios» box put http://www.helicontech.com/zoo/feed and click Add feed
  

If you now go to Zoo, Applications (in WebPI interface) you’ll find some new apps: Blank Django Project, Blank Rails Project, Blank Perl Project, Blank Mojolicious Project:

Blank Django Project is the simplest “Hello, World!” app used to set up all dependencies needed for Django-app to run and empty Django-project itself. For installation press “Add” and “Install”; then you’ll be shown the list of dependencies:

To accept licenses and start downloading and installing the project press “I Accept”. When done, the newly-installed project can be run by clicking “Launch application in browser”:

This screen symbolizes that Python, Django and the rest of things needed to run the apps were successfully installed on your PC. Now, if it’s a dev environment, you can start creating based on this empty “Hello World” app. After you make some changes to the app, you may upload it to production server with all files, folders, necessarily web.config, and if Blank Django Project was previously installed on the server (regardless on the destination folder as only its dependencies matter), the app is going to work. Every time you upload changes to the server restart Application Pool. This is needed for Django (and Rails) only, other frameworks probably won’t need it. Upon restart automatic data migration will occur – manage.py syncdb. Owing to this fact solution may be used on shared hostings where the user can’t access console.

Using WebMatrix

Another way to go is to use WebMatrix for development. Instead of installing “Blank Django Project” go to Zoo->Packages and install “WebMatrix Templates” and “Python Hosting Package”.

After that run WebMatrix and create new application from template. You will be proposed number of templates to choose from. Please select “Django site” and create it.

Then to deploy your web site to a remote IIS server “Python Hosting Package” has to be installed on this server to run Django applications.

Under the hood

Helicon Zoo core is represented by native IIS module acting as a bridge between IIS web-server and frameworks using Ruby, Python, Perl etc. the module makes use of FastCGI protocol which is recognized as quick and accurate means for interaction between web-apps and web-server. This interaction is asynchronous and uses I/O Completion Port technology. For transport Zoo uses either named pipes or TCP-sockets. Web-servers supported include IIS 7, IIS 7.5 and IIS Express.

Basic Helicon Zoo configuration is stored in <heliconZooServer> section of applicationHost.config. This section includes descriptions of all FastCGI-drivers to be operated via Zoo. Here’s an example of driver description for running Python wsgi-apps (Django-apps in particular), zoofcgi.py is a worker written in Python that provides transport through named pipes:

<engine name="python.2.7.pipe"
     fullPath="c:\python27\python.exe"
     arguments="-O %SystemDrive%\Zoo\Workers\python\zoofcgi.py"
     transport="pipe" />

Web.config

Responsible for configuration of web apps working through Zoo is web.config file. Following is the example configuration of Django-app to work in 32-bit pool:

<?xml version="1.0" encoding="UTF-8"?>
 <configuration>
  <system.webServer>
   <heliconZoo>
    <application name="django.project.x86" >
     <environmentVariables>
      <add name="PYTHONPATH" value="%APPL_PHYSICAL_PATH%;%PYTHONPATH%" />
      <add name="DJANGO_SETTINGS_MODULE" value="settings" />
          <add name="DEPLOY_FILE" value="deploy.py" />
          <add name="DEPLOY_LOG" value="log\deploy.log" />
     </environmentVariables>
    </application>
   </heliconZoo>
   <handlers>
     <add name="django.project.x86"
                scriptProcessor="python.2.7.pipe"  path="*" verb="*"
                modules="HeliconZoo_x86" preCondition="bitness32"
                resourceType="Unspecified" requireAccess="Script" />
    </handlers>
  </system.webServer>
 </configuration>

Some important comments on the above config:

• scriptProcess="python.2.7.pipe" is a reference to FastCGI-drived defined in applicationHost.config;
• PYTHONPATH environment variable contains a path where Python looks for its modules; in or case we are adding a path to our Django-app into PYTHONPATH;
• DJANGO_SETTINGS_MODULE environment variable storing the path to Python configuration file settings.py for Django-app, e.g. ‘mysite.settings’
• ‘DEPLOY_FILE’ environment variable contains the path to the file within the app which will be run before the first request to the app after restart of IIS-app. This file may be used for DB synchronization, for example: ‘manage.py syncdb –noinput’;
• ‘DEPLOY_LOG’ environment variable contains the path to the file within the app which is the destination for debugging info generated by DEPLOY_FILE command. The folder containing this file must be granted Write permissions for the user running this IIS-app (usually it’s IIS_IUSRS group).

Static content

Django framework is not suitable for fast and safe processing of static files (images, scripts, css). Statics must be processed directly by web-server. To do so all static files must be stacked in one folder and Django request handler for this folder is then switched off like this:

<?xml version="1.0" encoding="UTF-8"?>
 <configuration>
  <system.webServer>
   <handlers>
    <remove name="django.project.x86" />
   </handlers>
  </system.webServer>
 </configuration>

Sample Django-project for IIS 7

Let’s have a Django-project consisting of 3 apps:

• site — the site itself with templates, statics, urls.py nad settings.py
• blog
• store

Static files located in site/media are available in the /media/ virtual folder (i.e. MEDIA_URL=’/media/’)

Having installed Blank Django Project in the root of the site and having copied our django-project, we’ll get the following structure:

/media/ is a virtual folder pointing at site/media; Zoo module is switched off for this folder as mentioned above. PYTHONPATH in root web.config points at site root and DJANGO_SETTINGS_MODULE is set to ‘site.settings’:

...
  <heliconZoo>
    <application name="django.project.x86" >
     <environmentVariables>
      <add name="PYTHONPATH" value="%APPL_PHYSICAL_PATH%" />
      <add name="DJANGO_SETTINGS_MODULE" value="site.settings" />
     </environmentVariables>
    </application>
   </heliconZoo>
...

Performance

And now performance test for dessert. Testing machine acting as server was Core 2 Quad 2.4 GHz, 8 Gb RAM, Gigabit LAN. To generate load we used more powerful PC with Apache Benchmark. To measure Apache and Nginx performance Ubunthu 11.04 Server x64 was used. IIS 7 tests ran on Windows Server 2008 R2. No virtual machines – only bare hardware. As transport on Nginx we used the most advanced uwsgi, as well as wsgi and fast_cgi for comparison. On IIS 7 we’ve also compared with PyISAPIе.

There where two Django scripts created as testing pages. The first one outputs current time in high resolution; this is done to ensure pages are not taken from cache. The second one does the same but previously saves the result into database. It’s all done using templates in order to apply real Django infrastructure; DB used is MySQL. All settings were left default as the task was to test the most common configurations. Here are results (in requests per second):

No surprises here as Python performance on Windows may be slower than Ubunthu version. Taking this into consideration Helicon Zoo transport performance should be really high. Uwsgi is ahead probably due to closer integration with Django.

The results for the second script are not that smooth. Why Nginx + fcgi + MySQL showed only 175 requests per second remains unknown. MySQL on Windows score is also frustrating, although on shared hosting the problem might not be that critical. The thing is that performance drops due to internal MySQL locks, while the server is not even loaded for 20% while generating these 104 requests per seocnd. It’s reasonable to assume that by increasing number of sites on the server and consequently the number of DBs, if they do not interlock with each other, the total server performance will be acceptable.

Thus we decided to add MS SQL Express into the tests. The result was easy to explain with Python and its database driver being the bottleneck, though in general the picture is quite promising. Unfortunately PyISAPIe was unable to work with MS SQL Express and was excluded from tests.

It is worth to mention the ability of IIS 7 to handle great number of connections. IIS 7 + Helicon Zoo easily held thousands of concurrent connections, we simply didn’t have testing powers to generate enough connections to trigger any problems in this test. Ubuntu with default settings started throwing connection failures when the number of connections increased. Moreover, Apache appeared to be greedy for memory. During the test with the number of connections going up Apache swallowed about 3 GB in 20 seconds.

Resume

Suggested solution proved stable and efficient. It will perfectly fit development environment as well as production. Advantageous is possibility of using Helicon Zoo by different Windows shared hosting providers to offer Django services to their clients. Hope is that with the growth number of Django servers on Windows Python developers will pay more attention to code debugging and optimization for Windows platform. And the army of existing Windows-developers can also contribute to current open-source projects.