The quest for MacOSX 10.5, Apache, Python, Django, and mod_wsgi

Posted by Andy on October 02, 2009
Computers, Internet, Software Development

It all started when I decided to learn the Django web framework for Python.  I ran through a bunch of examples on the Django site, and in the book Practical Django Projects, Second Ed. and was overall really impressed with the framework and the provided tools.  It has a lot of really useful features, and makes web development a lot of fun.  During my “evaluation” of Django, I was testing everything using the out-of-the-box dev web server that comes with Django, but I eventually came to the point where I wanted to try to host a Django site “for real” in Apache.  I checked the Django deployment documentation, and decided to try to get the suggested mod_wsgi module setup in my local Apache web server, on my Mac Mini (version 10.5.8 – “Leopard”).

At that time (a few hours ago), I was using the default install of python that came with my Mac Mini: version 2.5.1.  I thought it would be a good idea to download and install the latest version of Python (at least the latest 2.x version, as the 3.x versions are still not fully supported by most Python toolkits), so I could be on the bleeding edge.  Little did I know that the latest version 2.6.3 was being released as I spoke (Oct 2)!  I went to download the 2.6.3 version, and found that the MacOSX .dmg download link was broken on the Python download page.  (This will probably be fixed by the time anyone reads this, but this link was throwing a 404 error when I tried!).  Since the .dmg was not available, I had to give up on the “easy route.”  The only download of 2.6.3 available was the source code .tar.gz.  After download this and screwing around with it for awhile (couldn’t figure out exactly which options to use in configure and make…), I decided to chicken out and look for the most recent version that provided a .dmg install for Mac.  2.6.2 from April 2009 had a .dmg, so I went ahead an installed that.  After removing the default /usr/bin/python related links, I now had a new install of Python 2.6.2 in /usr/local/bin.  So far so good.

The next step would be to get mod_wsgi setup in my current Apache install (version 2.2).  I followed the directions on the mod_wsgi install documentation, and almost got through them.  I read through the MacOSX-specific install instructions, and didn’t really see anything that applied, so I figured I was okay.  I modified the httpd.conf file to load the mod_wsgi module, and this is where the fun started.  When I tried to restart Apache, I got the following error:

httpd: Syntax error on line 117 of /private/etc/apache2/httpd.conf:
Cannot load /usr/libexec/apache2/mod_wsgi.so into server:
dlopen(/usr/libexec/apache2/mod_wsgi.so, 10):
Symbol not found: _PyExc_RuntimeError
  Referenced from: /usr/libexec/apache2/mod_wsgi.so
  Expected in: dynamic lookup

I searched around for some more info, and after reading through a bunch of blog posts and forums, I learned that this was most likely happening because the Python shared library was only build in 32-bit mode, whereas, the default build of Apache on MacOSX 10.5 runs in 64-bit.  At least that’s what I made of it, not sure if that’s completely accurate…  The docs on the mod_wsgi site for MacOSX mention that you need to add extra -arch flags to the Makefile when building mod_wsgi for 64-bit, but that wasn’t the problem.  Those flags were correctly generated by the configure script, and my mod_wsgi.so library was built with full 32 and 64-bit architecture support.  The problem appeared to be that when Apache tried to load the mod_wsgi.so, the mod_wsgi.so was trying to load the default Python library, which was only built for 32-bit.

I searched around some more, and found this excellent blog post:  http://blog.captnswing.net/2009/04/19/python-mod_wsgi-64bit-mac-os-x-105/ on this very subject.  The post describes the exact problem I was having, and provides the steps necessary to build Python in the full “fat” architecture (all 4 archs), as well as mod_wsgi.  I was about to try out the described steps, when I noticed a link to another blog:  http://gidden.net/tom/2008/06/30/mysql-and-pdo-on-os-x-leopard-intel/comment-page-1/#comment-16477.  This procedure seemed much simpler, so I decided to try that first.  It would involve over-riding the default launchd configuration for Apache, so that the process would be launched in 32-bit mode, rather than 64-bit when the Mac starts up.  I tried this out and eventually got it to work, but the problem with this is that if you do a “sudo apachectl restart”, it ignores your launchd configuration and starts apache up in the normal 64-bit mode.  I decided to give up on this and keep looking.

I then found this blog post:  http://codesnippets.joyent.com/posts/show/1328.  This post presents a novel idea:  avoid all the trouble with 64-bit, and just dumb Apache down to the sane level of 32-bits!  I’m not sure why Apple decided to have Apache run in 64-bits when everything else basically runs in i386 (32-bit) mode.  I thought this might be a nice, simple approach, so I’m going to give it a whirl…

This post is already too long, so I’ll post an update if I come up with any new information!

Tags: , ,

4 Comments to The quest for MacOSX 10.5, Apache, Python, Django, and mod_wsgi

  • Rather than search through all those blogs, you could have just read:

    http://code.google.com/p/modwsgi/wiki/InstallationOnMacOSX

    It mentions how to build fully fat framework from source code as well as mention tricks for thinning Apache etc if absolutely needed.

    The particular error you are getting also possibly not entirely related to that anyway and may be because wrong version of Python framework being picked up at run time. You need to check with ‘otool’ what mod_wsgi.so thinks it will find. If it isn’t right, you need to change how mod_wsgi is linked against Python framework.

    Easiest thing to do is make sure using mod_wsgi 2.6 and if standard configure doesn’t work, use the ‘–disable-framework’ option to configure to force linking via old style library reference. Don’t know why but different Python installations and operating system versions appear not to be consistent as to what works.

    Also read:

    http://code.google.com/p/modwsgi/wiki/ChangesInVersion0206

    as it mentions the linking issues that may be causing this and that configure option.

    BTW, asking for help on the mod_wsgi list on Google Groups is always a good idea.

  • Hey Graham, thanks for your comment! I’m fairly new to Apache, modules, and dealing with 64-bit on Mac, among other things, so this was a good learning experience. I did read through that wiki for wsgi on MacOSX, and the problem was not with mod_wsgi.so. I verified that the Makefile was building mod_wsgi with the full “fat” set of architectures. I believe the problem I was seeing was that the Python library was only built in 32-bit mode. I figured the solution to this problem would be to either try to build the Python library with all archs, or thin Apache down to 32-bits. I was hoping to not have to thin Apache down, but that seems to be the easiest solution. What are your thoughts?

  • I looked into this and decided (like you) for now to thin Apache in the ‘reversible’ manner suggested, which works under my 32 bit Python 2.6 and OS X 10.5 Intel system:

    http://groups.google.com/group/modwsgi/browse_thread/thread/cacddadc0c3c8e93

    However, the Python 2.7 OS X installer is now ‘full fat’ (unlike the 32 bit 2.6 version), so – as Graham suggests – I could restore the 64 bit Apache and try again with everything full fat. Although, the 32/64 bit architecture issue first surfaced when I used easy_install for MySQL-Python, so I’d have to re-install that too, but (at time of writing) it is only officially supported up to Python 2.6.

    Then there’s the re-install and configure MySQL as 64 bit, and time soon runs out, when all I really wanted to do (oh, about 3 months ago) was to get that little Django blog app working locally under mod_wsgi before deploying… well, at least I can now do that. The rest will have to wait for another slack weekend or so…

  • a quick way to find out which archs a binary supports is to use ‘file’. Works on shared libs and executables.
    $ file mod_userdir.so
    mod_userdir.so: Mach-O universal binary with 4 architectures
    mod_userdir.so (for architecture ppc7400): Mach-O bundle ppc
    mod_userdir.so (for architecture ppc64): Mach-O 64-bit bundle ppc64
    mod_userdir.so (for architecture i386): Mach-O bundle i386
    mod_userdir.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
    $ file bin/python
    bin/python: Mach-O universal binary with 2 architectures
    bin/python (for architecture ppc): Mach-O executable ppc
    bin/python (for architecture i386): Mach-O executable i386

    as you can see I have the same problem. maybe i’ll just recompile Python when I get another free day.

Leave a Reply

Your email address will not be published. Required fields are marked *