The quest for MacOSX 10.5, Apache, Python, Django, and mod_wsgi

Posted by Andy on October 02, 2009
Computers, Internet, Software Development / 4 Comments

It all started when I decided to learn the Django web framework for Python.  I ran through a bunch of examples on the Django site, and in the book Practical Django Projects, Second Ed. and was overall really impressed with the framework and the provided tools.  It has a lot of really useful features, and makes web development a lot of fun.  During my “evaluation” of Django, I was testing everything using the out-of-the-box dev web server that comes with Django, but I eventually came to the point where I wanted to try to host a Django site “for real” in Apache.  I checked the Django deployment documentation, and decided to try to get the suggested mod_wsgi module setup in my local Apache web server, on my Mac Mini (version 10.5.8 – “Leopard”).

At that time (a few hours ago), I was using the default install of python that came with my Mac Mini: version 2.5.1.  I thought it would be a good idea to download and install the latest version of Python (at least the latest 2.x version, as the 3.x versions are still not fully supported by most Python toolkits), so I could be on the bleeding edge.  Little did I know that the latest version 2.6.3 was being released as I spoke (Oct 2)!  I went to download the 2.6.3 version, and found that the MacOSX .dmg download link was broken on the Python download page.  (This will probably be fixed by the time anyone reads this, but this link was throwing a 404 error when I tried!).  Since the .dmg was not available, I had to give up on the “easy route.”  The only download of 2.6.3 available was the source code .tar.gz.  After download this and screwing around with it for awhile (couldn’t figure out exactly which options to use in configure and make…), I decided to chicken out and look for the most recent version that provided a .dmg install for Mac.  2.6.2 from April 2009 had a .dmg, so I went ahead an installed that.  After removing the default /usr/bin/python related links, I now had a new install of Python 2.6.2 in /usr/local/bin.  So far so good.

The next step would be to get mod_wsgi setup in my current Apache install (version 2.2).  I followed the directions on the mod_wsgi install documentation, and almost got through them.  I read through the MacOSX-specific install instructions, and didn’t really see anything that applied, so I figured I was okay.  I modified the httpd.conf file to load the mod_wsgi module, and this is where the fun started.  When I tried to restart Apache, I got the following error:

httpd: Syntax error on line 117 of /private/etc/apache2/httpd.conf:
Cannot load /usr/libexec/apache2/mod_wsgi.so into server:
dlopen(/usr/libexec/apache2/mod_wsgi.so, 10):
Symbol not found: _PyExc_RuntimeError
  Referenced from: /usr/libexec/apache2/mod_wsgi.so
  Expected in: dynamic lookup

I searched around for some more info, and after reading through a bunch of blog posts and forums, I learned that this was most likely happening because the Python shared library was only build in 32-bit mode, whereas, the default build of Apache on MacOSX 10.5 runs in 64-bit.  At least that’s what I made of it, not sure if that’s completely accurate…  The docs on the mod_wsgi site for MacOSX mention that you need to add extra -arch flags to the Makefile when building mod_wsgi for 64-bit, but that wasn’t the problem.  Those flags were correctly generated by the configure script, and my mod_wsgi.so library was built with full 32 and 64-bit architecture support.  The problem appeared to be that when Apache tried to load the mod_wsgi.so, the mod_wsgi.so was trying to load the default Python library, which was only built for 32-bit.

I searched around some more, and found this excellent blog post:  http://blog.captnswing.net/2009/04/19/python-mod_wsgi-64bit-mac-os-x-105/ on this very subject.  The post describes the exact problem I was having, and provides the steps necessary to build Python in the full “fat” architecture (all 4 archs), as well as mod_wsgi.  I was about to try out the described steps, when I noticed a link to another blog:  http://gidden.net/tom/2008/06/30/mysql-and-pdo-on-os-x-leopard-intel/comment-page-1/#comment-16477.  This procedure seemed much simpler, so I decided to try that first.  It would involve over-riding the default launchd configuration for Apache, so that the process would be launched in 32-bit mode, rather than 64-bit when the Mac starts up.  I tried this out and eventually got it to work, but the problem with this is that if you do a “sudo apachectl restart”, it ignores your launchd configuration and starts apache up in the normal 64-bit mode.  I decided to give up on this and keep looking.

I then found this blog post:  http://codesnippets.joyent.com/posts/show/1328.  This post presents a novel idea:  avoid all the trouble with 64-bit, and just dumb Apache down to the sane level of 32-bits!  I’m not sure why Apple decided to have Apache run in 64-bits when everything else basically runs in i386 (32-bit) mode.  I thought this might be a nice, simple approach, so I’m going to give it a whirl…

This post is already too long, so I’ll post an update if I come up with any new information!

Tags: , ,

Kayak log: The Numbers 2009-07-25

Posted by Andy on July 29, 2009
Kayaking / No Comments

Date: 2009-07-25 10am

Run: The Numbers to Buena Vista Park (Arkansas River)

Flow: ~600cfs

Class: IV-

Temperature: Warm at first, then rainy

Buddies: Fino, Chadwick Shoults

Description:  A fun run down from Numbers to Buena Vista.  This was Chadwick’s first time down The Numbers, and he did really well.  At this flow, The Numbers becomes much more “creeky” than at the previous flow of 1800 or 2000.  I thought #1, #5, and #6 were the best rapids.  #1 was way less pushy than at higher flows, but there were a lot more rocks and boily eddies to maneuver around.  #2 was fun and not too bad.  #3 and the supposed #3.5 were still a bit of a mystery as to which was which.  After #2 there is a decent-sized drop, which we thought might be #3, followed by a longer, wavy drop which we thought might be #3.5.  I don’t really know if there is a #3.5, maybe the longer one was actually just #3.  I guess I’ll have to go with a guide sometime and figure that out.  #4 was good, I went down the chute on the right side, just to the left of where the irrigation channel starts on river-right.  Fino and Chadwick went down the left side, which was a bit bumpier, but didn’t look too bad.  #5 was good, it’s much easier at this flow.  After the initial drop-in, you just have to make you way around a few rocks and small holes and it’s over.  #6 was a surprise, I had forgotten about this one – it might have actually been the best rapid of the day.  This is where the “photogs” hang out for the commercial trips.  The lead in is a little tricky with some holes and rocks, and then there’s a sizable hole near the bottom.  Cheez had a little trouble in the hole area, but made it out fine.  The rest of the run was fun.  After the railroad bridge, I guess we went through #7, but I’m not sure which one it was.  The rest of Fractions was fun, but then the river kind of died.  Frog Rock seemed much different at this flow.  You can clearly see the nasty pin spot on the right, but the left side was cake.  House Rock was the last rapid of the day.  I went down the left side and slipped between two pointy rocks.  Fino and Cheez went down the right.  Chadwick kind of slammed into the wall and had to roll.  He then proceeded to give the finger to House Rock, his life-long nemesis.  The rest of the run got a bit painful, a lot of flat water and those annoying waves that splash you just enough to be annoying.

Overall, it was a great run, but I think at this flow, it’s best to just get out at Railroad Bridge and skip Fractions/Frog Rock.  The river really peters out, and it becomes a bit too long for one sitting.  However, low flow is a great time to take a first-timer down the numbers, so they can get a good feel for what it might look like when the flows come up.

Carnage:  I made it completely clean, which was nice.  Chadwick had a few rolls, but not bad at all for a first time!

Tags: , ,

Kayak log: Upper Clear Creek 2009-07-07

Posted by Andy on July 08, 2009
Kayaking / No Comments

Date: 2009-07-07 6:30pm

Run: Upper Clear Creek (Kermitt’s to mile marker 260.6)

Flow: 550cfs

Class: IV

Temperature: Warm

Buddies: Fino

Description: This was my first run down Upper Clear Creek.  The first two drops (beaver 1 and 2) were fun and not too difficult.  The first one was more straight-forward.  We scouted this one before running – left of center, there is a pillow of water that builds up.  A good line is to basically try to get on top of the pillow and ride it all the way down, and miss the hole at the bottom.  We didn’t scout beaver 2, so we just made a best-guess line.  I went over a small drop on the left, and rode the tongue down to the bottom.  At the bottom there is a wall-like pillow of water on the left, which unexpectedly flipped both Fino and I.  Next time, I’ll be ready.  After that, there was some bony boogie water, leading up to the biggest rapid on the run.  Fino and I thought that this large drop was called “Cabbage Shredder,” but after looking at CRC and WWSR tonight, I think it actually was “Double Knife.”  We were thinking Double Knife was the last big rapid on the run, but I think that’s actually called “Terminator.”  Anyway, the first major rapid was definitely the hardest.  There is a nasty shelf pour-over at the top on the right.  We both took the left line at the top and worked to the center and right near the bottom, avoiding some nasty, sharp rocks and holes.  After that, there was some more boogie water, finally leading up to what I guess is Terminator.  This rapid has a large boulder on the right at the top, and another large boulder at the bottom on the left.  A large hole develops at the bottom left, but can be easily avoided (this must be the terminator hole.)  The rest of the run is easy class III-.  The takeout is a large asphalt pullout at mile market 260.6, but it seems like you can go quite a bit further before Blackrock starts.

Overall, it was a fun run.  I’d say it’s a little more white-knuckle than Lower CC, due to the nasty rocks, and larger drops, but less continuous class IV compared to LCC.

Carnage:  Rolled at beaver 2 (pillow wall got me), but cleaned the rest.

Tags: , ,

RWSD: Service “good-practices”

Posted by Andy on June 30, 2009
RWSD, Software Development / No Comments

I thought I’d jot down a list of what I would consider to be good practices when developing software services.  I’m mainly a C#/.NET developer, so I’ll focus on exposing services via WCF, but most of these concepts are language “agnostic.”  A lot of these are pretty obvious and well-known; however, I’ve seen each of these violated enough times in the span of the last two-to-three months to make me think that maybe they are not so well-known…

  1. Don’t use a raw “object” type for any parameter, property, return value, or really anything else exposed as a service. Using a raw object makes it exceedingly difficult to achieve automatic serialization of messages coming in or out of the service method.  If you expose an “object” in your service, the caller has no idea what to pass in, and you have no way to deserialize whatever XML you end up receiving, unless you import XSDs from the client, which is a pain.  Importing XSDs from another company in your service creates an undesirable dependency between your service and the client.  Sometimes this is necessary, but dealing with WSDLs and XSDs is almost always more painful than it should be.
  2. Use low-level (primitive) datatypes as much as possible for the the properties of your input and output types, and avoid any objects that have properties that go more than two levels deep in object complexity. Web services are supposed to make it easy to achieve interop between any two languages, but I’ve never seen any web-service integration that was easy.  Internal services (where you can control all the datatypes) are usually less painful, but when dealing with external partners, trying to integrate with a foreign WSDL with a complex object model almost always requires some hacking.  The simpler the objects are, the easier it is to integrate.
  3. Don’t expose any object in your service that has anything to do with your backend data-access-layer. This creates a very undesirable dependency between your clients and your data-access layer.  If you expose a data object over the wire, and an external party integrates with your service, it now becomes very difficult to make any breaking changes to your data access layer.  Always create a simple data-transfer object to expose in your service, and hide everything having to do with your domain data.  Another reason to use simple DTOs is it forces any lazily-loaded data objects to be hydrated as you are copying the data to the DTO.
  4. If you need to leave room for extension in a service method, try to define concrete parameters for as many of the fields as possible, and add a string-string dictionary as the last parameter to your service method so the caller can attach additional data.  Another option is to expose an argument that can be subclassed to add functionality. I personally prefer to use the string-string dictionary, because dealing with subtypes adds an extra level of complexity for your integration.  The string-string dictionary is slightly painful, because it’s not really type-safe, requires clunky casting and validation, and is not easily documentable or intuitive to use.  This string-string dictionary is also a very useful technique for receiving and returning client-specific data in a service call.  A lot of times, the client will have their own identifiers for messages, and if you end up needing to perform asynchronous processing on the message, with a callback to the client, it becomes simple to store the data on your side, then return it back to the client with the callback, so they know what the response corresponds to on their side.
  5. With #4 said, avoid “untyped” data as much as possible. By “untyped,” I mean any data that has to be converted/cast at the receiving end before it can be used.  I’ve seen several services where the only argument exposed is a string-string dictionary.  With this, the client can pass whatever arbitrary keys they want, and you have to validate and cast the corresponding values before doing anything.  Try to use concrete, serializable types as much as possible.
  6. Don’t abbreviate anything! One of my coding pet-peeves is when people abbreviate arbitrary words in a method or variable name.  It takes a bit longer to type out the full word, but if you type it all out, you’ll never have to remember what you abbreviated, and the code becomes much more obvious to read and work with.  It is acceptable to abbreviate common acronyms, such as “Http,” or “Sql” however, but try to avoid abbreviating any domain-specific terms.
  7. Don’t define an argument or return type, then share it across multiple methods. There are a few cases where this might be useful, like if your methods all need to return the exact same data, but I’ve seen cases where the shared object ends up needing some custom parameters for certain methods.  When this happens, you end up adding some additional properties, which are coincidentally “given” to all the other methods.  Now you’re stuck with an overblown object that has too many properties for any one method, and requires additional validation code and error handling.
  8. Avoid throwing exceptions out of service methods. If possible, try to catch all exceptions within your service method, and return back some sort of status object indicating what went wrong.  Dealing with SOAP faults is yet another thing that adds an extra level of difficulty to any third-party integration.  I’m not a huge fan of dealing with error codes, but I think it’s actually cleaner and easier to deal with on the client end.
  9. Keep the service interface as compact and intuitive as possible. Avoid any generic methods that can be used to accomplish different types of business use cases.  Avoid “super” interfaces that provide all of the methods you need for an application.  Creating an interface with 20+ utility methods might seem like an easier way to deploy and integrate the service, but it creates a ton of undesirable dependencies between your client and your service.  Follow the “I” principle in the SOLID principles: “Interface Segregation” – segregate the parts of your interface into the smallest possible units that a client needs.  If a client only needs a few methods in your interface, don’t make them integrate with the entire thing.  By doing this you also give yourself the option to move the different services to different locations in your backend, without moving the entire thing, and gives you the ability to scale out parts of your system that might get hit harder.
  10. For heavy service loads, use a “HTTP-Queue” bridge to process messages in an asynchronous fashion. If you have any part of your system that has to process varying loads of messages, you can use an HTTP bridge to receive and enqueue incoming messages.  The HTTP layer exposes a web service to receive the messages in an interop-friendly format (SOAP, etc.), but simply passes the message directly onto a queue for processing by a backend service.  WCF makes this dead simple, you can setup a WCF web service application (hosted in IIS) to listen for messages, then setup a NET MSMQ binding between the web app and your backend Windows Service app.  Using queues makes it slightly more difficult to handle the response, because it requires a web-service callback to the client, but it’s a really solid pattern.
  11. If possible, don’t do anything overly fancy with the serialization of objects in your service. In many cases, your client will be using a different language than you to integrate with your service.  If you make the serialization of types complicated or non-standard, the client will most likely have problems integrating with your service using the default tools of their language.
  12. Keep everything as simple as possible! Try to avoid using any of the more advanced features that might be provided by your language.  I guess WCF is sort of an exception to this rule, since it abstracts away almost the entire service protocol layer, and does a ton of work behind the scenes; however, try to keep your service interface as clean, simple, and standard as possible.  Even the simplest of services can give your clients headaches to integrate with, so just try your best to make it easy on them.

Tags: , ,

iPhone 3GS not receiving incoming calls

Posted by Andy on June 28, 2009
iPhone / 1 Comment

I thought I’d blog this experience, in case anyone else runs into it. I just got a brand new iPhone 3GS, and had an issue receiving incoming calls. I was a T-Mobile customer up until now, so I had to transfer my phone number from T-Mobile.  On my new AT&T account, I could send and receive text messages, and make outgoing calls, but whenever someone would try to call me, it would go straight to voicemail, and if the person left a voicemail, I would not see it in my Voicemail list on the iPhone. If I dialed my own number, I could listen to the messages, but something was definitely not right. Apparently, this is not an uncommon issue for people with iPhones, there are a lot of articles and forum posts from people seeing the same problem.

I went through the following troubleshooting steps, with no success:

  1. Hard restart of iPhone (hold home and lock buttons simultaneously for 10 seconds)
  2. Disable WiFi
  3. Disable Bluetooth
  4. Disable 3G
  5. Reset network settings
  6. Reset all settings

I finally called the AT&T support (dial 611) on the phone.  I first talked to an Apple rep about the issue, and he went through the above steps, and I told him that I had tried all of those things.  He concluded that it was probably an issue with the phone number transfer from T-Mobile.  I called 611 again, and talked to an AT&T rep, and she tried something without success.  She decided that I needed to talk to a tech rep from AT&T, so I called 611 once again and got in touch with a tech rep.  He looked up my account and discovered that the “port” of my number from T-Mobile was still sitting in a pending state.  He was able to push the request through the system, and I think that fixed the problem.

The AT&T and Apple reps were really friendly and helpful.  If anyone is experiencing this problem, I’d highly recommend calling AT&T tech support, they were able to fix the problem for me quickly and easily.

Tags: ,