Python as a Platform

Something that recently occurred to me is that the only operating system that doesn’t come with Python pre-installed on it is Windows.

While Linux and OS X both view Python as essentially a first-class development platform–i.e., as something that shrink-wrap applications can be built on–Windows does not. Instead, it’s generally expected that a Python-based Windows application be “frozen”: bundled into a self-contained package that includes a copy of the Python interpreter and whatever libraries it uses, which are private to the particular application. While this ensures that the application will function as expected and not run into “dependency hell”, it also results in a relatively large download–distributing a simple “Hello World” program is at least a megabyte in size, and makes extending the program’s functionality more difficult.

During the summer of 2007, I made simple Python-based tool for some friends who played World of Warcraft; for those that played on the Mac, the situation was easy because their computers already had Python on them. For the Windows users, however, I was faced with the dilemma of either “freezing” the program into a fairly large download which would be difficult to test and update, or making my friends download the Python installer from python.org and running my Python script from source. Either solution seemed too cumbersome.

So instead, I opted for a different solution: I’d create a tiny executable that would first check for the existence of a Python installation on the end-user’s system. If Python was already installed, the executable would simply extract a Python script from itself and run it. That script was much like ez_setup.py; along with a few tricks, it essentially ensured that my friends would transparently auto-update to the latest version of the tool whenever they launched the program.

If there was no Python installation on the end-user’s system, however, the executable would simply transparently download the latest Python installer from python.org, silently install it on the end-user’s system without any prompting, and then proceed as outlined above.

So, in other words, a small executable file–only about 50k in size–was essentially responsible for “bootstrapping” my program by downloading and silently installing Python if necessary, and then downloading and running the latest version of my program’s source code. This ultimately meant that the user would be able to run a self-updating Python program from a tiny executable without having to click through any installation prompts or dialog boxes.

I’m glossing over a few details here and there are definitely some holes in my implementation–for instance, if the user uses a web proxy, they’re hosed–but I thought the general idea was interesting.

The source for the executable was compiled using the Nullsoft Scriptable Install System, along with its InetLoad plug-in. The source file is below for those interested.

# This is a NSIS script that builds an executable that launches the
# WoW addon updater, installing Python beforehand if necessary.

# Requires the InetLoad plugin:
# http://nsis.sourceforge.net/InetLoad_plug-in

;--------------------------------
; General Attributes

Name "WoW Addon Updater"
OutFile "WowAddonUpdater.exe"

;--------------------------------
;Interface Settings

  !include "MUI.nsh"

  !insertmacro MUI_PAGE_INSTFILES
  !insertmacro MUI_LANGUAGE "English"


;--------------------------------
;Installer Sections

Function .onInit
    # Figure out if Python is installed by seeing if there's any
    # program associated with opening files with the .py extension.
    EnumRegKey $1 HKCR .py 0
    IfErrors InstallPython 0
        # Python is installed, so we want to just run the addon updater
        # as though this is the executable for launching it.
        SetSilent silent
        Return
    InstallPython:
        # Python isn't installed, so we want to act like an installer and
        # install it.
        SetSilent normal
FunctionEnd

Section "InstallAndRun"
    # If at all possible, extract the addon updater to WoW's root dir, just
    # so it's always running from the same location.
    ReadRegStr $0 HKLM "SOFTWARE\Blizzard Entertainment\World of Warcraft"      
                  "InstallPath"
    IfErrors 0 +2
        StrCpy $0 "$TEMP"
    SetOutPath $0

    IfSilent ExecutePyFile
        # Download and install Python.
        InetLoad::load "http://www.python.org/ftp/python/2.5.1/python-2.5.1.msi"
                        "$TEMP\python-2.5.1.msi"
        Pop $0
        StrCmp $0 "OK" +2
            Abort "Downloading of Python installer aborted."
        DetailPrint "Downloading of Python installer returned $0"
        # Install Python 'passively' so that the user doesn't have to click
        # anything; we want the installatio to be as unintrusive as possible.
        ExecWait '"msiexec.exe" /package "$TEMP\python-2.5.1.msi" /passive' $0
        DetailPrint "Python installer returned $0"
    ExecutePyFile:
        # Run the addon updater.
        File "WowAddonUpdater.py"
        ExecShell "open" "$OUTDIR\WowAddonUpdater.py"
SectionEnd

5 Responses to “Python as a Platform”

  1. Mike Kent Says:

    “Something that recently occurred to me is that the only operating system that doesn’t come with Python pre-installed on it is Windows.

    While Linux and OS X both view Python as essentially a first-class development platform–i.e., as something that shrink-wrap applications can be built on–Windows does not.”

    You’re making the same incorrect assumption that so many others are making: that if the platform isn’t Windows, you can count on Python already being installed on it.

    But in the real world, we have to install our Python-2.5-dependent-apps on machines running AIX 4.2, HP-UX, Red Hat boxes that only have Python 2.3, and ancient SCO boxes. You CAN NOT assume that a *nix box has Python installed at all, much less the correct version of Python.

    If you want Python as a platform, available just like Java is, you have to be prepared to install it yourself. On possibly thousands of boxes already out in the field. Many of which have no internet connection. And have the manpower and the hundreds of man-hours available to do it.

    Or you use freeze, and distribute your apps that way. And have to listen to clients and your management saying “OMG, look how huge these apps are!”

    Oh yes, and BTW, freeze is deprecated. No-one is interested in maintaining it anymore, and it may not even work in the future. Because “any computer already has it installed.”

  2. Paul Moore Says:

    Windows is not the only platform without a pre-installed Python. I support databases on Windows, Solaris, HP-UX and AIX systems, all of which do not have Python installed (and as I’m not the admin, I’d have to provide a serious business case for installing Python).

    Windows, on the other hand, is the only platform of these with a reliable way of bundling the Python interpreter with the script (py2exe).

    As regards providing just the script, what if the script relies on Python 2.5 features, and I am running 2.4 because of another application’s dependency? You mention this “dependency hell”, but I don’t get the impression that it has been a major issue for you (whereas it is a pain for me).

    Ultimately, the situation isn’t clear-cut. It depends on your environment, and that of your users. But I will say that you seem to be working in an environment where “Unix” means “OSX or a recent version of Linux”. I wish I was…

  3. Atul Says:

    Sorry, I should’ve clarified the kind of use-case I’m talking about here, which is that of shrinkwrap software—so I’m only really considering Windows, OS X, and distributions of Linux intended to be used by people with little to no prior computing experience, such as Ubuntu and the OS that comes with the Asus Eee PC.

    Dependency hell is certainly an issue, and one of the things I had to do with my World of Warcraft tool was actually test it out on Python 2.3 to make sure it still worked there. And I definitely ran into some kinks; the constants for integer status codes in httplib, for instance, weren’t in 2.3, so I couldn’t use those.

  4. Ubiquity’s Python Feed Plugin at Toolness Says:

    [...] come with Python built-in like the other two operating systems do, we’ll have to bootstrap it first, which we haven’t gotten around to [...]

  5. Jonathan Morgan Says:

    I would be concerned about an application that downloads things without telling me (though I would probably get annoyed if it told me – don’t ask me to be consistent). Probably many of your users will be bandwidth limited in some way, and they just might not be happy with your magic auto-updaters and downloading of Python (yes, I know you wouldn’t be the first to do it).