Build Python Yourself

Building Python 3 on Windows with Visual Studio Express

So, you want to build your own Python3 on Windows, from source, including the newest 3rd party libraries and using the newest version of Visual Studio (or Visual Studio Express) available?
Look no further.

First off, credit were credit is due:
This guide re-uses a lot of information from an excellent post by Gerson Kurz that was a tremendous help. If the following does not work for you — his instructions might.

Secondly, this approach is for building a working 32bit Python interpreter. Building a 64bit version seems to be largely covered as well, but the FCI.LIB cannot be used for 64bit builds and (at least on my machine) the openssl build script fails because of nasm not working correctly. I am sure there are ways to fix those issues as well, but as I am only interested in a 32bit version, I did not investigate very far.

Thirdly, I have never used Tkinter at all so far and with PySide readily available, I don’t feel like I will anytime soon. Therefore, this guide ignores all things Tkinter / Tk / Tcl and produces a Python without tkinter libraries.

And lastly, this guide has been tested with:

  • bzip2-1.0.6
  • openssl-1.0.1j
  • sqlite-amalgamation-3080704
  • xz-5.2.0

Newer versions might break it in ways that I cannot imagine… let me know if they do!

Let’s get started.
There are two ways you can take: the easy or the correct way, in which I will take you through the process of setting up everything manually. You will be older but wiser for it … interpreter-wise, the end result is the same.

1. Download and install prerequisites

  • Download and install Perl from http://strawberryperl.com/ (get the 32bit version).
    We will need it to build openssl.
  • Download the latest Python3.x source from https://www.python.org/downloads/source/ (get the Gzipped source tarball).
  • Extract the folder Python-3.x.x to wherever you want to build. We will call this folder the root-folder.

2. The easy way

  • Here is where you can take a shortcut. If you want to get home early, download Python3.4.2_build_patch.7z, extract the archive into the root-folder overwriting any existing files.
    The patch includes aforementioned versions of 3rd party sources (bzip2-1.0.6, openssl-1.0.1j, sqlite-amalgamation-3080704, xz-5.2.0) and updated Python build configuration and project files. If newer versions were released or you just want to know exactly what is going on… it’s the hard way for you. Jump over to step 3. Otherwise read on.
  • Open Python-3.4.2\PCbuild\pcbuild.sln with Visual Studio Express 2013 and say “yes” to upgrading the all project files.
  • Remove the _tkinker project — no worries, the rest will work fine without it.
  • … And we are basically done. Continue with step 7 for final build advice.

3. Download 3rd party libraries

  • In the root-folder, create a folder called “externals“. You can use any other name or place for the directory as well, as long as you are consistent with it in the following steps.
    Usually, Python building tutorials recommend placing 3rd party libraries in the same directory as the root-folder, but I prefer to keep the externals within the directory of the application I am trying to build. That’s just me, though.
  • Download and extract the latest bzip from http://www.bzip.org/downloads.html into the externals folder (the resulting folder at the time of writing was: Python-3.4.2\externals\bzip2-1.0.6)
  • Download and extract the latest OpenSSL from https://www.openssl.org/source/ (the one with the [LATEST] next to it, resulting in: Python-3.4.2\externals\openssl-1.0.1j)
  • Download and extract the latest SQLite from https://www.sqlite.org/download.html (the source zip archive below the yellow box: Python-3.4.2\externals\sqlite-amalgamation-3080704)
  • Download and extract the latest xz from http://tukaani.org/xz/ (the pre-built binaries, about halfway down the page. You’ll have to create your own folder for it: Python-3.4.2\externals\xz-5.2.0-windows)
  • Next, we need a file called FCI.LIB originally released by Microsoft but since vanished from their download sites. So we’ll have to improvise.
    Download cabsdk.exe (the “Microsoft Cabinet SDK version 4.71.410.0”) from http://www.pixelsplasher.com/_downloads/software/Microsoft-Cabinet-SDK/ and temporarily save it somewhere.
    Luckily, we don’t have to install it – just open the file using 7zip (for example). Find the file under LIB/FCI.LIB and copy it into a new folder Python-3.4.2\externals\misc.

4. Update project directory variables

  • Open Python-3.4.2\PCBuild\pyproject.props.
    In line 19, change the <externalsDir> variable to the relative path from Python-3.4.2\PCBuild to your externals-folder. If you stuck with the suggested name and place, the line should read:

  •  In the lines below, change the names to the names of the folders of step 3. In my case they read:

  •  Ignore the tcltk* variables, as we won’t build Tkinter anyway.

5. Updating build_ssl.py

  • Open Python-3.4.2\PCBuild\build_ssl.py. There are only two minor modifications here. Change line 71 from

to

 Here, we are just redirecting the search to our <externalsDir> from the previous step. Adjust if necessary.

  • In line 153, change

to

This is tricky. It seems like the nasm compiler does not work…? Whatever. This does.

6. Updating the Visual Studio solution

  • It is finally time to fire up the old Visual Studio.
    Open Python-3.4.2\PCbuild\pcbuild.sln with Visual Studio Express 2013 and say “yes” to upgrading the all project files.
  • Remove the _tkinker project — no worries, the rest will work fine without it.
  • Open the project properties of the _msi project, and switch to Configuration: All Configurations.
    This is the project that is missing the mysterious FCI.lib, to change that go to Configuration Properties -> Linker -> General -> Additional Libraries Directories and append   ;$(externalsDir)\misc;
    Then hit enter and apply.
  • If you were to built now, 4 projects would return the same error LNK2026: module unsafe for SAFESEH image. Fortunately, this one is easy to fix.
    Still in the properties of the _msi project, go to Configuration Properties -> Linker -> Advanced and change the value of Image Has Safe Exception Handlers to No (a simple double-click does the trick).
  • Repeat the same modification for the projects: _ssl, _hashlib and finally _lzma.
  • Since xz-5.2.0, we need to change another detail in the _lzma project, so go to Configuration Properties -> Linker -> Input -> Additional Dependencies and change bin_i486 to bin_i686.

7. Building Python

  • Hit F7 and watch Python build.
  • As the readme.txt in Python-3.4.2\PCBuild states, the official Python releases are built using the PGInstrument and PGUpdate configurations that take advantage of the “Profile Guided Optimization” of Visual Studio. This option is however is only available in the non-express, commercial versions of VS. Take at look at the section “Profile Guided Optimization” in the readme.txt for further details.
  • If (usually when building in Release mode) you get error LNK1104: cannot open file ‘python3.lib’ in project xxlimited, just build again and it works the second time.

And that is is. Let me know if you have any trouble building Python with this guide and I always appreciate a short shout-out if it worked for you!

The next post deals with creating a deployment from the built.