Hi, I'm Daniel Roy Greenfeld, and welcome to my blog. I write about Python, Django, and much more.

Building Conda Packages for Multiple Operating Systems

Thursday, January 29, 2015 (permalink)

On the Cookiecutter project, recently we added conda to the open source packaging systems we officially support (You can find Cookiecutter on PyPI, homebrew, and apparently some Linux distros).

Creating a conda recipe from a PyPI package

Prequisites:

Once those are ready, create a conda recipe for Cookiecutter.

$ conda skeleton pypi cookiecutter

This will create a conda recipe, which is a directory named cookiecutter that contains several text files.

Inside the new cookiecutter recipe directory, find the meta.yaml file and change the appropriate sections to have this content:

source:
    # Change to match the most recent release
    git_tag: 0.9.1
    git_url: https://github.com/audreyr/cookiecutter.git

package:
    name: cookiecutter
    version: {{ environ['GIT_DESCRIBE_TAG'] }}

build:
    number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}

    # Note that this will override the default build string with the Python
    # and NumPy versions
    string: {{ environ.get('GIT_BUILD_STR', '') }}

Building a conda package

Use the conda recipe to build a package for my operating system (in this case, Mac OS X):

$ conda build cookiecutter

This creates a Cookiecutter conda package at ~/miniconda/conda-bld/osx-64/cookiecutter-0.9.1_BUILDNUM.tar.bz2.

Note: The official conda recipe for cookiecutter is at https://github.com/conda/conda-recipes/tree/master/cookiecutter.

Converting the conda package to other systems

Let's convert that to Windows and Linux systems:

$ conda convert ~/miniconda/conda-bld/osx-64/cookiecutter-0.9.1_BUILDNUM.tar.bz2 -p all

This creates five new directories, each with a new package. It looks something like this:

$ ls
linux-32
linux-64
osx-64
win-32
win-64

Each one of these directories contains a conda build also named cookiecutter-0.9.1_BUILDNUM.tar.bz2.

Note: I never left the Mac OSX operating system, yet I have packages that are pretty much garaunteed to work on Windows and Linux. That said, Cookiecutter is pure python and it's dependencies already have conda packages. I haven't tried this yet on anything that includes compiling C or C++, much less Fortran.

Uploading conda packages to Binstar

With these packages created, it's time to upload them to binstar, the primary conda package index.

First, register your binstar account.

Then use conda to install the binstar client:

$ conda install binstar

Finally, start uploading the new packages:

$ binstar upload linux-32/cookiecutter-0.9.1-BUILDNUM.tar.bz2
$ binstar upload linux-64/cookiecutter-0.9.1-BUILDNUM.tar.bz2
$ binstar upload osx-64/cookiecutter-0.9.1-BUILDNUM.tar.bz2
$ binstar upload win-32/cookiecutter-0.9.1-BUILDNUM.tar.bz2
$ binstar upload win-64/cookiecutter-0.9.1-BUILDNUM.tar.bz2

Check out the results of my work or take a look right below at what's on binstar:

https://pydanny.com/static/packages.png

Try installing Cookiecutter with conda!

If you have conda installed, you should be able to get Cookiecutter thus:

$ conda config --add channels https://conda.binstar.org/pydanny
$ conda install cookiecutter

Summary

Writing about how to package software is hard, so figuring this out was a bit of detective work. I think that's going to change, as the company behind conda, Continuum Analytics has stated their intentions to improve conda's documentation. Furthermore, just as many for-python cookiecutter templates include carefully researched setup.py modules for use with distutils, in 2015 I think we'll begin to see many of these templates include carefully research conda recipes and instructions.

Many thanks go to Fernando Perez for inspiring me to actually delve into conda. Travis Swicegood gave me some useful pointers. Last, but not least, none of this would have been figured out without the help of Wes Turner.

Updates

  • 2015/01/31 - Fixed a broken binstar link thanks to Russ Ferriday.
  • 2015/01/30 - Wes Turner corrected a couple typos in the conda command statements.

Comments