Python_application的打包和发布

[Packaging and Distributing
Projects]{.ul} 介绍了
python application 打包和发布的规范，只有满足这种规范的 package
才能被最为常用的包管理工具 pip
所管理。[setuptools]{.ul} 是常用的打包工具，其文档 [Building
and Distributing Packages with
Setuptools]{.ul} 同样详细的阐述打包规范。本文以名为
packagedemo 的 application 为例(官网例子请见 [PyPA sample
project]{.ul})，展示如何编写一个标准的
package，期间主要涉及三个步骤：

• Configure: packagedemo 的文件和目录组织

• Package: 打包 packagedemo

• Distribute: 把打包后的 packagedemo 发布到 Pypi 中

Configuring your Project

编写 packagedemo 时，需要注意两方面的规范：

• 文件目录组织：包含目录组织规范、某些必需的文件

• setup.py：打包的参数信息

Initial Files

我们以 packagedemo 为例，这是一个非常简单的
application，其目录结构及文件名称如下所示：

packagedemo

├── setup.py

├── setup.cfg

├── README.rst

├── MANIFEST.in

└── packagedemo

├── init.py

   └── main.py

从上可知，packagedemo 的根目录必须包含四个基本文件：

• setup.py:

  最为重要的文件，包含了打包的参数和基本信息，详细介绍请见下节

• setup.cfg: setup.py

  的配置文件，其格式为 [INI]{.ul}

• README.rst: [reStructuredText]{.ul} 格式的文件，用于介绍项目

• MANIFEST.in: 记录某些需要被打包但未被 setup.py

  包含的文件，其格式和规范请见 [The MANIFEST.in
  template]{.ul}

setup.py 文件内容请见下节。

setup.cfg 和 MANIFEST.in 在本例中均为空文件。

README.rst 文件的内容如下：

$ cat README.rst

===========

packagedemo

===========

This project provides a demo for packaging and distributing python
application.

packagedemo/init.py 为空文件，packagedemo/main.py
文件的内容如下：

$ cat packagedemo/main.py

def main():

print(“A demo for python package.”)

Setup.py

setup.py 有两个非常重要的函数：

• setup(): 该函数囊括了打包的参数和信息

• 打包命令的入口: python setup.py –help 可以查看可用的命令

本例 setup.py 文件的内容如下:

import os

import setuptools

setuptools.setup(

name**=**’packagedemo’,

version**=**’2015.09.1’,

keywords**=**’demo’,

description**=**’A demo for python packaging.’,

long_description**=**open(

os.path.join(

os.path.dirname(file),

‘README.rst’

)

).read(),

author**=**’vanderliang’,

author_email**=**‘vanderliang@gmail.com‘,

url**=**’[https://github.com/DeliangFan/packagedemo]{.ul}‘,

packages**=**setuptools.find_packages(),

license**=**’MIT’

)

• name: 即项目名称，本例为 packagedemo

• version: 即版本号，关于版本号的取法请见 [[Choosing a versioning

  scheme]{.ul}](https://packaging.python.org/en/latest/distributing/#choosing-a-versioning-scheme)

• keywords: 描述项目的关键字

• description: 项目简介

• long_description: 项目详细介绍

• author: 作者名称

• author_email: 作者邮箱

• url: 项目的 homepage

• packages: 项目包括的 python package，setuptools.find_packages()

  可自动找出包含的 package

• license: 如 MIT, APACHE, GNU 等

本例暂未用到如下参数，这些参数的使用请见[下篇]{.ul}。

• classifiers:

• install_requires:

• package_data:

• data_files:

• scripts:

• entry_points

• console_scripts:

Build your Package

采用 pip 可安装 source distribution(sdist) 和 wheels 这两种格式 python
package，如果二者同时存在，pip 优选选择 wheel。

• [[Source

  Distributions(sdist)]{.ul}](http://wsfdl.com/python/2015/09/06/(https:/packaging.python.org/en/latest/glossary/#term-source-distribution-or-sdist)):
  即源码包

• [Wheel]{.ul}:

  采用预编译格式，相比之下具有安装速度快的特点

Source Distribution

采用如下命令即可编译成 source distribution:

$ python setup.py sdist

running sdist

running egg_info

creating packagedemo.egg-info

writing packagedemo.egg-info/PKG-INFO

…

creating dist

Creating tar archive

removing ‘packagedemo-2015.09.1’ (and everything under it)

$ ls dist/

packagedemo-2015.09.1.tar.gz

Wheel

根据 application 包含的代码类型以及其所支持的 python 版本， wheel
格式可细分为三种

• [[Universal

  wheel]{.ul}](http://python-packaging-user-guide.readthedocs.org/en/latest/distributing/#universal-wheels):
  纯 python 代码，并且支持 python 2 和 3

• [[Pure python

  wheel]{.ul}](http://python-packaging-user-guide.readthedocs.org/en/latest/distributing/#pure-python-wheels):
  纯 python 代码，不同时支持 python2 和 3

• [[Platform

  wheel]{.ul}](http://python-packaging-user-guide.readthedocs.org/en/latest/distributing/#platform-wheels):
  非纯 python 代码

采用如下命令可编译成 universal wheel

$ python setup.py bdist_wheel –universal

running bdist_wheel

running build

installing to build/bdist.macosx-10.10-intel/wheel

……

running install_scripts

creating
build/bdist.macosx-10.10-intel/wheel/packagedemo-2015.09.1.dist-info/WHEEL

$ ls dist

packagedemo-2015.09.1-py2-none-any.whl

采用如下命令可编译成非 universal wheel(即 pure python wheel 或 platform
wheel):

$ python setup.py bdist_wheel

running bdist_wheel

running build

installing to build/bdist.macosx-10.10-intel/wheel

……

running install_scripts

creating
build/bdist.macosx-10.10-intel/wheel/packagedemo-2015.09.1.dist-info/WHEEL

$ ls dist

packagedemo-2015.09.1-py2.py3-none-any.whl

Others

Python application 还可以被编译成其它类型的 package，如 rpm, egg
等，只是这些类型的 package 不被 pip 支持。

+—————+——————————————————–+

| bdist | create a built (binary) distribution |

| bdist_dumb | create a “dumb” built distribution |

| bdist_rpm | create an RPM distribution |

| bdist_wininst | create an executable installer for MS Windows |

| bdist_egg | create an “egg” distribution |

+—————+——————————————————–+

Uploading your Project to PyPI

安装 twine，用于上传打包好的 package 至 Pypi：

$ pip install twine

1 .
在 [Pypi]{.ul} 创建一个账户。

2 . 把 packagedemo.egg-info/PKG-INFO 上传至 [Pypi
submit]{.ul}，用于注册该项目。

$ cat packagedemo.egg-info/PKG-INFO

Metadata-Version: 1.0

Name: packagedemo

Version: 2015.09.1

Summary: A demo for python packaging.

Home-page:
[https://github.com/DeliangFan/packagedemo]{.ul}

Author: vanderliang

Author-email: vanderliang@gmail.com

License: MIT

Description: ===========

packagedemo

===========

This project provides a demo for packaging and distributing python
application.

Keywords: demo

Platform: UNKNOWN

3 . 上传至 Pypi:

创建如下配置文件 ~/.pypirc：

[distutils]

index-servers=pypi

[pypi]

repository =
[https://pypi.python.org/pypi]{.ul}

username =

password =

上传 package：

$ twine upload dist/*****

Uploading distributions to
[https://pypi.python.org/pypi]{.ul}

Uploading packagedemo-2015.09.1-py2-none-any.whl

Uploading packagedemo-2015.09.1-py2.py3-none-any.whl

Uploading packagedemo-2015.09.1.tar.gz

Test

$ pip install packagedemo

Downloading/unpacking packagedemo

Downloading packagedemo-2015.09.1-py2.py3-none-any.whl

Installing collected packages: packagedemo

Successfully installed packagedemo

Cleaning up…

复杂些的 application 可能要求以下功能。

• CLI: 提供命令行入口

• Data file: 数据文件

• Dependency: 依赖其它的 package

CLI

现在往 packagedemo 添加一个名为 packagedemo_cli 的 CLI，执行
packagedemo_cli 后，直接调用 main.py 的 main 函数，新增如下文件和目录。

packagedemo

├── …

└── bin

   └── packagedemo_cli

文件 packagedemo_cli 的内容如下：

#!/usr/bin/python

from packagedemo import main

main.main()

并为 packagedemo_cli 设置可执行权限：

$ chmod 755 packagedemo_cli

更新 setup.py：

setuptools.setup(

…

scripts**=**[‘bin/packagedemo_cli’],

…

)

安装后，测试如下：

$ which packagedemo_cli

/usr/local/bin/packagedemo_cli

$ packagedemo_cli

A demo for python package.

除此方法以外，还可以利用 [entry_points]{.ul} 中的 [console_points]{.ul} 配置
CLI，详情请见 [Command Line
Scripts]{.ul}

Data File

有些 python application 会依赖一些非 *.py 数据文件，比如 image,
documentation 和 data tables 等，我们把这些文件统称为 data
file，所以打包时需将这些文件包含在内。现在往 packagedemo 添加 data
目录和相关文件，如下：

packagedemo

├── …

└── data

   └── data.json

在上篇为空文件的 MANIFEST.in 终于派上用场，更新如下：

$ cat MANIFEST.in

include data/data.json

更新 setup.py：

setuptools.setup(

…

include_package_data**=**True,

…

)

Dependency and Others

随着 packagedemo 功能越来越丰富，不免会依赖其它的 package，所以需要往
setup.py 中写入所依赖的 package 名称。假定 packagedemo 依赖 wsgiref 这个
package，并且要求其 version >=0.1.2，那么需往 setup.py 增添如下的参数：

setuptools.setup(

…

install_requires**=**[‘wsgiref>=0.1.2’],

…

)

最后一个被介绍的参数是 [classifiers]{.ul}，它包含
package 的成熟度，类型，以及支持的平台等信息。

setuptools.setup(

…

classifiers**=**[

# How mature is this project? Common values are

# 3 - Alpha

# 4 - Beta

# 5 - Production/Stable

‘Development Status :: 3 - Alpha’,

# Indicate who your project is intended for

‘Intended Audience :: Developers’,

‘Topic :: Software Development :: Build Tools’,

# Pick your license as you wish (should match “license” above)

‘License :: OSI Approved :: MIT License’,

# Specify the Python versions you support here. In particular, ensure

# that you indicate whether you support Python 2, Python 3 or both.

‘Programming Language :: Python :: 2’,

‘Programming Language :: Python :: 2.6’,

‘Programming Language :: Python :: 2.7’,

‘Programming Language :: Python :: 3’,

‘Programming Language :: Python :: 3.3’,

‘Programming Language :: Python :: 3.4’,

‘Programming Language :: Python :: 3.5’,

],

…

)

Summary of Python packaging tools

Python 有多种打包工具，如 setuptools, distutils 等等，stackoverflow
有篇名为 [differences-between-distribute-distutils-setuptools-and-distutils2]{.ul} 的帖子对这些打包工具做了详细的对比，原文如下：

Here’s a summary of the Python packaging landscape in September 2014:

• Distutils is still the standard tool for packaging in Python. It

  is included in the standard library (Python 2 and Python 3.0 to
  3.4). It is useful for simple Python distributions, but lacks
  features. It introduces the distutils Python package that can be
  imported in your setup.py script.

• Setuptools was developed to overcome Distutils’ limitations, and

  is not included in the standard library. It introduced a
  command-line utility called easy_install. It also introduced the
  setuptools Python package that can be imported in your setup.py
  script, and the pkg_resources Python package that can be imported
  in your code to locate data files installed with a distribution.
  One of its gotchas is that it monkey-patches the distutils Python
  package. It should work well with pip. The latest version was
  released in August 2014.

• Distribute was a fork of Setuptools. It shared the same

  namespace, so if you had Distribute installed, import setuptools
  would actually import the package distributed with Distribute.
  Distribute was merged back into Setuptools 0.7, so you don’t need
  to use Distribute any more. In fact, the version on Pypi is just a
  compatibility layer that installs Setuptools.

• Distutils2 was an attempt to take the best of Distutils,

  Setuptools and Distribute and become the standard tool included in
  Python’s standard library. The idea was that Distutils2 would be
  distributed for old Python versions, and that Distutils2 would be
  renamed to packaging for Python 3.3, which would include it in its
  standard library. These plans did not go as intended, however, and
  currently, Distutils2 is an abandoned project. The latest release
  was in March 2012, and its Pypi home page has finally been updated
  to reflect its death.

• Distlib is a tool that aims to implement a subset of the

  previous tools’ functionality, but only functionality that is very
  well-defined in accepted PEPs. It should hopefully be included
  eventually in the Python standard library. It is still being
  developed and is not recommended for end-users yet.

• Bento is a packaging solution designed to replace Distutils,

  Setuptools, Distribute and Distutils2, written from the ground up.
  Its primary developer is also a core developer of numpy/scipy, so
  he’s familiar with non-simple use-cases for packaging systems. Its
  first commit was in October 2009, and the latest commit as of
  writing was in August 2014, although the authors are not updating
  its Pypi page correspondingly. It’s in active development but it
  is not mature yet, and it is not as widely known as Setuptools
  yet.

关于选择哪种工具，大神 [flimm]{.ul} 推荐
setuptools：

So in conclusion, out of all these options, I would recommend
Setuptools, unless your requirements are very basic and you only need
Distutils. Setuptools works very well with Virtualenv and Pip, tools
that I highly recommend.

虽然 python package 能满足多数 application
的需求，但是随着项目的越来越繁杂，RPM 或者 Debian
会是更好的包管理方式，它们的功能更为强大丰富。