August 22nd, 2017

Bruno Rocha: Publish your Python packages easily using flit

Programing, Python, by admin.

Deploying Python Packages to PyPI using Flit

The traditional way of package deployment on Python is using a setup.py script located in the root of your project and then running python setup.py sdist upload to release a new version.

It works by using distutils, setuptools or distribute and there is also twine which is a command line application to manage uploads.

History

distutils is the standard way of Python package distribution included in standard library since Python 2.x then setuptools was created to overcome distutils limitations and also introduces a command line application called easy_install (which is now pip) and a very handy feature called pkg_resources and one of the characteristics of setuptools is that it uses Monkey Patching over the standard distutils.

Some forks of setuptools has been created to fix some issues and ad common developers preferences and well known forks like distribute and distutils2 and distlib has been merged back to the original setuptools

Lots of other packaging tools has been created to try to fix the distribution problems.

How it works in standard way

using one of the above you should create a file called setup.py in the root of your project, e.g:

from <my_favorite_dist_tool> import setup # Example taken from Django's repository
setup( name='Django', version=version, url='https://www.djangoproject.com/', author='Django Software Foundation', author_email='foundation@djangoproject.com', description=('A high-level Python Web framework that encourages ' 'rapid development and clean, pragmatic design.'), license='BSD', packages=find_packages(exclude=EXCLUDE_FROM_PACKAGES), include_package_data=True, scripts=['django/bin/django-admin.py'], entry_points={'console_scripts': [ 'django-admin = django.core.management:execute_from_command_line', ]}, install_requires=['pytz'], extras_require={ "bcrypt": ["bcrypt"], "argon2": ["argon2-cffi >= 16.1.0"], }, zip_safe=False, classifiers=[ 'Development Status :: 2 - Pre-Alpha', 'Environment :: Web Environment', 'Framework :: Django', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: OS Independent', 'Programming Language :: Python', 'Programming Language :: Python :: 3', 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Topic :: Internet :: WWW/HTTP', 'Topic :: Internet :: WWW/HTTP :: Dynamic Content', 'Topic :: Internet :: WWW/HTTP :: WSGI', 'Topic :: Software Development :: Libraries :: Application Frameworks', 'Topic :: Software Development :: Libraries :: Python Modules', ],
)

As you can see it is very confusing to decide which of them to addopt and how the setup.py should be writen because there are different examples over the github most famous Python repositories.

Making things easier with Flit

NOTE: Forget about all that history you read above and consider Flit.

Flit is a simple way to Package and deploy Python projects on PyPI, Flit makes it easier by using a simple flit.ini file and assumes common defaults to save your time and typing.

I knew Flit when I was taking a look at Mariatta Wijaya game called Tic Tac Taco Pizza and noted that she used flit to deploy the game, so we also asked her the reason on the podcast we recorded so I decided to try porting my projects to Flit.

How it works?

Instead of a setup.py you put a flit.ini in the root of your project and it looks like:

[metadata]
module = markdocs
author = Bruno Rocha
author-email = rochacbruno@gmail.com
maintainer = rochacbruno
maintainer-email = rochacbruno@gmail.com
home-page = https://github.com/rochacbruno/markdocs
requires = mistune click
description-file = README.md
classifiers = Programming Language :: Python :: 3.6 Intended Audience :: Developers License :: OSI Approved :: MIT License Topic :: Documentation Topic :: Software Development :: Documentation Topic :: Software Development :: Quality Assurance
requires-python = >=3.6 [scripts]
markdocs = markdocs:main

Now you only need to have flit installed in your local machine pip3 install flit as it works only in Python3+ and also is recommended to have pandoc and pypandoc also installed because it can convert your README.md into the .rst format still used by PyPI.

The advantages are:

  • No more complicated setup.py
  • If you omit some fields it will assume common used defaults
  • It is easier to read and write
  • Will convert your README.md
  • Will take the __version__ included in your program/__init__.py
  • Can develop to TestPyPI
  • Avoids over engineering on setup.py

Development installation

To install your package during development use

flit install --symlink --python path/to/virtualenv/bin/python

the --python is optional, by default will take the current which python

Registering and deploying

It is easy and will register the new project for you if doesn’t exist on PyPI

flit puslish

Flit packages a single importable module or package at a time, using the import name as the name on PyPI. All subpackages and data files within a package are included automatically.

Important!

  • Flit will use the data from ~/.pypirc to authenticate and to find the server deployment addresses
  • You can also set FLIT_USERNAME and FLIT_PASSWORD and FLIT_INDEX_URL as environment variables which makes flit good for CI deployment (e.g: TravisCI)

What is missing?

NOTE: Flit is open-source, so some of this things are already under consideration.

  • Flit will not bump your project version automatically, you can still use tools like bumpversion but this feature would be better if builtin
  • Flit will not parse a requirements.txt file and would be nice to have it as tools like pyup.io can track those files but not flit.ini yet
  • Flit does not create a .lock frozen file with the version used on specific release and it is interesting just like pipenv does

Conclusion

Flit is the easier way to deploy packaged to PyPI following 3 steps (1. install flit, 2. describe flit.ini 3. run flit publish) your library is released to PyPI.

But Python still needs better standards because you still need separated tools to make common tasks and using a single tool to pack, install, deploy, create would be better (just like what cargo/crates does for Rust) for example, in Python you have:

  • Flit to deploy packages to PyPI for distribution
  • bumpversion to bump your semver number
  • pip to install and update packages from PyPI (or pipenv/WIP to do the same with more powers)
  • Cookiecutter to create a new Python Package (from strucured templates)
  • safety to check dependencies security
  • flake or pylint to static and styling checks
  • venv, pyenv or virtualenvwrapper to manage isolated environments
  • pytest to run tests
  • pyup to watch for depdendency updates

So many tools brings a lot of confusion and makes it hard to choose, why not having a single tool, based on different plugins sharing the same API?

Just an idea

python -m manage [install packagename] # <-- calls pip or pipenv [publish --options --bump=version] # <-- calls `flit` and `bumpversion` [new packagename templatename] # <-- calls cookiecutter [safecheck] # <-- calls safety [checkupdates] # <-- checks in the same way as Pyup [test path] # <-- calls pytest, nose or unittest [lint path] # <-- calls flake, pylint [venv options] # <-- calls the existing venv module All above configurable via config file or env vars.

and each of that endpoints would be provided by many plugins sharing the same API, so you could choose between flit or twine as your publish manager etc..

So maybe I can implement that features in manage

Please share in comments if you know some other Python management tool

Back Top

Leave a Reply