What is this article about, and why does it exist?
I'm writing this article for those who are just starting out or want to publish their package on PyPI. When I first tried publishing a PyPI package myself using the official guide, I had a lot of questions about why and how.
Furthermore, I felt this article was a bit overwhelming for a beginner, and it would be a good idea to split it into two parts:
- A quick start with the bare minimum.
- For those who are more familiar with the topic, there will be a comprehensive and in-depth guide. (Under development)
I'd also like to point out that I'll be writing about a new (modern) approach to publishing packages, namely, using the pyproject.toml configuration file, which contains all the necessary configurations for packaging your code.
Introduction
Why bother with packaging, who needs it, and what exactly is PyPI? Let’s start at the beginning.
This packaging standard is relatively new. It came into widespread use around 2016 with the introduction of PEP 518, which established a unified way to publish and package Python projects via pyproject.toml.
Who needs this system? Mainly developers, testers, and anyone working with Python. Packaging standardizes the way Python projects are distributed and deployed on PyPI, making life easier for everyone involved.
But why do we need PyPI at all? Why not just use platforms like GitHub or Bitbucket to store source code? The answer depends on perspective. Python differs significantly from languages like C++ or JavaScript. For example, C++ lacks an official package index like PyPI, while JavaScript uses the Node Package Manager (NPM).
Each language ecosystem follows different norms and levels of complexity. If you want to quickly and easily install functionality with a single command, you need a central hub with specific rules and standards for package structure and operation. That’s exactly what PyPI provides.
For instance, if you want to build web applications or an entire website, you just install Django from PyPI—and you’re ready to go.
An example how to install a package from PyPI
Or if you want to add support for the HTMx library to your templates (html pages), simply install the corresponding node module:
An example how to install a package from NPM
Setting Up the Basics
To create your own package for PyPI, you need to have Python and pip installed. Additionally, it’s recommended to create virtual environments both for the package you’re developing and for any projects where you plan to use that package.
Here are the steps to get started:
- Install Python from the official website: https://www.python.org/downloads/. Pip will be installed automatically along with Python.
- Create a directory and a virtual environment where you will develop your package. For example, call this directory UsefulPackage.
- Register an account on pypi.org. If you’re new to packaging, it’s also a good idea to register on test.pypi.org, a test PyPI server.
If you’re unfamiliar with virtual environments, here’s how you create one:
- python specifies the Python version you want to use to create the environment.
- -m venv tells Python to use the virtual environment module.
- .venv is the directory where the virtual environment will be saved.
That's all we need for what follows. So, at this point, our package structure is as follows:
- UsefulPackage
- .venv
Additionally, a test project
You can also create a test project where we'll install the newly created package, just to make sure we've done everything correctly and the package actually works as expected. To do this, create a UsefulProgram directory. Then, create a virtual environment there, as described above.
We'll also need a main.py file, which we'll run through the interpreter.
Create it and paste the following test code:
Instead of YOURNAME, you'll need to substitute your own name or any other string. This is necessary so that everyone can create their own package and avoid conflicts of interest. Alternatively, you can come up with your own package name and use that.
That's all we need for now. So, at this point, our project structure is as follows:
- UsefulProgram
- .venv
- main.py
If you run it now, you'll get an error about invalid package imports; the package doesn't exist yet.
A Quick Start to Creating a Basic Python Package for PyPI
This chapter will guide you through the basic steps to publishing your first package on PyPI. I'll also cover all the nuances and specifics of our actions as we create the package.
Creating the Minimal Project Structure
First, you'll need to create the appropriate structure for the package. It will look like this:
- UsefulPackage/
- .venv/
- useful_package_timthewebmaster/
- LICENSE <- License file, you can choose here
- pyproject.toml <- Let's talk about it later, configuration file
- dist/ <- The folder where source distribution and build distribution gonna be placed
- src/
- useful_package_timthewebmaster/ <- Source of your package
- __init__.py
- main.py
In the src/useful_package_timthewebmaster our scripts will be located there, which we want to use in other projects.
Configuring the pyproject.toml file
After creating the package structure, we need to define the configuration file, pyproject.toml. This file has only four required parameters.
In the [build-system] section, we specify the backend (and version) to use for building the package. And in the [project] section, we specify only two required fields: the package name and its version. All other fields are optional.
Although the configuration file above is the bare minimum for creating a package, I highly recommend familiarizing yourself with the other configuration fields. Plus, all the possible classifiers will help you define your package's niche and find the right users accordingly.
I usually start from this basic template, created by me and for myself, and simply change the different fields:
Using as an example from one of my project
Generating a Python Package
We have everything we need to build our first package. But first, let's install the Python interpreter in the current virtual environment (please remember to create and activate it):
Go to the directory with the Python package settings file, i.e. pyproject.toml, and start the package creation process:
And this is what the build module's output will look like. More specifically, two archives will be created: the source distribution and the build distribution:
We'll now have the following archives: useful_package_timthewebmaster-0.0.1-py2.py3-none-any.whl and useful_package_timthewebmaster-0.0.1.tar.gz, which are now ready to be indexed by PyPI.
Uploading the package to PyPI
To upload them, you'll need to install another module, the twine module (don't forget to create and activate a virtual environment).
If you're still working with the test repository, to push a package to the test index, you need to specify it using the --repository flag with the testpypi argument. Finally, specify all the archives you want to push.
In my case, I will send all the archives in the dist directory
Well, if you no longer need to test the methods and ways of publishing your packages, then publishing a package to the open index looks like this:
At this stage, you'll be asked for an API token to confirm who wants to publish their package and where. You'll need to create one. There's a small caveat: each token has a scope, so to speak. You can specify a scope for all packages, or you can specify a scope for each one.
To create a token for the test index, go here: https://test.pypi.org/manage/account/#api-tokens
To create a token for the public index, go here: https://pypi.org/manage/account/#api-tokens
This is what the result of successfully publishing a package to the PyPI index will look like (regardless of whether it's a test server or not):
Congratulations, you've successfully published your first Python package, and now everyone can use it. Welcome to the community!
A complete guide to creating Python packages for PyPI.
Checking if you've built a working package
After successfully building and uploading the package to the PyPI index (or the test index), it's highly recommended to check whether your module works and imports at all. To do this, create a simple project, create and activate a virtual environment, and finally install the package:
If the package was uploaded to the PyPI test index, then to install it, you need to:
Don't forget to replace useful-package-timthewebmaster with your package name.
To update it, add the --upgrade flag.
Don't forget to replace useful-package-timthewebmaster with your package name.
If you already upload packages to public PyPI servers, install the packages as usual:
Don't forget to replace useful-package-timthewebmaster with your package name.
Next, you just need to run the project that uses your package and test it. In our case, simply run the script from the terminal:
