Managing Python projects with Poetry

Install Poetry

Available versions and requirements

For available Poetry versions, have a look at Poetry releases.

Also, before running the installation commands, have a look at the system requirements (python version...).

Use Pyenv to install multiple system independent Python versions

If the required version of Python for installing Poetry is not available on your system packages repositories, you can use Pyenv to install that Python version alongside your current system's Python version. Here is a quick guide on how to setup and use Pyenv.

Install required packages for Pyenv

Use this: Required packages for Pyenv.

Install Pyenv

# Run installation script
curl https://pyenv.run | bash

# Load pyenv automatically by appending
# the following to ~/.bash_profile if it exists, 
# otherwise ~/.profile (for login shells) and 
# ~/.bashrc (for interactive shells)

export PYENV_ROOT="$HOME/.pyenv"
[[ -d $PYENV_ROOT/bin ]] && export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

# Restart your shell for the changes to take effect.
exec $SHELL

Update Pyenv and manage Python versions

# To update pyenv, its plugins including the list of available python versions
pyenv update

# List available python versions that can be installed
pyenv install --list

# Install a specific python version
pyenv install <python_version>

# Uninstall a specific python version
pyenv uninstall <python_version>

# Set a specific pyenv installed python version as the default for all the system
pyenv global <python_version>

# Example + verification
# Get the default python version for the whole system
$ pyenv global
system # 'system' means that the python version we are using
       # is the one installed by default on the system

# Set python 3.12.2 for the whole system
$ pyenv global 3.12.2

# Verify
$ pyenv global
3.12.2

# Rollback to the system default python version
$ pyenv global system

Poetry installation

Poetry versions

# Latest version
curl -sSL https://install.python-poetry.org | python3 -

# Specific version
curl -sSL https://install.python-poetry.org | POETRY_VERSION=1.2.0 python3 -

# Add Poetry executable to $PATH
echo export PATH="$HOME/.local/bin:$PATH" >> ~/.bashrc
bash

# Verify
poetry --version

Initialize Python project with Poetry

A 'pyproject.toml' configuration file contains info about our python packages and required python versions and dependencies. To create a 'pyproject.toml' file for our project, we use:

poetry init

Here is an example generated 'pyproject.toml' content:

[tool.poetry]
name = "mypackage"
version = "0.1.0"
description = "illustrate pyproject.toml config generated with poetry"
authors = ["gmkziz"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.30.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

To create a Python project folder using the 'src' layout (the default), we can use the 'poetry new' command. The '--flat' option can also be used for a 'flat' layout. Details Here.

For more about Python 'src' and 'flat' layouts, have a look at Python src vs flat layouts.

Manage Python packages dependencies with Poetry

The 'poetry install' command installs all the packages dependencies from a 'poetry.lock' file. If there is no 'poetry.lock' file when running 'poetry install', it is created after all the specified dependencies inside the 'pyproject.toml' file have been installed.

The 'poetry.lock' file contains all the installed dependencies and version constraints. To only create the 'poetry.lock' file, use:

# Only create the poetry.lock file
poetry lock

The 'poetry add' or 'poetry remove' commands can be used to add or remove dependencies. Once the dependencies installed or removed, the 'pyproject.toml' and 'poetry.lock' files are updated.

By default, dependencies are implicitely added to or removed from the 'root' group (the main one). For details about the different forms we can use to specify dependencies packages, see the end of 'poetry add --help' or have a look at the corresponding online documentation: poetry add.

# Add click package as dependency for the root group
poetry add click

# Add pre-commit package as development dependency (dev group)
poetry add pre-commit --group=dev

# Add pytest package as test dependency (test group)
poetry add pre-commit --group=test

Here is how the previous command impacted the 'pyproject.toml' file:

(...)
[tool.poetry.dependencies]
python = "^3.8"
requests = "^2.30.0"
click = "^8.1.3" # added

# added
[tool.poetry.group.dev.dependencies]
pre-commit = "^3.3.2"

# added
[tool.poetry.group.test.dependencies]
pytest = "^7.3.1"
(...)

To remove dependencies, we use the following:

# If the group is not specified, will be removed from all groups
poetry remove pytest

# Specify a specific group for the removal
# Package will be removed only from that group
poetry remove pre-commit --group=dev

Execute Python code from Poetry virtualenvs

The 'poetry run' command can be used to run our python codes inside poetry virtual environment containing all the installed packages dependencies:

poetry run script.py [args]

The 'poetry shell' command can be used to spawn a shell within the virtual environment containing all the installed packages dependencies:

poetry shell
pip freeze
python ...
(...)

Update Poetry

To update Poetry version to the latest available one, simply run:

poetry self update

Containerise Python programs managed with Poetry

Here is an example 'Dockerfile' content that can be used to containerise Python programs managed with Poetry:

FROM python:3.11
ENV POETRY_VERSION=1.4.2
RUN pip install "poetry==$POETRY_VERSION"
WORKDIR /app
COPY poetry.lock pyproject.toml /app/
RUN POETRY_VIRTUALENVS_CREATE=false poetry install --without=dev --without=tests --no-interaction --no-ansi
COPY src /app
ENTRYPOINT ["python", "app.py"]