Python Modules and Packages

Python Modules

• A Python module is simply a container for functions and classes
• Any Python '.py' file can be considered as a module
• Here is an example Python module containing functions:

# File: greetings.py

def hello_en(name: str) -> None:
    """Say Hello in English"""
    print(f"Hello {name}!")

def hello_fr(name: str) -> None:
    """Say Hello in French"""
    print(f"Bonjour {name}!")

• The 'greetings.py' file represents a module named 'greetings' (name of the module file without the '.py' extension) containing the 'hello_en' and 'hello_fr' functions

• Here is how we use the functions provided by the 'greetings' module from another Python file (or module):

# File: main.py

# First form
import greetings
greetings.hello_fr("hackerstack.org")
greetings.hello_en("hackerstack.org")

# Second form
from greetings import *
hello_fr("hackerstack.org")
hello_en("hackerstack.org")

# Third form
from greetings import hello_fr, hello_en
hello_fr("hackerstack.org")
hello_en("hackerstack.org")

# Bonus: you can also rename functions or classes
# that you import from a module using aliases
from greetings import hello_fr as hello
hello("gmkziz")

• The different import syntaxes shown above are equivalent in terms of performance because the modules we use during imports will be fully loaded

• To speed up modules load during executions, Python always pre-compile modules the first time they are executed

• For our examples with the 'greetings' module, Python has created a '__pycache__' directory to store a Byte-compiled version of the module:

$ file __pycache__/greetings.cpython-312.pyc 
__pycache__/greetings.cpython-312.pyc: Byte-compiled Python module for CPython 3.12 or newer, timestamp-based, .py timestamp: ..., .py size: 186 bytes

• The '.pyc' Byte-compiled version of the module will be re-compiled the next time it runs, if its content has changed

• If you want to do something during the import of a module, simply add the task inside the module's '.py' file. Here is an example:

# File: greetings.py

def hello_en(name: str) -> None:
    """Say Hello in English"""
    print(f"Hello {name}!")

# This will run during module import
print(f"Module {__name__} loaded!") 

# File: main.py

from greetings import hello_en
hello_en("Hackerstack!")

# Run main.py

$ python main.py 
Module greetings loaded!
Hello Hackerstack!!

# Run greetings.py
$ python greetings.py 
Module __main__ loaded!

• We have added a new line inside the 'greetings.py' file to print the name of the module using the Python built-in '__name__' variable

• Now when we run a script importing the 'greetings' module, "Module greetings loaded!" is printed. The value of the '__name__' variable corresponds to the name of the imported module

• But, when we run the module file itself ('greetings.py'), "Module __main__ loaded!" is printed. The value of the '__name__' variable corresponds to '__name__'

• That's why you will see in some Python codes that are used for scripting, something like this:

# File: greetings.py

"""
Say hello in many languages. Awesome!
"""

# Using this file for scripting, which means we will 
# run greetings.py directly to accomplish specific tasks

def hello_en(name: str) -> None:
    """Say Hello in English"""
    print(f"Hello {name}!")
    
# The condition below simply means: 
# if you run this module directly (not through imports)

if __name__ == '__main__':
    hello_en("gmkziz")
    
# Run

$ python greetings.py 
Hello gmkziz!

• Python looks for the modules that we import in the current directory first. If not found, it then looks in order, at the locations listed by the 'sys.path' function. Here are the paths on my setup. They won't necessarily be the same for you:

# Open Python interpreter and run sys.path

$ python
>>> import sys
>>> sys.path
['', '/usr/lib/python312.zip', '/usr/lib/python3.12', '/usr/lib/python3.12/lib-dynload', '/usr/local/lib/python3.12/dist-packages', '/usr/lib/python3/dist-packages']

• You can use the 'dir' function to list all the components of a module (or attributes of the module object) as show below:

# From the directory where the greetings.py file resides

$ python
>>> import greetings
>>> dir(greetings)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'hello_en']

• The names of the built-in attributes that will be available for all Python modules start and end with two underscores (__). Here are example outputs for some of those built-in attributes of Python modules:

# Path to the source file of the module
>>> greetings.__file__
'/home/gmkziz/code/python/greetings.py'

# Name of the module
>>> greetings.__name__
'greetings'

# Path to the Byte-compiled file of the module.
# Used to speed up imports
>>> greetings.__cached__
'/home/gmkziz/code/python/__pycache__/greetings.cpython-312.pyc'

# Documentation of the module
>>> greetings.__doc__
'\nSay hello in many languages. Awesome!\n'

# Name of the package in which the module is included
# Empty because the greetings module is not include in a package
# More on Python packages next.
>>> greetings.__package__

Python packages

• A Package in Python is simply a container for Python modules (a directory containing modules '.py' files)

• The name of the directory will be the name of the package and a '__init__.py' file should also be created inside the directory to tell Python that the directory is a package

• Let's create a package called 'language' containing the 'fr' and 'en' modules that we will use to say hello in English and French:

├── language
│   ├── en.py
│   ├── fr.py
│   └── __init__.py
├── main.py

• Here are the contents of the 'en' and 'fr' modules included inside the 'language' package and also the content of the main.py file:

# File: en.py

def say_hello():
    """Say hello in English"""
    print("Hello")

# File: fr.py

def say_hello():
    """Say hello in French"""
    print("Salut")

• Here is the content of the 'main.py' file and the result after running the file:

# File: main.py

from language import fr, en
en.say_hello()
fr.say_hello()
print(f"The en module is included in the package named {en.__package__}")
print(f"The source file of the en module is located at {en.__file__}")

# Result after running main.py

$ python main.py
Hello
Salut
The en module is included in the package named language
The source file of the en module is located at /home/gmkziz/code/python/language/en.py

• Once your package is ready, you can create a source distribution package (sdist) and/or a binary distribution package (bdist) in order to share it.

• The source distribution package is an archive of the source code and the binary distribution uses the Python package binary format called wheel ('.whl' extension)

• Also, the wheel command line tool can be used to manipulate wheel files (unpack, repack, etc)

• Setuptools is one of the build backends you can use to create Python packages.

• The recommended way for creating Python packages today is this:

  • declare a build backend like setuptools inside a pyproject.toml configuration file

  • declare your packaging configuration and use the appropriate build utility to create the packages sources and binaries distributions. If you use setuptools as your build backend for instance, you will use build

• The pyproject.toml configuration file is becoming the standard for Python projects management. It is used by popular Python projects packaging and dependencies management tools like Peotry and uv

• Next, we are going to create a source and binary distributions for our 'language' package by using a 'pyproject.toml' file and setuptools as the build backend.

• For that, we first need to ensure that the build utility is installed:

pip install --upgrade build

• Now we can create the pyproject.toml configuration file:

# File: pyproject.toml

[project]
name = "language"
description = "My super language package"
version = "0.0.1"
readme = "README.md"
authors = [
    {name = "gmkziz", email = "gmkziz@hackerstack.org"}
]
license-files = ["LICEN[CS]E*", "vendored/licenses/*.txt", "AUTHORS.md"]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

• It is also recommended to add a 'README.md' and a 'LICENCE' file at the root of your project. Here is the final directory tree before the package creation:

.
└── language
    ├── language
    │   ├── en.py
    │   ├── fr.py
    │   └── __init__.py
    ├── LICENCE
    ├── pyproject.toml
    └── README.md

• Now to create the source and binary distributions for the package, we simply need to run the following command:

$ python -m build
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools
* Getting build dependencies for sdist...
(...)
adding 'language/__init__.py'
adding 'language/en.py'
adding 'language/fr.py'
adding 'language-0.0.1.dist-info/licenses/LICENCE'
adding 'language-0.0.1.dist-info/METADATA'
adding 'language-0.0.1.dist-info/WHEEL'
adding 'language-0.0.1.dist-info/top_level.txt'
adding 'language-0.0.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Successfully built language-0.0.1.tar.gz and language-0.0.1-py3-none-any.whl

• The resulting packages are available inside the 'dist' directory:

$ tree language/dist/
language/dist/
├── language-0.0.1-py3-none-any.whl
└── language-0.0.1.tar.gz

• Now you can share the distribution files through the public Python Package Index (PyPI) repository, your private PyPI repository or by making the files available elsewhere

• Publishing your package to the public PyPI repository will make it installable directly using pip

• You can publish your package to PyPI from the command line using twine

• When you share the package binary distribution file directly, people can install your package as follows:

# Is there any package whose name starts
# with language already installed ? No
$ pip freeze | egrep ^language

# Install the language package
$ cd dist && pip install language-0.0.1-py3-none-any.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./language-0.0.1-py3-none-any.whl
Installing collected packages: language
Successfully installed language-0.0.1

# Is there any package whose name starts
# with language already installed ? Yes
$ pip freeze | egrep ^language
language @ file:///home/gmkziz/code/python/language/dist/language-0.0.1-py3-none-any.whl#sha256=61c5e9414e30d0ff6e212bd71fa1d6b812f2e3c17717cc438e2d4da405b9c929

• The language package is now included inside my Python installation. I can import it from any Python module. Here is an example usage from Python interpreter:

# Run Python interpreter from a random directory
$ python
>>> from language import fr
>>> fr.say_hello()
Salut
>>> # The package has been loaded from here
>>> language.__path__
['/home/gmkziz/.local/lib/python3.12/site-packages/language']

That's all. Hope you better understand Python modules and packages now.

Want to report a mistake or ask questions ? Feel free to email me at gmkziz@hackerstack.org.

If you like my articles, consider registering to my newsletter in order to receive the latest posts as soon as they are available.

Take care, keep learning and see you in the next post 🚀