Packages, Modules and Libraries

Import external libraries and organize your code into functional chunks. For interactive reading and executing code blocks and find b05-pypckg.ipynb, or install Python and JupyterLab locally.

Import Packages or Modules¶

Importing a package or module in Python makes external functions and other elements (such as objects) of modules accessible in a script. The functions and other elements are stored within another Python file (.py) in the site-packages folder (directory) of the interpreter environment. Thus, to use a non-standard package, it needs to be downloaded and installed first. Standard Python packages (e.g., os, math) are always accessible, and others can be added with conda or pip (read more pip-installing).

The os package provides basic system-terminal-like commands, for example, to manage folder directories. So let’s import this essential package:

Overview of Import Options¶

Here is an overview of options to import packages or modules (hierarchical parts of packages):

Example¶

What is the best way to import a package or module?¶

There is no global answer to this question. However, be aware that from package-name import * overwrites any existing variable or other items in the script. Thus, only use * when you are aware of all contents of a module or package. This is also why PEP 8 discourages wildcard imports and recommends placing all imports at the top of a script. The import statement in the middle of the following example is for demonstration purposes only:

What items (attributes, classes, functions) are in a module?¶

Sometimes we want to explore modules or check variable attributes. This is achieved with the dir() command:

Create a new Module¶

In object-oriented programming and code factorization, writing custom, new modules is an essential task. To write a new module, first, create a new script. Then, open the new script and add some parameters and functions.

icecreamdialogue.py can now either be executed as a script (nothing will happen visibly) or imported as a module to access its variables (e.g., icecreamdialogue.flavors):

Make Script Stand-alone¶

As an alternative, we can append the call to items in icecreamdialogue.py in the script and run it as a stand-alone script by adding an if __name__ == "__main__": block:

Now we can run icecreamdialogue_standalone.py in a terminal (e.g., Linux Terminal, PyCharm’s Terminal tab at the bottom of the window, or VS Code’s integrated terminal).

Standalone Scripts with Input Parameters¶

To make the script more flexible, we can define, for instance, scoops_wanted as an input variable of a function.

Now, we can run icecreamdialogue_standalone_withinput.py in a terminal.

Initialization of a Package (Hierarchically Organized Module)¶

Good practice involves that one script does not exceed 50-100 lines of code (except inline docs and multiline variables). Consequently, a package will most likely consist of multiple scripts that are stored in one folder and one core script serves for the initiation of the scripts. This core script is called __init__.py and Python will always invoke this script name in a package folder. Example structure of a package called icecreamery:

• icecreamery (folder name)

  • __init__.py - package initiation Python script

  • icecreamdialogue.py - dialogue producing Python script

  • icecream_maker.py - virtual ice cream producing Python script

To automatically invoke the two relevant scripts (sub-modules) of the icecreamery package, the __init__.py needs to include the following:

Do you remember the dir() function? Applied to a package (e.g., dir(icecreamery)), it lists the items that are currently defined in the package namespace. However, to control which sub-modules a wildcard import (from icecreamery import *) loads, define an __all__ list in the __init__.py:

The full example of the icecreamery_all package is also available in an icecream repository.

Package Creation Summary¶

A hierarchically organized package contains an __init__.py file with an __all__ list to invoke relevant module scripts. The structure of a module can be more complex than the above example list (e.g., with sub-folders). When you write a package, consider using meaningful script and variable names, along with appropriate documentation.

Reload (Re-import) a Package or Module¶

Since Python 3, reloading a module requires importing the importlib module first. Reloading only makes sense if you are actively writing a new module. To reload a module, type:

Package Development & PyPI (pip) Deployment¶

The icecreamery example shows how a package works internally. To make a package installable for anyone through pip install icecreamery, it needs to be deployed to PyPI, the Python Package Index that pip queries in the background (recall pip-installing). This section first summarizes the deployment workflow, including automation with GitHub workflows and documentation on Read the Docs, and then explains good practice for developing a package collaboratively.

From Local Code to a pip-installable Package¶

Modern Python packaging is driven by a single pyproject.toml file, which replaces the formerly used setup.py (see PEP 621). A deployment-ready repository resembles the following structure, which is known as the src layout:

icecreamery/                  (repository root)
    src/
        icecreamery/          (the package itself)
            __init__.py
            icecreamdialogue.py
            icecream_maker.py
    tests/                    (automated tests, e.g., for pytest)
    docs/                     (documentation source, e.g., for Sphinx)
    examples/                 (functional usage examples)
    pyproject.toml            (package metadata and build configuration)
    README.md
    LICENSE

The pyproject.toml file defines how pip (or any other installer) builds and installs the package:

[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

[project]
name = "icecreamery"
version = "0.1.0"
description = "Virtual ice cream sales dialogues"
readme = "README.md"
license = "BSD-3-Clause"
requires-python = ">=3.10"
dependencies = [
    "matplotlib",
    "numpy",
]

While developing, install the package in editable mode into the active environment:

pip install -e .

The -e (editable) flag makes Python import the package directly from the local development clone instead of a static copy in the site-packages folder, so code modifications take effect immediately without re-installing.

To deploy a release on PyPI manually:

1. Register (for free) at pypi.org and, for rehearsing uploads, at test.pypi.org.

2. Build the distribution archives (a source archive and a wheel) with python -m build (install the builder once with pip install build). The archives land in a new dist/ folder.

3. Upload the archives with twine: python -m twine upload dist/*. Best practice: rehearse the upload with python -m twine upload --repository testpypi dist/* first.

Done. From now on, everyone can pip install icecreamery.

Automate Testing and Deployment with GitHub Workflows¶

Manually building and uploading every release is error-prone. GitHub Actions automate such recurring jobs with so-called workflows, which are YAML files stored in the .github/workflows/ folder of a repository. Two workflows are particularly useful for package development:

• A test (continuous integration) workflow that runs the test suite (e.g., with pytest) for every push and pull request, ideally on multiple Python versions and operating systems. Thus, broken code is flagged before it is merged into main (recall the Collaboration & Branches section).

• A publish workflow that builds and uploads the package to PyPI whenever a new release (version tag) is published on GitHub.

Best practice for the publish workflow is PyPI’s Trusted Publishing, which links the GitHub repository directly to the PyPI project (a one-time setup in the PyPI account settings), so that no API tokens need to be stored in the repository secrets:

# .github/workflows/publish.yml
name: Publish to PyPI

on:
  release:
    types: [published]

jobs:
  publish:
    runs-on: ubuntu-latest
    environment: pypi
    permissions:
      id-token: write   # required for PyPI Trusted Publishing
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - name: Build distribution archives
        run: |
          python -m pip install build
          python -m build
      - name: Upload to PyPI
        uses: pypa/gh-action-pypi-publish@release/v1

With this workflow in place, publishing a new version reduces to increasing the version number in pyproject.toml and clicking on Draft a new release (with a tag such as v0.1.1) on GitHub.

Documentation on Read the Docs (Free Plan)¶

A pip-installable package without documentation will hardly be used by anyone. The de-facto standard for hosting Python package documentation is Read the Docs, which builds and hosts documentation of public (open-source) repositories for free at PACKAGE-NAME.readthedocs.io (the free plan shows small ads). The workflow:

1. Write consistent docstrings (e.g., in numpy or Google style) for all modules, classes, and functions, so that documentation generators can render the API reference automatically.

2. Create a docs/ folder with a Sphinx project (sphinx-quickstart) and enable the sphinx.ext.autodoc and sphinx.ext.napoleon extensions in docs/conf.py to pull the docstrings into the documentation. MkDocs with the mkdocstrings plugin is a popular alternative.

3. Add a .readthedocs.yaml configuration file (required by Read the Docs) to the repository root:

# .readthedocs.yaml
version: 2

build:
  os: ubuntu-24.04
  tools:
    python: "3.12"

sphinx:
  configuration: docs/conf.py

python:
  install:
    - method: pip
      path: .
    - requirements: docs/requirements.txt

4. Sign in at readthedocs.org with a GitHub account and import the repository. Read the Docs installs a webhook, so every push to main triggers an automatic rebuild of the documentation, and every release tag can be published as a version-specific documentation build.

Done. The documentation now updates itself with every push.

Collaborative Package Development¶

As soon as several developers (e.g., a research group) push code to the same package repository, working on dedicated branches with pull requests is only half of the story. The following good practice rules keep a growing package maintainable (this list stems from painful experience with real-world research code):

Strictly follow PEP 8:

• Naming conventions: make sure that all script (module) filenames and variable names follow good practice, that is, short, lowercase_with_underscores names for modules, functions, and variables, CamelCase for classes, and UPPERCASE for constants (recall meaningful script and variable names).

• Module shadowing: never name an internal script or folder after an installed library or module (e.g., math.py, numpy.py, or a folder called matplotlib/). Because Python searches the script’s own directory before the site-packages folder, the local file gets imported instead of the intended library, which leads to seemingly inexplicable import failures. Overly generic names, such as plots.py or plots/, are risky for the same reason: they easily collide with third-party modules and get confused with plotting libraries like matplotlib.pyplot.

• Docstrings: equip every module and every function with a docstring, so that collaborators (and documentation generators, see above) understand what the code does without reverse-engineering it.

• File length and code redundancy: a modular package structure means that scripts should remain concise. For context, we once had to refactor a plotting script that had grown to more than 3500 lines, partly because of copy-pasted (redundant) code blocks. Break large files down into logical sub-modules and strictly follow the DRY (Don’t Repeat Yourself) principle.

Use dedicated folders for examples and templates to keep the core package (i.e., the src/ directory) clean:

• dev-examples/: ongoing research or development cases (e.g., dev-examples/cylinder-flume-telemac/). Configure the repository to block large files (e.g., anything above 20 MB), because large simulation outputs do not belong in a git repository and should be backed up separately.

• examples/: finalized, cleaned-up, and functional examples, along with README information on how to run them.

• templates/: generalized versions of the example scripts that use keyword arguments instead of hardcoded paths.

Maintain strict top-level cleanliness: do not add or move files directly into the repository root or the package source directory. For instance, keep environment activation scripts in a dedicated folder (e.g., env-scripts/) and invoke them from your local environment or example directories as needed.

Stay synchronized with main: pull the latest main branch regularly and always before creating a new branch. In addition, install the package from the local development clone in editable mode (pip install -e .) instead of manipulating relative imports or sys.path so that example scripts import your latest local code rather than a globally installed site-packages copy. AI assistants (e.g., Claude Code or Codex) can help to robustly refactor legacy scripts with broken imports, but review their modifications as critically as any other pull request.

Learning Success Check-up¶

Take the learning success test for this Jupyter notebook.

Unfold QR Code