Packages, Modules and Libraries

Packages, Modules and Libraries#

Import external libraries and organize your code into functional chunks. For interactive reading and executing code blocks and find b05-pypckg.ipynb, or install Python and JupyterLab locally.

Requirements

Make sure to understand Python functions.

Watch this section as a video

Watch this section as a video on the @Hydro-Morphodynamics channel on YouTube.

Import Packages or Modules#

Importing a package or module in Python makes external functions and other elements (such as objects) of modules accessible in a script. The functions and other elements are stored within another Python file (.py) in some /site_packages/ folder (directory) of the interpreter environments. Thus, to use a non-standard package, it needs to be downloaded and installed first. Standard Python packages (e.g., os, math) are always accessible and other can be added with conda (read more about conda-installing) or pip (read more pip-installing).

The Difference between a Module and a Package

Modules are single or multiple files that can be imported independently (e.g. import a script named a_module.py with import a_module). Packages are a collection of modules in a directory with a defined hierarchy, which also enables the import of individual modules (e.g. from a_package.module import a_function). Packages are therefore also modules, but with a hierarchy definition (i.e., a __path__ attribute and an __init__.py file). Sounds fuzzy? Read this section down to the bottom and come back here to re-read this note.

The os package provides basic system-terminal-like commands, for example, to manage folder directories. So let’s import this essential package:

import os
print(os.getcwd()) # print current working directory
print(os.path.abspath('')) # print directory of script running

Overview of Import Options#

Here is an overview of options to import packages or modules (hierarchical parts of packages):

Command	Description	Usage of attributes
`import package-name`	Import an original module	`package.item()`
`import package-name as nick-name`	Import module and rename (alias) it in the script	`nick-name.item()`
`from package-name import item`	Import only a function, class or other items	`item()`
`from package-name import *`	Import all items	`item()`

Example#

import matplotlib.pyplot as plt # import pyplot from the matplotlib module and alias it with plt
import math as m

x = []
y = []

for e in range(1, 10):
    x.append(e)
    y.append(e**2)

plt.plot(x, y)

What is the best way to import a package or module?#

There is no global answer to this question. However, be aware that from package-name import * overwrites any existing variable or other items in the script. Thus, only use * when you are aware of all contents of a module or package.

pi = 9.112 # define a float called pi 
print("Pi is not %1.3f." % pi)

from math import pi # this overwrites the before defined variable pi
print("Pi is %1.3f." % pi)

Tip

Define default import packages for JupyterLab’s IPython kernel (read more in the Python installation section).

What items (attributes, classes, functions) are in a module?#

Sometimes we want to explore modules or check variable attributes. This is achieved with the dir() command:

import sys
print(sys.path)
print(dir(sys.path))

a_string = "zabaglione"
print(", ".join(dir(a_string)))

Create a new Module#

In object-oriented programming and code factorization, writing custom, new modules is an essential task. To write a new module, first, create a new script. Then, open the new script and add some parameters and functions.

# icecreamdialogue.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"

icecreamdialogue.py can now either be executed as a script (nothing will happen visibly) or imported as a module to access its variables (e.g., icecreamdialogue.flavors):

import icecreamdialogue as icd
print(icd.welcome_msg)
scoops_wanted = 2
print("That makes {0} please".format(icd.price_scoops[scoops_wanted]))

Make Script Stand-alone#

As an alternative, we can append the call to items in icecreamdialogue.py in the script and run it as a stand-alone script by adding an if __name__ == "__main__": block:

# icecreamdialogue_standalone.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"


if __name__ == "__main__":
    print(welcome_msg)
    scoops_wanted = 2
    print("That makes {0} please".format(price_scoops[scoops_wanted]))

Now we can run icecreamdialogue_standalone.py in a terminal (e.g., Linux Terminal, PyCharm’s Terminal tab at the bottom of the window, or in Atom using platformio-ide-terminal).

C:\temp\ python icecreamdialogue_standalone.py

Note

Depending on the definition of system variables used in the Terminal environment, Python must be called with a different variable name than python (e.g., python3 on some Linux platforms).

Standalone Scripts with Input Parameters#

To make the script more flexible, we can define, for instance, scoops_wanted as an input variable of a function.

# icecreamdialogue_standalone_withinput.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"

def dialogue(scoops_wanted): #formerly in the __main__ statement
    print(welcome_msg)
    print("That makes {0} please".format(price_scoops[scoops_wanted]))

if __name__ == "__main__":
    # import the terminal function emulator sys
    import sys
    if len(sys.argv) > 1: # make sure input is provided
        # if true: call the dialogue function with the input argument
        dialogue(int(sys.argv[1]))

Now, we can run icecreamdialogue_standalone_withinput.py in a terminal.

C:\temp\ python3 icecreamdialogue_standalone.py 2

Initialization of a Package (Hierarchically Organized Module)#

Good practice involves that one script does not exceed 50-100 lines of code (except inline docs and multiline variables). In consequence, a package will most likely consist of multiple scripts that are stored in one folder and one core script serves for the initiation of the scripts. This core script is called __init__.py and Python will always invoke this script name in a package folder. Example structure of a module called icecreamery:

icecreamery (folder name)
- __init__.py - package initiation Python script
- icecreamdialogue.py - dialogue producing Python script
- icecream_maker.py - virtual ice cream producing Python script

To automatically invoke the two relevant scripts (sub-modules) of the icecreamery module, the __init__.py needs to include the following:

# __init__.py
print(f'Invoking __init__.py for {__name__}') # not absolutely needed ..
import icecreamery.icecreamdialogue, icecreamery.icecream_maker

# example usage of the icecreamery package
import icecreamery
print(icecreamery.icecreamdialogue.welcome_msg)

Do you remember the dir() function? It is intended to list all modules in a package, but it does not do so unless we defined an __all__ list in the __init__.py.

# __init__.py with __all__ list
__all__ = ['icecreamdialogue', 'icecream_maker']

ThThe full example of the icecreamery_all package is also available in an icecream repository.

# example usage of the icecreamery package
from icecreamery_all import *
print(icecreamdialogue.welcome_msg)

Package Creation Summary#

A hierarchically organized package contains an __init__.py file with an __all__ list to invoke relevant module scripts. The structure of a module can be more complex than the above example list (e.g., with sub-folders). When you write a package, consider using meaningful script and variable names, along with appropriate documentation.

Exercise with logging

Implement a custom logger in your module with logger = logging.getLogger(__name__) (replace __name__ with for example my-module-log).

Reload (Re-import) a Package or Module#

Since Python3, reloading a module requires importing the importlib module first. Reloading makes only sense if you are actively writing a new module. To reload any module type:

import importlib
importlib.reload(my-module)