Packages, Modules and Libraries#
Import external libraries and organize your code into functional chunks. For interactive reading and executing code blocks and find b05-pypckg.ipynb, or install Python and JupyterLab locally.
Requirements
Make sure to understand Python functions.
Watch this section as a video
Watch this section as a video on the @Hydro-Morphodynamics channel on YouTube.
Import Packages or Modules#
Importing a package or module in Python makes external functions and other elements (such as objects) of modules accessible in a script. The functions and other elements are stored within another Python file (.py
) in some /site_packages/ folder (directory) of the interpreter environments. Thus, to use a non-standard package, it needs to be downloaded and installed first. Standard Python packages (e.g., os
, math
) are always accessible and other can be added with conda (read more about conda-installing) or pip (read more pip-installing).
The Difference between a Module and a Package
Modules are single or multiple files that can be imported independently (e.g. import a script named a_module.py
with import a_module
). Packages are a collection of modules in a directory with a defined hierarchy, which also enables the import of individual modules (e.g. from a_package.module import a_function
). Packages are therefore also modules, but with a hierarchy definition (i.e., a __path__
attribute and an __init__.py
file). Sounds fuzzy? Read this section down to the bottom and come back here to re-read this note.
The os
package provides basic system-terminal-like commands, for example, to manage folder directories. So let’s import this essential package:
import os
print(os.getcwd()) # print current working directory
print(os.path.abspath('')) # print directory of script running
Overview of Import Options#
Here is an overview of options to import packages or modules (hierarchical parts of packages):
Command |
Description |
Usage of attributes |
---|---|---|
|
Import an original module |
|
|
Import module and rename (alias) it in the script |
|
|
Import only a function, class or other items |
|
|
Import all items |
|
Example#
import matplotlib.pyplot as plt # import pyplot from the matplotlib module and alias it with plt
import math as m
x = []
y = []
for e in range(1, 10):
x.append(e)
y.append(e**2)
plt.plot(x, y)
What is the best way to import a package or module?#
There is no global answer to this question. However, be aware that from package-name import *
overwrites any existing variable or other items in the script. Thus, only use *
when you are aware of all contents of a module or package.
pi = 9.112 # define a float called pi
print("Pi is not %1.3f." % pi)
from math import pi # this overwrites the before defined variable pi
print("Pi is %1.3f." % pi)
Tip
Define default import packages for JupyterLab’s IPython kernel (read more in the Python installation section).
What items (attributes, classes, functions) are in a module?#
Sometimes we want to explore modules or check variable attributes. This is achieved with the dir()
command:
import sys
print(sys.path)
print(dir(sys.path))
a_string = "zabaglione"
print(", ".join(dir(a_string)))
Create a new Module#
In object-oriented programming and code factorization, writing custom, new modules is an essential task. To write a new module, first, create a new script. Then, open the new script and add some parameters and functions.
# icecreamdialogue.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"
icecreamdialogue.py
can now either be executed as a script (nothing will happen visibly) or imported as a module to access its variables (e.g., icecreamdialogue.flavors
):
import icecreamdialogue as icd
print(icd.welcome_msg)
scoops_wanted = 2
print("That makes {0} please".format(icd.price_scoops[scoops_wanted]))
Make Script Stand-alone#
As an alternative, we can append the call to items in icecreamdialogue.py
in the script and run it as a stand-alone script by adding an if __name__ == "__main__":
block:
# icecreamdialogue_standalone.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"
if __name__ == "__main__":
print(welcome_msg)
scoops_wanted = 2
print("That makes {0} please".format(price_scoops[scoops_wanted]))
Now we can run icecreamdialogue_standalone.py
in a terminal (e.g., Linux Terminal, PyCharm’s Terminal tab at the bottom of the window, or in Atom using platformio-ide-terminal
).
C:\temp\ python icecreamdialogue_standalone.py
Note
Depending on the definition of system variables used in the Terminal environment, Python must be called with a different variable name than python
(e.g., python3
on some Linux platforms).
Standalone Scripts with Input Parameters#
To make the script more flexible, we can define, for instance, scoops_wanted
as an input variable of a function.
# icecreamdialogue_standalone_withinput.py
flavors = ["vanilla", "chocolate", "bread"]
price_scoops = {1: "two euros", 2: "three euros", 3: "your health"}
welcome_msg = "Hi, I only have " + flavors[0] + ". How many scoops do you want?"
def dialogue(scoops_wanted): #formerly in the __main__ statement
print(welcome_msg)
print("That makes {0} please".format(price_scoops[scoops_wanted]))
if __name__ == "__main__":
# import the terminal function emulator sys
import sys
if len(sys.argv) > 1: # make sure input is provided
# if true: call the dialogue function with the input argument
dialogue(int(sys.argv[1]))
Now, we can run icecreamdialogue_standalone_withinput.py
in a terminal.
C:\temp\ python3 icecreamdialogue_standalone.py 2
Initialization of a Package (Hierarchically Organized Module)#
Good practice involves that one script does not exceed 50-100 lines of code (except inline docs and multiline variables). In consequence, a package will most likely consist of multiple scripts that are stored in one folder and one core script serves for the initiation of the scripts. This core script is called __init__.py
and Python will always invoke this script name in a package folder. Example structure of a module called icecreamery
:
icecreamery
(folder name)__init__.py
- package initiation Python scripticecreamdialogue.py
- dialogue producing Python scripticecream_maker.py
- virtual ice cream producing Python script
To automatically invoke the two relevant scripts (sub-modules) of the icecreamery
module, the __init__.py
needs to include the following:
# __init__.py
print(f'Invoking __init__.py for {__name__}') # not absolutely needed ..
import icecreamery.icecreamdialogue, icecreamery.icecream_maker
# example usage of the icecreamery package
import icecreamery
print(icecreamery.icecreamdialogue.welcome_msg)
Do you remember the dir()
function? It is intended to list all modules in a package, but it does not do so unless we defined an __all__
list in the __init__.py
.
# __init__.py with __all__ list
__all__ = ['icecreamdialogue', 'icecream_maker']
ThThe full example of the icecreamery_all
package is also available in an icecream repository.
# example usage of the icecreamery package
from icecreamery_all import *
print(icecreamdialogue.welcome_msg)
Package Creation Summary#
A hierarchically organized package contains an __init__.py
file with an __all__
list to invoke relevant module scripts. The structure of a module can be more complex than the above example list (e.g., with sub-folders). When you write a package, consider using meaningful script and variable names, along with appropriate documentation.
Exercise with logging
Implement a custom logger in your module with logger = logging.getLogger(__name__)
(replace __name__
with for example my-module-log
).
Reload (Re-import) a Package or Module#
Since Python3, reloading a module requires importing the importlib
module first. Reloading makes only sense if you are actively writing a new module. To reload any module type:
import importlib
importlib.reload(my-module)