Standard version

Python 3.6.8 is installed on the cluster as the standard version. You do not need to load a module to use it. Some packages such as numpy, scipy and mpi4py are available specifically for this Python version if the corresponding modules are loaded as listed below:

Latest version

>>> help("modules")

$ module load python
Loading Python 3.11.2 Environment

$ python --version
Python 3.11.2

$ which python
/cm/shared/omni/apps/python/3.11.2/bin/python

Attention: For compatibility reasons, Python 2.7.18 is still available on the operating system side. Python 2 is no longer supported since the beginning of 2020 and should not be used! All of the following explanations refer to Python 3 and may not be available in Python 2.

A good resource for Python beginners, for example, are the tutorials on the Python website.

Note: Example commands beginning with $ refer to the Linux console, commands beginning with >>> refer to the Python console. These characters must be omitted when entering commands.

There are other Python installations installed on the cluster as part of Miniconda or Jupyter (with Miniconda you can even install your own Python installations in your home directory). How to use them is described on the corresponding pages for Miniconda
and Jupyter
are described.

Python in the cluster

You can use the python command to call up the interactive console.

$ python
Python 3.6.8 (default, Apr 16 2020, 01:36:27)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

The python2 command calls the Python2 console.

To execute a Python script, enter

$ python 

in. You can also execute individual Python commands using the -c option:

$ python -c "print('Hello world.'); print('Use semicolons to enter multiple commands.')"

Note that the entire command or the entire chain of commands must be enclosed in quotation marks (either single or double).

The commands described here can of course also be used in shell scripts (such as job scripts).

Python packages

Python code is organized in packages, which contain individual modules. Every file with the extension .py is a module, every directory that contains a file init.py is a package. A large number of packages are available in the central Python Package Index (PyPI).

Pre-installed packages
You can display a list of all installed packages by entering in the Python console:

>>> help("modules")

in the Python console.

Certain Python packages that are often used on the cluster are available by loading the corresponding modules. These are numpy, scipy and mpi4py in particular. You can display these packages with:

module spider py3

Install packages yourself

Note: The package manager Conda is also installed on the OMNI cluster, which can also install many Python modules. If you are already using Conda or the Anaconda distribution, you can use these to install packages instead. Anaconda on the OMNI cluster is described here
described here.

If you need a package that is not installed, you can install it yourself using the pip3 program. To do this, enter the following in the console

$ pip3 install --user 

in the console. The package and any dependencies that are not yet installed will then be installed in your user directory, by default in:

/home//.local/lib/python/site-packages

Packages in this directory are found directly by Python. Please note that packages can only be used by you in this way. You can use a Python module or package in another Python script by importing it:

>>> import examplemodule

or

>>> from examplepackage import examplemodule

Custom software and environment variables

If you want to compute on the cluster with software you have written yourself in Python (or manually downloaded packages), you must ensure that Python finds your software. A Python script that integrates your software as in this example:

runthis.py
import mysoftware

# Do stuff with mysoftware...

will abort with the following error message:

>>> import mysoftware
Traceback (most recent call last):
  File "", line 1, in 
ImportError: No module named 'mysoftware'

This is because Python only searches for the module name in defined paths when importing modules (or packages). You can view the list of these paths by entering the following in the Python console

>>>import sys; print(sys.path)

in the Python console. You can add the directory of your software within Python:

>>> sys.path.append(/home//mysoftware)

It is more advisable to use the environment variable PYTHONPATH by setting it at the beginning of your job script and exporting it:

$ export PYTHONPATH=$`PYTHONPATH:/home//mysoftware

In both cases, the addition to the search path is only temporary, i.e. as long as Python (in the case of sys.path) or your terminal (in the case of PYTHONPATH) are open. You can add paths permanently to sys.path by creating a file with the extension .pth in a directory that already belongs to sys.path, in which you list the additional directories (one per line).

By default, inclusion is not recursive, which means that even if you add a directory to sys.path or PYTHONPATH, you cannot import Python modules in subdirectories. However, if the subdirectory has a file named __init__.py, Python treats the directory as a package (see above) and you can import either the package or individual modules from this package.