Standard Version
Python 3.6.8 is installed on the cluster as the default version. You do not need to load any modules to use it. Several packages specific to this Python version, such asnumpy,scipy, andmpi4py, are available if the corresponding modules listed below are loaded:
| Python Modules | Environment Modules |
|---|---|
| numpy | py3-numpy |
| scipy | py3-scipy |
| mpi4py | py3-mpi4py |
Newer version
The latest version, Python 3.11.2, is available via the `python` module. This module already includes compatible, pre-installed packages such as NumPy, SciPy, and mpi4py. Therefore, unlike the default version, these do not need to be loaded separately.
Using
>>> help("modules")you can display an overview of the preinstalled packages directly in the Python console.
You can load the ` python ` module using the following command:
module load pythonYou can then check which version of Python is being used. You can also determine the installation directory using the appropriate Python commands.
Alternatively, Python 3.14.3 is also available. You can load this version using the following command:
module load python/3.14.3Python 2 Compatibility
$ module load python
Loading Python 3.11.2 Environment
$ python --version
Python 3.11.2
$ which python
/cm/shared/omni/apps/python/3.11.2/bin/pythonNote: For compatibility reasons, Python 2.7.18 is still available through the operating system. Python 2 has not been supported since early 2020 and should not be used! All of the following explanations refer to Python 3 and may not be available in Python 2.
A good resource for Python beginners, for example, are the tutorials on thePython website.
Note: Example commands beginning with$refer to the Linux console; commands beginning with>>>refer to the Python console. These characters must be omitted when entering commands.
There areadditional Python installations available on the cluster as part of Miniconda or Jupyter (with Miniconda, you can even install your own Python versions in your home directory). Instructions on how to use them are provided on the respective pages forMiniconda andJupyter.
Python on the Cluster
You can open the interactive console using the`python` command.
$ python
Python 3.6.8 (default, Apr 16 2020, 01:36:27)
[GCC 8.3.1 20191121 (Red Hat 8.3.1-5)] on linux
Type "help", "copyright", "credits", or "license" for more information.
>>>The `python2` command opens the Python 2 console.
To run a Python script, type
$ python . You can also execute individual Python commands using the-c option:
$ python -c "print('Hello world.'); print('Use semicolons to enter multiple commands.')"Note that the entire command or chain of commands must be enclosed in quotes (either single or double).
The commands described here can, of course, also be used in shell scripts (such as job scripts).
Python Packages
Python code is organized into packages, which contain individual modules. Every file with the .py extension is a module; every directory containing an init.py file is a package. A large number of packages are available in the central Python Package Index (PyPI).
Preinstalled Packages
You can view a list of all installed packages by entering the following in the Python console:
>>> help("modules").
Certain Python packages that are frequently used on the cluster are available by loading their corresponding modules. These include, in particular,numpy,scipy, andmpi4py. You can view these packages by entering:
module spider py3Installing Packages Yourself
Note: The OMNI cluster also has theConda package manager installed, which can install many Python modules as well. If you already use Conda or the Anaconda distribution, you can use them instead to install packages. Anaconda on the OMNI cluster is describedhere.
If you need a package that isn’t installed, you can install it yourself using thepip3 program. To do so, enter the following in the console
$ pip3 install --user . The package and any dependencies that are not yet present will then be installed in your user directory, by default in:
/home//local/lib/python/site-packages Packages in this directory are found directly by Python. Note that packages installed this way are only available to you. You can use a Python module or package in another Python script by importing it:
>>> import examplemoduleor
>>> from examplepackage import examplemoduleYour Own Software and Environment Variables
If you want to run software you’ve written yourself in Python (or manually downloaded packages) on the cluster, you must ensure that Python can find your software. A Python script that imports your software, as shown in this example:
runthis.py
import mysoftware
# Do stuff with mysoftware…will terminate with the following error message:
>>> import mysoftware
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named 'mysoftware'
This is because when importing modules (or packages), Python only searches for the module name in defined paths. You can view the list of these paths by entering the following in the Python console:
>>>import sys; print(sys.path). You can add your software’s directory within Python:
>>> sys.path.append(/home//mysoftware) It is recommended to use thePYTHONPATHenvironment variable by setting and exporting it at the beginning of your job script:
$ export PYTHONPATH=$`PYTHONPATH:/home//mysoftware In both cases, the addition to the search path is only temporary—that is, it lasts only as long as Python (in the case of`sys.path`) or your terminal (in the case of `PYTHONPATH`) is open. You can permanently add paths to`sys.path` by creating a file with the`.pth`extension in a directory that is already part of`sys.path`, listing the additional directories (one per line).
By default, this inclusion is not recursive; that is, even if you add a directory to `sys.path` or `PYTHONPATH`, you cannot import Python modules from its subdirectories. However, if the subdirectory contains a file named__init__.py, Python treats the directory as a package (see above), and you can import either the package itself or individual modules from that package.