Python

Miniconda
Miniforge
Qiime1

Miniconda

The default installation of Miniconda may have restrictive licensing on some packages (see https://stackoverflow.com/a/78766306 for some explanation). The drop-in replacement "miniforge" will set up a similar minimal conda environment that defaults to the "conda-forge" channel of open packages.

As the packages in Python distributions can change rapidly, it is difficult to have a single, system-wide installation that is useful to everyone. We will continue to install some basic ones in /sopt, but they are not likely to change after initial build.

To let users customize a minimal Python environment that won't disturb others, we suggest installing the "Miniconda" version of the Anaconda Python distro. (See https://conda.io/miniconda.html for additional info.)

Example Install 1) Python3-based conda installed in the default $HOME/miniconda3/ directory:

cd
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod u+x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh -b

Example Install A) Same as option1, but installed to /DFS-L/DATA/$group/$user/miniconda3/ directory, with link in home directory (allows installing very large numbers of packages that would make your home directory go over quota):

cd
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod u+x Miniconda3-latest-Linux-x86_64.sh
mkdir /DFS-L/DATA/$(id -gn)/$USER/miniconda3
ln -s /DFS-L/DATA/$(id -gn)/$USER/miniconda3 miniconda3
./Miniconda3-latest-Linux-x86_64.sh -bu

As of 23-November-2020, this installs Miniconda3 4.9.2, which uses Python 3.8.5. It also installs the minimal set of packages:

_libgcc_mutex-0.1-main
brotlipy-0.7.0-py38h27cfd23_1003
ca-certificates-2020.10.14-0
certifi-2020.6.20-pyhd3eb1b0_3
cffi-1.14.3-py38h261ae71_2
chardet-3.0.4-py38h06a4308_1003
conda-4.9.2-py38h06a4308_0
conda-package-handling-1.7.2-py38h03888b9_0
cryptography-3.2.1-py38h3c74f83_1
idna-2.10-py_0
ld_impl_linux-64-2.33.1-h53a641e_7
libedit-3.1.20191231-h14c3975_1
libffi-3.3-he6710b0_2
libgcc-ng-9.1.0-hdf63c60_0
libstdcxx-ng-9.1.0-hdf63c60_0
ncurses-6.2-he6710b0_1
openssl-1.1.1h-h7b6447c_0
pip-20.2.4-py38h06a4308_0
pycosat-0.6.3-py38h7b6447c_1
pycparser-2.20-py_2
pyopenssl-19.1.0-pyhd3eb1b0_1
pysocks-1.7.1-py38h06a4308_0
python-3.8.5-h7579374_1
readline-8.0-h7b6447c_0
requests-2.24.0-py_0
ruamel_yaml-0.15.87-py38h7b6447c_1
setuptools-50.3.1-py38h06a4308_1
six-1.15.0-py38h06a4308_0
sqlite-3.33.0-h62c20be_0
tk-8.6.10-hbc83047_0
tqdm-4.51.0-pyhd3eb1b0_0
urllib3-1.25.11-py_0
wheel-0.35.1-pyhd3eb1b0_0
xz-5.2.5-h7b6447c_0
yaml-0.2.5-h7b6447c_0
zlib-1.2.11-h7b6447c_3

This uses about 323MB of disk space.

Once installed, you can set up the environment paths to your private version (with either install option) using:

ml miniconda/3/own

Installing new packages (e.g. numpy) within your miniconda3 directory is as simple as:

conda install numpy

Miniconda2 can be installed in a similar way with:

cd
wget https://repo.continuum.io/miniconda/Miniconda2-latest-Linux-x86_64.sh
chmod u+x Miniconda2-latest-Linux-x86_64.sh
./Miniconda2-latest-Linux-x86_64.sh -b

There is also the corresponding miniconda/2/own module. Only one python module (miniconda/anaconda/python/Intel-python, all with versions 2 or 3) can be loaded at a time.

Miniforge

To let users customize a minimal Python environment that won't disturb others, we suggest installing the "Miniforge" version of the Conda Python environment. (See https://conda-forge.org/download/ for additional info.)

Example Install 1) Python3-based conda installed in the default $HOME/miniforge3/ directory:

cd
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
chmod u+x Miniforge3-Linux-x86_64.sh
./Miniforge3-Linux-x86_64.sh -b

Example Install A) Same as option1, but installed to /DFS-L/DATA/$group/$user/miniforge3/ directory, with link in home directory (allows installing very large numbers of packages that would make your home directory go over quota):

cd
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
chmod u+x Miniforge3-Linux-x86_64.sh
mkdir /DFS-L/DATA/$(id -gn)/$USER/miniforge3
ln -s /DFS-L/DATA/$(id -gn)/$USER/miniforge3 miniforge3
./Miniforge3-Linux-x86_64.sh -bu

As of 23-January-2025, this installs Conda 24.11.3, which uses Python 3.12.8.

This uses about 169MB of disk space.

Once installed, you can set up the environment paths to your private version (with either install option) using:

ml miniforge/3/own

Updating will upgrade conda to 25.1.0

conda update --all

Installing new packages (e.g. numpy) within your miniforge3 directory is as simple as:

conda install numpy

Create a new environment for CPU-based PyTorch

conda create --name torch-CPU pytorch torchvision torchaudio cpuonly -c pytorch

Only one python module (miniforge/miniconda/anaconda/python/Intel-python) can be loaded at a time.

Qiime1

Instructions: http://qiime.org/install/install.html

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod u+x Miniconda3-latest-Linux-x86_64.sh
./Miniconda3-latest-Linux-x86_64.sh

When installing Miniconda it will prompt you for a pathname if not it
will default to $HOME/miniconda3 if you started from your home directory.

Python is lots of small files so it is best to use /DFS-B/DATA on the new system,
in my case /DFS-B/DATA/staff/santucci/miniconda3 or /DFS-B/DATA/<group/pi-name>/<username>.

Results in updating:

/export/home/santucci/.bashrc

Original .bashrc can be found:

/export/home/santucci/.bashrc-miniconda3.bak

# source .bashrc
. ~/.bashrc

# update your conda environment
conda update -n base conda

# create virtual environment 
conda create -n qiime1 python=2.7 qiime matplotlib=1.4.3 mock nose -c bioconda

# activate qiime1 in python virtual environment
source activate qiime1

# deactive qiime1 in python virtual environment
source deactivate

# destroy/revert back to original conda environment
conda remove --name qiime1 --all