Conda

Anaconda is a distribution that provides a collection of packages for scientific computing (data science, machine learning applications, large-scale data processing, predictive analytics, etc.) in the Python and R programming languages.

Anaconda distribution is used by over 12 million users and includes more than 1,400 popular data science packages suitable for Windows, Linux, and MacOS.

To install Anaconda, visit the https://www.anaconda.com website and download the version appropriate for your OS (e.g., Download and install the appropriate distribution for your OS (Windows, MacOS, Linux)

Currently, Anaconda offers versions based on Python 3.7 and Python 2.7.

Conda is an application provided by Anaconda for managing package versions.

Conda helps Python users easily resolve dependency issues, which are often encountered during package installation.

This document introduces how to use Conda packages on the KISTI system for Python users.

The "/home01/userID" on the introduction page is the home directory for a test account. Modify it to the appropriate path for use.

A. Use of Conda

For Miniconda, you can download a version suitable for each OS from the website: https://docs.conda.io/en/latest/miniconda.html
For Anaconda, you can download a version suitable for each OS from the website: https://www.anaconda.com/distribution/#download-section

Command

Description

clean

Remove unused packages and caches.

config

Modify configuration values in .condarc.

This is modeled after the git config command.

Writes to the user .condarc file (/home01/userID/.condarc)

by default.

create

Create a new conda environment from a list of specified packages.

help

Displays a list of available conda commands and their help strings.

info

Display information about current conda install.

init

Initialize conda for shell interaction. [Experimental]

install

Installs a list of packages into a specified conda environment.

list

List linked packages in a conda environment.

package

Low-level conda package utility. (EXPERIMENTAL)

remove

Remove a list of packages from a specified conda environment.

uninstall

Alias for conda remove.

run

Run an executable in a conda environment. [Experimental]

Search for packages and display associated information.

The input is a MatchSpec, a query language for conda packages.

update

Updates conda packages to the latest compatible version.

upgrade

Alias for conda update

How to Initialize Conda You can add settings to the .bashrc file in your home directory using the conda init command.

$ source /apps/applications/Miniconda/23.3.1/etc/profile.d/conda.sh
$ conda init
$ source ~/.bashrc

How to Change the Conda PathThe conda environment and package paths are set to the home directory by default, but they can be changed to other paths, such as scratch.

$ echo "export CONDA_ENVS_PATH=/scratch/$USER/.conda/envs" >> /home01/$USER/.bashrc
$ echo "export CONDA_PKGS_DIRS=/scratch/$USER/.conda/pkgs" >> /home01/$USER/.bashrc
$ source ~/.bashrc

B. Create Conda Environment

A conda environment creates an independent virtual execution environment for Python, making it easy to manage package versions.
You can create a conda environment using the command conda create –n [ENVIRONMENT].
By default, the environment will be created under the envs path in the conda directory with the name you specify.

- Example -

※ If you have not initialized conda, run the initialization command once.
$ source /apps/applications/Miniconda/23.3.1/etc/profile.d/conda.sh
$ conda init
$ source ~/.bashrc

$ conda create -n scikit-learn_0.21 
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home01/userID/.conda/envs/scikit-learn_0.21

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > conda activate scikit-learn_0.21
#
# To deactivate an active environment, use:
# > conda deactivate
#

$ conda activate scikit-learn_0.21
(scikit-learn_0.21) $

C. Install and Check Packages in Conda Environment

Packages can be installed using conda install [package name]
Packages in the conda channel can be installed using "conda install -c [channel name] [package name]".
The above item ("B. Creating a Conda Environment") is where the packages will be installed under the path of the conda environment created.

- Example -

$ source /apps/applications/Miniconda/23.3.1/etc/profile.d/conda.sh
$ conda activate scikit-learn_0.21
(scikit-learn_0.21) $ conda install scikit-learn
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home01/userID/.conda/envs/scikit-learn_0.21
  added / updated specs:
    - scikit-learn

The following packages will be downloaded:
    package                  |            build
    ------------------------|-----------------
    ca-certificates-2019.1.23  |                0         126 KB
    ...
      wheel-0.33.1               |           py37_0          39 KB
    -------------------------------------------------------
                                           Total:       277.6 MB

The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl
 ...
   zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3

Proceed ([y]/n)? y

Downloading and Extracting Packages
setuptools-40.8.0    | 643 KB    | ##################################### | 100% 
...
openssl-1.1.1b       | 4.0 MB    | ##################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(scikit-learn_0.21) $ python -c "import sklearn"
(scikit-learn_0.21) $

D. Check Conda Environment List

You can check the conda environment list using "conda-env list" or "conda env list."

- Example -

(scikit-learn_0.21) $ conda env list
# conda environments:
#
base                     /apps/applications/PYTHON/3.7
scikit-learn_0.21     *  /home01/userID/.conda/envs/scikit-learn_0.21
(scikit-learn_0.21) $ conda deactivate
$

E. Export Conda Environment

The conda-pack package is required before exporting a conda environment.

※ (Reference) https://conda.github.io/conda-pack

You can use the command "conda pack -n [ENVIRONMENT] -o [file name]" to utilize a conda environment on a different system.

(Example) When you need to use the same conda environment on a system that is not connected to the internet or when the same environment is required on another system without reinstalling, you can export the conda environment as shown below.

- Example -

$ conda activate scikit-learn_0.21
(scikit-learn_0.21) $ conda install -c conda-forge conda-pack
(scikit-learn_0.21) $ conda pack -n scikit-learn_0.21 -o scikit-learn_test.tar.gz
Collecting packages...
Packing environment at '/home01/userID/.conda/envs/scikit-learn_0.21' to 'scikit-learn_test.tar.gz'
[########################################] | 100% Completed |  4min 18.8s
(scikit-learn_0.21) $ ls -l scikit-learn_test.tar.gz
-rw-------. 1 userID in0162 1459826406 Mar 28 15:03 scikit-learn_test.tar.gz
(scikit-learn_0.21) $

F. Import Conda Environment

You can import the conda environment that was created using conda pack, as shown in the example below, and use it after setting the environment.

- Example -

$ mkdir -p $HOME/.conda/envs/scikit-learn_test
$ tar xvzf scikit-learn_test.tar.gz -C $HOME/.conda/envs/scikit-learn_test
※ If the scratch directory is the conda path, enter it as /scratch/$USER/.conda

$ conda activate scikit-learn_test
(scikit-learn_0.21) $ conda-unpack
(scikit-learn_0.21) $ conda deactivate
$

G. Deleting a Conda Environment

You can delete a conda environment using either "conda-env remove -n [ENVIRONMENT]" or "conda env remove -n [ENVIRONMENT]".

- Example -

$ conda env remove -n scikit-learn_test
Remove all packages in environment /home01/userID/.conda/envs/scikit-learn_test:

Last updated on November 08, 2024.

PreviousMain Keywords for Job Scripts NextSingularity Container

Last updated 1 month ago