Conda

Anaconda is a distribution that provides a collection of packages for scientific computing in Python and R, including fields such as data science, machine learning applications, large-scale data processing, and predictive analytics.

The Anaconda distribution is used by over 12 million users and includes more than 1,400 popular data science packages suitable for Windows, Linux, and macOS.

To install Anaconda, download the appropriate distribution for your OS from https://www.anaconda.com and install it.

(Examples) Windows, macOS, Linux

Anaconda currently offers versions based on Python 3.7 and Python 2.7.

Conda is an application provided by Anaconda for managing package versions.

Conda helps Python users easily resolve dependency issues, which are often encountered during package installation.

This document introduces how to use conda packages for Python users on the KISTI system.

The "/home01/userID" in the introduction page refers to the home directory of the test account, and should be appropriately replaced with your own path.

A. Use of Conda

Miniconda can be downloaded for each OS from the following site: https://docs.conda.io/en/latest/miniconda.html.

Anaconda can be downloaded for each OS from the following site: https://www.anaconda.com/distribution/#download-section.

Command	Description
clean	Remove unused packages and caches.
config	Modify configuration values in .condarc. This is modeled after the git config command. Writes to the user .condarc file (/home01/userID/.condarc) by default.
create	Create a new conda environment from a list of specified packages.
help	Displays a list of available conda commands and their help strings.
info	Display information about current conda install.
init	Initialize conda for shell interaction. (Experimental)
install	Installs a list of packages into a specified conda environment.
list	List linked packages in a conda environment.
package	Low-level conda package utility. (Experimental)
remove	Remove a list of packages from a specified conda environment.
uninstall	Alias for conda remove.
run	Run an executable in a conda environment. (Experimental)
search	Search for packages and display associated information. The input is a MatchSpec, a query language for conda packages.
update	Updates conda packages to the latest compatible version.
upgrade	Alias for conda update

How to Initialize Conda

You can add settings to the .bashrc file in your home directory by using the conda init command.

$ source /apps/applications/Miniconda/23.3.1/etc/profile.d/conda.sh
$ conda init
$ source ~/.bashrc

How to Change the Conda Path

By default, the conda environment and package paths are set to the home directory, but they can be changed to other locations, such as scratch.

$ echo "export CONDA_ENVS_PATH=/scratch/$USER/.conda/envs" >> /home01/$USER/.bashrc
$ echo "export CONDA_PKGS_DIRS=/scratch/$USER/.conda/pkgs" >> /home01/$USER/.bashrc
$ source ~/.bashrc

B. Create Conda Environment

Conda environments create independent virtual execution environments in Python, making it easy to manage package versions.

You can create a conda environment using the command conda create “-n [ENVIRONMENT]”.

By default, it will be created under the envs directory within the conda path, with the specified environment name.

If you use the “--use-local” option, it will be created in the user’s home directory (${HOME}/.conda/envs/[environment_name]).

※ If you have not initialized conda, run the initialization command once.
$ source /apps/applications/Miniconda/23.3.1/etc/profile.d/conda.sh
$ conda init
$ source ~/.bashrc

$ conda create -n scikit-learn_0.21 
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home01/userID/.conda/envs/scikit-learn_0.21

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use:
# > conda activate scikit-learn_0.21
#
# To deactivate an active environment, use:
# > conda deactivate
#

$ conda activate scikit-learn_0.21
(scikit-learn_0.21) $

C. Install and Check Packages in Conda Environment

You can install a package using conda install [package name].

Packages from a specific conda channel can be installed with the command “conda install -c [channel name] [package name]”.

The packages will be installed under the conda environment path created in step "2" above.

Example :

$ source activate scikit-learn_0.21
(scikit-learn_0.21) $ conda install scikit-learn
Collecting package metadata: done
Solving environment: done

## Package Plan ##

  environment location: /home01/userID/.conda/envs/scikit-learn_0.21
  added / updated specs:
    - scikit-learn

The following packages will be downloaded:
    package                  |            build
    ------------------------|-----------------
    ca-certificates-2019.1.23  |                0         126 KB
    ...
      wheel-0.33.1               |           py37_0          39 KB
    ------------------------------------------------------------
                                           Total:       277.6 MB
The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl
 ...
   zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3

Proceed ([y]/n)? y

Downloading and Extracting Packages
setuptools-40.8.0    | 643 KB    | ##################################### | 100% 
...
openssl-1.1.1b       | 4.0 MB    | ##################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(scikit-learn_0.21) $ python -c "import sklearn"
(scikit-learn_0.21) $

D. Check Conda Environment List

You can view the list of environments using “conda-env list” or “conda env list”.

(scikit-learn_0.21) $ conda env list
# conda environments:
#
base                     /apps/applications/PYTHON/3.7.1
scikit-learn_0.21     *  /home01/userID/.conda/envs/scikit-learn_0.21
(scikit-learn_0.21) $ conda deactivate
$

E. Export Conda Environment

The conda-pack package is required before exporting.

※ (Reference) https://conda.github.io/conda-pack

You can use “conda pack -n [ENVIRONMENT] -o [file name]” to package a conda environment for use on another system.

(Example) When the external internet is not available or when you need to use the same conda environment on another system.

$ conda activate scikit-learn_0.21
(scikit-learn_0.21) $ conda install -c conda-forge conda-pack
(scikit-learn_0.21) $ conda pack -n scikit-learn_0.21 -o scikit-learn_test.tar.gz
Collecting packages...
Packing environment at '/home01/userID/.conda/envs/scikit-learn_0.21' to 'scikit-learn_test.tar.gz'
[########################################] | 100% Completed |  4min 18.8s
(scikit-learn_0.21) $ ls -l scikit-learn_test.tar.gz
-rw-------. 1 userID in0162 1459826406 Mar 28 15:03 scikit-learn_test.tar.gz
(scikit-learn_0.21) $

F. Import Conda Environment

You can use a conda environment that was packaged with conda pack by setting it up as shown in the example below.

$ mkdir -p $HOME/.conda/envs/scikit-learn_test
$ tar xvzf scikit-learn_test.tar.gz -C $HOME/.conda/envs/scikit-learn_test
※ If the scratch directory is the conda path, enter it as /scratch/$USER/.conda

$ conda activate scikit-learn_test
(scikit-learn_0.21) $ conda-unpack
(scikit-learn_0.21) $ conda deactivate
$

G. Deleting a Conda Environment

You can delete a conda environment using either "conda-env remove -n [ENVIRONMENT]" or "conda env remove –n [ENVIRONMENT]".

$ conda env remove -n scikit-learn_test
Remove all packages in environment /home01/userID/.conda/envs/scikit-learn_test:

Last updated on November 11, 2024.