# Lustre stripe

## A. Basic Settings for the Neuron Lustre Striping

The Neuron Lustre File System supports file striping, which distributes a single file across multiple OSTs (Object Storage Targets, i.e., physically distributed disks) to reduce bottlenecks and improve I/O performance. In particular, the Progressive File Layout (PFL) feature, supported from Lustre 2.10 onward, is applied to the /scratch file system. This feature automatically adjusts the stripe count based on the file size, enhancing I/O performance without requiring the user to manually configure striping. The striping settings for the Neuron file system are as follows:

![](https://2000842020-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtvPPtXiAYArocipMRdp9%2Fuploads%2Fgit-blob-186cdffafcf047087a5763e7bfc4ce80c6abaf73%2Fbasic_settings_for_the_neuron_lustre_striping.png?alt=media)

## B. Lustre Striping Concept

![Figure 1 The Concept of Lustre Striping](https://2000842020-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FtvPPtXiAYArocipMRdp9%2Fuploads%2Fgit-blob-88c43e8e7e8370c95d8e1ca351e8dd07579b9d97%2FByuiN89DGA7hjDU.png?alt=media)

Lustre maximizes I/O performance for large files by dividing data across OSTs, with the maximum effective degree of parallelism being equal to the number of OSTs. As shown in the diagram, even a single file is stored in parallel on OSTs using the Lustre Striping feature.

## C. Stripe Settings and Verification

```shell-session
$ lfs setstripe [--stripe-size|-s size] [--stripe-count|-c count] filename|dirname
```

A command to apply striping settings to a file or directory. All files created in the directory where this command has been applied, or any files created with this command, will have the striping settings applied

* \--stripe-size
  * Set the size of data to be stored on each OST
  * Once the specified size is stored, the data is saved to the next OST
  * The default value is 1MB, and setting stripe\_size to 0 uses the default
  * stripe\_size must be set as a multiple of 64KB and should be less than 4GB
* \--stripe-count
  * Set the number of OSTs to be used for striping
  * The default value is 1, and setting stripe\_count to 0 uses the default
  * If stripe\_count is set to -1, all available OSTs will be used

```shell-session
$ lfs getstripe filename|dirname
```

※ Command to check the striping settings applied to a file or directory

## D. Recommendations and Tips

If you specify setstripe for the directory where the model’s result files will be saved within a job script, all subsequent subdirectories and files will inherit these settings.

Setting --stripe-count to 4 for files larger than 1GB typically improves performance. Testing is recommended when using larger values

\--stripe-size is only effective for files larger than several terabytes, so using the default value is generally sufficient

{% hint style="info" %}
Last updated on November 11, 2024.
{% endhint %}