Lustre stripe

A. Basic Settings for the Neuron Lustre Striping

The Neuron Lustre File System supports file striping, which distributes a single file across multiple OSTs (Object Storage Targets, i.e., physically distributed disks) to reduce bottlenecks and improve I/O performance. In particular, the Progressive File Layout (PFL) feature, supported from Lustre 2.10 onward, is applied to the /scratch file system. This feature automatically adjusts the stripe count based on the file size, enhancing I/O performance without requiring the user to manually configure striping. The striping settings for the Neuron file system are as follows:

B. Lustre Striping Concept

Figure 1 The Concept of Lustre Striping

Lustre maximizes I/O performance for large files by dividing data across OSTs, with the maximum effective degree of parallelism being equal to the number of OSTs. As shown in the diagram, even a single file is stored in parallel on OSTs using the Lustre Striping feature.

C. Stripe Settings and Verification

$ lfs setstripe [--stripe-size|-s size] [--stripe-count|-c count] filename|dirname

A command to apply striping settings to a file or directory. All files created in the directory where this command has been applied, or any files created with this command, will have the striping settings applied

• --stripe-size

  • Set the size of data to be stored on each OST

  • Once the specified size is stored, the data is saved to the next OST

  • The default value is 1MB, and setting stripe_size to 0 uses the default

  • stripe_size must be set as a multiple of 64KB and should be less than 4GB

• --stripe-count

  • Set the number of OSTs to be used for striping

  • The default value is 1, and setting stripe_count to 0 uses the default

  • If stripe_count is set to -1, all available OSTs will be used

$ lfs getstripe filename|dirname

※ Command to check the striping settings applied to a file or directory

D. Recommendations and Tips

If you specify setstripe for the directory where the model’s result files will be saved within a job script, all subsequent subdirectories and files will inherit these settings.

Setting --stripe-count to 4 for files larger than 1GB typically improves performance. Testing is recommended when using larger values

--stripe-size is only effective for files larger than several terabytes, so using the default value is generally sufficient

Last updated on November 11, 2024.