-
Notifications
You must be signed in to change notification settings - Fork 908
Description
Some large supercomputing systems have IBM Spectrum Scale (a.k.a GPFS) for the parallel file-system. The code in openmpi-4.1.2 for GPFS support in romio321 has one minor issue. It will not compile unless one changes "stat64" to "stat" in source file ad_gpfs_open.c. To enable GPFS support in the current openmpi's romio321 path, one must pass the following option to the top-level configure script : configure ... --with-io-romio-flags='--with-file-system=gpfs', then "make" will attempt to build support for GPFS. Doing this helps significantly with MPI-IO performance on systems that have GPFS, when adding -mca io romio321 to the mpirun command. My experience has been that romio321 when properly configured provides better performance on GPFS than the current "ompio" layer. I am familiar with this issue because I was involved in getting romio321 support into IBM' Spectrum MPI for the Summit system at Oak Ridge.
I mention that current versions of MPICH have somewhat newer ROMIO code for GPFS, which does not have this issue. I don't think it is necessary to update the GPFS code in openmpi, other than to fix the issue with stat64 => stat in ad_gpfs_open.c .