Numerical Weather Prediction on AWS & AWS-Graviton2 with WRF

Storage options

One of the many advantages of using AWS infrastructure is the availability of many storage solutions meeting different needs and criteria. It is not the purpose of this section to cover every available option, which has its own voluminous documentation, but to describe a few options of interest to WRF users. The two most common options are EBS (Elastic Block Store) and S3 (Simple Storage Service), which should meet most demands. Two more advanced options include the use of Lustre filesystems for high performance I/O attached to instances or clusters, and Amazon EFS (Elastic File System), which can provide a general filesystem attached to several instances or clusters.

The use of EBS is the most common as AWS AMIs are imprinted in this format at the time of instance creation. The AMI contains, the OS, apps, and dependencies, and constitute the root volume associated to the instance. In the case of AMIs containing WRF, the minimum root volume is usually around 75 GB with little free space. When launching an instance, increasing the available disk space can be accomplished either through increasing the root volume size (the system will not accept a smaller size) or by attaching another EBS to the instance as shown in the screenshot.

In this example, we are adding 150 GB of disk space with the caveat that our choice is of the type gp3, which offers better performance than the standard gp2 type. A discussion on the technical specifications of different EBS types is available, for example, at https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html. The advantage of using this extra EBS volume (instead of simply increasing the root volume) is that the data in this filesystem can be used separately from the instance including its attachment to other instances or clusters. This extra space is not immediately available, but it needs to be mounted. An example of how to mount this EBS volume on the data subdirectory would be:

$ lsblk                                                ! To check the name of the volume

$ sudo file -s /dev/nvme1n1

$ sudo mkfs -t xfs /dev/nvme1n1

$ sudo mkdir data

$ sudo mount /dev/nvme1n1 data

$ sudo chown centos:centos data

Close Menu