Numerical Weather Prediction on AWS with WRF
The following guide helps you to deploy an EC2 instance running WRF.
- Numerical Weather Prediction on AWS with WRF can be used by any user with an active AWS account. If you are new to AWS, follow the instructions at https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/ to create a new account. You will also need to create an IAM user (https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html). You do not need the aws_access_key_id and aws_secret_access_key for running WRF on a single instance or downloading data from S3 buckets, but you will need them for more complex actions such as launching a cluster or using storage capabilities. Either way, keep these keys safely as required by AWS.
- The AMI with the preinstalled software can be downloaded from the AWS Marketplace (https://aws.amazon.com/marketplace/). Choose the right architecture (x86_64 or AArch64) for your type of instance and subscribe. Make sure to understand the charges for AWS infrastructure and for the AMI.
- Once your subscription is active, you can launch available instances based on your choices of architecture and configuration. More information about launching EC2 instances is available at https://docs.aws.amazon.com/quickstarts/latest/vmlaunch/step-1-launch-instance.html. The AMI is available in most regions and AZs. Contact us if you wish to use WRF on AWS GovCloud (US). A few tips for instance launch follow:
a) Spot instances are available per the usual conditions.
b) The AMI is 75 GB but only about 10 to 15 GB are free. Users have several options to increase available space. Two easy options are to increase the storage at the time of launching instances or to use EBS space. The latter can be particularly useful for large datasets to be recycled with different instances and clusters. The ‘storage option’ links describes these options in greater detail.
c) Choose the SG according to you own needs. As a minimum, it should have port 22 open. If you are new to AWS, this is the SG configuration by default.
d) Once you have selected the region, you will need to use your own existing keypair or create a new one for launching WRF.
- Connect to your instance using a SSH client or similar (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AccessingInstances.html). Your username is ‘centos’ and you have ‘sudo’ privileges.
- NICE-DCV (https://docs.aws.amazon.com/dcv/index.html) is available for high-performance display and preinstalled in the AMI. However, and unlike for clusters, you will need to manually start the server ($ sudo systemctl start dcvserver) and create the session ($ dcv create-session wrfdisplay). The session will be available in most web browsers via https://External_IP_instance:8443/#wrfdisplay.
- WPS and WRF are installed at the /home/centos/WPS and /home/centos/WRF-4.2.2 directories, respectively. After all preprocessing tasks are complete, the WRF app can be run with ‘$mpirun -np N wrf.exe’ where N is the number of MPI ranks if you are from a directory with both case files and the executable. If not, you will need to either create a symlink with the file or include the directory “/home/centos/WRF-4.2.2/main/wrf.exe.” For Graviton-2 instances, the number of cores equals the number of vCPUs and you should use this value as the number of MPI ranks. For Intel-powered instances, the number of cores is half the number of vCPUs. We recommend running one MPI rank per core (e.g. it would be ‘mpirun -np 36’ for a c5n.18xlarge instance) but running in hyperthreaded mode requires adding the ‘–oversubscribe’ flag’ (e.g. ‘mpirun –oversubscribe -np 72’). Running jobs in clusters requires the use of Slurm (see specific instructions for cluster).
- The subdirectory /home/centos/DATA contains several preprocessing data of interest:
a) Preprocessing data for the January 2000 example case (https://www2.mmm.ucar.edu/wrf/OnLineTutorial/CASES/JAN00/index.php).
b) A file detailing the content of the CESM LENS (Community Earth System Model Large Ensemble). Files can be downloaded with the command aws s3 (e.g. aws sp cp s3://ncar-cesm-lens//filename). Add the flag ‘–no-content’ if you still have not configured your AWS CLI credentials.
c) A file OpenData_list with the endpoints of available datasets. Other datasets might become available and the list will be updated accordingly.
- NCL (NCAR Command Language) is available for post-processing duties. The official website (https://www.ncl.ucar.edu/) furnishes documentation on the language itself and how to create high-quality graphics from WRF data.
Return to AWS-Marketplace main menu