# Notes on the Oct-2013 CODE.tar Version¶

These are notes on building and running the version of NEMO configured for the Salish Sea as received in the CODE.tar archive prepared by J-P Paquin and downloaded at UBC on 2-Oct-2013.

The “Notes on NEMO/OPA Usage” document in the DOCUMENTATION.tar archive provided guidance for the creation of the notes below.

Todo

## What’s in CODE.tar¶

The CODE.tar tarball unpacks to a directory called CONCEPTS110_WCSD_OW_NOBC_tide which apparently indicates:

• CONCEPTS110: Canadian Operational Network of Coupled Environmental PredicTion Systems model at the version 110 level

• WCSD: MEOPAR West Coast Sub-Domain configuration

• OW: Open West boundary

• NOBC: ?? maybe refers to reading OBC file data instead of initial conditions ??

• tide: focus of the model runs is to calculate tides in the domain

CONCEPTS110_WCSD_OW_NOBC_tide contains jcompile.sh, as bash script that runs the build tool chain described below, and the modipsl/ directory.

modipsl is the framework that was used to obtain and build the NEMO code prior to NEMO up to v3.2. modipsl is available via anonymous svn checkout from http://forge.ipsl.jussieu.fr/igcmg/svn/modipsl/. However, it is unclear what revision/tag of modipsl the tarball contains.

The documented (in “Notes on NEMO/OPA Usage” and in the NEMO Quick Start Guide) means of obtaining the NEMO code is to run:

cd modipsl/util
model NEMO


which automates the process of doing checkouts of code from a collection of cvs and svn repositories.

Examination of the modipsl/util/log file that the modipsl/util/model script creates and the the modipsl/util/mod.def definitions file for the script indicate that model was run on 18-Mar-2010 and that the nemo_v3_1 tag was used to obtain the NEMO code.

## Build Notes¶

Note

The build process described here was attempted on several platforms with the following results:

• MacBook running OS/X 10.8.5 with Xcode installed: fait_AA_make reported sed: RE error: illegal byte sequence numerous times,

• salish: fait_AA_make failed because ksh was not installed

• jasper: build was successful

The build NEMO 3.1 for a new configuration the following steps are required:

• modeles/NEMO/OPA_SRC/par_oce.F90 must be edited to set the number of processors to be used, and include configuration parameters via a .h90 file. The edits are done in C-preprocessor (CPP) blocks. J-P Paquin did this, using the CPP key key_wc3 to specify an 8x8 processor layout and parameters in par_WC3.h90. The edits are flagged with comments that start with !JPP.

• Based on modeles/NEMO/OPA_SRC/par_ORCA_R2.h90, create the configuration parameters header file included via the new CPP key in modeles/NEMO/OPA_SRC/par_oce.F90. The file created by J-P Paquin is par_WC3.h90.

• Edit modeles/UTIL/fait_config to define the NEMO source files that are required for the build. The configuration name, WC3 in the present case, must be added to the LIST shell variable, a newline-separated list of configuration names. A set -A statement must also be added to the collection that follows the LIST. The configuration is referenced in the set -A statement as DIR_configname; i.e. DIR_WC3 for the present case. The set -A statement lists the source code directories that are to be included in the build. Those edits are present in fait_config in the CODE.tar tarball.

• Run:

cd modeles/UTIL
./fait_config WC3


That results in the creation of the modeles/NEMO/WORK/ directory in which the source files from the directories given in the set -A DRI_WC3 ... statement in fait_config are symlinked so as to appear to be all in the same directory. The modipsl/config/WC3/ directory is also created. Its scripts/ directory contains the BB_make and BB_make.ldef files.

• Edit BB_make.ldef to activate/deactivate CPP keys for the configuration. The keys are listed in a single, space-separated line as the value of the P_P variable. modipsl/config/WC3/scripts/BB_make.ldef in CODE.tar contains three P_P lists identified with #-- JPP comments. The uncommented one (and so presumably most recently used) is:

#-- JPP 20130717 Run compilation keys for TEST1
P_P = key_wc3 key_dtatem key_dtasal key_flx_core key_vvl key_zrefsurf key_zdftke key_traldf_c2d key_dynldf_c3d key_mpp_mpi key_ldfslp key_dynspg_ts2 key_dtatem_month key_dtasal_month key_obc_mer key_tide key_diaharm


Also ensure that there is a prefix for preprocessing line for the build/run target, for example:

#-Q- jasper  prefix = -D

• Ensure that there is an appropriate set of definitions in modipsl/util/AA_make.gdef for the build/run target, for example:

#-Q- jasper     #- Global definitions for jasper.westgrid.ca using Linux Compiler Intel v8
#-Q- jasper     LIB_MPI = MPI2
#-Q- jasper     LIB_MPI_BIS =
#-Q- jasper     M_K = make
#-Q- jasper     P_C = cpp
#-Q- jasper     P_O = -P -C -traditional $(P_P) #-Q- jasper F_C = mpiifort -c -fpp #-Q- jasper #-D- MD F_D = -g #-Q- jasper #-D- MN F_D = #-Q- jasper #-P- I4R4 F_P = -i4 #-Q- jasper #-P- I4R8 F_P = -i4 -r8 #-Q- jasper #-P- I8R8 F_P = -i8 -r8 #-Q- jasper #-P- ?? F_P = -i4 -r8 #-Q- jasper F_O = -O3$(F_P)  -I$(MODDIR) -I$(MODDIR)/oce -module $(MODDIR) -assume byterecl -convert big_endian -I$(NCDF_INC)
echo working dir: $(pwd) module load compiler/intel/12.1 module load library/intelmpi/4.0.3.008 module load library/netcdf/4.1.3 module load library/szip/2.1 mpiexec ./opa  If that script is stored as jasper.pbs, a run is submitted with the command: qsub jasper.pbs  As an initial test, the run duration was set to 720 time steps via the &namrun.nitend namelist item. The run completed in just over 2 minutes. A subsequent 4320 time step run took about 17 minutes. ## Post-Processing¶ The results of the runs described above are groups of 64 netCDF files (one for each processor) for each of the calculated quantities: • U, V, W, and T • output.init: initial time step output ?? • restart and open boundary condition restart • 2D slice timeseries results • tidal harmonics disagnostic results ### NOCSCOMBINE¶ Google lead to the NOCSCOMBINE tool at ftp://ftp.soc.soton.ac.uk/omfftp/NEMO/NOCSCOMBINE.tar. Building it on jasper required creation of a new makefile with NCHOME and LIBS variable set to: NCHOME = /lustre/jasper/software/netcdf/netcdf-4.1.3 LIBS = -L$(NCHOME)/lib -I\$(NCHOME)/include -lnetcdf -lnetcdff -lhdf5_hl -lhdf5 -lz -lsz


Commands like:

cd WCSD_RUN_tide_M2_OW_ON_file_DAMP_ANALY/
../../NOCSCOMBINE/nocscombine -f WC3_CU60_20020102_20020104_grid_U_0000.nc


result in the 64 pre-processor files of u velocity results being combined into a single WC3_CU60_20020102_20020104_grid_U.nc file. The process takes over 10 minutes per quantity for U, V, and T for the 72 hour run, and nearly 30 minutes for W.