NEMO-3.6 Code Repo Maintenance

Warning

This section is completely out of date because NEMO now uses Git for version control rather than SVN, and we now use Git and GitHub for our Salish Sea NEMO code repository rather than Mercurial and Bitbucket.

This section is retained to provide guidance for development of a future maintenance workflow when we move to a newer version of NEMO.

Set-up

The /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repository is an svn checkout of http://forge.ipsl.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE and also a read-only Mercurial repository. It was initialized with:

$ cd /ocean/sallen/hg_repos
$ svn co -r 5072 http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE NEMO-3.6-hg-mirror
$ hg init NEMO-3.6-hg-mirror
$ cd NEMO-3.6-hg-mirror
$ cat > .hgignore
syntax: glob
.svn
ctrl-d
$ hg add
$ hg ci -m"Initialize NEMO-3.6 svn mirror at r5072 of ^/trunk."

svn v1.8.8 was used on skookum for the svn part of the initialization.

Doug maintains an NEMO-3.6-mirror-merge repo on his laptop. That repo is used to merge changes from the upstream Subversion repository that are brought in via the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo, and from the Salish Sea team NEMO-3.6-code repo on Bitbucket. The NEMO-3.6-mirror-merge repo was created by cloning the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo:

hg clone ssh://skookum.eos.ubc.ca//ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror NEMO-3.6-code-mirror-merge

and setting the paths in its .hg/hgrc to:

[paths]
bb = ssh://hg@bitbucket.org/salishsea/nemo-3.6-code
default-push = ssh://hg@bitbucket.org/salishsea/nemo-3.6-code
mirror = ssh://skookum.eos.ubc.ca//ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror

Those paths mean that the repo for hg pull and hg incoming commands must be specified explicitly. The bb and mirror paths are provided to facilitate pulling from nemo-3.6-code on Bitbucket and /ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror, respectively. hg push and hg outgoing commands will act on the nemo-3.6-code repo on Bitbucket, unless otherwise specified.

The Salish Sea team NEMO-3.6-code repo on Bitbucket was created via the Bitbucket web interface and populated there by an hg push from Doug’s NEMO-3.6-mirror-merge repo.

A working copy was then created by cloning the Salish Sea team NEMO-3.6-code repo on Bitbucket as NEMO-3.6-code. The workflow sections below explain how these 4 repo clones are used to pull changes from upstream and merge them with changes that Salish Sea team members push to Bitbucket.

../../../_images/NEMO-3.6CodeRepoMaint.svg

NEMO-3.6 code repositories and workflow to update and merge SVN and local changes

Workflow to Pull Changes from NEMO-3.6 svn Repo

The workflow to pull changes from the master NEMO svn repo and commit them to our NEMO-3.6-hg-mirror repo was somewhat automated by the (now deprecated) Marlin tool that we developed.

  1. Review the upstream changes in the source browser at http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE to select a range of changes to be pulled into our NEMO-3.6-hg-mirror repo.

    Note

    Pay special attention to changes in the OPA_SRC/ and TOP_SRC/ trees that involve files that have been copied into NEMOGCM/CONFIG/SalishSea/MY_SRC/ or team members’ MY_SRC/ directories. Those files must be manually merged with their MY_SRC/ counterparts.

  2. Working on salish in the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo with an activated virtualenv in which marlin is installed:

    $ ssh salish
$ workon marlin
(marlin)$ cd /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror

  3. Use marlin incoming information about the next SVN revision that will be pulled from upstream and confirm that it is the expected revision:

    (marlin)$ marlin incoming
r6482 2016-04-19 09:59:19 UTC
  #1687 - Add a consistency check for the setting of ln_useCT and nn_eos

    The --limit option can be used to see more incoming revisions; see marlin help incoming for details.

  4. Use marlin update to update the working copy to the next upstream commit and commit the SVN update as a Mercurial changeset with the SVN commit message as the body of the Mercurial commit message and echo that message:

    (marlin)$ marlin update
Update to svn r6482.

#1687 - Add a consistency check for the setting of ln_useCT and nn_eos

    The --to-rev option can be used to apply a series of upstream updates, committing them to Mercurial one at a time; see marlin help update for details.

Workflow to Merge NEMO-3.6 svn Repo and Salish Sea Revisions

Merging changes from NEMO svn and the Salish Sea team NEMO-3.6-code repo on Bitbucket is done in a repo that is used for only that purpose. Doug does the merges on his laptop. The repo in which the merging is done was created by cloning the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo as described in the Set-up section.

After the Workflow to Pull Changes from NEMO-3.6 svn Repo has been completed the workflow to merge those changes with Salish Sea MEOPAR project revisions is:

  1. Pull and update recent changes from the Salish Sea team NEMO-3.6-code repo into NEMO-3.6-mirror-merge:

    cd NEMO-3.6-mirror-merge
hg pull --update bb

  2. Pull and update the changes from /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror into NEMO-3.6-mirror-merge:

    hg pull mirror

  3. Because the changesets pulled from the Salish Sea team NEMO-3.6-code repo are public a branch merge is necessary:

    hg merge
hg commit -m"Merge svn updates."

  4. Manually merge and commit changes that involve files that have been copied into NEMOGCM/CONFIG/SalishSea/MY_SRC/ or team members’ MY_SRC/ directories. Those files are most likely to be in OPA_SRC/ and TOP_SRC/.

  5. Push the result of the updates and merges to the Salish Sea team NEMO-3.6-code repo:

    hg push bb

    If other users have pushed changes to the Salish Sea team NEMO-3.6-code repo while merge conflicts were being handled hg pull --rebase can be used to bring in those changes and deal with any additional merge conflicts.

  6. Notify team members of the upstream merge, especially if manual merges of MY_SRC/ files were required, so that they can manage merging changes into any untracked MY_SRC/ files they may have.

Local commits

We are maintaining several local commits to files that do not fit in MY_SRC/ directories. They are:

  1. Import changeset 7751 from upstream NEMO: 900e1ce4066f2a64223b502167a9a15a6d7c7b2d

  2. Make rebuild_nemo write compressed netCDF4 restart files: 28e2d9cd2a82b88e475a16804837eee22ea342f2

  3. Sort cfg.txt in makenemo, 9f0335b59c780d055b7466e5ffd641d4ab0234aa

  4. Escape ‘ls’ in Fmake_WORK.sh, 4979fc15317c1c899697c1b89d5a64d688917253

Some of these are fixed in NEMO upstream:

#. Escape ‘ls’ in Fmake_WORK.sh, https://forge.ipsl.fr/nemo/ticket/1895, https://forge.ipsl.fr/nemo/changeset/8527