NEMO-3.6 Code Repo Maintenance

Set-up

The /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repository is an svn checkout of http://forge.ipsl.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE and also a read-only Mercurial repository. It was initialized with:

$ cd /ocean/sallen/hg_repos
$ svn co -r 5072 http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE NEMO-3.6-hg-mirror
$ hg init NEMO-3.6-hg-mirror
$ cd NEMO-3.6-hg-mirror
$ cat > .hgignore
syntax: glob
.svn
ctrl-d
$ hg add
$ hg ci -m"Initialize NEMO-3.6 svn mirror at r5072 of ^/trunk."

svn v1.8.8 was used on skookum for the svn part of the initialization.

Doug maintains an NEMO-3.6-mirror-merge repo on his laptop. That repo is used to merge changes from the upstream Subversion repository that are brought in via the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo, and from the Salish Sea team NEMO-3.6-code repo on Bitbucket. The NEMO-3.6-mirror-merge repo was created by cloning the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo:

hg clone ssh://skookum.eos.ubc.ca//ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror NEMO-3.6-code-mirror-merge

and setting the paths in its .hg/hgrc to:

[paths]
bb = ssh://hg@bitbucket.org/salishsea/nemo-3.6-code
default-push = ssh://hg@bitbucket.org/salishsea/nemo-3.6-code
mirror = ssh://skookum.eos.ubc.ca//ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror

Those paths mean that the repo for hg pull and hg incoming commands must be specified explicitly. The bb and mirror paths are provided to facilitate pulling from nemo-3.6-code on Bitbucket and /ocean/sallen/hg_repos/NEMO-3.6-code-hg-mirror, respectively. hg push and hg outgoing commands will act on the nemo-3.6-code repo on Bitbucket, unless otherwise specified.

The Salish Sea team NEMO-3.6-code repo on Bitbucket was created via the Bitbucket web interface and populated there by an hg push from Doug’s NEMO-3.6-mirror-merge repo.

A working copy was then created by cloning the Salish Sea team NEMO-3.6-code repo on Bitbucket as NEMO-3.6-code. The workflow sections below explain how these 4 repo clones are used to pull changes from upstream and merge them with changes that Salish Sea team members push to Bitbucket.

../../../_images/NEMO-3.6CodeRepoMaint.svg

NEMO-3.6 code repositories and workflow to update and merge SVN and local changes

Workflow to Pull Changes from NEMO-3.6 svn Repo

The workflow to pull changes from the master NEMO svn repo and commit them to our NEMO-3.6-hg-mirror repo is somewhat automated by the Marlin – Salish Sea NEMO svn-hg Maintenance Tool.

  1. Review the upstream changes in the source browser at http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE to select a range of changes to be pulled into our NEMO-3.6-hg-mirror repo.

    Note

    Pay special attention to changes in the OPA_SRC/ and TOP_SRC/ trees that involve files that have been copied into NEMOGCM/CONFIG/SalishSea/MY_SRC/ or team members’ MY_SRC/ directories. Those files must be manually merged with their MY_SRC/ counterparts.

  2. Working on salish in the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo with an activated virtualenv in which marlin is installed:

    $ ssh salish
$ workon marlin
(marlin)$ cd /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror

  3. Use marlin incoming information about the next SVN revision that will be pulled from upstream and confirm that it is the expected revision:

    (marlin)$ marlin incoming
r6482 2016-04-19 09:59:19 UTC
  #1687 - Add a consistency check for the setting of ln_useCT and nn_eos

    The --limit option can be used to see more incoming revisions; see marlin help incoming for details.

  4. Use marlin update to update the working copy to the next upstream commit and commit the SVN update as a Mercurial changeset with the SVN commit message as the body of the Mercurial commit message and echo that message:

    (marlin)$ marlin update
Update to svn r6482.

#1687 - Add a consistency check for the setting of ln_useCT and nn_eos

    The --to-rev option can be used to apply a series of upstream updates, committing them to Mercurial one at a time; see marlin help update for details.

Workflow to Merge NEMO-3.6 svn Repo and Salish Sea Revisions

Merging changes from NEMO svn and the Salish Sea team NEMO-3.6-code repo on Bitbucket is done in a repo that is used for only that purpose. Doug does the merges on his laptop. The repo in which the merging is done was created by cloning the /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror repo as described in the Set-up section.

After the Workflow to Pull Changes from NEMO-3.6 svn Repo has been completed the workflow to merge those changes with Salish Sea MEOPAR project revisions is:

  1. Pull and update recent changes from the Salish Sea team NEMO-3.6-code repo into NEMO-3.6-mirror-merge:

    cd NEMO-3.6-mirror-merge
hg pull --update bb

  2. Pull and update the changes from /ocean/sallen/hg_repos/NEMO-3.6-hg-mirror into NEMO-3.6-mirror-merge:

    hg pull mirror

  3. Because the changesets pulled from the Salish Sea team NEMO-3.6-code repo are public a branch merge is necessary:

    hg merge
hg commit -m"Merge svn updates."

  4. Manually merge and commit changes that involve files that have been copied into NEMOGCM/CONFIG/SalishSea/MY_SRC/ or team members’ MY_SRC/ directories. Those files are most likely to be in OPA_SRC/ and TOP_SRC/.

  5. Push the result of the updates and merges to the Salish Sea team NEMO-3.6-code repo:

    hg push bb

    If other users have pushed changes to the Salish Sea team NEMO-3.6-code repo while merge conflicts were being handled hg pull --rebase can be used to bring in those changes and deal with any additional merge conflicts.

  6. Notify team members of the upstream merge, especially if manual merges of MY_SRC/ files were required, so that they can manage merging changes into any untracked MY_SRC/ files they may have.

Local commits

We are maintaining several local commits to files that do not fit in MY_SRC/ directories. They are:

  1. Import changeset 7751 from upstream NEMO: 900e1ce4066f2a64223b502167a9a15a6d7c7b2d

  2. Make rebuild_nemo write compressed netCDF4 restart files: 28e2d9cd2a82b88e475a16804837eee22ea342f2

  3. Sort cfg.txt in makenemo, 9f0335b59c780d055b7466e5ffd641d4ab0234aa

  4. Escape ‘ls’ in Fmake_WORK.sh, 4979fc15317c1c899697c1b89d5a64d688917253

Some of these are fixed in NEMO upstream:

  1. Escape ‘ls’ in Fmake_WORK.sh, Ticket 1895, Changeset 8527