Merging with Subversion

Note: Nik's actively updating this page. I'll remove this note when I've finished.

A common argument levied against Subversion is that it provides no mechanism to assist in repeated merging. Specifically, if you merge a change from one branch to another in Subversion, and then try and repeat the merge some time later, Subversion does not record the revisions that have already been merged.

This means that the end user has to do some additional work to record information about the revisions that have been merged, otherwise the merge operation will generate many conflicts that need to be resolved by hand.

This argument is, like many, true and false.

It is true that the Subversion API provides no specific mechanism to record this information. And the “standard” command line tools that ship with Subversion do not do this.

However, the Subversion API provides a mechanism to attach arbitrary key/value pairs to entries in the repository. These are called “properties” in the Subversion parlance. These properties provide an “escape hatch” through which extra information can be added to the repository without it needing to be a “first class” item.

Various third party tools use properties to record additional information in the repository. Of particular relevance to this discussion is svnmerge. This ships in the contrib/client-side directory of the Subversion distribution, and is available in both sh(1) and Python implementations. svnmerge uses properties to record merge information, allowing repeated merges to be performed easily.

svnmerge is not the only tool that does this. SVK does something similar. There are probably others.

This walkthrough shows how svnmerge would be used to implement all the MFC merging functionality required by the FreeBSD project.

First, we need to create a repository and populate it with some files. This example follows the suggested layout described in [“SVN Repo Layout”]. We create freebsd/src/trunk/, and a projects directory, ready to be populated with two long lived projects.

% svnadmin create repo
% svn checkout file://`pwd`/repo wc
Checked out revision 0.
% cd wc
% svn mkdir freebsd
% svn mkdir freebsd/src
% svn mkdir freebsd/src/trunk
% svn mkdir projects
% svn mkdir projects/trustedbsd
% svn mkdir projects/geom
% svn commit -m 'Initial tree creation'
Adding         freebsd
Adding         freebsd/src
Adding         freebsd/src/branches
Adding         freebsd/src/trunk
Adding         projects
Adding         projects/geom
Adding         projects/trustedbsd

Committed revision 1}}}

With that done we can add some sample files to `trunk/`.

{{{
% cd freebsd/src/trunk/
% echo 'foo' > foo
% echo 'bar' > bar
% echo 'baz' > baz
% svn add foo bar baz
A         foo
A         bar
A         baz
% svn commit -m 'Add initial three files'
Adding         trunk/bar
Adding         trunk/baz
Adding         trunk/foo
Transmitting file data ...
Committed revision 2.}}}

=== Creating a stable branch ===

In order to demonstrate MFCs we need a branch to hold -stable.

{{{
% cd ../
% svn update
At revision 2.
% svn mkdir branches
A          branches
% svn copy trunk branches/8.x
A          branches/8.x
% svn commit -m 'Create 8.x branch in preparation for 8.0 release'
Adding         src/branches
Adding         src/branches/8.x

Committed revision 3.}}}

=== Changing -current ===

Before a change can be MFCd we need a change to MFC.  Accordingly, create a simple change to -current.

{{{
% cd trunk
% echo 'This must be MFCd' > foo
% svn commit -m 'A change to MFC'
Sending   trunk/foo
Transmitting file data .
Committed revision 4.}}}

As you can see, this change is '''revision 4'''.  Remember this.

=== Carrying out MFCs ===

Now that we have a change to MFC we can go and MFC it.

The first thing we need to do (and this has to be done once per branch) is tell `svnmerge` that this is a directory we'll be merging to.  `svnmerge` has a similar command structure to `svn` and `cvs`.  Namely, "svnmerge ''subcommand'' ''subcommand options''".  The initialisation subcommand is `init`, and needs to be run in the directory that you will be merging '''to'''.

Accordingly, change in to the -stable directory (`branches/8.x`), and run `svnmerge`.

{{{
% cd ../branches/8.x/
% svnmerge init
property 'svnmerge-integrated' set on '.'

% svn commit -m 'Property changes from svnmerge init'
Sending      8.x

Committed revision 5.}}}

Because Subversion has recorded that `branches/8.x` was copied from `src/trunk` we don't need to tell `svnmerge` where we're merging from -- unless told otherwise it will assumme that the merge is going to happen from the source of the copy.

Running this command has created a new property in this directory called `svnmerge-integrated`.  This property change needs to be committed.

Now that `svnmerge` knows where we're merging from, we can use the `avail` subcommand to find out if there are any revisions that we can merge.

{{{
% svnmerge avail
4}}}

Indeed there are.  As you can see, this has shown that there has been one change on `src/trunk`, which is revision 4.  This is correct.  We want to MFC this change, so we use `svnmerge` to help with this.

{{{
% svnmerge merge -r 4
U    foo

property 'svnmerge-integrated' set on '.'}}}

Instead of using the `svn merge` command we use `svnmerge merge`.  As well as merging the change in to the working copy this also updates the `svnmerge-integrated` property to note that r4 has been merged.  We can now commit the MFC from the working copy.

{{{
% svn commit -m 'MFC r4 from trunk'
Sending        8.x
Sending        8.x/foo
Transmitting file data .
Committed revision 6.}}}

=== More complicated MFCs ===

You don't have to merge every change, as this example shows.

First, change back on the trunk, and commit three changes.  Once of these changes we definitely don't want to MFC, the other two are MFC candidates.

{{{
% cd ../../trunk
% echo 'foo - not for MFC' > foo
% svn commit -m 'Change foo.    Not for MFC'
Sending    trunk/foo
Transmitting file data .
Committed revision 7.
% echo 'bar change, for MFC' > bar
% echo 'baz change, for MFC' > baz
% svn commit -m 'Fix bar, MFC after: ...' bar
Sending       bar
Transmitting file data .
Committed revision 8.
% svn commit -m 'Fix baz, MFC after: ...' baz
Sending       baz
Transmitting file data .
Committed revision 9.}}}

Now, go back to the stable branch, and see what we can commit.

{{{
% cd ../branches/8.x
% svnmerge avail
7-9}}}

This shows that revs 7, 8, and 9 are available for merging.  But we know that one of them (r7) is not an MFC candidate.  We can inform `svnmerge` of this with the `block` subcommand.  We block the revision from showing (which is a property change, so should be committed).

{{{
% svnmerge block -r 7
property 'svnmerge-blocked' set on '.'

% svn commit -m 'Prevent r7 from showing as a merge candidate'
Sending       8.x

Committed revision 10.}}}

Now we can use the `avail` subcommand again to see what's available for merging.

{{{
% svnmerge avail
8-9}}}

That no longer shows r7 as a merge candidate.

Now we need to decide how to merge these two revisions.  We can either merge and commit them one after another, or we can lump them in to one large change that's going to be merged en-masse.  Opting for the later choice, we can use the `8-9` syntax with the `merge` subcommand.

{{{
% svnmerge merge -r 8-9
U    bar
U    baz

property 'svnmerge-integrated' set on '.'

% svn commit -m 'MFC r8 and r9'
Sending        8.x
Sending        8.x/bar
Sending        8.x/baz
Transmitting file data ..
Committed revision 11.}}}

That's it.  Pretty much everything you need to know about carrying out MFCs.  `svnmerge` has some additional features that are useful.

 * `svnmerge integrated` shows you which revisions have already been merged (essentially the inverse of `svnmerge avail`).
 * `svnmerge unblock` makes revisions blocked with `svnmerge block` available for merge again.
 * `svnmerge avail` supports `--diff` and `--log` options (amongst others) that will show the available changes as a diff, or as a list of log messages, instead of the default list of revisions.  This is very handy when you want to check which change a particular rev corresponds to.

== Project branch merge walkthrough ==

MFCs are relatively simple, as they're always one-way (from -current, to one or more of the stable branches).  Project branch merges are more complex.  A long lived project, like trustedbsd or geom is going to want to periodically carry out MFCs.  But they're also going to want to merge their functionality back to -current, either all in one go, or piece by piece as the work becomes appropriate for inclusion in to -current.

This walkthrough assumes you've already carried out the basic MFC walkthrough.

=== Merge from current to project branch ===

Lets start the trustedbsd project, make some changes to `src/trunk`, and merge them in.

Start the project by taking a copy of `src/trunk`.

{{{
% cd ../../../..
% svn update
At revision 11.
% svn copy freebsd/src/trunk projects/trustedbsd
A         projects/trustedbsd/trunk
% svn commit -m 'Start the trustedbsd project'
Adding       projects/trustedbsd/trunk

Committed revision 12.}}}

Now continue making changes to `src/trunk`.

{{{
% cd freebsd/src/trunk
% echo 'ongoing development' > foo
% svn commit -m 'Work continues on trunk' foo
Sending   trunk/foo
Transmitting file data .
Committed revision 13.}}}

{{{
% cd ../../../projects/trustedbsd/trunk/
% svn update
At revision 13}}}