GitWorkflow [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
The official %%FreeBSD%% Git repo is now at https://cgit.freebsd.org/src/

Clone TL;DR:
(public-mirror)
https://git.FreeBSD.org/src.git
ssh:%%//%%anongit@git.%%FreeBSD%%.org/src.git
(developers-only)
ssh:%%//%%git@gitrepo.%%FreeBSD%%.org/src.git

The content of this page needs to be updated, please refer to the latest doc from git migration WG at https://github.com/bsdimp/freebsd-git-docs/ , and the discussion on https://lists.freebsd.org/mailman/listinfo/freebsd-git .

Update: 2020-12-14: These are not the repositories for the 2020/2021 svn to git conversion, please see the note above. The below is mostly historic information and will soon be entirely obsolete.

====== Introduction ======

This page used to describe the inofficial git mirrors of the SVN repositories of the %%FreeBSD%% project that could be used as common repositories to
base other work on.

The repositories were read-only mirrors of the [[https://www.freebsd.org/doc/handbook/svn.html|src, doc and, ports FreeBSD Subversion repositories]]

Bear in mind that git does not replace use of Subversion in %%FreeBSD%% and the
repository is currently offered on a best-effort basis only. The steps to recreate the conversion are outlined below, so that people can replicate the process in-house should they wish to.

CAVEAT EMPTOR: It is possible that the repositories might need to be re-done due to bugs in the converter.
This would result in changed commit IDs for the impacted branches.
As the actual tree objects for a certain SVN revision will remain identical, it will always be possible to rebase or merge your work on these new branches and not run into any merge conflicts.
Advance notice will be provided if this proves necessary, along with advice on the required git commands.

In fact, "bad SVN metadata" escaped into the published repos on github.com, so they **will** be re-rolled at some future date.

====== Repositories ======

The repository mirrors are updated at least hourly. Should they lag the SVN repository by more than a day, please create a [[https://bugs.freebsd.org/bugzilla/enter_bug.cgi?component=Git%20Integration&product=Services|new Bugzilla Issue: Git Integration]]. Monitoring still needs to be put into place, help wanted btw.

As of 2020-12-14, canonical repository of doc has been migrate to git:

  * https://cgit.freebsd.org/doc/ (web repository browser)
  * ''%%https://git.FreeBSD.org/doc.git%%'' (readonly, geographically distributed mirror for cloning through https)
  * anongit@git.%%FreeBSD%%.org:doc.git (readonly, geographically distributed mirror for cloning through anonymous ssh)
  * git@gitrepo.%%FreeBSD%%.org:doc.git (read/write for committers to push)

ports:
  * **[[https://github.com/freebsd/freebsd-ports.git]]** (the preferred sources for cloning and browsing)
  * https://cgit-beta.freebsd.org/ports/ (the canonical sources, web repository browser)
  * ''%%https://cgit-beta.freebsd.org/ports.git%%'' (the canonical sources for cloning)

src:
  * **[[https://github.com/freebsd/freebsd.git]]** (the preferred sources for cloning and browsing)
  * https://cgit-beta.freebsd.org/src/ (the canonical sources, web repository browser)
  * ''%%https://cgit-beta.freebsd.org/src.git%%'' (the canonical sources for cloning)

====== Quick Start ======

  * Fork your own copy of https://github.com/freebsd/freebsd.git: [[https://help.github.com/articles/fork-a-repo/|Fork A Repo]]
  * Keep your fork in sync: [[https://help.github.com/articles/syncing-a-fork/|Syncing a fork]]

<code>
$ git clone --config remote.origin.fetch='+refs/notes/*:refs/notes/*' https://github.com/YourGitUserName/freebsd.git
$ cd freebsd && git pull   # For some reason, the notes will not be fetched during the clone operation ...
(hack, hack, hack)
$ git commit
</code>

Advanced, useful examples to show the diff between a subdir of a branch, or the commits that introduced these diffs. It is basically what ''%%svn mergeinfo --show-revs=eligible%%'' does. See ''%%git-log(1)%%'' for more info.

<code>
$ git diff origin/stable/9 master -- usr.sbin/acpi
$ git log --graph --format=oneline --right-only --cherry-pick --no-merges origin/stable/9...origin/master -- usr.sbin/acpi
</code>

See any other git howto or documentation if you're stuck on the basics.
A useful reference for the issue with notes is [[https://git-scm.com/book/en/v2|Pro Git book, written by Scott Chacon and Ben Straub and published by Apress]].

===== Adding SVN Revision Notes to an Existing Clone =====

The git notes in the Github repositories contain useful information, and show up in ''%%git-log%%'' if present. ''%%git-clone%%'' without the ''%%--config ...%%'' line from above will default to **not** cloning ''%%refs/notes%%''. You can configure the local git repo to fetch notes like so:

<code>
$ cd freebsd
$ git config --add remote.origin.fetch '+refs/notes/*:refs/notes/*'
$ git fetch   # It's many MB for src, will take some time
</code>

It will add lines to your ''%%git log%%'' output like this:

<code>
Notes:
    svn path=/head/; revision=277815
</code>

====== Integration ======

===== Using git-svn =====

[[GitWorkflow/GitSvn#Using_git-svn|Using git-svn]]

===== Merging pull requests from GitHub using git-svn =====

Note: This only applies to, and works for committers

[[GitWorkflow/GitSvn#Merging_pull_requests_from_github_using_git-svn_.28FreeBSD_committers_only.29|Merging pull requests from GitHub]]

====== FAQ ======

  * I want authornames! Use the .gitauthors file in the root of the ports tree.
  * Why the full history and not only since day X It would make the history incompatible with other people's repository and the space savings are simply not worth the hassle. As of Oct/2012, all revisions make up about 933MB.

Everything done is so that everybody can verify the integrity of the conversion using the tools mentioned in this article. Hence the focus on repeat-ability of the process.

====== Known Problems ======

  * SVN merges for single revisions (that span only parts of the tree then), are not really representable as merges in git and instead are called cherry-picks there. The converter tries to be smart about whole tree merges however -- this heuristic is only activated for the head/, user/, and projects/ branches. A whole-tree merge of project/foo into head/ should thus be correctly represented in git. Merges from head/ to stable/ or from contrib/ to head/ will be plain commits, i.e. "cherry-picks".

===== September 2019 notes/commits force push =====

Sometime around September 2019, refs/notes/commits was force-pushed, breaking fast-forward fetch.  If you had a copy of notes before this, ''%%git fetch%%'' won't automatically update notes do to this change.  To force reset your local copy to the new copy, use:

<code>
$ git fetch -f <freebsd> refs/notes/commits:refs/notes/<commits_note_name>
</code>

Where ''%%freebsd%%'' is the name of your %%FreeBSD%% Git remote (if you're not sure, try ''%%origin%%'').  Here, ''%%commits_note_name%%'' is the name of your local copy of %%FreeBSD%% ''%%refs/notes/commits%%'' (if you followed the procedure in this wiki page, try ''%%commits%%'').

====== Verification ======

===== Content =====
Verifying that the git export is identical to the SVN tree (in content), can be done like this:

<code>
$ svn export --ignore-keywords svn://svn.freebsd.org/base/head freebsd.svn
...
Exported revision 243246.
$ git clone --config remote.origin.fetch='+refs/notes/*:refs/notes/*' https://github.com/freebsd/freebsd.git freebsd.git
$ cd freebsd.git && git pull
$ git log
<We see that revision 243246 is git commit f9ebae3>
$ git checkout f9ebae3
$ cd ../freebsd.svn
$ git --git-dir ../freebsd.git/.git diff
<There should be no output, i.e. no diff>
</code>

===== History =====

To verify the history and integrity of the conversion, you are advised to run such a conversion yourself and compare (and report) any discrepancies that you find.

  * Install the ports ''%%devel/subversion%%'', ''%%devel/git%%'', ''%%devel/qt4-corelib%%'', ''%%devel/qmake4%%''
  * Get an svnsync clone of the repository you want to verify, the converter can only run against a local copy
  * Get the converter and rules: ''%%svn co http://svn.freebsd.org/base/user/uqs/git_conv && cd git_conv%%''
  * Remove conflicting git configs, these are not for you: ''%%rm -rf doc-head.git freebsd-doc.git freebsd-ports.git freebsd-base.git ports-head.git src-head.git%%''
  * Build svn2git: ''%%cd svn2git/src && make && cd ../..%%''
  * Either tweak and run the ''%%git_conv%%'' script, or do it manually: ''%%svn2git/svn-all-fast-export --add-metadata-notes --identity-domain FreeBSD.org --rules freebsd-base.rules /path/to/freebsd-base.svn%%''
  * Fetch upstream commits into your repo: ''%%cd freebsd-base.git && git remote add origin git://github.com/freebsd/freebsd.git && git fetch origin%%''
  * Find out where/if their history diverges: ''%%git merge-base master origin/master%%''

To redo this at a later stage, you simply need to
  * ''%%svnsync sync file:///path/to/freebsd-base.svn%%''
  * ''%%svn2git/svn-all-fast-export --add-metadata-notes --identity-domain FreeBSD.org --rules freebsd-base.rules /path/to/freebsd-base.svn%%''
  * ''%%cd freebsd-base.git && git fetch origin%%''
  * ''%%git merge-base master origin/master%%''

====== Further Reading ======

It is *really* recommended, that you read
[[http://eagain.net/articles/git-for-computer-scientists/|Git for computer scientists]]
and skim [[UlrichSpoerlein/GitTalkDevSummit|GitTalkDevSummit]], although it's really outdated.

It really helps to understand the data structure of a git commit, because then
you know how merging/rebasing works and can fix snafus easily.

A good visual git reference can be found at [[http://marklodato.github.com/visual-git-guide/index-en.html|A Visual Git Reference]].

This is a decent book on getting started with Git ($16 USD for ebook) [[http://pragprog.com/book/pg_git/pragmatic-guide-to-git|Pragmatic Guide to Git]]

====== Implementation Details ======

The software used for the conversion is a slightly modified fork of svn2git, as used by the KDE project. It can be found at https://gitorious.org/~uqs/svn2git/uqs-svn2git

It requires a rules file to map svn trees and/or revisions to git branches or tags. The current rules don't make use of tags but simply store them as branches.

See the project branch at [[http://svn.freebsd.org/base/user/uqs/git_conv/]] for extra patches, rules and the scripts used to do these conversions.

----

{{tag>HowTo}}