comm-central future as a branch
mkmelin+mozilla at iki.fi
Tue Oct 9 08:06:45 UTC 2018
I'd like feedback on the below plan for having Thunderbird in a single
Let's have follow-ups go to dev-planning
TL;DR version: planning to have current comm-central live as a branch
instead of a separate repository.
When the Mozilla move from CVS to hg took place in 2008, the code
related to Thunderbird was not included in mozilla-central and instead
placed in the comm-central repository. Under the hood Thunderbird uses
the mozilla-central code, so to build it currently has to check out both
repositories. This is not ideal, for many many reasons.
I wrote [an email to
last year listing some of the drawbacks of this setup - but essentially
what it comes down to is that version control was not meant to be used
like this and it makes workflows error prone, in addition to having to
do various hacks in the build automation. Enabling autoland, phabricator
and other tools used by Mozilla core could also be done much more easily
for comm-central with a single repository.
The mail spurred some related discussion on git vs hg and importing of
history, but basically there was agreement among the comm-central
developers to go ahead with this change.
Thunderbird has now migrated its build automation to Taskcluster, and
also moved to building with mozilla-central as topdir, with a comm/
sub-folder containing comm-central code. (We used to instead have a
mozilla subdir checkout of mozilla-central as child the comm-central
In 2014 there was [a proposal to merge comm-central code into
but this was rejected since the powers that be did not want 1) the
confusion about what code belongs to what project, 2) pull/tree size,
and 3) it was seen as irrelevant to Firefox. The proposal below has none
of these obstacles, and depending on the physical repository location,
may not change anything at all for a Firefox developer.
I think it's now time to revisit the single repository idea and pull
that through in one way or the other. So, on to the proposal. For
reference, on disk comm/ is on currently 122 MB (+ comm/.hg 192 MB), so
size-wise it is comparatively small, with a mozilla-central checkout
weighing in at 2.3 GB. Including the relevant CVS history, the comm/.hg
would be 317 MB in total.
To capture the essence of what Thunderbird is, we would make
comm-central a named branch of the mozilla-central. Let's call this
Initially, we'd pull in a hg converted version of the current
comm-central default branch, putting all the files into a comm
sub-folder in the base directory of the comm/default branch. This is the
way that the Thunderbird build system already wants the code laid out.
With the comm sub-folder it will be clear to everyone involved which
code is on mozilla-central code and which is not. It is possible to
import the CVS history for the related directories too while we're at
it, but it adds some size. Importing related CVS history only adds
around 125 MB extra so I'm inclined to include it. With the new
comm/default branch pulled in, we merge the curent mozilla-central
default branch to comm/default. The resulting code is now the same as
status quo (of the both repos pulled in), just that it's all version
controlled in the same repository.
Once this is all set up, Thunderbird development work would move on on
the comm/default branch, and there would be periodical merges with the
(mozilla-central) default branch. These merges could be automated if wanted.
Quite often it's the case that Thunderbird code needs to be adjusted to
build, due to changes in mozilla-central code. For cases of known
incoming bustage Thunderbird now would have the possibility of waiting
to do the merge until a fix is available. I would suggest never to back
out bustage causing mozilla-central changesets from the comm/default
branch, but to detect other incoming bustage by doing builds from a
comm/band-aid branch (branced from comm/default). The details on this
can be discussed later though, not to derail discussion about the main
issue here. There are many alternatives, and it's also completely
possible to do what we currently do.
In the earlier discussions i's become clear there is a bunch of
confusion around what is a branch and what is a repository, so please
note the difference. Aa repository is the physical location where the
history of a project is stored. Every repository does not have to carry
every branch. I.e., the mozilla-central repository could carry the
comm/default branch, or the comm/default branch could exist only in a
comm-central2 repository somewhere else. In all cases it's trivial for
developers who have mozilla-central checked out, to add the comm/default
branch to their local checkout.
So for the location there are at least three options:
A: branch in the mozilla-central repository
B: branch in the mozilla-unified repository
C: branch in a repository elsewhere
Not to disrupt operations too much I think it would be preferable to use
option B - create the branch in the mozilla-unified repository. It does
depend a bit on what plans Mozilla has for these repositories.
Creating the branch in mozilla-unified of course needs buy-in from the
Mozilla hg people, so please let me hear your opinions.
What about history?
For the proposed approach (going through `hg convert`) commits are
preserved, but AFAIK it's not possible to preserve actual commit hashes.
`hg convert` will graft the original commits, and this adds the original
hash as an extra field to the new commit (use `hg log --debug` to see
it). I don't think this is such a big problem. The original commits
would be linked in the pushlog, the same way there is "converted from"
for mozilla-central-cvs, like
I don't see them e.g. in
so perhaps there is some server side feature that needs turning on?
Can I try it?
You can check
carries the comm/default branch.
In your mozilla tree, pull in the branch like this:
hg pull -u -b comm/default
To completely remove it again, use `hg strip "branch(comm/default)`
How did you do it?
git clone https://github.com/ehsan/mozilla-cvs-history.git
echo default comm/default | tee branchmap.txt
echo rename . comm | tee filemap.txt cc0.txt
hg --cwd=$CC_HG_REPO up 0
(cd $CC_HG_REPO && ls -d */ | cut -f1 -d'/' | sed 's/^/include /')
hg convert --filemap=cc0.txt --branchmap=branchmap.txt
CVS_TIP=`hg id --cwd=single-repo -i -r tip --debug`
CC_0=`hg id --cwd=$CC_HG_REPO -i -r 0 --debug`
echo $CC_0 $CVS_TIP | tee splicemap.txt
hg convert --config convert.hg.saverev=True
--branchmap=branchmap.txt $CC_HG_REPO single-repo/
For fun, see all the history down to 1999 is there, like
hg log comm/mailnews/mime/src/mimei.cpp
Then go on and add the default branch from mozilla-central and do the merge
hg pull -f -b default $CC_HG_REPO/..
hg up comm/default
hg merge default && hg commit -m "Merge default to comm/default branch"
hg push -f -b comm/default --new-branch thunderbird-central-push
Thoughts, comments, feedback appreciated!
More information about the tb-planning