<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
[Cross-posted to several groups to make sure anyone whose input is
desirable will see it]<br>
<br>
cc-rework, as you may or may not recall, is the attempt of
comm-central to divorce ourselves from having to maintain a partial
copy of mozilla-central's build system, by placing comm-central as a
subdirectory of mozilla-central instead of the parent directory. So
where our tree now would be comm-central/mozilla, it would be
mozilla-central/comm after these changes.<br>
<br>
As of right now, I have enough patches queued up to be able to make
this build configuration work. The patches I need for this amount
to:<br>
1. Fix makefiles to account for stricter rules in mozilla-central's
build system (presence of IS_COMPONENT is the only one I'm aware of
at the moment) [This can done prior to the switch].<br>
2. Automatically rewrite paths in Makefiles and jar.mn files to
account for the change in the top-source directory.<br>
3. Fix paths that are not touched by those automatic rules.<br>
4. Make some rules (like testsuite targets) work when COMM_BUILD is
not defined [This sounds like it could be done before the switch,
but it really can't].<br>
5. Remove the remnants of the current build system in comm-central
(client.mk, config/*, configure.in, top-level Makefile/moz.build, a
few things from build/*). Not necessary, but clean up is good.<br>
<br>
With all of these applied, I can not only build a working
Thunderbird on Linux, I can also run through all the build steps
that buildbot uses and get more or less the same results--such as a
working installer and a working test package. This means that the
biggest blocker to doing this is to make all of the necessary
changes to the releng configs to get this to work properly. I
suspect only the builders and the l10n packaging would need changes,
largely due to shifts in paths that affect where things need to be
checked out.<br>
<br>
Now one change which I have not yet attempted to make is client.py.
The projects that use comm-central can need up to 6 repositories:
comm-central, mozilla-central, ldap/sdks, extensions/irc,
extensions/inspector, and extensions/venkman (the latter three are
used by SeaMonkey but not Thunderbird). The present form of getting
the source code amounts to the following steps:<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/comm-central/">http://hg.mozilla.org/comm-central/</a><br>
$ cd comm-central && python client.py checkout<br>
<br>
Under the final form as envisioned by cc-rework, these steps are no
longer necessarily sufficient. Here are several options and my
thoughts on their tenability:<br>
A. Keep client.py largely the same and expect users to set
directories up correctly:<br>
$ mkdir mozilla && cd mozilla<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/comm-central">http://hg.mozilla.org/comm-central</a> comm<br>
$ cd comm && python client.py checkout<br>
<br>
This requires as little changes as possible, but requires users to
know to correctly name the comm-central directory, which requires
checking out comm-central to a non-default repository name. It also
means that client.py has slightly greater risk to screw you over if
you don't follow steps properly.<br>
<br>
B. A variant of A, where client.py does not attempt to update
mozilla-central.<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/mozilla-central">http://hg.mozilla.org/mozilla-central</a> mozilla<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/comm-central">http://hg.mozilla.org/comm-central</a> mozilla/comm<br>
$ cd mozilla/comm && python client.py checkout<br>
<br>
Now client.py won't ever attempt to touch .., which gives it safer
error handling (at worst, it only affects the current directory). On
the downside, mozilla-central must be manually updated when updating
a currently existing checkout in addition to client.py checkout.<br>
<br>
C. Nuke client.py, and expect users to update repositories manually.<br>
<br>
Since a regular comm-central requires effectively the tips of three
build systems [1], one of which updates almost never. However,
people who use all three of the other extensions need to remember to
update more repositories buried far deeper in the tree, even if they
update on relatively weak velocities.<br>
<br>
D. Merge the code into fewer repositories, and combine with any of
the above variants.<br>
<br>
The big downside here is that merging risks causing a political <i>avalanche
de merde</i>, particularly if it is the
comm-central-into-mozilla-central one. The three extension
repositories have some benefits to remaining separate repositories,
in that it allows them to update at faster velocities than riding
the trains to release; this is much harder to maintain if merged
into either comm-central or mozilla-central.<br>
<br>
It turns out that we appear to have roughly 2.5 repositories that
update on a frequent basis [2]: mozilla-central and comm-central
both see several checkins a day on average. The DOM Inspector addon
sees the most development of the other four repositories, followed
by Chatzilla, while the Venkman addon and the LDAP C SDK have seen
basically no development in years. Since Venkman relies on an API
that is slated to be removed (and there are no signs of developers
willing to pitch in to move it to the new API) and is largely
obsoleted by recent chrome debugging efforts, I would anticipate
much pushback on moving it into mozilla-central. I don't think many
people would object to merging the other two extensions into
mozilla-central (they're too small to really make much of a
difference), but I don't know if the maintainers of those add-ons
are willing to see them merge into mozilla-central or not. I'm
against merging the LDAP SDKs in for other reasons [3]. Merging
comm-central into mozilla-central is something that needs its own,
full discussion anyways, so I'm not going to suggest this as an
immediate alleviation measure.<br>
<br>
E. Merge the comm-central checkout code into mozilla-central's
client.py.<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/mozilla-central">http://hg.mozilla.org/mozilla-central</a> mozilla<br>
$ cd mozilla && python client.py checkout-comm<br>
<br>
This would require some approval from mozilla-central's build peers,
and the current use of mozilla-central's client.py is largely to
update external repositories that are checked into mozilla-central
like NSS, NSPR, etc. However, I do not think it would be difficult
to get such approval for this action, and this has roughly minimal
changes from the current workflow for obtaining or updating the
build tree.<br>
<br>
F. Like E, but use mach instead<br>
$ hg clone <a class="moz-txt-link-freetext" href="http://hg.mozilla.org/mozilla-central">http://hg.mozilla.org/mozilla-central</a> mozilla<br>
$ cd mozilla && ./mach checkout-comm<br>
<br>
... Unless build peers would rather I use mach instead of client.py.
Since releng currently uses client.py to prepare source copies for
building, this would be the first use of mach in production releng
that I am aware of, and I don't know how much I trust throwing that
switch right now.<br>
<br>
<br>
Personally, I think the only tenable long-term solution is to merge
everything into mozilla-central. As it stands right now, we are
effectively suffering from the submodules repository problem, and we
currently make a wonderful case study on how annoying this problem
can be. However, this is a potentially contentious decision, and
earlier discussions with principals leads to the conclusion that
this step, if taken at all, should be done as a separate step from
the cc-rework step. Since that is out of the question for the
moment, options E or F seem to me to be the best way to go.<br>
<br>
<br>
Thoughts/questions/comments/concerns/flames?<br>
<br>
[1] Actually, LDAP C-SDKS is checked out to a specific revision.
However, that revision has effectively been tip for the existence of
LDAP SDKs in a mercurial repository, so the distinction is minor.<br>
[2] This list, believe it or not, is completely ordered by the date
of the last substantive checkin--those that exclude build fixes or
compatibility updates. In particular, Venkman's last change was
January 2012, while LDAP's last change was December 2010.<br>
[3] I gave a lot of plausible future paths for LDAP support in
Thunderbird in a very detailed blog post a few months ago. Since the
only way to have a shot at decent testing is to use OpenLDAP as a
server, if we integrate and LDAP code into comm-central, I'd rather
integrate OpenLDAP instead of the current C SDKs.<br>
<pre class="moz-signature" cols="72">--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist</pre>
</body>
</html>