Thunderbird and the LDAP c-sdk

Joshua Cranmer pidgeot18 at gmail.com
Sun Nov 18 15:01:45 UTC 2012


On 8/13/2012 8:24 AM, Mark Banner wrote:
> There's two possible routes I think we could go down:
>
>  1. Replace the sdk with the OpenLDAP version, I believe this would
>     need some API tweaks and obviously testing to ensure no functional
>     difference, or
>  2. Replace the sdk with a javascript version, which would obviously
>     need some rewriting.
>

It's been a few months with no one other than me interjecting in this 
thread (am I really the only other person who has opinions on LDAP 
code?), so I thought I'd try investigating consequences of switching to 
OpenLDAP. The following is a list of potential issues that I could come 
up with and my investigative results:

1. License compatibility.
    OpenLDAP's license is essentially equivalent to three-clause BSD, so 
there's no incompatibility there.
2. Ability to use NSPR/NSS as underlying base library.
    There is a configure option to use nss, --withtls=moznss. Compiling 
with this flag and pointing to libraries in Thunderbird's dist directory 
worked, but careful investigation of build ordering leads me to believe 
that platform configure happens too early (it looks for the nssutil.h 
include header, which doesn't happen in the right place). Tweaking the 
-I paths may help here. But, given a completed Thunderbird build, the 
program built correctly, and I verified via ldd that all of the library 
dependencies were their proper NSS/NSPR equivalents.
3. Size.
    OpenLDAP, as measured by the unpacked version of 
openldap-2.4.33.tgz, is 27MB of disk space. ldap/sdks is 21MB, 6.5MB of 
which are .hg's metadata files. OpenLDAP's official repository is in 
git, which makes doing something akin to our current client.py checkout 
model annoying, so I assume we would switch to OpenLDAP by importing a 
static copy of its build code into the tree. By comparison, the 
mailnews, mail, and suite directories are each about 22-25MB, which 
means we would be increasing the size of comm-central by roughly 1/3 if 
we imported it to hg.
    That said, it's still possible to trim down the directory-size of 
OpenLDAP. It has 7.2MB in a doc directory which could be removed; 
another 3.5MB in a contrib directory which is even easier to delete (no 
references in scripts); 2.7MB in the test directory; 7.0MB of code for 
the server-side components. Removing any code other than the contrib 
directory would require modifying the configure and Makefile.in files to 
remove references to these directories, but this is not a difficult 
task. I'm assuming that the contrib and doc code would definitely be 
deleted, as they contribute nothing to Thunderbird, bringing import size 
down to 16MB. The OpenLDAP server-side code, though large, would be 
useful to have for test purposes. Being able to validate the code of 
comm-central-built OpenLDAP via the tests is potentially useful, but the 
test suites are not fast to run and definitely require server-side code.
4. Integration in build system.
    As I alluded to earlier, there are multiple options for handling 
OpenLDAP. We could:
    a) Treat it like a system library, and provide scripts for building 
it before our build.
    b) Add a --with-system-ldap that uses the system LDAP library, and 
build OpenLDAP via subconfigure if not present. Distros would probably 
love us. Note that --with-system-ldap does provide some interesting 
issues if we wish to reuse slapd for LDAP tests.
    c) Unconditionally build OpenLDAP as a subconfigure, or perhaps 
guarded with an --enable-ldap option.
    Note that mconley's addressbook work would eventually reduce our 
dependency on LDAP to a runtime library dependency via use of ctypes. 
Until then, however, we would need OpenLDAP to precede ldap/xpcom in our 
build configuration. OpenLDAP, for our purposes, can easily build via 
|make| followed by |make install| (we would override prefix to install 
libraries into dist/lib and headers into dist/include).
5. Other requirements
    OpenLDAP apparently relies on the Cyrus SASL library for SASL 
support. This means that just checking out OpenLDAP won't give us SASL 
support, which is a bigger problem on Windows and OS X than on Linux, 
where I suspect we'd use system OpenLDAP libraries (likely configured 
with SASL support) instead. For comparison, the ldapjs library also 
lacks SASL support, while the Mozilla LDAP SDKs does include SASL 
support. There's also some other includes apparently necessary for a 
subset of the server backends, like db.h; I haven't played around with 
backend server configs, though.

Thoughts on ways forward:
Replacing our underlying LDAP library is a big change that shouldn't be 
taken lightly, especially when we have no test coverage of our LDAP code 
as-is. It has been suggested in the past (via a comment on one of my 
blog posts) that we rely on OpenLDAP's server for test code; if people 
indicate that switching to OpenLDAP is a desirable outcome, then this 
would be an acceptable way to start writing LDAP tests IMHO. The way to 
move forward would then look like this:
1. Include basic functionality tests for LDAP using a version of LDAP 
that is assumed to be installed. Obviously, these wouldn't run on 
tinderbox as-is, but they would work for people working on doing the 
migration. Perhaps these tests could be enabled if a configure option 
like --enable-ldap-tests=/path/to/slapd. I already have some ideas about 
how the slapd binaries could be hooked up to a JS controller (it 
involves a lot of nsIProcess.run, so this may cause trouble if used in 
mozmill tests :( ).
2. Include a --with-system-ldap flag that would use system-installed 
libraries instead of ldap/sdks for our ldap/xpcom code. Equivalent 
correctness could be verified with the LDAP tests, and Linux distros 
could use these flags instead and fix the biggest problem with Mozilla's 
LDAP implementation right now. It's also possible that this could be the 
end state of our LDAP switch-out: we'd support both versions of LDAP, 
and fallback on Mozilla's implementation if OpenLDAP isn't installed.
3. Drop a copy of OpenLDAP into our tree and delete the code for using 
Mozilla LDAP SDKs.


Thoughts/questions/comments/concerns?

-- 
Joshua Cranmer
News submodule owner
DXR coauthor

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/tb-planning/attachments/20121118/deecab62/attachment.html>


More information about the tb-planning mailing list