[rust-dev] Deprecating rustpkg

Vladimir Matveev dpx.infinity at gmail.com
Fri Jan 31 06:07:27 PST 2014


Hi all,

> 4) If an incompatible change happens, then it's not fulfilling the same library API any more, so you stop trying to force square pegs into round holes, and **just rename the damn thing** ;) ;)

According to this logic Rust itself should have been renamed at least
nine times already :) Franky, this is rather weird approach. Libraries
are constantly being developed and improved, and it will lead to a
whole lot of the same projects but with slightly different names. Or a
part of version number will migrate to library name, which is even
worse. Versions are intended exactly for marking project milestones,
and API changes are natural part of project life, especially if it is
in active development.

> 5) Library names have namespaces, something like Java's (and go's?) com.org.libname system

The main language I work with is Java, and I also use Scala and
Clojure, so I think I can explain how Java packages and artifacts
work.

The kind of namespaces mentioned in 5) is a historical leftover which
causes a LOT of pain now. Java packages are more like modules in Rust
- except that these modules are not isolated at all in the running
program, but can be freely intermixed into a global classpath
assembled from multiple JAR files. This essentially means that there
is no module system at all, because nothing prevents you from e.g.
putting a class into a package you do not own (and then use
package-private and protected members of that package). Implementing
proper module system in Java is very hard, and current attempts like
OSGi or JBoss Modules have various drawbacks. Anyone who developed
complex application with a lot of dependencies for a JavaEE
application server would know what I mean.

Java has several dependency management systems for libraries, however.
Most notable are Ivy and Maven. These systems help in assembling
complete classpath for an application by downloading so-called
artifacts, which are just JAR archives with classes and attached
metadata. Both Ivy and Maven have two-level naming system, which
consists of group identifier and artifact identifier, plus version.
Sometimes there is also a classifier, but it is completely optional
and is used rarely. Complete artifact identifier usually looks like
this: com.google.inject:guice:3.0. Note that "com.google.inject" part
is just a string, no one forces you to use domain names, though that
may affect internal layout of Maven repository. This is most prominent
in Clojure community where libraries are usually named in one or two
words, like [project/artifact "version"] or even [project "version"]
(the latter is equivalent to [project/project "version"]). Clojure
main build tool, Leiningen, uses Maven for dependency resolution, and
aforementioned 'project' becomes group id, and 'artifact' becomes
artifact id.

So, package in Java loosely corresponds to a module in Rust, and
artifact corresponds to crates. In my opinion, Rust system is superior
to Java one because Rust crates are isolated from one another, and you
just cannot have conflicts in modules and items names. Java
conventional artifact system, on the other hand, is really great. It
has its deficiencies, but they are usually tied to concrete build
system (for example, Maven scopes). Java packages is not something
which should be copied, but artifacts systems are mostly nice and
worth learning from.

---

Personally I don't think that rustpkg should be deprecated. The fact
that it has bugs does not mean that it has got something fundamentally
wrong. As far as I understand, it was built using loosely the same
concepts as Go package manager but with own additions like
project-local builds and versions. I think this is really great
approach in its core, and we can start from it.

To summarize, I think the following features should be present in a
decent dependency management/build system:
1. Easy declaration of project dependencies.
2. Automatic downloading of project dependencies from a variety of
sources (version control systems, binary repositories etc.) with an
ability to flexibly define these sources and their priorities over
each other.
3. Support for project-level and user-level packages (this is bare
minimum; system-level packages would also be great, but they are not
required at all).
4. Support for package versions, preferably not tied to version control.
5. Fine-grained control over which dependencies should be used for the
given build.
6. Probably various build scopes with different settings.

These points are a kind of extract of my image of an ideal build
system based on my impressions of various build systems I have worked
with (Maven, Ivy, SBT, Gradle, Leiningen, Cabal, Go).

Most of these items are already present in rustpkg (disregarding bugs
for a moment):
1. Crate dependencies declaration is already present in the language,
it is `extern crate abcd = "crate id";` syntax.
2. rustpkg can use version control to download dependencies
automatically. Other kinds of repositories are not supported yet.
3. Support for various levels of installation is already present, and
it is really great. Maybe it can be tweaked a bit, but it is a solid
base.
4. Currently versions are tied to VCS. This can be changed, I think.
6. Partially supported by the Rust language itself via #[cfg(...)] annotation.

The most difficult part, I think, is 5. As far as I understand,
currently dependencies are resolved automatically, and the user cannot
override or exclude transitive dependencies. I don't know how it could
be implemented without metadata files, which means additional
complexity in the package manager. Item 2 is the second in difficulty,
because concrete format of various kinds of repositories should be
defined, and there are none now. A central, official repository would
be needed too, I think. These points have to be considered very
thoroughly so they won't result into long-standing problems in the
future.

I don't know about flexibility of rustpkg architecture because I
haven't looked through its code, so I don't know its ability to evolve
further, but I think that the main part of the functionality is
already present in it, and it will be a shame not to take advantage of
it.

2014-01-31 Lee Braiden <leebraid at gmail.com>:
> On 31/01/14 08:05, Gaetan wrote:
>
> Le vendredi 31 janvier 2014, Val Markovic <val at markovic.io> a écrit :
>>
>> .This is a huge problem in large C++ codebases. It is not fun. An example:
>> every version of Xerces-C++ puts its code in a new C++ namespace, so code is
>> in xerces_3_0, xerces_3_1, xerces_3_2 etc to prevent these kinds of issues.
>>
>
> We did that at work, this seems to be the unique, practical solution. I
> don't like when I see a hash in the library file name or symbol name, but
> this very efficient for easily manage inter dependency.
>
>
> This seems like a very wrong-headed approach to me.  The Amiga had a very
> simple and effective library system, which makes me wonder why other systems
> overcomplicate it.  It followed rules something like these, iirc:
>
> 1) If a minor change or bugfix happens, increment the minor version.
> 2) If a major change, which is backwards compatible (i.e., new features)
> happens, then increment the major version.
> 3) When loading libraries, you can specify a major and a minor version, with
> 0 for the minor version if you like.  You get at least that version, or
> better, or the loading fails.
> 4) If an incompatible change happens, then it's not fulfilling the same
> library API any more, so you stop trying to force square pegs into round
> holes, and **just rename the damn thing** ;) ;)
>
> Rule 4 seems to be where every other OS's libraries makes a big mistake.
>
>
> For the internet age, there are new complexities of decentralised forks, I
> think we'd need a few  more rules:
>
> 5) Library names have namespaces, something like Java's (and go's?)
> com.org.libname system
> 6) Anything unofficial (i.e., your patch to version 1.3, bringing it to an
> UNOFFICIAL version 1.4) goes in your own namespace, until accepted into the
> official codebase, OR you fork your own, NEW, incompatible library, as in
> (4)+(5).
>
>
> --
> Lee
>
>
> _______________________________________________
> Rust-dev mailing list
> Rust-dev at mozilla.org
> https://mail.mozilla.org/listinfo/rust-dev
>


More information about the Rust-dev mailing list