[rust-dev] Fwd: [racket] Planet2 questions

John Clements clements at brinckerhoff.org
Wed May 1 09:44:35 PDT 2013

Forgive me for cross-posting. The message below sheds light on some hard-earned lessons about versioning in the Racket package system. There may well be large differences between Rust and Racket in this regard, but the decoupling of version numbers from in-code dependencies was, I believe, an insight that took about five years to emerge.

Apologies if this is well-known or off-topic.


Begin forwarded message:

> From: Matthew Flatt <mflatt at cs.utah.edu>
> Subject: Re: [racket] Planet2 questions
> Date: April 30, 2013 9:24:39 AM PDT
> To: Berthold Bäuml <berthold.baeuml at dlr.de>
> Cc: Racket Mailing List <users at lists.racket-lang.org>, Matthias Felleisen <matthias at ccs.neu.edu>
> At Tue, 9 Apr 2013 18:18:35 +0200, Berthold Bäuml wrote:
>> I hope to find time to write up the model this weekend. But what is
>> absolutely clear to us is that a package management system has to
>> have a notion of versions (or variants as I called them) for a
>> package at the lowest level -- otherwise such simple things as
>> roll-backs or composing a system from packages in a deterministic way
>> (i.e., by precisely specifying a version for all packages) is not
>> possible. Of course, on top of such a low level system one can add
>> more convenient interfaces for users for the more standard use cases,
>> e.g., mimicking the version disregarding behavior of planet2 -- but
>> it is not possible to do it the other way round.
> I have been working with Jay on the package manager recently, and so I
> have some further info on this point.
> Building versions into the lowest level could mean different things:
> 1. It could mean that a packages are referenced in programs with a
>    specific version number.
> 2. It could mean that the package system has a built-in way to declare
>    dependencies not only on a particular package, but on a particular
>    revision of a package --- either to ensure that certain features
>    are available or that certain bugs are not available.
> 3. It could mean that there's a fine-grained way to say which
>    implementation of a package should be used in a given
>    installation/configuration.
> #1 was a prominent feature of Planet1 that we've dropped in the new
> package manager. In fact, packages are not referenced at all in program
> sources; only collections are referenced, and some collections turn out
> to be supplied by packages in a given installation/configuration. That
> is, package management and program execution are more separate.
> #2 is built into the new package system, though in an intentionally
> simplified form compared to Planet1. All version X specifications in
> dependencies mean "at least version X", and those specifications are
> intended to ensure the availability of new functionality or fixes that
> were added in a backwards-compatible way. Backwards incompatibility is
> handled by creating a new package name.
> #3 is at the core of the new package system, and the rest of this
> message is about that one. I think it's probably close to what you have
> in mind; I'm interested to hear more about whether you mean something
> different.
> ----------------------------------------
> To be clear about #3, we have to distinguish "package name" from
> "package implementation". A package name is something like "mischief",
> which you use for installing and declaring dependencies. A package
> implementation is something that you download from, say,
> https://github.com/carl-eastlund/mischief/tarball/     fe7119517a4dcd3f5c509735a7b5a5664151c14f
> Note that a package implementation in this sense corresponds to
> specific revision of a pile of code, such as a particular commit in a
> git repository. The package manager includes the concepts of a "package
> source" and a "checksum", which together tell you how to get a package
> implementation. (That implementation may have its own version number,
> which corresponds to #2 above, but such a version number is in
> principle orthogonal to the package implementation's checksum.)
> The mapping from a package name to a package implementation is provided
> by a "catalog". (This is recent terminology; until last week, a
> "catalog" was variously called a "package name resolver" or "index".)
> PLT provides a catalog server at pkg.racket-lang.org, but you can make
> your catalog (as a server or on a local filesystem), and so you can
> precisely control the mapping from package names to packages.
> Furthermore, we've recently added tools to `raco pkg' to make it easier
> to manage catalogs. For example, if you want to take a snapshot of the
> current pkg.racket-lang.org and use that from now on (so that the
> mapping from package names to packages doesn't change), use these
> commands:
> raco pkg catalog-copy  https://pkg.racket-lang.org /full/path/to/catalog/
> raco pkg config --set catalogs file:///full/path/to/catalog/
> You can modify the files generated at "/full/path/to/catalog/" by hand
> in a fairly obvious way. Or you can upload the directory to a
> file-serving HTTP site and point installations to the uploaded
> directory as the catalog. There's also an option to use an SQLite
> database as the format for a catalog, which is a better option if you
> want to modify the catalog programmatically via `pkg/db', but an SQLite
> database is less easy to use from a file-serving HTTP site.
> In particular, I can imagine having a project whose source code
> includes a package catalog. To upgrade a particular package, I'd change
> the catalog and `raco pkg update'. When I commit a particular revision
> of the source code to a git repository, the package catalog is saved;
> then, I can roll pack the project (including its references to specific
> package implementations) to any previous version with its associated
> package implementation via a `git checkout' (or whatever) plus `raco
> pkg update'. Working this way, the package catalog acts a lot like git
> submodules.
> ____________________
>  Racket Users list:
>  http://lists.racket-lang.org/users

More information about the Rust-dev mailing list