[rust-dev] Announcing the new Rust package manager, Cargo

Mon Mar 17 22:17:17 PDT 2014

On Mon, Mar 17, 2014 at 8:24 PM, Brendan Zabarauskas <bjzaba at yahoo.com.au>wrote:

> This is incredible news. Thank you for agreeing to help us out in our time
> of need!
>
> I have a couple of questions:
>
> - For those of us who are less familiar with Bundler, could you offer a
> quick run-down of how it works?
>

The most important thing about bundler is that it's structured around
allowing you to define the list of dependencies that a particular project
needs, and handles fetching them and making sure they are available to the
project *across multiple machines*.

In other words, a major goal of bundler is that you should be able to clone
a Ruby repository, run `bundle install` at its root, and have all of its
Ruby dependencies taken care of. This includes both packages published to
the Rubygems repository and packages that are still being developed (or
versions of packages not yet published) that live on Github.

The unusual features of Bundler that I think are most important are:

   - The "lockfile", which stores the actual versions of every dependency
   that the dependency resolver found, including SHAs for git dependencies.
   This file is generated every time you run `bundle install`, is meant to be
   checked into version control, and guarantees that the code that I run on my
   machine is the same as the code that you run on your machine, and the code
   that is run in tests and production (obviously, modulo any explicit test-
   or production-specific changes).
   - Conservative updating, which builds on the "lockfile" to guarantee
   that changes that you make to the primary manifest (known as the "Gemfile")
   only have the smallest possible effect on the list of packages you were
   using before. This is a solution to the "rebuild the universe" problem that
   many package managers have: when you make a small tweak to the list of
   packages you were using (even a small utility library), you are suddenly
   opted in to the latest versions of every package that match your original
   dependency range. In principle, Semver should moot this problem; in
   practice, changing the actual code you're running as little as possible is
   a good thing.

Bundler, in general, also insisted on proper dependency resolution (using
real CS), which makes the above two features possible.

TL;DR Bundler Workflow:

```
$ git clone https://github.com/my/repo
$ cd repo
$ bundle install # uses the lockfile to use last-known-good versions of
everything
$ bundle exec some-executable # run some-executable from the app's packages
$ vim Gemfile # edit the Gemfile to add a new gem or update an existing one
$ bundle install # re-calculate dependencies and fetch any new ones locally
$ git add Gemfile.lock # commit the (poorly named) lockfile to version
control
$ git push # now other machines and production can use these dependencies
```

Rust obviously has compilation to contend with, but a lot of this basic
workflow (especially around maintaining a list of last-known-good
dependencies) applies here as well.

>
> - Are there any things you learned from doing Bundler that you would do
> differently the next time round? That is, in addition to the specific
> changes you would make to tailor it to the Rust language.
>

Yes. I'll reply to this tomorrow :)

>
> ~Brendan
>
> On 18 Mar 2014, at 12:26 pm, Yehuda Katz <wycats at gmail.com> wrote:
>
> > Hello Rustlers,
> >
> > I'll be writing here more with more details soon. For now, a few quick
> comments:
> >       * I'm really glad that Mozilla and the Rust team are prioritizing
> package management. An open source language ecosystem really lives or dies
> based on how easy it is to share code, and writing a world-class package
> manager (as brson put it) takes time, especially when you account for the
> inevitable and important iteration that comes from real-world usage.
> >       * There's a lot about package management that's well-understood
> and somewhat language agnostic. On the other hand, there are things that
> are specific to native code or even more specific to Rust that a Rust
> package manager need to account for. My goal is to use well-known best
> practices for the former, and avoid reinventing the wheel, without losing
> track of what makes Rust unique or different. Carl and I are both the
> creators of the predominant Ruby package manager (bundler) and day-to-day
> production users of Rust (really!) at the company we founded. We think that
> mix should enable us to balance both of those priorities.
> >       * Over the next month or two, we plan to prioritize getting to
> regular, working milestones with Cargo. These milestones will not always
> reflect our plan for the final workflow that we expect with Cargo, but
> having real-world working code is very important when building something
> the size and scope of Cargo. We plan to share design documents (both on the
> internal architecture and expected workflows) as we work. We started work
> 10 days ago, and we already have a primitive "cargo" compiling one of our
> libraries based on its Cargo manifest, but so far only via very simple
> plumbing commands that don't reflect the actual workflow we intend.
> > In general, some guiding principles for the project:
> >       * It should be possible for new users of Rust to use Cargo to
> experiment with Rust and its library ecosystem and have success quickly.
> >       * Users of Cargo should get deterministic builds out of the box.
> If I build an artifact on one machine, it should be possible to build that
> same artifact on another machine with exactly the same source of every
> dependency, and exactly the same options.
> >       * Users of Cargo should be able to update dependencies with as
> minimal impact on other dependencies as possible. If I update a utility
> library, Cargo should avoid updates to other, unrelated libraries by design.
> >       * Cargo should support cross-compilation out of the box. As long
> as your version of Rust and its standard library are compiled for the
> expected targets, a single line of configuration should be enough to get
> builds for those targets
> >       * Cargo should support the common lifecycle for packages: a
> package starts out as a part of an existing project, moves to Github for
> easier sharing across multiple projects and eventually the open source
> community, and finally is published to a central repository with a version
> number. This means that all three kinds of "sources" (local file system,
> github, and central package repository) should be supported by the default
> distribution of Cargo.
> > We plan to publish more detail really soon, as well as more information
> on what we've already built.
> >
> > Please feel free to ask questions :)
> >
> > Yehuda Katz
> > (ph) 718.877.1325
> >
> >
> > On Mon, Mar 17, 2014 at 5:48 PM, Brian Anderson <banderson at mozilla.com>
> wrote:
> > Dearest Rustlers,
> >
> > Today I'm super-excited to announce that Mozilla has arranged to develop
> a world-class package manager for Rust. Yehuda Katz and Carl Lerche, from
> Tilde Inc., have previously built the popular Ruby package manager,
> Bundler, and now they are going to build Rust's package manager, Cargo.
> They are *experts* at the management of packages, and will deliver a tool
> that builds off both their experience in Ruby and ours with pevious
> iterations of rustpkg.
> >
> > The two of them will be talking about this project in more detail very
> soon. Please welcome them into your hearts.
> >
> > Regards,
> > Brian
> > _______________________________________________
> > Rust-dev mailing list
> > Rust-dev at mozilla.org
> > https://mail.mozilla.org/listinfo/rust-dev
> >
> > _______________________________________________
> > Rust-dev mailing list
> > Rust-dev at mozilla.org
> > https://mail.mozilla.org/listinfo/rust-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140317/72ed3a31/attachment.html>