[rust-dev] Why focus on single-consumer message passing?

Daniel Micay danielmicay at gmail.com
Sun Jan 26 05:13:35 PST 2014


On Fri, Jan 24, 2014 at 2:37 PM, Brian Anderson <banderson at mozilla.com> wrote:
> Thanks for your well thought-out considerations about Rust's message passing
> strategy.
>
>
> On 01/24/2014 07:32 AM, Daniel Micay wrote:
>>
>> The language documentation currently takes a very opinionated view on
>> concurrency. It focuses on message passing and at times makes the
>> claim that Rust does not have shared memory between tasks. I don't
>> think the language should be taking a position like this but rather
>> providing useful tools to implement a concurrent application as the
>> developer sees fit.
>
>
> Yes, Rust has always promoted message passing as the preferred way to do
> concurrency. Rust also provides useful tools to implement concurrency as the
> developer sees fit. I believe that recommending message passing by default
> is reasonable since it is widely applicable and easy to use. Perhaps we can
> update language in various documentation to not claim absolutely that Rust
> has no shared memory, but it is important to express that Rust protects
> developers from the pitfalls of shared memory.

I think shared memory is widely applicable and easy to use too. It's
often much harder to use message passing.

>> The library should be offering the `Arc` and `MutexArc` types in
>> `libstd` along with other useful concurrent data structures. A
>> concurrent hash table split into shards is a very scalable primitive
>> and quite trivial to implement. There's no reason to encode keyed
>> inserts/searches/removals with message passing when it's faster and
>> easier to do it directly.
>
>
> Possibly, yes. Instead of putting everything in std though I would rather
> foster a culture of using small crates. It is easier to accept more
> experimental code into the standard distribution if they are in seperate
> packages that can be developed independently. A concurrency crate would be
> more attractive to me than putting specialized data structures in the
> standard library.

Why not move message passing to another crate then? It doesn't have to
be in the standard library either. The standard library could provide
only native I/O and threading like C++, with the contentious stuff
moved to other crates and built on top of it. If it proves to be
useful, then the community will use it.

Shared memory is already far more solid and less experimental than the
existing concurrency and I/O support.

> I don't know what 'API sacrifices made at the performance altar' means. This
> sort of unspecific, inflammatory criticism is not necessary or welcome. Many
> people (myself included) have put an enormous amount of work into
> discovering how to build abstractions in this new language. It is a long,
> iterative process, there are many tradeoffs, and mistakes are occassionally
> made. Please try to be nice.

Rust is making a lot of semantic sacrifices for the sake of
performance. I see most of it as being under the assumption that it
will eventually perform well rather than being based on hard numbers.

> I believe the single-consumer restriction has to do with the complexity of
> implementing 'select' with multiple consumers. Do you have that implemented
> in rust-core?

I have an implementation for Windows and Linux but haven't pushed it
into rust-core. I haven't set up a FreeBSD install so I don't have a
working implementation with kqueue. It's significantly (20-30%+)
slower in the case where the condition variable is being hit a lot due
to imbalance in consumption/production. It also currently only
supports selection over channels as I don't really see a way to do
cross-platform AIO with acceptable performance. Linux is only going to
be able to select on non-blocking sockets, unless it's all shoved
behind more `eventfd` descriptors hooked up to a thread pool...


More information about the Rust-dev mailing list