[rust-dev] Update on I/O progress
banderson at mozilla.com
Wed Apr 24 15:38:32 PDT 2013
As you may know, I'm in the process of redesigning the I/O module on top
of a new task scheduler. With my [latest pull request] I've built enough
infrastructure that I'm about ready to start implementing TCP streams.
Before I get too deep though I want to make sure that everybody who
cares is aware of the design.
[latest pull request]: https://github.com/mozilla/rust/pull/6046
I had a grand idea that I would be able to produce a full [design doc],
but that's not going to happen I think. What I do have is a fair bit of
[API documentation], along with a growing list of [unresolved issues],
some of which I'll present here.
[design doc]: https://github.com/mozilla/rust/wiki/Lib-io
What I am focused on now is the design of a synchronous API for sending
and receiving streams of bytes, along with an implementation built on an
internal *asynchronous* I/O interface attached to the scheduler event
loop. This includes reading and writing files, TCP, UDP, Unix sockets,
stdio, pipes, adapters for string parsing, compression, memory buffers, etc.
There are plenty of important and related issues that I have not
considered yet, such as file and directory traversal, application-level
protocols like HTTP, async I/O. We need to take important use cases into
account, but don't necessarily need to implement them now.
I am particularly interested in making sure that common operations are
convenient and don't require a lot of boilerplate. To that end I've been
collecting [aspirational examples] of succinctly written Rust I/O code
using the hypothetical API. The goal will be to turn these into test
cases. I welcome additional examples.
The API I am proposing has a structure based on what we have now, with a
Reader and Writer trait that define a few primitive operations, along
with a number of types that implement them. Types that implement Reader
and Writer are called 'streams' and automatically implement `Stream`.
The types are influenced primarily by the existing `core::io`, .NET, Go,
I recommend reading the previously linked module documentation for the
Here I'll discuss some of the issues I've encountered so far.
## Error handling
I've spent a lot of time thinking about - and talking to Patrick about -
how I/O should deal with errors (the old `io` doesn't have a consistent
strategy). The [error handling strategy] I'm proposing is intended to
eliminate boilerplate and allow efficient recovery from errors, but it
uses some uncommon patterns.
[error handling strategy]:
In particular, it forces all return values to be 'nullable' or
'zeroable', which will lead to the usual logic errors under the right
circumstances. It also uses conditions, which aren't used much yet in Rust.
## String encoding
I have not yet put much thought about how to deal with string encoding
and decoding. The existing `io` module simply has Reader and Writer
extension traits that add a number of methods like `read_str`, etc. I
think this is the wrong approach because it doesn't allow any extra
state for the encoding/decoding, e.g. there's no way to customize the
way newline is handled. Probably we'll need some decorator types like
Do we need to have `close` methods or do we depend solely on RAII for
## Constructor options
There tend to be a lot of ways to open I/O types. Some potential examples:
* File::open(Path("diary.txt"), Create, WriteOnly)
The one you want to put at the front of the tutorial is the simple
`File::open("diary.txt")`, but then how do you use the other variations?
I am considering defining `open` more like `fn open:<S: OpenSpec>(spec:
S)`, then implementing `OpenSpec` for e.g. `&str`, `Path`, `(Path,
FileFlags, FileAccess), `Url`, etc.
Does this seem reasonable?
Sorry for the barrage of random notes. Again I encourage those
interested to read the [rt::io module docs] and the rest of the [rt::io
module] to get a clearer idea of where I'm going with this.
[rt::io module docs]:
[rt::io module]: https://github.com/brson/rust/tree/io/src/libcore/rt/io
More information about the Rust-dev