[rust-dev] robots.txt prevents Archive.org from storing old documentation

Ivan Kozik ivan at ludios.org
Thu Jul 10 10:15:16 PDT 2014


On Thu, Jul 10, 2014 at 3:49 PM, Jonas Wielicki
<j.wielicki at sotecware.net> wrote:
> While this is a good thing /all/ software projects should be doing imo,
> one could still explicitly allow Archive.org by prepending:
>
>     User-agent: ia_archiver
>     Disallow:
>
> ?

It looks like documentation for the old versions e.g.
http://doc.rust-lang.org/0.9/ is a 404 anyway.

Ivan


More information about the Rust-dev mailing list