[rust-dev] robots.txt prevents Archive.org from storing old documentation
banderson at mozilla.com
Mon Jul 14 11:16:43 PDT 2014
Can somebody file an issue described exactly what we should do and cc me?
On 07/14/2014 01:13 AM, richo wrote:
> On 10/07/14 17:15 +0000, Ivan Kozik wrote:
>> On Thu, Jul 10, 2014 at 3:49 PM, Jonas Wielicki
>> <j.wielicki at sotecware.net> wrote:
>>> While this is a good thing /all/ software projects should be doing imo,
>>> one could still explicitly allow Archive.org by prepending:
>>> User-agent: ia_archiver
>> It looks like documentation for the old versions e.g.
>> http://doc.rust-lang.org/0.9/ is a 404 anyway.
> Which is kinda the point of letting archive.org keep a copy for
> Not that I have a bunch of sway, but I'm +1 on not letting old docs be
> searchable, and also +1 on making an exception for archive.org
> Rust-dev mailing list
> Rust-dev at mozilla.org
More information about the Rust-dev