Source maps (was: Multiline Strings)

[K. Gadd]

I agree that the current source maps format does a poor job of
handling many translation/transpiling/js-as-compilation-target
scenarios. I do think it is worthwhile to try and build on the
existing format instead of reinvent it or significantly overhaul it,
though. The things it does, it does a pretty decent job of [1], and it
seems to be composable, so you can update your source map at each
stage in your JS build pipeline. It just needs to do a good job of
mapping other things like scopes, stack frames, etc.

I'll also point out that the current proposal (rev 3, as linked) is
very opaque, and despite multiple readings I failed to understand core
features until someone explained them to me. Even now, I can't really
tell how variable name mapping is accomplished with the current
proposal, and VLQ is barely understandable. A clearer description of
the spec would probably make it much easier for people to generate
usable source maps or write software that uses them.

My particular projects would definitely benefit greatly from a usable
version of the source maps spec, so I would love to help contribute to
the process of improving them. I have not seen any obvious way to
contribute in the past as the format seems to have been the result of
private development at Google and/or Mozilla. I do applaud the
original developers' attempts to keep the spec narrowly focused in
terms of what problems it solves and how it solves them.

-kg

[1] One exception to this: Dear god, why are the mappings a giant
string full of oddly-encoded variable-length data represented as
base64? I cannot imagine a worse format. This fails on multiple
counts:

a) I can't see any way to efficiently stream in this data; you have to
parse it all at once (especially since it's JSON)
b) You have to perform multiple passes on this data to make sense of
it, unless you create a customized parser: First you have to parse the
JSON, then you have to parse the mapping string to find the split
points (the semicolons), and then once you've done that you have to
undo the weird base64+VLQ encoding to get actual integers that you can
use as an internal representation. The third step can be deferred, at
least.
c) The fact that the data is all in one big chunk (even if it is
condensed) means that you have to generate it all at once and load it
all at once. That seems intrinsically awful for large source files and
it increases memory demands on each stage of your compilation pipeline
(this may be a big issue given that compilation can already be really
demanding on memory.)

The alternate 'sections' format seems to address some of this, but it
is not clear if it addresses all of it. It is unclear to me why
mappings is a single string instead of an array of smaller strings. In
all honesty, I think this entire format could be replaced with a
considerably simpler protobuf-based format, and all the
parsing/generation code could be automatically generated by the tools
that handle protobuf formats. It would be a win for everyone (but then
we'd have to figure out a way to roll the new format out; yuuuuck.)
JSON + base64 is simply not appropriate for a file format that will
largely be handled by JS *runtimes* and debuggers with significant
performance and memory concerns.

On Wed, Mar 12, 2014 at 2:00 PM, Nick Fitzgerald <fitzgen at gmail.com> wrote:
>> 4) Browsers are still all over the place in how they report Error stack trace information.
>
> We (Firefox Developer Tools) purposefully don't expose source mapped
> stacks to the web because it would require adding some kind of API to
> know when source maps are done being fetched or blocking(!) on
> Error.prototype.stack until the source map is fetched. We also avoid
> fetching source maps unless the debugger is open, so this would expose
> to the web if the debugger is open. Furthermore, we wouldn't want to
> only have the source mapped stack, you would also want the plain JS
> stack if you think that the source map could be bogus or if you are
> debugging the source maps you are generating as a tool author. This
> would further complicate the stack string.
>
>> 1) There is not yet a standard for sourcemaps. But see <https://docs.google.com/a/google.com/document/d/1U1RGAehQwRypUTovF1KRlpiOFze0b-_2gc6fAH0KY0k>,
>> <https://developers.google.com/chrome-developer-tools/docs/javascript-debugging#source-maps>,
>> and <https://github.com/mozilla/source-map/>. Would someone care to champion this for inclusion
>> in ES7?
>
> If a debug format for targeting JavaScript were to be standardized, it
> should do more than simply file/line/column translation. My thoughts
> on this subject outgrew an email reply, so I have collected them here:
>  http://fitzgeraldnick.com/weblog/55/
>
> _Nick_
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss