Shepherd.js - Implementing Harmony modules for today's browsers

Russell Leggett russell.leggett at
Thu Apr 19 06:11:35 PDT 2012

On Wed, Apr 18, 2012 at 10:21 AM, Xavier CAMBAR <xcambar at> wrote:

> Thanks for your comments, I'm glad you liked it.
> About CommonJS, the compatibility is the other way round. Shepherd can
> load commonJS modules without the addition of the in-comment syntax
> declaration. In such a case, require is wrapped to load whether an already
> loaded ES6 module or use commonJS's require. furthermore, exports or
> module.exports is used as the public API of the ES6 module (which is
> basically the definition :) ). This is really useful for 3rd party modules.
> Removing the comments would be possible, but it would require a much
> heavier parsing phase, and would probably require a full-blown JS parser,
> such as Esprima. it would be probably computationnally too heavy for the
> browser (read: for the user) without a systematic and automatic
> code-rewrite to ES5, optionnaly accompagned with some minification.

Yes, that's what I was getting at in my response. I know it would be a much
bigger effort, which is why I understand the approach. If I were to do the
project, though, I would assume that if it were done in the browser, that
would be purely for development - it should all be precompiled for
production. You could even use the module imports to do script

> On the other hand, the great advantage of using comments is that they act
> as placeholders for the syntax declaration, and are very easy to locate
> with a single regular expression.

This is beneficial for the implementation, but not the user.

> More realistically, I plan on allowing multiple module declarations in a
> single file (currently one only), where module implementations would be
> syntactically separated by the numerous definitions (ie, the implementation
> of a module ends at the beginning of the next module declaration).
> But yes, I am looking for an efficient way to remove the comments, which
> would be the only way Shepherd could be used as an efficient
> Harmony:modules polyfill. Maybe I'll have to play around with more
> minimalist placeholders, such as:

Personally, I think you'll want the full polyfill as a target. You can get
there incrementally by removing comments, but until it matches the full
spec, I don't think you'll see widespread adoption. IMHO anyway.

> //s6d
> module myModule {
>   import x from X;
>   export a, b, c, d;
> }
> //-s6d
> ...though the ending comment doesn't seem necessary.
> And please don't blame me for the possible terrible idea, I'm thinking
> aloud on this one! ;)

I think I would actually prefer comments over this. I wrote some code long
before module proposals that basically made my JavaScript files work like
Java files (Java is the other language we use), basically import
statements, exports were just based on the filename matching the class or
object definition inside. I would probably change it to be more like
modules if I did it now, maybe even how you did it. Anyway, point is, I put
the imports in the comments just like you for two reasons.

   1. easier to find with regexes, didn't have to parse full js
   2. didn't break existing JavaScript tooling i.e. syntax highlighting

Its the second reason that I would prefer your current comment based
solution over the new one you propose. Unless you support writing modules
the way I expect them to be written, I would rather have them in comments.

> The tradeoff to be found is the following: As of today, no browser has a
> stable release that allows harmony modules (yeah, the latest V8 has an
> option, well...). Whatsmore, the syntax is just a proposal which is updated
> almost every quarter. So do we want to allow files that work on today's
> engines and can be enhanced with Harmony:modules' features thankd to
> Shepherd, or do we want Harmony compliant files that won't run natively on
> any stable engine we can find today?
> I chose the first path, but the discussion remains open. I'll be happy to
> hear your thoughts on that point.

Support is coming. I look at it this way. Some day relatively soon, ES6
modules will be in node. Soon after that they will start showing up in
browsers. Over the course of the next year, I bet you'll see module support
in at least firefox and chrome. If you set your sights on full modules now,
you'll have code that doesn't need to be rewritten a year from now. A
polyfill will happen, do you want to write it?

- Russ

> Xavier
> On Wed, Apr 18, 2012 at 3:11 PM, Russell Leggett <
> russell.leggett at> wrote:
>> This is great! I've been considering doing the same thing, but I haven't
>> found the time. When you say it is compatible with CommonJS modules, does
>> that mean that you can do an import using ES6 syntax and have the result do
>> a CommonJS require?
>> The big thing I'm noticing is that all of the examples are inside of
>> comments, and your module definitions including exports are all separate
>> from the actual code for those exports. I'm assuming that means somebody
>> couldn't actually take the ES6 module examples and make them work, for
>> example. I appreciate the difference in difficulty level. Parsing just the
>> module syntax vs being an ES5 compliant parser + modules is a much
>> different task. The problem is, I don't see myself using it unless it
>> actually used full module syntax. Do you plan on going in that direction?
>> - Russ
>> On Wed, Apr 18, 2012 at 8:20 AM, Xavier CAMBAR <xcambar at> wrote:
>>> Hi,
>>> I wanted to announce that I've been working on a project called Shepherd
>>> (, a pure Javascript
>>> implementation of Harmony modules.
>>> Why such a project ? Fun first. Second, I was really looking forward to
>>> use harmony modules. Third, I wanted an efficient way to use my modules on
>>> the server and the client. And it seems to me that current module loaders
>>> and APIs available will be, at the end, superseded by the module syntax
>>> being defined at ECMA for the future versions of ECMAScript.
>>> The syntax used is as of 2012-02-27 (
>>> I couldn't find the time to implement the latest proposal (besides I would
>>> have had to choose one of the two variants), but apart from the syntax, it
>>> is usable both on the client and the server, and tested.
>>> The parser/lexer has been developped using JISON (
>>> and is available as a separate project (
>>> For backward-compatibility, the module declarations have to be put into
>>> comments (which you will discover in the examples provided on the site), it
>>> is compatible with CommonJS modules (user-defined as well as native modules
>>> in Node.js), and, although not critical to the project, a compatibility
>>> wrapper for the AMD API is on its way.
>>> Regarding production-level requirements, an optimizer has been
>>> implemented, but it still requires testing before being released.
>>> I've had and I'm still having a really good time working on this project
>>> and I would really appreciate if I could have some feedback from the
>>> readers and contributors of the mailing-list.
>>> A mailing list has been created for the project, still empty of messages
>>> simply because it has been created yesterday ;)
>>> Regards,
>>> --
>>> Xavier CAMBAR
>>> @xcambar <!/xcambar>
>>> _______________________________________________
>>> es-discuss mailing list
>>> es-discuss at
> --
> --
> Xavier CAMBAR
> @xcambar <!/xcambar>
> T: +33
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the es-discuss mailing list