Native Functions and FFI in Tamarin

Edwin Smith edwsmith at adobe.com
Sun Sep 30 12:46:47 PDT 2007


(tipping the hat to Ben G, and filling in some more parts)

In Tamarin, all methods are invoked with a standard cdecl calling
convention:

   int|double (*)(MethodEnv*, int argc, void* args)

return type varies (int-sized type, or double), and args* points to
natively typed values laid out in memory, vararg style.

the job of the generated thunks is to adapt this calling convention to
the one used for the C++ code.  The C++ code will always be "thiscall"
convention because they're methods on various native classes that
subclass ScriptObject.  However, that need not be so; other C++ calling
convention and arg-type conversions can be supported either via more
information stored in the native method's AbstractMethod, or A.M.
subclass (NativeMethod, NativeMethodV).

> * I deduce from NativeFunction.h that native methods are all
non-virtual. Am I correct? Do they use the "thiscall" calling
convention?

Yes, the C++ methods use thiscall conventions.  The jit-generated thunks
that call them are called with cdecl conventions (per above).

In terms of AS3, native methods can be virtual, static, or final.
doesnt change their calling conventions.

> * What is the purpose of a native method with a cookie?

The cookie lets you use the same native method to support more than one
different AS3 method.  An example is the various property setters in the
Date class.  Check out Date.as and the Date native method decls.  it's a
bit clumbsy but its one way of doing more with less native boilerplate.

> * How do native methods throw exceptions?

call AvmCore::throwException() with an Atom value to throw.  Tamarin's
exception
handling is done with setjmp/longjmp, so a throw amounts to a longjmp.
beware,
this means that C++ destructors traversed by the throw, will not be
called.

> I see that native methods have an ABC representation... how does this
representation match the native methods?

All methods in ABC have a MethodInfo record whose contents are kept in
memory in
an AbstractMethod object (or a subclass).  Some methods also have a
MethodBody record,
but native methods dont.  MethodInfo records have the signature info.

during ABC parsing, we use the method_id (index into the MethodInfo
record array) to
find the corresponding native method table entry, which in turn contains
the C++
function pointer and an enum telling what the calling convention is.
(cookie or not, rest args or not, etc).  other calling conventions could
be added of course.

> * Is it possible to dynamically generate (at runtime) ABC for a native
method?

Yes, dynamic loading from the web is done that way for example.  The VM
doesnt care where
ABC bytecodes come from.  you could load a dll, generate an abc for it
with one
native function declaration per dll method, also generate the native
method tables
with function pointers, etc.

There is a flag on PoolObject that says whether native methods are
allowed (you dont want them in stuff you download from the web).  so
you'd want to set that obviously.

the abc would need to define "global script" whose traits declares those
methods, and whose body could be trivially empty. so the abc would have
the cpools containing function names, etc, the MethodInfo records.

> Mozilla currently has some intrinsic IDL types and calling conventions
that are not supported by Tamarin:

yup, and the various string conversions will be just as expensive as
they are for SpiderMonkey.   you'll just know statically which
conversion to call.

> as far as I can tell the NativeMethod call signature is determined
entirely by the actionscript signature declared in the ABC file, is this
correct

Yes, correct.

> I must be just missing it, but how does one call JS functions from
binary code?

grep for coerceEnter().  basically you need to get the MethodEnv* for
the JS method you want to invoke, and initialize an array of Atom's for
the args.  Array.sort contains an example, as well as various places in
the VM where we call toString(), valueOf(), or a getter/setter method.

if you just have an object and a property name (and args), the best way
to call is via Toplevel->callproperty()

> CTYPES for TAMARIN:

I think the world would be a better place if we had a standard way of
calling existing C/C++ api's directly from AS3 without writing native
methods and without writing nasty C++ code with lots of dependencies on
the internals of the VM.  it seems in-depth enough to develop as a
branch or a patch, but i'd defer on the ifdef question until it existed
in prototype form.   

Ed



-----Original Message-----
From: tamarin-devel-bounces at mozilla.org
[mailto:tamarin-devel-bounces at mozilla.org] On Behalf Of Benjamin
Smedberg
Sent: Friday, September 28, 2007 1:58 PM
To: Tamarin Discussion
Subject: Native Functions and FFI in Tamarin

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I've been poking through the code and trying to figure out how native
functions work in Tamarin and how Mozilla is going to work with that
scheme.
Please bear with the long and involved post:

MOZILLA BACKGROUND:

Mozilla scripting uses a technology called XPConnect to provide
scriptable access to native objects through XPCOM interfaces.

XPCOM interfaces are declared in the IDL language like so:

interface nsIFoo : nsISupports
{
  readonly attribute nsIBar bar;

  nsIBaz createNewBaz();
};

All interfaces inherit from the common nsISupports interface which
provides for dynamic type queries and reference counting. In binary
form, the interface above translates into a C++ pure-virtual class:

class nsIFoo : public nsISupports
{
  virtual nsresult GetBar(nsIBar* *result) = 0;
  virtual nsresult CreateNewBaz(nsIBaz* *result) = 0; };

The xpidl compiler also converts the IDL into a typelib file which
contains the signatures of the interface methods.

We have the ability to call and implement arbitrary IDL signatures at
runtime using "xptcall", which is written in assembly.

At runtime, XPConnect wraps XPCOM objects in a JS-to-binary wrapper.
This wrapper exposes the IDL attributes and method names. When a method
is called, xpconnect first converts the JS types to IDL types and then
uses xptcall to make the binary method call.

In the other direction, JS objects can implement XPCOM interfaces by
simply implementing the necessary methods/attributes. XPConnect will
create an xptcall wrapper for the JS object.

This is a vast oversimplification and there's lots of cacheing and magic
going on, but I hope I don't need to get into those details yet.

PROBLEM:

Because all DOM methods are implemented as XPCOM interfaces and the
xpconnect type conversions are expensive, xpconnect is a major
bottleneck in Mozilla performance. Instead of performing the type
conversion in a loop, we are hoping to use the Tamarin JIT facilities to
perform direct method calls from JS into the relevant binary code.

I'm still trying to understand how Tamarin's NativeMethods work, so bear
with me if my questions are underdeveloped.

INITIAL QUESTIONS:

* I deduce from NativeFunction.h that native methods are all
non-virtual. Am I correct? Do they use the "thiscall" calling
convention?

* What is the purpose of a native method with a cookie?

* How do native methods throw exceptions?

* I see that native methods have an ABC representation... how does this
representation match the native methods?

* Is it possible to dynamically generate (at runtime) ABC for a native
method?

* As a first step for Mozilla, I'd like to wrap a raw XPCOM interface
pointer as a Tamarin type and make method calls on it. This will involve
at least giving NativeMethod and codegen the ability to call virtual
methods on a pointer. It also involves translating our "nsresult" return
type into JS exceptions.

* Mozilla currently has some intrinsic IDL types and calling conventions
that are not supported by Tamarin:

IDL         Binary      JS        Notes
string      char*       String    truncating wide chars to ASCII
wstring     PRUnichar*  String    PRUnichar is a 16-bit wide-char type
AString     nsAString&  String    nsAString& is a complex mozilla string
                                  class that supports copy-on-write and
                                  operates on PRUnichar buffers
AUTF8String nsACString& String    nsACString is like nsAString but with
char
                                  buffers
ACString    nsACString& String    the JS string is truncated to ASCII:
this
                                  type will be replaced by ByteArray
nsIFoo      nsIFoo*     object    an object implementing the nsIFoo
                                  interface short unsigned short long
unsigned long
long long   NSPR integer types PRUint16/PRInt16/etc

I'm hoping that we will eventually move away from the nsAString types to
either tamarin strings or C++ strings, but that will require some
intense automated rewriting.

We'll probably want to keep the various number types; as far as I can
tell the NativeMethod call signature is determined entirely by the
actionscript signature declared in the ABC file, is this correct? That
is, declaring these all as Number won't work.

* I must be just missing it, but how does one call JS functions from
binary code?

CTYPES for TAMARIN:

In similar vein, mfinkle and I have been working on a Mozilla port of
python ctypes. This is an arbitrary FFI interface from script, see
http://starkravingfinkle.org/blog/2007/09/hello-js-ctypes-goodbye-binary
-components/
for some simple examples. Ideally we'd like to see something like this
API done using JIT instead of using libffi. Is this something we would
hook in (with ifdefs etc) to core codegen or write a separate
ctypes-codegen system?

- --BDS

- --

Benjamin Smedberg
Platform Guru
Mozilla Corporation
benjamin at smedbergs.us
http://benjamin.smedbergs.us/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG/UCMSSwGp5sTYNkRAnlOAJ4uTk6ji5HjcRQpWiXmIocepjxjbACgh2h6
jXlxvHysTBEvn2BBvnx/x0w=
=M7sJ
-----END PGP SIGNATURE-----
_______________________________________________
Tamarin-devel mailing list
Tamarin-devel at mozilla.org
https://mail.mozilla.org/listinfo/tamarin-devel


More information about the Tamarin-devel mailing list