Webkit's Sunspider JS benchmark on ActionMonkey JS

Guoxin Fan gfan at sta.samsung.com
Fri Jan 4 10:03:14 PST 2008


 Hi, Dan:

   Done with the convert test cases to typed. I removed the previous
Error cases. 

    Some variables not clear how to convert. I will send you the
converted cases. Could you please fix them and send me back your fixes.

   I run TC on my Linux/Debian. For TT, I run Windows Debugger_Debugger
build (I can not build TT on Linux).

   For TC, most of the cases have gain. But for TT, there are more crash
cases and most of cases are actually worse than untyped.

 Typed Cases Results: (original untyped results in ().
------------------------------------------------------------
Test Cases                   TC-VM                 TT-VM
3d-cube                      127 (295)             Crash (211898)
3d-morph                     337 (437              30325 (30017)
3d-raytrace                  214 (337)             crash (231023)
access-binary-trees          142 (138)             35576 (23845)
access-nbody                 438 (430)             19553 (19219)
access-nsieve                141 (166)             10506 (103550)
bitops-3bit-bits-in-byte     37  (54)              crash (crash)
bitops-bits-in-byte          75  (83)              8043  (281)
bitops-bitwise-and           125 (914)             105005 (4096)
bitops-nsieve-bits           111 (218)             85708 (84018)
controlflow-recursive        120 (109)             stack overflow 
crypto-md5                   115 (125)             38872 (30720)
crypto-sha1                  113 (146)             17866 (1240)
math-cordic                  201 (371)             17727 (14048)
math-partial-sums            641 (561)             17276 (16501)
math-spectral-norm           128 (218)             17104 (7172)
string-fasta                 286 (292)             16168 (13359)
string-unpack-code           18167 (17452)         crash (crash)
string-validate-input        291 (257)             crash (36392)
------------------------------------------------------------------------
-

Thanks,
-Guoxin
  
-----Original Message-----
From: Dan Schaffer [mailto:dschaffe at adobe.com] 
Sent: Thursday, January 03, 2008 11:22 AM
To: Guoxin Fan
Cc: tamarin-devel at mozilla.org
Subject: RE: Webkit's Sunspider JS benchmark on ActionMonkey JS

Hi Guoxin,

Thanks for running the tests in these environments.  I can confirm your
tamarin-central and tamarin-tracing results.

Also I'm interested in comparing how the tamarin vms perform when
modifying all the code to typed variables (rather than the current JS1
untyped).  Let me know if you have time and are interested in working
together on this.

On the errors the only case I see different results was:

bitops-3bit-bits-in-byte - I see this running ok on TT on WinXP,  If you
build the Release-Debugger you can get stack traces of the javascript
code.  Also compiling with debug (asc -d ...)  will give you the line
numbers in the stack traces.  

For
date-format-tofte, date-format-xparb, string-tagcloud:  these tests use
eval.  Not yet implemented in TC and TT.  So #1065 and #1069 are
variable not defined and property not defined.  I use this link to see
runtime error codes
http://livedocs.adobe.com/flex/2/langref/runtimeErrors.html

I logged a few bugs in bugzilla for other problems:

(TC + TT) regexp-dna:  large string.match(regexp) verifier error:1068
https://bugzilla.mozilla.org/show_bug.cgi?id=409206

(TC + TT) string-base64.as: string indexing throws ReferenceError "z"[0]
https://bugzilla.mozilla.org/show_bug.cgi?id=409210

(TT only) controlflow-recurse.as stack overflow from ack function
https://bugzilla.mozilla.org/show_bug.cgi?id=409217

string-unpack-code.as (runs 15x slower on tamarin vs spidermonkey)
javascript compressor code runs slowly
https://bugzilla.mozilla.org/show_bug.cgi?id=409225
additional notes on this issue from Werner:
causes for slowness: lots of UTF8to16 and 16to8 roundtripping in our
pcre implementation,  also in pcre we are using NO_RECURSE to avoid
possible stack overflows, if rebuild with NO_RECURSE turned off in pcre
config.h how much faster, do we crash more often in regexp tests?


Dan

-----Original Message-----
From: tamarin-devel-bounces at mozilla.org
[mailto:tamarin-devel-bounces at mozilla.org] On Behalf Of Guoxin Fan
Sent: Thursday, January 03, 2008 10:57 AM
To: tamarin-devel at mozilla.org
Cc: Guoxin Fan
Subject: Webkit's Sunspider JS benchmark on ActionMonkey JS

    Hello:

      Run Sunspider test cases mentioned by Dan Schaffer:
https://mail.mozilla.org/pipermail/tamarin-devel/2007-December/000202.ht
ml
On ActionMonkey-JS, Mozilla-Central-JS, Tamarin-Central and
Tamarin-Tracing.

ActionMoneky-JS is slower than Mozilla-Central JS.

Here are the results:
------------------------------------------------------------------------
----
Test Cases			MC-JS		AM-JS 	TC-VM 	TT-VM
------------------------------------------------------------------------
----
3d-cube 		    	781		1542		295
211898
3d-morph			839		1371		437
30017
3d-raytrace			714		1069		337
231023
access-binary-trees	513		556		138
23845
access-nbody		863		1814		430
19219
access-nsieve		672		676		166
103550
bitops-3bit-bits-in-byte572		578		54
Crash 
bitops-bits-in-byte	649		654		83
281
bitops-bitwise-and	647		636		914
4906
bitops-nsieve-bits	834		942		218
84018
controlflow-recursive	498		508		109
Stack Overflow
crypto-md5			519		591		125
30720
crypto-sha1			484		549		146
1240
date-format-tofte		830		904		Error
#1065	Error #1065
date-format-xparb		493		665		Error
#1065	Error #1065
math-cordic			971		1413		371
14048
math-partial-sums		672		1170		561
16501
math-spectral-norm	582		817		218
7172
regexp-dna			1447		1547		Error
#1068	Crash 
string-base64		554		916		Error #1069
Error #1069
string-fasta		684		850		292
13359
string-tagcloud		676		764		Error #1065
Crash 
string-unpack-code	1285		1576		17452
Crash 
string-validate-input	327		512		257
36392
------------------------------------------------------------------------
----

Thanks,
-Guoxin

_______________________________________________
Tamarin-devel mailing list
Tamarin-devel at mozilla.org
https://mail.mozilla.org/listinfo/tamarin-devel


More information about the Tamarin-devel mailing list