Webkit's Sunspider JS benchmark on ActionMonkey JS

Dan Schaffer dschaffe at adobe.com
Mon Jan 7 14:25:24 PST 2008


Hi Guoxin,

I made some updates the the typed sunspider tests.  On tamarin-central
all the typed tests are faster or same compare to the untyped js1
format.

Run on WinXP, intel core 2 duo, 2.33ghz
(does not include Edwin Smith's early binding update with bugzilla
411139)

test                     TC    TC-typed   TT    TT-typed

3d-cube                  141   62         2450  2450
3d-morph                 156   125        500   500
3d-raytrace              188   110        3158  1891
access-binary-trees      94    94         1203  1329
access-fannkuch          172   94         3031  2953
access-nbody             281   134        533   533
access-nsieve            93    47         1313  1313
bitops-3bit-bits-in-byte 32    16         32    32
bitops-bits-in-byte      47    32         110   90
bitops-bitwise-and       344   312        985   100
bitops-nsieve-bits       78    47         1220  1220
crypto-aes               109   78         240   2400
crypto-md5               63    63         4203  4250
crypto-sha1              63    62         610  1610
math-cordic              141   110        406   406
math-partial-sums        313   125        935   313
math-spectral-norm       219   78         359   328
string-fasta             156   125        670   625
string-unpack-code       10750 10750      slow...
string-validate-input    141   125        5547  5547 


-----Original Message-----
From: Guoxin Fan [mailto:gfan at sta.samsung.com] 
Sent: Monday, January 07, 2008 4:40 PM
To: Dan Schaffer
Cc: tamarin-devel at mozilla.org
Subject: RE: Webkit's Sunspider JS benchmark on ActionMonkey JS

Hi,

  Previously, I was wrongly running TT and TC with Debugger_Debugger
build, which made the results look unreasonably bad.
   Here are the results of TT and TC with Release build, both on the
same WinXP machine for a better comparison.

   With Release, the controlflow-norm now has Error #1023 instead of
stack overflow. Date-format-tofe, date-format-xparb, regexp-dna
string-based64 and string-tagcloud still have the same errors as before.

Original untyped results in ().
----------------------------------------------------------------
Test Cases			     TC-VM            TT-VM
----------------------------------------------------------------
3d-cube                      63 (157)         2542 (2551)
3d-morph                     172 (219)        455 (485)
3d-raytrace                  125 (219)        1962 (3224)
access-binary-trees          94 (94)          1397 (1330)
access-nbody                 250 (266)        565 (610)
access-nsieve                47 (94)          1302 (1330)
bitops-3bit-bits-in-byte     15 (31)          63 (62)
bitops-bits-in-byte	     32 (47)          110 (141)
bitops-bitwise-and           329 (595)        126 (1064)
bitops-nsieve-bits           47 (78)          1255 (1283)
controlflow-recursive        78 (63)          Error #1023
crypto-md5                   62 (62)          4457 (4351)
crypto-sha1                  78 (63)          1695 (1675)
math-cordic                  109 (156)        424 (454)
math-partial-sums            328 (344)        753 (751)
math-spectral-norm           78 (125)         298 (313)
string-fasta                 126 (156)        643 (689)
string-unpack-code           11261 (11336)    130580 (131731)
string-validate-input        157 (140)        5274 (5052)
------------------------------------------------------------------

Thanks,
-Guoxin


-----Original Message-----
From: Dan Schaffer [mailto:dschaffe at adobe.com] 
Sent: Thursday, January 03, 2008 11:22 AM
To: Guoxin Fan
Cc: tamarin-devel at mozilla.org
Subject: RE: Webkit's Sunspider JS benchmark on ActionMonkey JS

Hi Guoxin,

Thanks for running the tests in these environments.  I can confirm your
tamarin-central and tamarin-tracing results.

Also I'm interested in comparing how the tamarin vms perform when
modifying all the code to typed variables (rather than the current JS1
untyped).  Let me know if you have time and are interested in working
together on this.

On the errors the only case I see different results was:

bitops-3bit-bits-in-byte - I see this running ok on TT on WinXP,  If you
build the Release-Debugger you can get stack traces of the javascript
code.  Also compiling with debug (asc -d ...)  will give you the line
numbers in the stack traces.  

For
date-format-tofte, date-format-xparb, string-tagcloud:  these tests use
eval.  Not yet implemented in TC and TT.  So #1065 and #1069 are
variable not defined and property not defined.  I use this link to see
runtime error codes
http://livedocs.adobe.com/flex/2/langref/runtimeErrors.html

I logged a few bugs in bugzilla for other problems:

(TC + TT) regexp-dna:  large string.match(regexp) verifier error:1068
https://bugzilla.mozilla.org/show_bug.cgi?id=409206

(TC + TT) string-base64.as: string indexing throws ReferenceError "z"[0]
https://bugzilla.mozilla.org/show_bug.cgi?id=409210

(TT only) controlflow-recurse.as stack overflow from ack function
https://bugzilla.mozilla.org/show_bug.cgi?id=409217

string-unpack-code.as (runs 15x slower on tamarin vs spidermonkey)
javascript compressor code runs slowly
https://bugzilla.mozilla.org/show_bug.cgi?id=409225
additional notes on this issue from Werner:
causes for slowness: lots of UTF8to16 and 16to8 roundtripping in our
pcre implementation,  also in pcre we are using NO_RECURSE to avoid
possible stack overflows, if rebuild with NO_RECURSE turned off in pcre
config.h how much faster, do we crash more often in regexp tests?


Dan

-----Original Message-----
From: tamarin-devel-bounces at mozilla.org
[mailto:tamarin-devel-bounces at mozilla.org] On Behalf Of Guoxin Fan
Sent: Thursday, January 03, 2008 10:57 AM
To: tamarin-devel at mozilla.org
Cc: Guoxin Fan
Subject: Webkit's Sunspider JS benchmark on ActionMonkey JS

    Hello:

      Run Sunspider test cases mentioned by Dan Schaffer:
https://mail.mozilla.org/pipermail/tamarin-devel/2007-December/000202.ht
ml
On ActionMonkey-JS, Mozilla-Central-JS, Tamarin-Central and
Tamarin-Tracing.

ActionMoneky-JS is slower than Mozilla-Central JS.

Here are the results:
------------------------------------------------------------------------
----
Test Cases			MC-JS		AM-JS 	TC-VM 	TT-VM
------------------------------------------------------------------------
----
3d-cube 		    	781		1542		295
211898
3d-morph			839		1371		437
30017
3d-raytrace			714		1069		337
231023
access-binary-trees	513		556		138
23845
access-nbody		863		1814		430
19219
access-nsieve		672		676		166
103550
bitops-3bit-bits-in-byte572		578		54
Crash 
bitops-bits-in-byte	649		654		83
281
bitops-bitwise-and	647		636		914
4906
bitops-nsieve-bits	834		942		218
84018
controlflow-recursive	498		508		109
Stack Overflow
crypto-md5			519		591		125
30720
crypto-sha1			484		549		146
1240
date-format-tofte		830		904		Error
#1065	Error #1065
date-format-xparb		493		665		Error
#1065	Error #1065
math-cordic			971		1413		371
14048
math-partial-sums		672		1170		561
16501
math-spectral-norm	582		817		218
7172
regexp-dna			1447		1547		Error
#1068	Crash 
string-base64		554		916		Error #1069
Error #1069
string-fasta		684		850		292
13359
string-tagcloud		676		764		Error #1065
Crash 
string-unpack-code	1285		1576		17452
Crash 
string-validate-input	327		512		257
36392
------------------------------------------------------------------------
----

Thanks,
-Guoxin

_______________________________________________
Tamarin-devel mailing list
Tamarin-devel at mozilla.org
https://mail.mozilla.org/listinfo/tamarin-devel


More information about the Tamarin-devel mailing list