space management

Benjamin Smedberg benjamin at smedbergs.us
Fri Dec 21 09:48:07 PST 2007


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thomas Reilly wrote:
> Part of our tamarin-tracing effort has been to re-evaluate MMgc.
> There's ascpects of MMgc's design that we knew right off the bat would
> make it non-ideal for embedded devices.  We have a prototype in shop
> to replace MMgc and I wanted to see what the folks who care about
> space management think about it.

I need to read this proposal more throughly before responding in detail,
which will have to wait until January. ActionMonkey is in a state where we
could prove real measurement of alternate allocators on a medium scale, but
the XPCOMGC experiment is not ready for the really large-scale testing that
we'll need to cover the "all of mozilla" case.

> First, the problems with MMgc:
> 
> We don't think the 4kb block architecture is suitable for embedded
> devices.  The size class (40) X memory type (3) model leads to pretty
> large base level heaps even for trivial applications.  Running in a
> fixed 128kb heap environment is never gonna happen with that model.

Is this the stated goal? Mozilla's mobile targets are a 64MB device, which
provides a heap significantly larger than 128k. I wonder if we will end up
needing a different allocator backend.

> Process wide GCHeap.  We've witnessed in the player that if you have
> two large applications when one goes away it leaves virtual memory
> looking like swiss cheese.  We want more compartmentalization of
> address space.

This is an interesting point: in Mozilla we have large object graphs that
have caused significant fragmentation, but this is not something we can
solve by having separate GC heaps: instead we will need to have solutions
such as a low-fragmentation allocator and arenas that associate objects of
similar lifetime together.

Let me introduce Stuart (pav), who has been heavily involved in measuring
Mozilla memory usage and has written up some blog posts on the topic:

http://blog.pavlov.net/2007/12/06/more-allocator-data-tcmalloc-edition/
http://blog.pavlov.net/2007/12/04/vlad-and-analysis-of-dtrace-was-used/

> MMgc spends most of its time in mark.  We want to explore exact
> marking techniques to speed up marking.  Mostly this b/c of being
> conservative but we can do more to speed it up than just going exact
> (separating gc/non-gc pointers in object layout, prefetching etc).

Is this penalty because we have to ask the page map whether it's a GC page
for every single pointer value? Or what is the profile for marking? It is
probably possible to provide exact mark information for large swaths of
mozilla objects, provided enough time, but it won't be simple and will
invove inventing at least one intermediate language.

> Finalizers and the gc callbacks (presweep/postsweep/prereap) have been
> problematic.  Ideally the memory manager won't have any re-entrancy.
> presweep and finalizers are kinda redundant, we think we want to
> remove finalization and force clients to make do with presweep.

Doesn't this mean that clients would have to keep an out-of-band cache of
objects that need finalization-like behavior? This seems drastic if there
are alternate solutions.

> So the protoype I've been working on is basically re-working MMgc to
> address these issues.  We started with a juicy center based on TLSF
> [1] and added the conservative/drc based collection around it.  TLSF
> works like traditional malloc implementations but it scales into
> larger objects well with a uniform algorithm which makes it small (ie none 
> of the tree stuff dlmalloc has for medium size allocations).

Pav has had significant sucess with tcmalloc. Is there a particular reason
you chose rtmalloc?

> There's no more managed vs. unmanaged distinction for write barriers,
> there's only one write barrier.  For tamarin's purposes this means
> AvmCore is just like any other object.  Unmanaged and managed objects
> are handled the same and both are scanned (unless they ask to be
> opaque).  Unmanaged just means the gc will never free it.

But if you have an "unmanaged" object, you aren't required to call
writebarriers on it, correct?

> The new library includes a "lot" abstraction to break up large
> allocations. The lot theory is that large allocations lead to

Do you have the code somewhere? I'm worried about this on a whole host of
levels, and it's hard to imagine without seeing the actual API.

> 1) accurate GC, heap is easy need to figure out native function
> frames, forth/Box stack and JIT stack.  Plan is to dump our thoughts
> on this to a bug if you want to particpate.

I'm sure we do, especially if accurate GC also implies a moving GC.

> Sorry for the long post, hopefully this prototype will land in the
> public soon as a tamarin-tracing branch, we'd like to air it out
> publically before committing it to TT.

You know, you can create a personal repository on hg.mozilla.org using these
instructions and post work-in-progress trees to it:

http://developer.mozilla.org/en/docs/Publishing_Mercurial_Clones

- --BDS

- --

Benjamin Smedberg
Platform Guru
Mozilla Corporation
benjamin at smedbergs.us
http://benjamin.smedbergs.us/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHa/xXSSwGp5sTYNkRAvtMAJ9MW4aMNGgZeP3MUUw68+9aLBXLTgCfWNls
Nz2ftQMiYxgJ7AEQCLywP7Q=
=xSk6
-----END PGP SIGNATURE-----


More information about the Tamarin-devel mailing list