Profile size: big offenders

Richard Newman rnewman at
Sat Feb 1 08:12:56 PST 2014

I pulled my living profile off my personal phone so I could take a snapshot of the disk usage, in aid of our campaign against fatfennec.

Android reports:
  Total:  386.00MB
  App:     33.88MB
  Data:   352.00MB

Historical peak is over 400MB.

Of that, my Mac reports that 240MB is my profile. I'll investigate the remaining 112MB when I can, but it's presumably "shit left in the raw app directory, prefs, and Sync metadata DBs".

Below is an annotated list of the big offenders, with my snide comments. "**" indicates something we should be attacking.

CCing Taras because this is the kind of stuff he digs, and it's ammo for his war on *.sqlite! This affects desktop, too.

High-level conclusions:

* We need to aggressively wipe caches. They're growing until we hit a storage limit, but that growth both makes us look bad and also affects cache seek time, to our detriment. Some of the telemetry numbers for cache hits and misses are bad -- we're looking at ninety-fifth percentiles in the multiple second ranges for both hits and misses, which is worse than having no cache at all.
* We have too many databases -- especially ones where both the memory and disk footprint would be reduced by just switching to JSON on disk.
* We don't control our WAL/journal sizes enough. We're routinely doubling the size of our DB footprint due to WALs.
* We need to clean shit up when it's dead.

I encourage y'all to pick something that particularly irks you and make some progress on figuring out why it's big and what we can do about it. I know Wes was working on some additional pruning, but this is a big and continuing effort, so help is appreciated.


 204M	Cache              -- ** This size is driving up cache hit _and_ miss times, and making us look fat.
                              We should be aggressively pruning the cache until our miss rates go up, trading that against the contribution to those times.
   1M	OfflineCache       -- Wat? Most of this is a 384KB DB for no purpose that I can discern.

Native DBs:

3.8M browser.db            -- ** A consequence of using Sync and thus having a big history, but certainly bloated by icons etc.
512K browser.db-wal
 64K tabs.db               -- Not sure what the disk block size is, but this is bigger than it needs to be for the data stored…
414K tabs.db-wal           -- ** … and wow, we should consider flushing more often!
 32K tabs.db-shm
148K health.db             -- Pretty small considering my FHR data.
285K health.db-journal     -- ** But our journals are big. Why?
288K key4.db               -- I don't even know what this is.

Gecko DBs:

 11M webappsstore.sqlite       -- ** Bug 857888. This needs to die.
1.5M webappsstore.sqlite-wal

1.4M formhistory.sqlite

1.4M cookies.sqlite-wal
1.4M cookies.sqlite

512K addons.sqlite
 23K addons.json               -- ** Does this means addons.sqlite can be deleted?

896K netpredictions.sqlite     -- ** Bug 947745 made this smaller (I asked them for a 2MB limit -- it was 40MB+!), but still, it's bad. It also causes a lot of slow queries in telemetry.

 11K extensions.json
448K extensions.sqlite          -- ** Why is this different to addons? Why is it so big? Why do we have so many DBs?!
 65K extensions.sqlite-journal

480K signons.sqlite            -- ** Bug 946857 will replace this, but we need to do that work. JSON is better, and Java should own it.


301K lightweighttheme-header
 26K lightweighttheme-footer   -- ** This is 320KB for an *uninstalled LWT*. That needs to be fixed.
160K stylish.sqlite            -- ** I uninstalled this add-on! Why do I still have its DB on disk?
3.3M safebrowsing/             -- Looks reasonable

More information about the mobile-firefox-dev mailing list