Bookmark sync improvements in Nightly

Kit Cambridge kcambridge at
Mon Nov 21 17:28:11 UTC 2016

Hi all,

Tomorrow's Nightly will include some improvements to how we track
bookmark changes for syncing [1]. I wanted to take some time to
summarize them in this email.

Previously, Sync listened for observer notifications from Places [2],
recorded the changes in memory, and periodically flushed them to disk.
This caused Sync to occasionally miss changes at startup or during a
sync, and lose changes from interrupted syncs.

The fix moves the change tracking logic into Places. Instead of using
observers, the new bookmarks tracker pulls the set of changed
bookmarks directly from the database. A new "change counter" column
increments every time a bookmark is modified in a way that should
trigger a sync, and decrements once the bookmark has been successfully
uploaded to the server. The counter is updated in a transaction,
ensuring we don't lose changes at startup or shutdown. This design
also avoids missing changes during a sync.

We think these lost changes are responsible for a large portion of
corrupt server bookmark trees. This corruption can spread to other
devices, and blocks us from enabling bidirectional syncing on iOS.
We've developed a two-pronged mitigation strategy: prevent Sync from
corrupting the server in the first place, and repair existing
corruption by uploading missing bookmarks. The tracker improvements
are a form of prevention, and repair is currently in the works [3].

If you're using Sync, I encourage you to install the About Sync add-on
[4], and validate your bookmarks [5]. We want to make sure the new
tracker doesn't introduce new errors. If you've been using Sync for a
while, it's likely you'll already have some validation errors. We want
to know about these, too, so that we can prioritize our repair
efforts. Please file new bugs in the "Firefox :: Sync" component on
Bugzilla, or reply to this thread.

Special thanks to Marco Bonardo, Mark Hammond, Richard Newman, and
Thom Chiovoloni for all their insightful feedback and reviews!

- kit

[2]: Places is the system that manages bookmarks and history in Firefox.
[5]: Once you've installed the add-on, go to Tools > About Sync, and
click the "Validation" tab under the "bookmarks" collection.

More information about the Sync-dev mailing list