[Go Faster] Intent to Implement: System Addon for Sync Bookmark Validation and Repair

Thom Chiovoloni tchiovoloni at mozilla.com
Wed Oct 18 19:29:00 UTC 2017


Hello!

The Sync team currently has some pieces of code in desktop Firefox that
would benefit from (more) rapid iteration. Specifically, the code
pertaining to bookmark validation and repair.

# Overview

Currently, a large portion of Sync users have corrupted bookmarks in their
server-side bookmark tree. There are a number of ways this could happen,
such as non-atomic syncs (the server only recently landed support for
atomic uploads across multiple requests), bugs in one or more clients, and
a number of other ways.

For a long time this prevented iOS devices from syncing bookmarks at all,
and while iOS has some ad-hoc fixes to enable it now, corruption still can
lead to data loss, and the fact that the tree is in a corrupt state still
limits our ability to avoid further corruption.

For reasons like these, Sync grew validation and repair code to measure and
fix the issue (the validation code came much before the repair code, and
the repair code was originally written due to iOS being blocked). The
repair code focused on a few specific issues, with the hope that the others
would get fixed automatically (there are a number of issues that imply
other issues will be present). In practice it was insufficient.

This code currently runs on Beta and Nightly. We've been reluctant to move
it to release due to it's relatively high cost (it's fairly slow, as it
necessarily requires fetching the full server bookmarks tree). We already
run for a small subset of users, and only in cases where we know the record
count is relatively low. (Specifically, we consider running validation at
most once a day, and when we consider it there's only a 10% chance we'll
actually do it, and even then it will only happen for users with < 1000
bookmarks).

# Timeline

The code is already written and lives in Sync, but needs to be factored
out. We also will need to develop mechanisms for handling it's presence or
lack thereof, etc. We don't expect this to be terribly time consuming, a
month or so seems safe (but I've been wrong before).

I don't think we plan on most of the iteration on repair/validation to be
done until Q1 2017, so riding the release train for 58 or 59 would likely
be fine (my understanding is that it's somewhat common for system addons to
ride the release train, after which they can be iterated upon more quickly
-- I can't find documentation confirming or denying this, however).

# References

The bug tracking this work, bug 1407067,
https://bugzilla.mozilla.org/show_bug.cgi?id=1407067

Most of the sync code in question:
-
http://searchfox.org/mozilla-central/source/services/sync/modules/bookmark_repair.js
-
http://searchfox.org/mozilla-central/source/services/sync/modules/bookmark_validator.js
-
http://searchfox.org/mozilla-central/source/services/sync/modules/doctor.js

Well, I hope that all makes sense.

- Thom Chiovoloni [:tcsc]
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/sync-dev/attachments/20171018/fad0f3b8/attachment.html>


More information about the Sync-dev mailing list