06/25/2015 Flame KK v18D-1 319 MB Gaia-ui Automation mozilla-central Mozilla RIL Full Flash [Locally Run] Build 42/44 smoketests, 121/128 non-smoketests

Oliver Nelson onelson at qanalydocs.com
Thu Jun 25 22:36:06 UTC 2015


Modified *requirements.txt *files in order to circumvent:
https://bugzil.la/1176882

*git/B2G/gaia/tests/python/gaia-ui-tests/requirements.txt*

*++marionette-transport==0.4*
*git/B2G/gecko/testing/marionette/client/requirements.txt *[gecko =>
mozilla-central hg repo]
--*marionette-transport>=0.4*

I based these* changes* off the comments from Martijn in the above bug, and
they appeared to fix any of the marionette issues I was encountering. My
repositories were updated to latest, then reset back to matching commits of
the engineering flame build that was flashed.


*Smoketests:*

* attached:
** jun25_smoke.html
** jun25_adhoc.html

*Test failures - 0 replicable manually:*

   - [Not reproduced manually] test_cost_control_data_alert_mobile --
   https://bugzil.la/1177553 - TimeoutException: Timed out after 20.1
   seconds
   - [Not reproduced manually] test_IMAP_email_notification -- KeyError:
   'smtp' encountered; https://bugzil.la/1176584 (maybe needs its own bug,
   but issue is identical)

*Non-smoketests:*

* attached:
** jun25_nonsmoke1.html
** jun25_nonsmoke2.html
** jun25_adhoc.html

*Test failures - 0 replicable manually:*

   - [No manual error; different UI flow] test_sync_facebook_contacts --
   TimeoutException: Timed out after 20.1 seconds
      - Appears different UI flow, but doesn't hit Jenkins server, will
      observe personal account for fault
   - [No manual error; different UI flow] test_unlink_facebook --
   TimeoutException: Timed out after 20.1 seconds
      - Appears different UI flow, but doesn't hit Jenkins server, will
      observe personal account for fault
   - [Not reproduced manually] test_rocketbar_offline_behavior -- Not
   reproduced locally - 5/5 passing
   - [Not reproduced manually] test_settings_device_info -- IndexError:
   list index out of range; reproduced via adhoc, 0/5 passing
   - [Not reproduced manually] test_fmradio_add_remove_from_favorites --
   User error initially, no headphones; test passed with headphones
   - [Not reproduced manually] test_fmradio_airplane_mode -- User error
   initially, no headphones; test passed with headphones



*Expected failures / disabled tests:*

   - [Expected fail] test_change_language_settings_rtl -
   https://bugzil.la/1137766 - Error in both test_settings_change_language
   tests: exception "Element '%s' could not be foundin select wrapper"
   - [Expected fail] test_change_language_settings -
   https://bugzil.la/1137766 - Error in both test_settings_change_language
   tests: exception "Element '%s' could not be foundin select wrapper"
   - [Expected fail] test_import_gmail_no_network.py -
   https://bugzil.la/1155763 -xfail
   - [Expected fail] test_call_log_groups.TestCallLogGroups -
   https://bugzil.la/1117804 - [v2.2] Investigate test_call_log_groups
   failure
   - [Disabled] test_settings_change_time_format -- Bug 1144868 - Disable
   test_settings_change_time_format consistently failing because of Bug 1144686
   - [Disabled] test_settings_airplane_mode -- https://bugzil.la/1131070
   - [Disabled] test_settings_bluetooth -- https://bugzil.la/1134244 -
   test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
   bluetooth_settings.enable_visible_to_all()
   - [Disabled] test_lockscreen_time_check -- Bug 1167848 - Disable
   test_lockscreen_time_check.py until bug 1141258 is fixed
   - [Disabled] test_homescreen_column_layout -- skip-if: device == "flame"
   - [Disabled] test_ftu_skip_tour_desktopb2g -- Bug 1147611 - Intermittent
   failures
   - [Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863 -
   Investigate test_fmradio_find_stations.py failure
   - [Disabled] test_receive_active_sync_email.py -
   https://bugzil.la/1028192 - Problems with ActiveSync accounts
   - [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 -
   Problems with ActiveSync accounts
   - [Disabled] test_dialer_set_up_conference_call.py -
   https://bugzil.la/1117813 - Make PlivoUtil use a valid 'answer_url' with
   a valid Plivo XML
   - [Disabled] test_call_log_all_calls.py - https://bugzil.la/1096385 -
   Investigate failure in test_call_log_all_calls.py, call log is displayed
   erroneously
   - [Disabled] test_cards_view_with_two_apps -- skip-if: device == "flame"
   - [Disabled] test_cards_view_kill_apps_with_two_apps -- skip-if: device
   == "flame"


*Build under test:*

Device firmware (base)    L1TC100118D0
Device firmware (date)    24 Jun 2015 12:38:44
Device firmware (incremental)    eng.cltbld.20150624.153831
Device firmware (release)    4.4.2
Device identifier    flame
Gaia date    23 Jun 2015 18:48:14
Gaia revision    eb0d4aefa62b
Gecko build    20150624080416
Gecko revision    4cdc1a95a672
Gecko version    41.0a1

*Summary:*

*Smoketests:*

44 tests ran in 4369 seconds.
42 passed, 135 skipped, 0 failed, 2 errors.
0 expected failures, 0 unexpected passes.

*Non-smoketest:*

65 tests ran in 5712 seconds.
61 passed, 60 skipped, 0 failed, 3 errors.
2 expected failures, 0 unexpected passes.

66 tests ran in 4552 seconds.
60 passed, 60 skipped, 1 failed, 3 errors.
2 expected failures, 0 unexpected passes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150625/52ec8aab/attachment-0005.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150625/52ec8aab/attachment-0006.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150625/52ec8aab/attachment-0007.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150625/52ec8aab/attachment-0008.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150625/52ec8aab/attachment-0009.html>


More information about the Fxosqa-report-watchlist mailing list