03/30/2015 Flame KK v18D-1 319 MB Gaia-ui Automation mozilla-central Mozilla RIL Full Flash Build 30/45 smoketests, 114/133 non-smoketests

Geo Mealer gmealer at mozilla.com
Mon Mar 30 18:57:26 UTC 2015


As mentioned in IRC, the wifi errors are due to the patch that landed in
https://bugzilla.mozilla.org/show_bug.cgi?id=1123680.

I note this has been backed out in b2g-inbound, per our request, but am
less clear as to whether/when this'll hit mozilla-central for the sake of
our automation reports. My hope is backouts are merged in the same way as
any other landing, in which case it should reflect in the results soon.

Some of the other stuff (the ones that didn't repro in ad hoc) look like
flukes. A couple of the "didn't reproduce manually" appear to be real
errors, but it's probably ok to let them ride until tomorrow or 4/1 to see
if they're consistent or whether they're intermittent.

Geo

On Mon, Mar 30, 2015 at 10:45 AM, Oliver Nelson <onelson at qanalydocs.com>
wrote:

>
> *Smoketests:*
>
> http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/129/
>
> *Test failures - 0 replicable manually:*
>
>    - [Not reproduced manually] test_gallery_flick - New error:
>    https://bugzil.la/1149202  test_gallery_flick failing to flick:
>    "image.flick_to_next_image()"
>    - [Not reproduced manually] test_music_artist_mp3 - New error:
>    https://bugzil.la/1149207 test_music_artist_mp3 fails to select
>    artist: "sublist_view = artists[0].tap_first_artist()"
>    - [Not reproduced manually] test_browser_cell_data - Did not repro
>    from Jenkins adhoc: 5/5 passing
>    - [Not reproduced manually] test_cost_control_data_alert_mobile - Did
>    not repro from Jenkins adhoc: 5/5 passing
>    - [Not reproduced manually]
>    test_dialer_receive_call_with_locked_screen - https://bugzil.la/1097754
>    test_dialer_receive_call_with_locked_screen.py sometimes doesn't wait until
>    the callscreen arrives at the bottom of the screen
>    - [Not reproduced manually] test_setup_and_send_imap_email-
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_sms_with_attachments -
>    https://bugzil.la/1109213 Camera fails to capture via 'Select' button
>    - [Not reproduced manually]
>    test_rocketbar_add_collection_save_bookmark - https://bugzil.la/1148453
>    Test errors when connecting to Wi-Fi: "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_setup_basic_gmail -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_camera_capture_video -
>    https://bugzil.la/1125798  Investigate camera switch source failure on
>    automation
>    - [Not reproduced manually] test_fmradio_frequency_dialer -
>    https://bugzil.la/1133810 test_fmradio_frequency_dialer failing to
>    scroll/flick
>    - [Not reproduced manually] test_homescreen_launch_app_packaged -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_homescreen_delete_app_packaged -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_marketplace_packaged_app -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>
>
> *Expected failures / disabled tests:*
>
>    - [Reproduces manually] test_ftu_skip_tour.TestFtu -
>    https://bugzil.la/1134257 Default value for 'About Firefox OS, Yes!
>    Send Data.' causing fail in ftu_skip_tour
>    - [Disabled] test_call_log_all_calls.py - https://bugzil.la/1096385 -
>    Investigate failure in test_call_log_all_calls.py, call log is displayed
>    erroneously
>    - [Disabled] test_dialer_set_up_conference_call.py -
>    https://bugzil.la/1117813 Make PlivoUtil use a valid 'answer_url' with
>    a valid Plivo XML
>    - [Disabled] test_setup_and_send_active_sync_email.py -
>    https://bugzil.la/1028192 - Problems with ActiveSync accounts
>    - [Disabled] test_receive_active_sync_email.py -
>    https://bugzil.la/1028192 - Problems with ActiveSync accounts
>    - [Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863
>    Investigate test_fmradio_find_stations.py failure
>    - [Disabled] test_ftu_skip_tour_desktopb2g.py - Desktop Only
>    - [Disabled] test_homescreen_column_layout.py -
>    https://bugzil.la/1131037 Settings OoM and placing test in unexpected
>    location
>    - [Disabled] test_lockscreen_time_check.py -
>    https://bugzilla.mozilla.org/show_bug.cgi?id=1140027
>    - [Disabled] test_settings_bluetooth.py - Bug 1134244 -
>    test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
>    bluetooth_settings.enable_visible_to_all()
>    - [Disabled] test_settings_airplane_mode.py -
>    https://bugzilla.mozilla.org/show_bug.cgi?id=1131070
>
>
> *Non-smoketests:*
>
>
> http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/124/
>
>
> http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/122/
>
> *Test failures - 0 replicable manually:*
>
>    - [Not reproduced manually] test_calendar_flick_through_months - New
>    error: https://bugzil.la/1149224 test_calendar_flick_through_months
>    failed to flick through months: "calendar.flick_to_next_month()"
>    - [Not reproduced manually] test_music_album_mp3 - New error:
>    https://bugzil.la/1149210 test_music_album_mp3 fails to select album:
>    "sublist_view = albums[0].tap_first_album()"
>    - [Not reproduced manually] test_browser_save_image -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not tested manually] test_browser_bookmark -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not tested manually] test_browser_clear_browsing_history -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_camera_delete_captured_video -
>    https://bugzil.la/1125798  Investigate camera switch source failure on
>    automation
>    - [Not tested manually] test_import_contacts_from_gmail -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not tested manually] test_unlink_facebook -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_rocketbar_search -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_persona_app -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_browser_navigation -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_browser_share_link -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_sync_facebook_contacts -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_import_gmail_no_network -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_email_one_header_displayed -
>    https://bugzil.la/1148453 Test errors when connecting to Wi-Fi:
>    "self.connect_to_local_area_network()"
>    - [Not reproduced manually] test_add_to_new_contact_from_messages -
>    Did not repro from Jenkins adhoc: 5/5 passing
>
>
> *Expected failures / disabled tests:*
>
>    - [Skipped] test_settings_change_time_format -
>    https://bugzil.la/1141654 test_settings_change_time_format
>    consistently failing, seems to hang and fail on the homescreen
>    - [Disabled] test_call_log_all_calls.py - Bug 1096385 - Investigate
>    failure in test_call_log_all_calls.py, call log is displayed erroneously
>    - [Disabled] test_dialer_set_up_conference_call.py -
>    https://bugzil.la/1117813 - Make PlivoUtil use a valid 'answer_url'
>    with a valid Plivo XML
>    - [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 -
>    Problems with ActiveSync accounts
>    - [Disabled] test_receive_active_sync_email.py - Bug 1028192 -
>    Problems with ActiveSync accounts
>    - [Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863
>    - Investigate test_fmradio_find_stations.py failure
>    - [Disabled] test_ftu_skip_tour_desktopb2g.py - Test runs only on
>    desktop
>    - [Disabled] test_homescreen_column_layout.py -
>    https://bugzil.la/1131037 - Homescreen settings sub-menu not displayed
>    when going to homecreen and back
>    - [Disabled] test_lockscreen_time_check.py -
>    https://bugzilla.mozilla.org/show_bug.cgi?id=1140027
>    - [Disabled] test_settings_bluetooth.py - Bug 1134244 -
>    test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
>    bluetooth_settings.enable_visible_to_all()
>    - [Disabled] test_settings_airplane_mode.py -
>    https://bugzilla.mozilla.org/show_bug.cgi?id=1131070
>    - [Expected fail] test_call_log_groups.TestCallLogGroups -
>    https://bugzil.la/1117804 - [v2.2] Investigate test_call_log_groups
>    failure
>    - [Expected fail] test_change_language_settings_rtl -
>    https://bugzil.la/1137766 - Error in both
>    test_settings_change_language tests: exception "Element '%s' could not be
>    foundin select wrapper"
>    - [Expected fail] test_change_language_settings -
>    https://bugzil.la/1137766 - Error in both
>    test_settings_change_language tests: exception "Element '%s' could not be
>    foundin select wrapper"
>
>
> *Build under test:*
>
> Device firmware (base)     L1TC100118D0
> Device firmware (date)     30 Mar 2015 01:33:44
> Device firmware (incremental)     eng.cltbld.20150330.043332
> Device firmware (release)     4.4.2
> Device identifier     flame
> Gaia date     29 Mar 2015 03:19:00
> Gaia revision     be25b16efa19
> Gecko build     20150330010204
> Gecko revision     dfe60814eda7
> Gecko version     39.0a1
>
>
> *Summary:*
>
> *Smoketests:*
>
> 45 tests ran in 4893 seconds.
> 30 passed, 134 skipped, 1 failed, 13 errors.
> 1 expected failures, 0 unexpected passes.
>
> *Non-smoketests:*
>
> 67 tests ran in 5805 seconds.
> 57 passed, 59 skipped, 0 failed, 8 errors.
> 2 expected failures, 0 unexpected passes.
>
> 66 tests ran in 6084 seconds.
> 57 passed, 59 skipped, 0 failed, 8 errors.
> 1 expected failures, 0 unexpected passes.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150330/1c535d39/attachment-0001.html>


More information about the Fxosqa-report-watchlist mailing list