06/05/2015 Flame KK v18D-1 319 MB Gaia-ui Automation mozilla-central Mozilla RIL Full Flash Build 37/43 smoketests, 110/129 non-smoketests

Peter Bylenga pbylenga at qanalydocs.com
Fri Jun 5 20:09:01 UTC 2015


Smoketests:

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/278/

Ad hoc testing of failed cases was derailed by
https://bugzilla.mozilla.org/show_bug.cgi?id=1147858 (
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/844)

Test Failures -

[Not manually reproduced] test_clock_create_new_alarm.py - Reproduces 2/3
on adhoc, received error "TimeoutException: TimeoutException: Timed out
after 5.3 seconds with message: b2g failed to stop."
[Not manually reproduced] test_setup_and_send_imap_email.py - Unable to
reproduce with further ad-hoc testing 0/4
[Not manually reproduced] test_homescreen_move_app.py - Unable to reproduce
with further ad-hoc testing 0/4
[Not manually reproduced] test_settings_usb_storage.py - blocked from
testing by https://bugzilla.mozilla.org/show_bug.cgi?id=1171768 - Issue
affected from automation is
https://bugzilla.mozilla.org/show_bug.cgi?id=1147858 - Unable to reproduce
with further ad-hoc testing 0/4
[Not manually reproduced] test_power_button_long_press.py - Unable to
reproduce with further ad-hoc testing 0/4
[Not manually reproduced] test_play_3gp_video.py - Unable to reproduce with
further ad-hoc testing 0/4


Expected failures / disabled tests:

[Expected fail] test_ftu_skip_tour.TestFtu - https://bugzil.la/1134257
Default value for 'About Firefox OS, Yes! Send Data.' causing fail in
ftu_skip_tour
[Disabled] test_call_log_all_calls.py - https://bugzil.la/1096385 -
Investigate failure in test_call_log_all_calls.py, call log is displayed
erroneously
[Disabled] test_dialer_set_up_conference_call.py - https://bugzil.la/1117813
Make PlivoUtil use a valid 'answer_url' with a valid Plivo XML
[Disabled] test_setup_and_send_active_sync_email.py -
https://bugzil.la/1028192 - Problems with ActiveSync accounts
[Disabled] test_receive_active_sync_email.py - https://bugzil.la/1028192 -
Problems with ActiveSync accountsc
[Disabled] test_rocketbar_add_collection_save_bookmark -
https://bugzil.la/1162112 - TimeoutException: TimeoutException: Timed out
after 30.1 seconds
[Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863
Investigate test_fmradio_find_stations.py failure
[Disabled] test_ftu_skip_tour_desktopb2g.py - Desktop Only
[Disabled] test_lockscreen_time_check.py - https://bugzil.la/1141258
disabled until bug fixed
[Disabled] test_settings_bluetooth.py - Bug 1134244 -
test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
bluetooth_settings.enable_visible_to_all()
[Disabled] test_settings_airplane_mode.py - https://bugzil.la/1131070
[Disabled] test_settings_change_time_format.py - https://bugzil.la/1144686

Non-smoketests:

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/273/
http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/273/

Ad hoc testing of failed cases was derailed by
https://bugzilla.mozilla.org/show_bug.cgi?id=1147858 (
http://jenkins1.qa.scl3.mozilla.com/view/UI/job/flame-kk.ui.adhoc/844)

Test Failures -

[Not manually reproduced] test_homescreen_column_layout.py - Unable to
reproduce with further ad-hoc testing 0/4
[Can reproduce manually] test_findmydevice.py -
https://bugzilla.mozilla.org/show_bug.cgi?id=1172027 - Reproducing on ad
hoc 2/4
[Not manually reproduced] test_settings_power_save_mode.py - Unable to
reproduce with further ad-hoc testing 0/4
[Not manually reproduced] test_settings_root.py - Unable to reproduce with
further ad-hoc testing 0/4
[Not manually reproduced] test_system_notification_bar.py - Unable to
reproduce with further ad-hoc testing 0/4
[Not manually reproduced] test_quick_settings.py - Unable to reproduce with
further ad-hoc testing 0/4
[] test_system_message_pending.py - Unable to reproduce with further ad-hoc
testing 0/4
[] test_privileged_app_audio_capture_prompt.py - Unable to reproduce with
further ad-hoc testing 0/4
[] test_privileged_app_contacts_prompt.py - Unable to reproduce with
further ad-hoc testing 0/4
[] test_privileged_app_device_picture_prompt.py - Unable to reproduce with
further ad-hoc testing 0/4
[] test_privileged_app_device_video_prompt.py - Unable to reproduce with
further ad-hoc testing 0/4
[Not manually reproduced] test_video_empty.py - Unable to reproduce with
further ad-hoc testing 0/4
[Not manually reproduced] test_play_webm_video.py - Unable to reproduce
with further ad-hoc testing 0/4
[Not manually reproduced] test_play_ogg_video.py - Unable to reproduce with
further ad-hoc testing 0/4
[Not manually reproduced] test_cost_control_reset_wifi.py  - Unable to
reproduce with further ad-hoc testing 0/4
[Not manually reproduced] test_MMI.py - Unable to reproduce with further
ad-hoc testing 0/4
[Not manually reproduced] test_rocketbar_offline_behavior.py - Reproducing
on ad hoc 2/4 received error "TimeoutException: TimeoutException: Timed out
after 20.3 seconds with message: b2g failed to stop."
[Not manually reproduced] test_sms_notification.py - Reproducing 2/4 on ad
hoc received error "AssertionError: False is not true"
[Not manually reproduced] test_sms_notification_removed_when_sms_deleted.py
 - Unable to reproduce with further ad-hoc testing 0/4


Expected failures / disabled tests:

[Expected fail] test_call_log_groups.TestCallLogGroups -
https://bugzil.la/1117804 - [v2.2] Investigate test_call_log_groups failure
[Expected fail] test_change_language_settings_rtl -
https://bugzil.la/1137766 - Error in both test_settings_change_language
tests: exception "Element '%s' could not be foundin select wrapper"
[Expected fail] test_import_gmail_no_network.py - https://bugzil.la/1155763
-xfail
[Expected fail] test_change_language_settings - https://bugzil.la/1137766 -
Error in both test_settings_change_language tests: exception "Element '%s'
could not be foundin select wrapper"
[Disabled] test_call_log_all_calls.py - https://bugzil.la/1096385 -
Investigate failure in test_call_log_all_calls.py, call log is displayed
erroneously
[Disabled] test_dialer_set_up_conference_call.py - https://bugzil.la/1117813
- Make PlivoUtil use a valid 'answer_url' with a valid Plivo XML
[Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 -
Problems with ActiveSync accounts
[Disabled] test_receive_active_sync_email.py - https://bugzil.la/1028192 -
Problems with ActiveSync accounts
[Disabled] test_rocketbar_add_collection_save_bookmark -
https://bugzil.la/1162112 - TimeoutException: TimeoutException: Timed out
after 30.1 seconds
[Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863 -
Investigate test_fmradio_find_stations.py failure
[Disabled] test_ftu_skip_tour_desktopb2g.py - Test runs only on desktop
[Disabled] test_lockscreen_time_check.py - https://bugzil.la/1141258
disabled until bug fixed
[Disabled] test_settings_bluetooth.py - https://bugzil.la/1134244 -
test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
bluetooth_settings.enable_visible_to_all()
[Disabled] test_settings_airplane_mode.py - https://bugzil.la/1131070
[Disabled] test_settings_change_time_format.py - https://bugzil.la/1144686


Device firmware (base) L1TC100118D0
Device firmware (date) 05 Jun 2015 08:37:39
Device firmware (incremental) eng.cltbld.20150605.043730
Device firmware (release) 4.4.2
Device identifier flame
Device memory 219772 kB
Device serial e47cd84d
Device uptime 0 days 0 hours 2 minutes 12 seconds
Gaia date 04 Jun 2015 20:28:16
Gaia revision 65369b217faa
Gecko build 20150605010203
Gecko revision 0496b5b3e9ef
Gecko version 41.0a1


Summary:

Smoketests:

179 tests ran in 4525.1 seconds.
37 passed, 135 skipped, 0 failed, 6 errors.
1 expected failures, 0 unexpected passes.

Non-smoketests:

126 tests ran in 7153.2 seconds.
52 passed, 58 skipped, 0 failed, 14 errors.
2 expected failures, 0 unexpected passes.

123 tests ran in 6650.6 seconds.
58 passed, 58 skipped, 1 failed, 4 errors.
2 expected failures, 0 unexpected passes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150605/2034183d/attachment.html>


More information about the Fxosqa-report-watchlist mailing list