03/24/2015 Flame KK v18D-1 319 MB Gaia-ui Automation mozilla-central Mozilla RIL Full Flash Build 23/45 smoketests, 74/134 non-smoketests

Oliver Nelson onelson at qanalydocs.com
Tue Mar 24 17:31:13 UTC 2015


*Notes:*
** https://bugzil.la/1145875 - Random tests failing with
JavascriptException: TypeError: this.element is null in app://
system.gaiamobile.org/gaia_build_defer_index.js line: 2337 affecting 99% of
the errors occurring today (anything not labeled with an additional error).
**
**
http://jenkins1.qa.scl3.mozilla.com/user/oliverthor/my-views/view/Smoke%20Team%20Reporting/job/flame-kk.ui.adhoc/746/testReport/
**
** Ran adhoc report through link above and saw huge failure rate again to
same error, nothing reproed manually **
*** NonSmoke test errors were not tested again in lieu of time ***

*Smoketests:*

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.smoke/
/

*Test failures - 0 replicable manually:*

   - [Not reproduced manually] test_browser_cell_data
   - [Not reproduced manually] test_cards_view_with_two_apps
   - [Not reproduced manually] test_clock_create_new_alarm
   - [Not reproduced manually] test_cost_control_data_alert_mobile
   - [Not reproduced manually] test_dialer
   - [Not reproduced manually] test_dialer_receive_call
   - [Not reproduced manually] test_setup_basic_gmail
   - [Not reproduced manually] test_setup_and_send_imap_email
   - [Not reproduced manually] test_IMAP_email_notification
   - [Not reproduced manually] test_rocketbar_add_collection_save_bookmark
   - [Not reproduced manually] test_ftu_status_bar
   - [Not reproduced manually] test_gallery_view
   - [Not reproduced manually] test_gallery_flick
   - [Not reproduced manually] test_homescreen_delete_app
   - [Not reproduced manually] test_lockscreen_unlock_to_homescreen
   - [Not reproduced manually]
   test_lockscreen_unlock_to_camera_with_passcode
   - [Not reproduced manually] test_music_artist_mp3
   - [Not reproduced manually] test_settings_wifi
   - [Not reproduced manually] test_settings_cell_data
   - [Not reproduced manually] test_settings_usb_storage
   - [Not reproduced manually] test_power_button_long_press
   - [Not reproduced manually] test_sms_contact -- Appeared to timeout,
   sent message but did not receive
   - [Not reproduced manually] test_sms_with_attachments -- Appeared to
   timeout while sending message
   - [Not reproduced manually] test_marketplace_packaged_app -- Appeared to
   timeout while searching for apps


*Expected failures / disabled tests:*

   - [Reproduces manually] test_ftu_skip_tour.TestFtu -
   https://bugzil.la/1134257 Default value for 'About Firefox OS, Yes! Send
   Data.' causing fail in ftu_skip_tour
   - [Disabled] test_call_log_all_calls.py - https://bugzil.la/1096385 -
   Investigate failure in test_call_log_all_calls.py, call log is displayed
   erroneously
   - [Disabled] test_dialer_set_up_conference_call.py -
   https://bugzil.la/1117813 Make PlivoUtil use a valid 'answer_url' with a
   valid Plivo XML
   - [Disabled] test_setup_and_send_active_sync_email.py -
   https://bugzil.la/1028192 - Problems with ActiveSync accounts
   - [Disabled] test_receive_active_sync_email.py -
   https://bugzil.la/1028192 - Problems with ActiveSync accounts
   - [Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863
   Investigate test_fmradio_find_stations.py failure
   - [Disabled] test_ftu_skip_tour_desktopb2g.py - Desktop Only
   - [Disabled] test_homescreen_column_layout.py - https://bugzil.la/1131037
   Settings OoM and placing test in unexpected location
   - [Disabled] test_lockscreen_time_check.py -
   https://bugzilla.mozilla.org/show_bug.cgi?id=1140027
   - [Disabled] test_settings_bluetooth.py - Bug 1134244 -
   test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
   bluetooth_settings.enable_visible_to_all()
   - [Disabled] test_settings_airplane_mode.py -
   https://bugzilla.mozilla.org/show_bug.cgi?id=1131070



*Non-smoketests:*

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.1/111/

http://jenkins1.qa.scl3.mozilla.com/job/flame-kk-319.mozilla-central.nightly.ui.functional.non-smoke.2/109/

*Test failures - 0 replicable manually:*

   - [Not tested manually] test_camera_delete_captured_video
   - [Not tested manually] test_delete_all_contacts
   - [Not tested manually] test_export_contacts_to_sdcard
   - [Not tested manually] test_import_contacts_from_gmail
   - [Not tested manually] test_unlink_facebook
   - [Not tested manually] test_dialer_find_contact
   - [Not tested manually] test_gallery_crop_photo
   - [Not tested manually] test_gallery_handle_valid_image_file
   - [Not tested manually] test_homescreen_change_wallpaper
   - [Not tested manually] test_email_keyboard
   - [Not tested manually] test_lockscreen_notification
   - [Not tested manually] test_sms_add_contact
   - [Not tested manually] test_sms_contact_match
   - [Not tested manually] test_sms_notification
   - [Not tested manually] test_inbox_to_settings
   - [Not tested manually] test_sms_forward_sent_message
   - [Not tested manually] test_sms_notification_removed_when_sms_deleted
   - [Not tested manually] test_settings_media_storage
   - [Not tested manually] test_settings_device_info
   - [Not tested manually] test_settings_battery_power_save_mode
   - [Not tested manually] test_settings_change_ring_tone
   - [Not tested manually] test_system_message
   - [Not tested manually] test_inter_app_comm
   - [Not tested manually] test_privileged_app_video_capture_prompt
   - [Not tested manually] test_priveleged_app_device_sdcard_prompt
   - [Not tested manually] test_play_mp4_video
   - [Not tested manually] test_browser_share_link
   - [Not tested manually] test_capture_and_delete_video
   - [Not tested manually] test_delete_contact
   - [Not tested manually] test_import_gmail_no_network
   - [Not tested manually] test_MMI
   - [Not tested manually] test_fmradio_turn_on_off
   - [Not tested manually] test_gallery_edit_photo
   - [Not tested manually] test_gallery_handle_invalid_file
   - [Not tested manually] test_gallery_share_picture
   - [Not tested manually] test_gallery_switch_to_camera_main_view
   - [Not tested manually] test_keyboard_bug_1073870
   - [Not tested manually] test_lockscreen_unlock_to_emergency_call_screen
   - [Not tested manually] test_add_to_new_contact_from_messages
   - [Not tested manually] test_sms_contact_input_validation
   - [Not tested manually] test_sms_semicolon_complets_the_entered_recipient
   - [Not tested manually] test_sms_auto_save_draft
   - [Not tested manually] test_findmydevice
   - [Not tested manually] test_settings_do_not_track
   - [Not tested manually] test_settings_wallpaper
   - [Not tested manually] test_settings_change_keyboard_language
   - [Not tested manually] test_settings_power_save_mode
   - [Not tested manually] test_quick_settings
   - [Not tested manually] test_system_message_pending
   - [Not tested manually] test_privileged_app_audio_capture_prompt
   - [Not tested manually] test_privileged_app_device_picture_prompt
   - [Not tested manually] test_play_ogg_video
   - [Not tested manually] test_dialer_airplane_mode
   - [Not tested manually] test_clear_phone_number
   - [Not tested manually] test_dialer_clear_miss_call_notification
   - [Not tested manually] test_redial_last_number
   - [Not tested manually] test_gallery_delete_image
   - [Not tested manually] test_gallery_open_valid_image_file_load_bmp
   - [Not tested manually] test_gallery_open_valid_image_file_load_gif
   - [Not tested manually]
   test_gallery_open_valid_image_file_load_transparent_background_png
   - [Not tested manually] test_gallery_switch_to_camera
   - [Not tested manually] test_homescreen_change_wallpaper_from_gallery
   - [Not tested manually] test_unlock_to_homescreen_with_passcode
   - [Not tested manually] test_sms_add_contact
   - [Not tested manually] test_sms_forward_sent_message
   - [Not tested manually] test_sms_notification_removed_when_sms_deleted
   - [Not tested manually] test_empty_music
   - [Not tested manually] test_enable_gps_via_settings_app
   - [Not tested manually] test_set_passcode_by_settings
   - [Not tested manually] test_settings_battery_power_save_mode
   - [Not tested manually] test_geolocation_prompt
   - [Not tested manually] test_get_sdcard_prompt
   - [Not tested manually] test_play_mp4_video
   - [Not tested manually] test_kill_app_from_cards_view
   - [Not tested manually] test_call_contact
   - [Not tested manually] test_import_contacts_from_facebook
   - [Not tested manually] test_cost_control_reset_wifi
   - [Not tested manually] test_dialer_add_contact
   - [Not tested manually] test_delete_call_log
   - [Not tested manually] test_rocketbar_offline_behavior
   - [Not tested manually] test_turn_radio_on_off
   - [Not tested manually] test_gallery_handle_load_corrupt_file_corrupt_jpg
   - [Not tested manually] test_gallery_handle_load_corrupt_file_text_file
   - [Not tested manually]
   test_gallery_handle_load_corrupt_file_truncated_bmp
   - [Not tested manually]
   test_gallery_handle_load_corrupt_file_truncated_jpg
   - [Not tested manually]
   test_gallery_handle_load_corrupt_file_truncated_png
   - [Not tested manually]
   test_gallery_handle_load_corrupt_file_zero_length_file
   - [Not tested manually] test_keyboard_predictive_key
   - [Not tested manually] test_lock_screen_wake_with_notification
   - [Not tested manually] test_sms_create_new_contact
   - [Not tested manually] test_sms_auto_save_draft
   - [Not tested manually] test_enable_do_not_track_via_settings_app
   - [Not tested manually] test_change_keyboard_language_settings
   - [Not tested manually] test_power_save_mode
   - [Not tested manually] test_settings_root_items
   - [Not tested manually] test_pending_system_message
   - [Not tested manually] test_audio_capture_prompt
   - [Not tested manually] test_get_pictures_prompt
   - [Not tested manually] test_empty_video



*Expected failures / disabled tests:*

   - [Skipped] test_settings_change_time_format - https://bugzil.la/1141654
   test_settings_change_time_format consistently failing, seems to hang and
   fail on the homescreen
   - [Disabled] test_call_log_all_calls.py - Bug 1096385 - Investigate
   failure in test_call_log_all_calls.py, call log is displayed erroneously
   - [Disabled] test_dialer_set_up_conference_call.py -
   https://bugzil.la/1117813 - Make PlivoUtil use a valid 'answer_url' with
   a valid Plivo XML
   - [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 -
   Problems with ActiveSync accounts
   - [Disabled] test_receive_active_sync_email.py - Bug 1028192 - Problems
   with ActiveSync accounts
   - [Disabled] test_fmradio_find_stations.py - https://bugzil.la/1000863 -
   Investigate test_fmradio_find_stations.py failure
   - [Disabled] test_ftu_skip_tour_desktopb2g.py - Test runs only on desktop
   - [Disabled] test_homescreen_column_layout.py - https://bugzil.la/1131037
   - Homescreen settings sub-menu not displayed when going to homecreen and
   back
   - [Disabled] test_lockscreen_time_check.py -
   https://bugzilla.mozilla.org/show_bug.cgi?id=1140027
   - [Disabled] test_settings_bluetooth.py - Bug 1134244 -
   test_settings_bluetooth.py:test_toggle_bluetooth_settings is failing on
   bluetooth_settings.enable_visible_to_all()
   - [Disabled] test_settings_airplane_mode.py -
   https://bugzilla.mozilla.org/show_bug.cgi?id=1131070
   - [Expected fail] test_call_log_groups.TestCallLogGroups -
   https://bugzil.la/1117804 - [v2.2] Investigate test_call_log_groups
   failure
   - [Expected fail] test_change_language_settings_rtl -
   https://bugzil.la/1137766 - Error in both test_settings_change_language
   tests: exception "Element '%s' could not be foundin select wrapper"
   - [Expected fail] test_change_language_settings -
   https://bugzil.la/1137766 - Error in both test_settings_change_language
   tests: exception "Element '%s' could not be foundin select wrapper"


*Build under test:*

Device firmware (base)     L1TC100118D0
Device firmware (date)     24 Mar 2015 01:37:38
Device firmware (incremental)     eng.cltbld.20150324.043727
Device firmware (release)     4.4.2
Device identifier     flame
Gaia date     23 Mar 2015 13:15:53
Gaia revision     efebbafd12fc
Gecko build     20150324010202
Gecko revision     840cfd5bc971
Gecko version     39.0a1


*Summary:*

*Smoketests:*

45 tests ran in 3776 seconds.
23 passed, 134 skipped, 0 failed, 21 errors.
1 expected failures, 0 unexpected passes.


*Non-smoketests:*

67 tests ran in 5275 seconds.
37 passed, 59 skipped, 0 failed, 28 errors.
2 expected failures, 0 unexpected passes.

66 tests ran in 5161 seconds.
37 passed, 59 skipped, 0 failed, 28 errors.
1 expected failures, 0 unexpected passes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/fxosqa-report-watchlist/attachments/20150324/4efaa5a8/attachment-0001.html>


More information about the Fxosqa-report-watchlist mailing list