Skip to content

[Android] Introduces API 34 emulators for subset of tests #4820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 59 commits into from
Closed

[Android] Introduces API 34 emulators for subset of tests #4820

Show file tree
Hide file tree
Changes from 37 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
dec3d69
Merge remote-tracking branch 'upstream/main' into camx_occ
camsim99 May 1, 2023
0e0333b
Merge remote-tracking branch 'upstream/main'
camsim99 May 2, 2023
bd7ac99
Merge remote-tracking branch 'upstream/main'
camsim99 May 3, 2023
5c3363b
Merge remote-tracking branch 'upstream/main'
camsim99 May 10, 2023
fed9621
Undo changes
camsim99 May 10, 2023
5aabe34
Merge remote-tracking branch 'upstream/main'
camsim99 May 12, 2023
2b9a352
Merge remote-tracking branch 'upstream/main'
camsim99 May 25, 2023
a1173da
Merge remote-tracking branch 'upstream/main'
camsim99 May 30, 2023
cbc3d6b
Merge remote-tracking branch 'upstream/main'
camsim99 May 30, 2023
cae5a4c
Merge remote-tracking branch 'upstream/main'
camsim99 Jun 1, 2023
72283db
Merge remote-tracking branch 'upstream/main'
camsim99 Jun 5, 2023
166a77c
Merge remote-tracking branch 'upstream/main'
camsim99 Jun 5, 2023
399780e
Merge remote-tracking branch 'upstream/main'
camsim99 Jun 14, 2023
8d5d0e7
Merge remote-tracking branch 'upstream/main'
camsim99 Jun 26, 2023
084d960
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 12, 2023
d2a59ac
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 17, 2023
a1422bf
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 17, 2023
bdd87a6
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 18, 2023
137a28b
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 19, 2023
bc0db5a
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 20, 2023
d04b466
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 21, 2023
a9cfe87
Merge remote-tracking branch 'upstream/main'
camsim99 Jul 24, 2023
a32def1
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 2, 2023
4785148
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 14, 2023
7a8fc69
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 15, 2023
b02e15f
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 15, 2023
c6e5868
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 17, 2023
0c0065a
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 28, 2023
9dfe259
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 29, 2023
bfcc0df
Merge remote-tracking branch 'upstream/main'
camsim99 Aug 30, 2023
7ee21ea
Bump tests to 34
camsim99 Aug 30, 2023
cc2957d
Bump linux android
camsim99 Aug 31, 2023
0026d0e
Try filtering packages
camsim99 Sep 13, 2023
0bc3706
Merge remote-tracking branch 'upstream/main' into test_34_1
camsim99 Sep 13, 2023
791dd78
Add filter script
camsim99 Sep 13, 2023
c687491
Add quick actions test fix
camsim99 Sep 13, 2023
d91007d
Add 6 back in
camsim99 Sep 13, 2023
4c628c7
Remove quick actions fix attempt, partially address review
camsim99 Sep 14, 2023
83afe66
Correct exclusions/inclusions
camsim99 Sep 14, 2023
89a968a
Remove exlude filter
camsim99 Sep 14, 2023
4430919
Path provider fails with 33, so put back to 34
camsim99 Sep 14, 2023
f39b0b0
Exclude webview
camsim99 Sep 18, 2023
cd977fe
Merge remote-tracking branch 'upstream/main' into test_34_1
camsim99 Sep 18, 2023
70c7aeb
Merge branch 'main' into test_34_1
camsim99 Sep 18, 2023
57117d8
Update still_requires_api_33_avd_drive_examples.yaml
camsim99 Sep 19, 2023
844c8cb
Add new tasks, reduce config files to one
camsim99 Sep 20, 2023
3861779
Merge branch 'test_34_1' of github.com:camsim99/packages into test_34_1
camsim99 Sep 20, 2023
a42b3e6
Fix comments
camsim99 Sep 20, 2023
ea10dde
Mark bringup true
camsim99 Sep 20, 2023
f4b839e
Merge remote-tracking branch 'upstream/main' into test_34_1
camsim99 Sep 26, 2023
685f810
Get rid of duplicate tasks
camsim99 Sep 26, 2023
5272d12
Update exclusions, delte unused files
camsim99 Sep 27, 2023
a764265
Fix integration test, try removing line that may be failing due to pe…
camsim99 Sep 28, 2023
c8883bc
Try using permissions, alternatively
camsim99 Sep 28, 2023
3cfd2b3
Remove experimental changes
camsim99 Oct 9, 2023
106db42
Merge remote-tracking branch 'upstream/main' into test_34_1
camsim99 Oct 9, 2023
7dab380
Undo my changes
camsim99 Oct 9, 2023
378c808
Add audio permission for music
camsim99 Oct 9, 2023
e3229a8
Add permissions and bump version
camsim99 Oct 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions .ci.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ platform_properties:
device_type: none
dependencies: >-
[
{"dependency": "android_sdk", "version": "version:33v6"},
{"dependency": "android_sdk", "version": "version:34v1"},
{"dependency": "open_jdk", "version": "version:17"},
{"dependency": "curl", "version": "version:7.64.0"}
]
Expand Down Expand Up @@ -250,7 +250,7 @@ targets:
# web_benchmarks needs Chrome.
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"},
{"dependency": "android_virtual_device", "version": "34"},
{"dependency": "clang", "version": "git_revision:5d5aba78dbbee75508f01bcaa69aedb2ab79065a"},
{"dependency": "chrome_and_driver", "version": "version:114.0"}
]
Expand All @@ -267,7 +267,7 @@ targets:
# See comments on 'master' version above.
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"},
{"dependency": "android_virtual_device", "version": "34"},
{"dependency": "clang", "version": "git_revision:5d5aba78dbbee75508f01bcaa69aedb2ab79065a"},
{"dependency": "chrome_and_driver", "version": "version:114.0"}
]
Expand Down Expand Up @@ -306,7 +306,7 @@ targets:
recipe: packages/packages
timeout: 60
properties:
target_file: android_platform_tests.yaml
target_file: android_platform_tests_api_33.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 0 --shardCount 6"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to changed the shard flags (the names of the tasks will be slightly confusing during the transition, but changing names is a huge pain with LUCI), so that each of the two partitions has their own shards, otherwise we may end up with a plugin assigned to a specific shard, but then excluded in that shard (since sharding is pre-exclusion, for slightly complicated reasons).

So I believe the setup we need is something like:

- name: ..._shard_1
  // set up for 34
  package_sharding: "--shardIndex 0 --shardCount 4"

- name: ..._shard_2
  // set up for 34
  package_sharding: "--shardIndex 1 --shardCount 4"

- name: ..._shard_3
  // set up for 34
  package_sharding: "--shardIndex 2 --shardCount 4"

- name: ..._shard_4
  // set up for 34
  package_sharding: "--shardIndex 3 --shardCount 4"

- name: ..._shard_5
  // set up for 33
  package_sharding: "--shardIndex 0 --shardCount 1"

- name: ..._shard_6
  // set up for 33
  package_sharding: "--shardIndex 1 --shardCount 2"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other option we could consider if we think we'll be in this state for a while is to bring up a new task or two for 33 tests, and leave the existing 6 shards as-is but with 34, so the names make more sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm going to be actively working on getting the tests failing on 34 working, but two of the failures have been really tricky, so could be a matter of months worst case. I'll start changing the shard flags but let me know if you think it makes more sense to add new tasks instead given that projection.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think on a potential timeline of months it's probably better to bring up 1-2 new shards for 33.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok hoping to be faster but I think that's probably safest.

Expand All @@ -325,7 +325,7 @@ targets:
package_sharding: "--shardIndex 1 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_3 master
Expand All @@ -338,14 +338,14 @@ targets:
package_sharding: "--shardIndex 2 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_4 master
recipe: packages/packages
timeout: 60
properties:
target_file: android_platform_tests.yaml
target_file: android_platform_tests_api_33.yaml
channel: master
version_file: flutter_master.version
package_sharding: "--shardIndex 3 --shardCount 6"
Expand All @@ -364,7 +364,7 @@ targets:
package_sharding: "--shardIndex 4 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_6 master
Expand All @@ -377,7 +377,7 @@ targets:
package_sharding: "--shardIndex 5 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_1 stable
Expand All @@ -391,7 +391,7 @@ targets:
package_sharding: "--shardIndex 0 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_2 stable
Expand All @@ -405,7 +405,7 @@ targets:
package_sharding: "--shardIndex 1 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_3 stable
Expand All @@ -419,7 +419,7 @@ targets:
package_sharding: "--shardIndex 2 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_4 stable
Expand All @@ -433,7 +433,7 @@ targets:
package_sharding: "--shardIndex 3 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_5 stable
Expand All @@ -447,7 +447,7 @@ targets:
package_sharding: "--shardIndex 4 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

- name: Linux_android android_platform_tests_shard_6 stable
Expand All @@ -461,7 +461,7 @@ targets:
package_sharding: "--shardIndex 5 --shardCount 6"
dependencies: >-
[
{"dependency": "android_virtual_device", "version": "33"}
{"dependency": "android_virtual_device", "version": "34"}
]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to add matching stable shards, or we'll lose coverage on stable since they are running the same scripts. That can be done in the same prequel PR.

# Device versions of Android integration tests, run via FTL.
Expand Down
2 changes: 1 addition & 1 deletion .ci/targets/android_platform_tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ tasks:
args: ["native-test", "--android", "--no-unit"]
- name: drive examples
script: script/tool_runner.sh
args: ["drive-examples", "--android", "--exclude=script/configs/exclude_integration_android.yaml,script/configs/exclude_integration_android_emulator.yaml"]
args: ["drive-examples", "--android", "--exclude=script/configs/exclude_integration_android.yaml,script/configs/exclude_integration_android_emulator.yaml,script/configs/still_requires_api_33_avd.yaml"]
29 changes: 29 additions & 0 deletions .ci/targets/android_platform_tests_api_33.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Same as android_platform_tests.yaml with only packages currently requiring
# Android 33 due to test failures caused by running on Android 34 AVDs.
tasks:
- name: prepare tool
script: .ci/scripts/prepare_tool.sh
infra_step: true # Note infra steps failing prevents "always" from running.
- name: download Dart and Android deps
script: script/tool_runner.sh
infra_step: true
args: ["fetch-deps", "--android", "--supporting-target-platforms-only"]
- name: build examples
script: script/tool_runner.sh
args: ["build-examples", "--apk"]
- name: lint
script: script/tool_runner.sh
args: ["lint-android"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these tasks aren't being passed either of the include/exclude flags, they will all run twice: once on the 34 shards and once on the 33 shards, even though there's no difference.

We should either pass the flags to everything, to keep all the tasks for a given plugin together, and running only in one place, or we should remove these two tasks from the 33 version, so all builds and lints run on the 34 shards.

I would vote for the former, for consistency, but either should work.

# Native unit and native integration are split into two steps to allow for
# different exclusions.
# TODO(stuartmorgan): Eliminate the native unit test exclusion, and combine
# these steps.
- name: native unit tests
script: script/tool_runner.sh
args: ["native-test", "--android", "--no-integration", "--exclude=script/configs/exclude_native_unit_android.yaml"]
- name: native integration tests
script: script/tool_runner.sh
args: ["native-test", "--android", "--no-unit"]
- name: drive examples
script: script/tool_runner.sh
args: ["drive-examples", "--android", "--exclude=script/configs/exclude_integration_android.yaml,script/configs/exclude_integration_android_emulator.yaml", "--filter-packages-to=script/configs/still_requires_api_33_avd.yaml"]
2 changes: 1 addition & 1 deletion ...oid/app/src/androidTest/java/io/flutter/plugins/quickactionsexample/QuickActionsTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ public void appShortcutLaunchActivityAfterStarting() {
.filter(s -> s.getId().equals(firstShortcut.getId()))
.findFirst()
.get();
Intent dynamicShortcutIntent = dynamicShortcut.getIntent();
Intent dynamicShortcutIntent = dynamicShortcut.getIntent().addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is too long and needs to be formatted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this fix attempt and excluded quick_actions for now since the test is still flaky.

AtomicReference<QuickActionsTestActivity> initialActivity = new AtomicReference<>();
scenario.onActivity(initialActivity::set);
String appReadySentinel = " has launched";
Expand Down
3 changes: 3 additions & 0 deletions script/configs/still_requires_api_33_avd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Running the tests of these packages on an AVD with Android 34 causes failures.
- file_selector
- path_provider