Skip to content
This repository was archived by the owner on Feb 22, 2023. It is now read-only.

[integration_test] Recommend tests to be in integration_test/, fix example #2986

Merged
merged 4 commits into from
Sep 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ task:
- ./chromedriver/chromedriver --port=4444 &
test_script:
- cd $INTEGRATION_TEST_PATH/example/
- flutter drive -v --target=test_driver/example_integration.dart -d web-server --release --browser-name=chrome
- flutter drive -v --driver=test_driver/integration_test_driver.dart --target=integration_test/example_test.dart -d web-server --release --browser-name=chrome
- name: build-apks+java-test+firebase-test-lab
env:
matrix:
Expand Down
91 changes: 54 additions & 37 deletions packages/integration_test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,78 +7,95 @@ and native Android instrumentation testing.
## Usage

Add a dependency on the `integration_test` and `flutter_test` package in the
`dev_dependencies` section of pubspec.yaml. For plugins, do this in the
pubspec.yaml of the example app.
`dev_dependencies` section of `pubspec.yaml`. For plugins, do this in the
`pubspec.yaml` of the example app.

Invoke `IntegrationTestWidgetsFlutterBinding.ensureInitialized()` at the start
of a test file, e.g.
Create a `integration_test/` directory for your package. In this directory,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the cirrus file we are still using test_driver/ directory even though it is not the recommended directory?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, I just read the recommended package structure.

Is there a reason for these recommendations? One idea that comes to my mind is: integration_test_driver.dart won't be the same for different tests.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't be the same but I would think for most use cases the bulk of the testing code should go into the test script that runs on the device, and the driver is mainly for reporting.

Currently, some packages place the test scripts in test/foo_e2e.dart or test_driver/foo_e2e.dart. This works today, because of the old e2e naming, but if they were to rename their tests to like test/foo_integration_test.dart and test_driver/foo_integration_test.dart, this would conflict with globbing rules today that check for the *_test.dart suffix, hence the recommendation of a new integration_test/ directory for such tests

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did the renaming like this last week in the for the web engine: flutter/engine#20954

create a `<name>_test.dart`, using the following as a starting point to make
assertions.

```dart
import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';

void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();

testWidgets("failing test example", (WidgetTester tester) async {
expect(2 + 2, equals(5));
});
}
```

## Test locations
### Driver Entrypoint

It is recommended to put integration_test tests in the `test/` folder of the app
or package. For example apps, if the integration_test test references example
app code, it should go in `example/test/`. It is also acceptable to put
integration_test tests in `test_driver/` folder so that they're alongside the
runner app (see below).
An accompanying driver script will be needed that can be shared across all
integration tests. Create a `integration_test_driver.dart` in the `test_driver/`
directory with the following contents:

## Using Flutter Driver to Run Tests
```dart
import 'package:integration_test/integration_test_driver.dart';

`IntegrationTestWidgetsTestBinding` supports launching the on-device tests with
`flutter drive`. Note that the tests don't use the `FlutterDriver` API, they
use `testWidgets` instead.
Future<void> main() => integrationDriver();
```

Put the a file named `<package_name>_integration_test.dart` in the app'
`test_driver` directory:
You can also use different driver scripts to customize the behavior of the app
under test. For example, `FlutterDriver` can also be parameterized with
different [options](https://api.flutter.dev/flutter/flutter_driver/FlutterDriver/connect.html).
See the [extended driver](https://github.com/flutter/plugins/tree/master/packages/integration_test/example/test_driver/integration_test_extended_driver.dart) for an example.

```dart
import 'dart:async';
### Package Structure

import 'package:integration_test/integration_test_driver.dart';
Your package should have a structure that looks like this:

Future<void> main() async => integrationDriver();
```
lib/
...
integration_test/
foo_test.dart
bar_test.dart
test/
# Other unit tests go here.
test_driver/
integration_test_driver.dart
```

To run a example app test with Flutter driver:
[Example](https://github.com/flutter/plugins/tree/master/packages/integration_test/example)

```sh
cd example
flutter drive test/<package_name>_integration.dart
```
## Using Flutter Driver to Run Tests

These tests can be launched with the `flutter drive` command.

To test plugin APIs using Flutter driver:
To run the `integration_test/foo_test.dart` test with the
`test_driver/integration_test_driver.dart` driver, use the following command:

```sh
cd example
flutter drive --driver=test_driver/<package_name>_test.dart test/<package_name>_integration_test.dart
flutter drive \
--driver=test_driver/integration_test_driver.dart \
--target=integration_test/foo_test.dart
```

You can run tests on web in release or profile mode.
### Web

Make sure you have [enabled web support](https://flutter.dev/docs/get-started/web#set-up)
then [download and run](https://flutter.dev/docs/cookbook/testing/integration/introduction#6b-web)
the web driver in another process.

First you need to make sure you have downloaded the driver for the browser.
Use following command to execute the tests:

```sh
cd example
flutter drive -v --target=test_driver/<package_name>dart -d web-server --release --browser-name=chrome
flutter drive \
--driver=test_driver/integration_test_driver.dart \
--target=integration_test/foo_test.dart \
-d web-server
```

## Android device testing
## Android Device Testing

Create an instrumentation test file in your application's
**android/app/src/androidTest/java/com/example/myapp/** directory (replacing
com, example, and myapp with values from your app's package name). You can name
this test file MainActivityTest.java or another name of your choice.
this test file `MainActivityTest.java` or another name of your choice.

```java
package com.example.myapp;
Expand All @@ -96,7 +113,7 @@ public class MainActivityTest {
```

Update your application's **myapp/android/app/build.gradle** to make sure it
uses androidx's version of AndroidJUnitRunner and has androidx libraries as a
uses androidx's version of `AndroidJUnitRunner` and has androidx libraries as a
dependency.

```gradle
Expand Down Expand Up @@ -160,7 +177,7 @@ You can pass additional parameters on the command line, such as the
devices you want to test on. See
[gcloud firebase test android run](https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run).

## iOS device testing
## iOS Device Testing

You need to change `iOS/Podfile` to avoid test target statically linking to the plugins. One way is to
link all of the plugins dynamically:
Expand Down Expand Up @@ -189,4 +206,4 @@ change the code. You can change `RunnerTests.m` to the name of your choice.
INTEGRATION_TEST_IOS_RUNNER(RunnerTests)
```

Now you can start RunnerTests to kick out integration tests!
Now you can start RunnerTests to kick-off integration tests!
22 changes: 14 additions & 8 deletions packages/integration_test/example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,21 @@

Demonstrates how to use the `package:integration_test`.

## Getting Started
To run `integration_test/example_test.dart`,

This project is a starting point for a Flutter application.
Android / iOS:

A few resources to get you started if this is your first Flutter project:
```sh
flutter drive \
--driver=test_driver/integration_test_driver.dart \
--target=integration_test/example_test.dart
```

- [Lab: Write your first Flutter app](https://flutter.dev/docs/get-started/codelab)
- [Cookbook: Useful Flutter samples](https://flutter.dev/docs/cookbook)
Web:

For help getting started with Flutter, view our
[online documentation](https://flutter.dev/docs), which offers tutorials,
samples, guidance on mobile development, and a full API reference.
```sh
flutter drive \
--driver=test_driver/integration_test_driver.dart \
--target=integration_test/example_test.dart \
-d web-server
```
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@

import 'package:integration_test/integration_test.dart';

import 'example_integration_io.dart'
if (dart.library.html) 'example_integration_web.dart' as tests;
import '_example_test_io.dart' if (dart.library.html) '_example_test_web.dart'
as tests;

void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// This is a Flutter widget test can take a screenshot.
//
// NOTE: Screenshots are only supported on Web for now.
// NOTE: Screenshots are only supported on Web for now. For Web, this needs to
// be executed with the `test_driver/integration_test_extended_driver.dart`.
//
// To perform an interaction with a widget in your test, use the WidgetTester
// utility that Flutter provides. For example, you can send tap and scroll
Expand All @@ -9,8 +10,8 @@

import 'package:integration_test/integration_test.dart';

import 'example_integration_io_extended.dart'
if (dart.library.html) 'example_integration_web_extended.dart' as tests;
import '_extended_test_io.dart' if (dart.library.html) '_extended_test_web.dart'
as tests;

void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import 'package:integration_test/integration_test_driver.dart';

Future<void> main() => integrationDriver();
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
import 'dart:async';

import 'package:flutter_driver/flutter_driver.dart';
import 'package:integration_test/integration_test_driver_extended.dart';

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
import 'dart:async';

import 'package:integration_test/common.dart' as common;
import 'package:flutter_driver/flutter_driver.dart';
import 'package:integration_test/common.dart' as common;
import 'package:test/test.dart';

Future<void> main() async {
Expand Down