Skip to content

Commit 97bad7e

Browse files
camsim99camsim99
authored
[camerax] Add fix for camera preview rotation on landscape-oriented devices and set up fix for Impeller support (flutter#7044)
Partially lands flutter/packages#6856. This PR specifically includes: #### 1: A fix for correctly rotating the camera preview This fix is required for `Surface`s not backed by a `SurfaceTexture` because they don't contain the transformation information needed to correctly rotate the camera preview. In that case, we use the logic described in https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation. The fix is **not currently used** (the logic is not reachable) as Impeller support has not been added back to the plugin, but has been tested in flutter/packages#6856 and will be turned on when flutter#149294 is fixed. Part of flutter#149294. #### 2: A fix for correctly rotating the camera preview on naturally landscape-oriented devices I believe this issue was caused because we assume that the natural orientation of the device is portrait up. We fix this here by adding an extra rotation for the camera preview based on the natural orientation of the device. Fixes flutter#149177.
1 parent 754de19 commit 97bad7e

25 files changed

+817
-34
lines changed

‎packages/camera/camera_android_camerax/CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
## 0.6.6
2+
3+
* Adds logic to support building a camera preview with Android `Surface`s not backed by a `SurfaceTexture`
4+
to which CameraX cannot not automatically apply the transformation required to achieve the correct rotation.
5+
* Adds fix for incorrect camera preview rotation on naturally landscape-oriented devices.
6+
* Updates example app's minimum supported SDK version to Flutter 3.22/Dart 3.4.
7+
18
## 0.6.5+6
29

310
* Updates Guava version to 33.2.1.

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/Camera2CameraInfoHostApiImpl.java

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,26 +29,30 @@ public class Camera2CameraInfoHostApiImpl implements Camera2CameraInfoHostApi {
2929

3030
/** Proxy for methods of {@link Camera2CameraInfo}. */
3131
@VisibleForTesting
32+
@OptIn(markerClass = ExperimentalCamera2Interop.class)
3233
public static class Camera2CameraInfoProxy {
3334

3435
@NonNull
35-
@OptIn(markerClass = ExperimentalCamera2Interop.class)
3636
public Camera2CameraInfo createFrom(@NonNull CameraInfo cameraInfo) {
3737
return Camera2CameraInfo.from(cameraInfo);
3838
}
3939

4040
@NonNull
41-
@OptIn(markerClass = ExperimentalCamera2Interop.class)
4241
public Integer getSupportedHardwareLevel(@NonNull Camera2CameraInfo camera2CameraInfo) {
4342
return camera2CameraInfo.getCameraCharacteristic(
4443
CameraCharacteristics.INFO_SUPPORTED_HARDWARE_LEVEL);
4544
}
4645

4746
@NonNull
48-
@OptIn(markerClass = ExperimentalCamera2Interop.class)
4947
public String getCameraId(@NonNull Camera2CameraInfo camera2CameraInfo) {
5048
return camera2CameraInfo.getCameraId();
5149
}
50+
51+
@NonNull
52+
public Long getSensorOrientation(@NonNull Camera2CameraInfo camera2CameraInfo) {
53+
return Long.valueOf(
54+
camera2CameraInfo.getCameraCharacteristic(CameraCharacteristics.SENSOR_ORIENTATION));
55+
}
5256
}
5357

5458
/**
@@ -105,6 +109,12 @@ public String getCameraId(@NonNull Long identifier) {
105109
return proxy.getCameraId(getCamera2CameraInfoInstance(identifier));
106110
}
107111

112+
@Override
113+
@NonNull
114+
public Long getSensorOrientation(@NonNull Long identifier) {
115+
return proxy.getSensorOrientation(getCamera2CameraInfoInstance(identifier));
116+
}
117+
108118
private Camera2CameraInfo getCamera2CameraInfoInstance(@NonNull Long identifier) {
109119
return Objects.requireNonNull(instanceManager.getInstance(identifier));
110120
}

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/DeviceOrientationManager.java

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,7 @@ interface DeviceOrientationChangeCallback {
5252
* Starts listening to the device's sensors or UI for orientation updates.
5353
*
5454
* <p>When orientation information is updated, the callback method of the {@link
55-
* DeviceOrientationChangeCallback} is called with the new orientation. This latest value can also
56-
* be retrieved through the {@link #getVideoOrientation()} accessor.
55+
* DeviceOrientationChangeCallback} is called with the new orientation.
5756
*
5857
* <p>If the device's ACCELEROMETER_ROTATION setting is enabled the {@link
5958
* DeviceOrientationManager} will report orientation updates based on the sensor information. If
@@ -124,7 +123,7 @@ static void handleOrientationChange(
124123
*/
125124
// Configuration.ORIENTATION_SQUARE is deprecated.
126125
@SuppressWarnings("deprecation")
127-
@VisibleForTesting
126+
@NonNull
128127
PlatformChannel.DeviceOrientation getUIOrientation() {
129128
final int rotation = getDefaultRotation();
130129
final int orientation = activity.getResources().getConfiguration().orientation;

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/DeviceOrientationManagerHostApiImpl.java

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -95,7 +95,8 @@ public void stopListeningForDeviceOrientationChange() {
9595
* for instance for more information on how this default value is used.
9696
*/
9797
@Override
98-
public @NonNull Long getDefaultDisplayRotation() {
98+
@NonNull
99+
public Long getDefaultDisplayRotation() {
99100
int defaultRotation;
100101
try {
101102
defaultRotation = deviceOrientationManager.getDefaultRotation();
@@ -106,4 +107,11 @@ public void stopListeningForDeviceOrientationChange() {
106107

107108
return Long.valueOf(defaultRotation);
108109
}
110+
111+
/** Gets current UI orientation based on the current device orientation and rotation. */
112+
@Override
113+
@NonNull
114+
public String getUiOrientation() {
115+
return serializeDeviceOrientation(deviceOrientationManager.getUIOrientation());
116+
}
109117
}

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/GeneratedCameraXLibrary.java

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1441,6 +1441,9 @@ void requestCameraPermissions(
14411441
@NonNull
14421442
String getTempFilePath(@NonNull String prefix, @NonNull String suffix);
14431443

1444+
@NonNull
1445+
Boolean isPreviewPreTransformed();
1446+
14441447
/** The codec used by SystemServicesHostApi. */
14451448
static @NonNull MessageCodec<Object> getCodec() {
14461449
return SystemServicesHostApiCodec.INSTANCE;
@@ -1508,6 +1511,29 @@ public void error(Throwable error) {
15081511
channel.setMessageHandler(null);
15091512
}
15101513
}
1514+
{
1515+
BasicMessageChannel<Object> channel =
1516+
new BasicMessageChannel<>(
1517+
binaryMessenger,
1518+
"dev.flutter.pigeon.SystemServicesHostApi.isPreviewPreTransformed",
1519+
getCodec());
1520+
if (api != null) {
1521+
channel.setMessageHandler(
1522+
(message, reply) -> {
1523+
ArrayList<Object> wrapped = new ArrayList<Object>();
1524+
try {
1525+
Boolean output = api.isPreviewPreTransformed();
1526+
wrapped.add(0, output);
1527+
} catch (Throwable exception) {
1528+
ArrayList<Object> wrappedError = wrapError(exception);
1529+
wrapped = wrappedError;
1530+
}
1531+
reply.reply(wrapped);
1532+
});
1533+
} else {
1534+
channel.setMessageHandler(null);
1535+
}
1536+
}
15111537
}
15121538
}
15131539
/** Generated class from Pigeon that represents Flutter messages that can be called from Java. */
@@ -1550,6 +1576,9 @@ void startListeningForDeviceOrientationChange(
15501576
@NonNull
15511577
Long getDefaultDisplayRotation();
15521578

1579+
@NonNull
1580+
String getUiOrientation();
1581+
15531582
/** The codec used by DeviceOrientationManagerHostApi. */
15541583
static @NonNull MessageCodec<Object> getCodec() {
15551584
return new StandardMessageCodec();
@@ -1634,6 +1663,29 @@ static void setup(
16341663
channel.setMessageHandler(null);
16351664
}
16361665
}
1666+
{
1667+
BasicMessageChannel<Object> channel =
1668+
new BasicMessageChannel<>(
1669+
binaryMessenger,
1670+
"dev.flutter.pigeon.DeviceOrientationManagerHostApi.getUiOrientation",
1671+
getCodec());
1672+
if (api != null) {
1673+
channel.setMessageHandler(
1674+
(message, reply) -> {
1675+
ArrayList<Object> wrapped = new ArrayList<Object>();
1676+
try {
1677+
String output = api.getUiOrientation();
1678+
wrapped.add(0, output);
1679+
} catch (Throwable exception) {
1680+
ArrayList<Object> wrappedError = wrapError(exception);
1681+
wrapped = wrappedError;
1682+
}
1683+
reply.reply(wrapped);
1684+
});
1685+
} else {
1686+
channel.setMessageHandler(null);
1687+
}
1688+
}
16371689
}
16381690
}
16391691
/** Generated class from Pigeon that represents Flutter messages that can be called from Java. */
@@ -4389,6 +4441,9 @@ public interface Camera2CameraInfoHostApi {
43894441
@NonNull
43904442
String getCameraId(@NonNull Long identifier);
43914443

4444+
@NonNull
4445+
Long getSensorOrientation(@NonNull Long identifier);
4446+
43924447
/** The codec used by Camera2CameraInfoHostApi. */
43934448
static @NonNull MessageCodec<Object> getCodec() {
43944449
return new StandardMessageCodec();
@@ -4481,6 +4536,33 @@ static void setup(
44814536
channel.setMessageHandler(null);
44824537
}
44834538
}
4539+
{
4540+
BasicMessageChannel<Object> channel =
4541+
new BasicMessageChannel<>(
4542+
binaryMessenger,
4543+
"dev.flutter.pigeon.Camera2CameraInfoHostApi.getSensorOrientation",
4544+
getCodec());
4545+
if (api != null) {
4546+
channel.setMessageHandler(
4547+
(message, reply) -> {
4548+
ArrayList<Object> wrapped = new ArrayList<Object>();
4549+
ArrayList<Object> args = (ArrayList<Object>) message;
4550+
Number identifierArg = (Number) args.get(0);
4551+
try {
4552+
Long output =
4553+
api.getSensorOrientation(
4554+
(identifierArg == null) ? null : identifierArg.longValue());
4555+
wrapped.add(0, output);
4556+
} catch (Throwable exception) {
4557+
ArrayList<Object> wrappedError = wrapError(exception);
4558+
wrapped = wrappedError;
4559+
}
4560+
reply.reply(wrapped);
4561+
});
4562+
} else {
4563+
channel.setMessageHandler(null);
4564+
}
4565+
}
44844566
}
44854567
}
44864568
/** Generated class from Pigeon that represents Flutter messages that can be called from Java. */

‎packages/camera/camera_android_camerax/android/src/main/java/io/flutter/plugins/camerax/SystemServicesHostApiImpl.java

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66

77
import android.app.Activity;
88
import android.content.Context;
9+
import android.os.Build;
910
import androidx.annotation.NonNull;
1011
import androidx.annotation.Nullable;
1112
import androidx.annotation.VisibleForTesting;
@@ -103,4 +104,18 @@ public String getTempFilePath(@NonNull String prefix, @NonNull String suffix) {
103104
null);
104105
}
105106
}
107+
108+
/**
109+
* Returns whether or not a {@code SurfaceTexture} backs the {@code Surface} provided to CameraX
110+
* to build the camera preview. If it is backed by a {@code Surface}, then the transformation
111+
* needed to correctly rotate the preview has already been applied.
112+
*
113+
* <p>This is determined by the engine, who uses {@code SurfaceTexture}s on Android SDKs 29 and
114+
* below.
115+
*/
116+
@Override
117+
@NonNull
118+
public Boolean isPreviewPreTransformed() {
119+
return Build.VERSION.SDK_INT <= 29;
120+
}
106121
}

‎packages/camera/camera_android_camerax/android/src/test/java/io/flutter/plugins/camerax/DeviceOrientationManagerWrapperTest.java

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,4 +93,16 @@ public void getDefaultDisplayRotation_returnsExpectedRotation() {
9393

9494
assertEquals(hostApi.getDefaultDisplayRotation(), Long.valueOf(defaultRotation));
9595
}
96+
97+
@Test
98+
public void getUiOrientation_returnsExpectedOrientation() {
99+
final DeviceOrientationManagerHostApiImpl hostApi =
100+
new DeviceOrientationManagerHostApiImpl(mockBinaryMessenger, mockInstanceManager);
101+
final DeviceOrientation uiOrientation = DeviceOrientation.LANDSCAPE_LEFT;
102+
103+
hostApi.deviceOrientationManager = mockDeviceOrientationManager;
104+
when(mockDeviceOrientationManager.getUIOrientation()).thenReturn(uiOrientation);
105+
106+
assertEquals(hostApi.getUiOrientation(), uiOrientation.toString());
107+
}
96108
}

‎packages/camera/camera_android_camerax/android/src/test/java/io/flutter/plugins/camerax/SystemServicesTest.java

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@
55
package io.flutter.plugins.camerax;
66

77
import static org.junit.Assert.assertEquals;
8+
import static org.junit.Assert.assertFalse;
89
import static org.junit.Assert.assertThrows;
10+
import static org.junit.Assert.assertTrue;
911
import static org.mockito.ArgumentMatchers.eq;
1012
import static org.mockito.Mockito.mock;
1113
import static org.mockito.Mockito.mockStatic;
@@ -24,12 +26,16 @@
2426
import java.io.IOException;
2527
import org.junit.Rule;
2628
import org.junit.Test;
29+
import org.junit.runner.RunWith;
2730
import org.mockito.ArgumentCaptor;
2831
import org.mockito.Mock;
2932
import org.mockito.MockedStatic;
3033
import org.mockito.junit.MockitoJUnit;
3134
import org.mockito.junit.MockitoRule;
35+
import org.robolectric.RobolectricTestRunner;
36+
import org.robolectric.annotation.Config;
3237

38+
@RunWith(RobolectricTestRunner.class)
3339
public class SystemServicesTest {
3440
@Rule public MockitoRule mockitoRule = MockitoJUnit.rule();
3541

@@ -130,4 +136,28 @@ public void getTempFilePath_throwsRuntimeExceptionOnIOException() {
130136

131137
mockedStaticFile.close();
132138
}
139+
140+
@Test
141+
@Config(sdk = 28)
142+
public void isPreviewPreTransformed_returnsTrueWhenRunningBelowSdk29() {
143+
final SystemServicesHostApiImpl systemServicesHostApi =
144+
new SystemServicesHostApiImpl(mockBinaryMessenger, mockInstanceManager, mockContext);
145+
assertTrue(systemServicesHostApi.isPreviewPreTransformed());
146+
}
147+
148+
@Test
149+
@Config(sdk = 29)
150+
public void isPreviewPreTransformed_returnsTrueWhenRunningSdk29() {
151+
final SystemServicesHostApiImpl systemServicesHostApi =
152+
new SystemServicesHostApiImpl(mockBinaryMessenger, mockInstanceManager, mockContext);
153+
assertTrue(systemServicesHostApi.isPreviewPreTransformed());
154+
}
155+
156+
@Test
157+
@Config(sdk = 30)
158+
public void isPreviewPreTransformed_returnsFalseWhenRunningAboveSdk29() {
159+
final SystemServicesHostApiImpl systemServicesHostApi =
160+
new SystemServicesHostApiImpl(mockBinaryMessenger, mockInstanceManager, mockContext);
161+
assertFalse(systemServicesHostApi.isPreviewPreTransformed());
162+
}
133163
}

‎packages/camera/camera_android_camerax/example/pubspec.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@ description: Demonstrates how to use the camera_android_camerax plugin.
33
publish_to: 'none'
44

55
environment:
6-
sdk: ^3.2.0
7-
flutter: ">=3.16.0"
6+
sdk: ^3.4.0
7+
flutter: ">=3.22.0"
88

99
dependencies:
1010
camera_android_camerax:

0 commit comments

Comments
 (0)