Repository navigation
[camera_android_camerax] Fix 90°-off preview rotation - #8629
Conversation
This comment was marked as off-topic.
This comment was marked as off-topic.
|
@camsim99 Do these fixes also address flutter/flutter#156974? Linking here in case this pull request fixes it as well. |
This wouldn't fix that issue, but I'll follow up there. |
matanlurey
left a comment
There was a problem hiding this comment.
From an implementation POV, LGTM, great work.
From a testing POV, we can rely a bit less on mocks and testing internal state, and more on testing the exported state of the code itself; this will make the system less fragile (we're doing closer to e2e tests in a unit test environment) and more easily understood.
There are external guides talking about API testing, but I'll message one in Flutter Android chat that is internal I like a lot.
| _subscriptionForDeviceOrientationChanges; | ||
| /// Whether or not the Android surface producer automatically handles | ||
| /// correcting the rotation of camera previews for the device this plugin runs on. | ||
| @visibleForTesting |
There was a problem hiding this comment.
I would try to not inspect the internal state of these variables as the test (in other words, not make them @visibleForTesting and make them private instead), and instead inspect the output of the widget tree. I'll leave more comments on the test itself.
There was a problem hiding this comment.
Left these as visible so that we can set them directly in the preview_rotation_test.dart. These values being fetched as expected is tested in android_camera_camerax_test.dart.
There was a problem hiding this comment.
Left these as visible so that we can set them directly in the preview_rotation_test.dart.
Is it possible to write that test without setting these variables directly? By setting them directly, we are bypassing the logic that is initially used to set them (in createCameraWithSettings) which if I remember correctly was a significant source of difficulty.
| expect((widget as Texture).textureId, cameraId); | ||
| }); | ||
|
|
||
| test( |
There was a problem hiding this comment.
This is a good start, but I'd split this into separate tests.
This is even complicated enough where I'd support a preview_rotation_test.dart.
Matrix of options I'd spect:
- When
handlesCropAndRotation = true, the preview is <not rotated or w/e it is>
- Sub test: Verify rotation the stream still works as expected
- When
handlesCropAndRotation = false
- Sub test: Based on the initial orientation; 1 test for each orientation
- Sub test: Preview changes as orientation changes (1 test is fine IMO)
- Sub test: How
sensorOrientationDegreesplays into things - Sub test: How
facingSignplays into things
Basically, we'd want to know that all of these parameters are being used to render the camera preview as expected. It should be possible to read this test suite and understand the contract of the camera preview.
There was a problem hiding this comment.
Added preview_rotation_test.dart with the intention of covering all of these cases.
Sub test: Verify rotation the stream still works as expected
Not 100% sure what you meant but I:
- Already have a test in
android_camera_camerax_test..dartthat tests theonDeviceOrientationChanged(). - Ensured that when
handlesCropAndRotationisfalse, noRotatedBoxis used inpreview_rotation_test.dart. - Added to
camera/camera'sCameraPreviewtest to ensure that its rotation behaves as expected. cc @bparrishMines because this is why your review was requested :)
3 made me realize a case that this PR will not cover; the preview rotation logic written here does not account for locked capture orientations. If it is locked, CameraPreview is rotated according to that instead of the device orientation. I'll file an issue and follow this PR up with a fix for that.
@matanlurey let me know if these cases don't cover what you were talking about!
There was a problem hiding this comment.
3 made me realize a case that this PR will not cover; the preview rotation logic written here does not account for locked capture orientations. If it is locked, CameraPreview is rotated according to that instead of the device orientation. I'll file an issue and follow this PR up with a fix for that.
Filed flutter/flutter#163857
matanlurey
left a comment
There was a problem hiding this comment.
Mostly LGTM. I'd like to see us try not to use @visibleForTesting at all to manipulate the internal state for tests, but it's possible that is not doable or requires a lot more work than I expect. PTAL.
| _subscriptionForDeviceOrientationChanges; | ||
| /// Whether or not the Android surface producer automatically handles | ||
| /// correcting the rotation of camera previews for the device this plugin runs on. | ||
| @visibleForTesting |
There was a problem hiding this comment.
Left these as visible so that we can set them directly in the preview_rotation_test.dart.
Is it possible to write that test without setting these variables directly? By setting them directly, we are bypassing the logic that is initially used to set them (in createCameraWithSettings) which if I remember correctly was a significant source of difficulty.
There was a problem hiding this comment.
These tests look really good.
My only other comment (left in a previous file) is it would be better to go through the createCameraWithSettings branch instead of carving out the @visibleForTesting members.
If that is really difficult (for mocking or testability reasons), and you want help, let's pair (it's possible I'll agree and give up this point, but I'd want to give it a shot).
There was a problem hiding this comment.
Done. Also, did a minor refactor to remove a duplicate call to get the sensor orientation. Also also, I'll note it requires a call to availableCameras and createCameraWithSettings, so all of the tests go through that path. PTAL
| flutterSurfaceProducer.release(); | ||
| return; | ||
| } | ||
| throw new IllegalStateException( |
There was a problem hiding this comment.
Where does our code catch IllegalStateException? if we dont where do we document for our users that they need to catch an exception?
Native code thrown exceptions I remember tripping up my previous team on a couple of occasions to the point where I think we made a rule that exceptions were not allowed to leave the plugin api and if they did it was considered a plugin bug.
I think our best guidance is here https://github.com/flutter/flutter/blob/master/docs/ecosystem/contributing/README.md#platform-exception-handling
There was a problem hiding this comment.
We throw exceptions like this throughout the camera_android_camerax implementation to signal that there may be an error in the plugin implementation to plugin authors, so this is not meant to be caught by users.
If you have greater concerns about this pattern, I suggest we file an issue and address this for the plugin as a whole in a follow up PR.
There was a problem hiding this comment.
OK, Non blocking question. Do we catch this (or other) exception(s) or is it thrown to users code?
There was a problem hiding this comment.
We don't catch it. They're really just supposed to help debug.
flutter/packages@c44c228...01d3d5c 2025-02-27 [email protected] Manual roll Flutter from 043b719 to 1659206 (19 revisions) (flutter/packages#8728) 2025-02-27 [email protected] Manual roll Flutter from 911aa75 to 043b719 (flutter/packages#8693) 2025-02-26 [email protected] Dependabot to update major and minor versions of test dependencies, ignore patch (flutter/packages#8712) 2025-02-26 [email protected] [local_auth] Update to use flutter.targetSdkVersion (flutter/packages#8695) 2025-02-26 [email protected] [go_router_builder]: Handle invaild params (flutter/packages#8405) 2025-02-26 [email protected] [pigeon] Timestamp test steps in CI (flutter/packages#8716) 2025-02-26 [email protected] [camera_android_camerax] Fix 90°-off preview rotation (flutter/packages#8629) 2025-02-26 [email protected] [go_router] Secured empty matches in canPop (flutter/packages#8557) 2025-02-26 [email protected] [tool] Update targetsdk version to 35 from 32 (flutter/packages#8694) 2025-02-26 [email protected] [various] Bump androidx.test:core to 1.4.0 (flutter/packages#8710) 2025-02-26 [email protected] [camera] Disable flaky tests (flutter/packages#8708) 2025-02-26 [email protected] [url_launcher][web] Prevent browser from navigating when followLink isn't called (flutter/packages#8675) 2025-02-26 [email protected] [various] Remove plugin-level `integration_test` dependencies (flutter/packages#8711) 2025-02-26 [email protected] [ci] Lengthen custom tests timeout (flutter/packages#8715) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages-flutter-autoroll Please CC [email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
…iented devices (#9097) Fixes the incorrect camera preview rotation on landscape-oriented devices (like some tablets). Really, this PR generalizes the fix added in #8629 for correcting the camera preview rotation by correcting the camera preview rotation according to the rotation from the natural device orientation instead of assuming the device is naturally oriented in portrait up. There are technically two different fixes in this PR; here's the breakdown for clarity: 1. For devices using the `SurfaceTexture` Impeller backend: Rotate the camera preview according to the current default display rotation (the rotation away from the natural device orientation) and ignore rotation added in `CameraPreview` widget completely (as it assumes the natural device orientation to be portrait up). 2. For devices using the `ImageReader` Impeller backend: Rotate the camera preview according to https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation, where `deviceOrientationDegrees` comes from [a call](https://developer.android.com/reference/android/view/Display?_gl=1*lysjx*_up*MQ..*_ga*NjY3Nzk0Mzg2LjE3NDU5NjY3ODc.*_ga_6HH9YJMN9M*MTc0NTk2Njc4Ni4xLjAuMTc0NTk2Njc4Ni4wLjAuMjEwMTI3NTg1Mg..#getRotation()) to get the current default display rotation (the rotation away from the natural device orientation) instead of assuming it to be the rotation degrees from portrait up. Also, fixes a potential regression that @bparrishMines discovered in the `DeviceOrientationManager`. The changes there essentially ensure that it resets its state every time it is "re-started" to listen into changes into device orientation changes. Fixes flutter/flutter#164493 *Seems to work on all physical devices I test, but not emulators (tried Pixel Tablet API 28, 35). I think that is an emulator bug? Otherwise tested on a new Pixel Tablet (API 34), Pixel Tablet C (API 12), Pixel 3A (phone, API 32). ## Pre-Review Checklist [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
Fixes 90°-off preview rotation without locked capture orientation. Part of flutter/flutter#154241. The major changes in this PR: 1. Ensures the preview widget is rebuilt when a new device orientation is detected by changing the `buildPreview` to return a `StatefulWidget` instead of a `Texture` that wouldn't pick up changes in device orientation. 2. Uses [`handlesCropAndRotation`](https://api.flutter.dev/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#handlesCropAndRotation()) to detect if the preview is already correctly rotated or not. This API was added exactly for that purpose. 3. Corrects the preview when needed using https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation (also subtracts rotation applied in `CameraPreview` widget). See flutter/flutter#154241 (comment) for slightly more context if desired.
Fixes 90°-off preview rotation without locked capture orientation. Part of flutter/flutter#154241. The major changes in this PR: 1. Ensures the preview widget is rebuilt when a new device orientation is detected by changing the `buildPreview` to return a `StatefulWidget` instead of a `Texture` that wouldn't pick up changes in device orientation. 2. Uses [`handlesCropAndRotation`](https://api.flutter.dev/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#handlesCropAndRotation()) to detect if the preview is already correctly rotated or not. This API was added exactly for that purpose. 3. Corrects the preview when needed using https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation (also subtracts rotation applied in `CameraPreview` widget). See flutter/flutter#154241 (comment) for slightly more context if desired.
…iented devices (flutter#9097) Fixes the incorrect camera preview rotation on landscape-oriented devices (like some tablets). Really, this PR generalizes the fix added in flutter#8629 for correcting the camera preview rotation by correcting the camera preview rotation according to the rotation from the natural device orientation instead of assuming the device is naturally oriented in portrait up. There are technically two different fixes in this PR; here's the breakdown for clarity: 1. For devices using the `SurfaceTexture` Impeller backend: Rotate the camera preview according to the current default display rotation (the rotation away from the natural device orientation) and ignore rotation added in `CameraPreview` widget completely (as it assumes the natural device orientation to be portrait up). 2. For devices using the `ImageReader` Impeller backend: Rotate the camera preview according to https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation, where `deviceOrientationDegrees` comes from [a call](https://developer.android.com/reference/android/view/Display?_gl=1*lysjx*_up*MQ..*_ga*NjY3Nzk0Mzg2LjE3NDU5NjY3ODc.*_ga_6HH9YJMN9M*MTc0NTk2Njc4Ni4xLjAuMTc0NTk2Njc4Ni4wLjAuMjEwMTI3NTg1Mg..#getRotation()) to get the current default display rotation (the rotation away from the natural device orientation) instead of assuming it to be the rotation degrees from portrait up. Also, fixes a potential regression that @bparrishMines discovered in the `DeviceOrientationManager`. The changes there essentially ensure that it resets its state every time it is "re-started" to listen into changes into device orientation changes. Fixes flutter/flutter#164493 *Seems to work on all physical devices I test, but not emulators (tried Pixel Tablet API 28, 35). I think that is an emulator bug? Otherwise tested on a new Pixel Tablet (API 34), Pixel Tablet C (API 12), Pixel 3A (phone, API 32). ## Pre-Review Checklist [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
…iented devices (flutter#9097) Fixes the incorrect camera preview rotation on landscape-oriented devices (like some tablets). Really, this PR generalizes the fix added in flutter#8629 for correcting the camera preview rotation by correcting the camera preview rotation according to the rotation from the natural device orientation instead of assuming the device is naturally oriented in portrait up. There are technically two different fixes in this PR; here's the breakdown for clarity: 1. For devices using the `SurfaceTexture` Impeller backend: Rotate the camera preview according to the current default display rotation (the rotation away from the natural device orientation) and ignore rotation added in `CameraPreview` widget completely (as it assumes the natural device orientation to be portrait up). 2. For devices using the `ImageReader` Impeller backend: Rotate the camera preview according to https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation, where `deviceOrientationDegrees` comes from [a call](https://developer.android.com/reference/android/view/Display?_gl=1*lysjx*_up*MQ..*_ga*NjY3Nzk0Mzg2LjE3NDU5NjY3ODc.*_ga_6HH9YJMN9M*MTc0NTk2Njc4Ni4xLjAuMTc0NTk2Njc4Ni4wLjAuMjEwMTI3NTg1Mg..#getRotation()) to get the current default display rotation (the rotation away from the natural device orientation) instead of assuming it to be the rotation degrees from portrait up. Also, fixes a potential regression that @bparrishMines discovered in the `DeviceOrientationManager`. The changes there essentially ensure that it resets its state every time it is "re-started" to listen into changes into device orientation changes. Fixes flutter/flutter#164493 *Seems to work on all physical devices I test, but not emulators (tried Pixel Tablet API 28, 35). I think that is an emulator bug? Otherwise tested on a new Pixel Tablet (API 34), Pixel Tablet C (API 12), Pixel 3A (phone, API 32). ## Pre-Review Checklist [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
Fixes 90°-off preview rotation without locked capture orientation. Part of flutter/flutter#154241. The major changes in this PR: 1. Ensures the preview widget is rebuilt when a new device orientation is detected by changing the `buildPreview` to return a `StatefulWidget` instead of a `Texture` that wouldn't pick up changes in device orientation. 2. Uses [`handlesCropAndRotation`](https://api.flutter.dev/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#handlesCropAndRotation()) to detect if the preview is already correctly rotated or not. This API was added exactly for that purpose. 3. Corrects the preview when needed using https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation (also subtracts rotation applied in `CameraPreview` widget). See flutter/flutter#154241 (comment) for slightly more context if desired.
…iented devices (flutter#9097) Fixes the incorrect camera preview rotation on landscape-oriented devices (like some tablets). Really, this PR generalizes the fix added in flutter#8629 for correcting the camera preview rotation by correcting the camera preview rotation according to the rotation from the natural device orientation instead of assuming the device is naturally oriented in portrait up. There are technically two different fixes in this PR; here's the breakdown for clarity: 1. For devices using the `SurfaceTexture` Impeller backend: Rotate the camera preview according to the current default display rotation (the rotation away from the natural device orientation) and ignore rotation added in `CameraPreview` widget completely (as it assumes the natural device orientation to be portrait up). 2. For devices using the `ImageReader` Impeller backend: Rotate the camera preview according to https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation, where `deviceOrientationDegrees` comes from [a call](https://developer.android.com/reference/android/view/Display?_gl=1*lysjx*_up*MQ..*_ga*NjY3Nzk0Mzg2LjE3NDU5NjY3ODc.*_ga_6HH9YJMN9M*MTc0NTk2Njc4Ni4xLjAuMTc0NTk2Njc4Ni4wLjAuMjEwMTI3NTg1Mg..#getRotation()) to get the current default display rotation (the rotation away from the natural device orientation) instead of assuming it to be the rotation degrees from portrait up. Also, fixes a potential regression that @bparrishMines discovered in the `DeviceOrientationManager`. The changes there essentially ensure that it resets its state every time it is "re-started" to listen into changes into device orientation changes. Fixes flutter/flutter#164493 *Seems to work on all physical devices I test, but not emulators (tried Pixel Tablet API 28, 35). I think that is an emulator bug? Otherwise tested on a new Pixel Tablet (API 34), Pixel Tablet C (API 12), Pixel 3A (phone, API 32). ## Pre-Review Checklist [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
Fixes 90°-off preview rotation without locked capture orientation. Part of flutter/flutter#154241. The major changes in this PR: 1. Ensures the preview widget is rebuilt when a new device orientation is detected by changing the `buildPreview` to return a `StatefulWidget` instead of a `Texture` that wouldn't pick up changes in device orientation. 2. Uses [`handlesCropAndRotation`](https://api.flutter.dev/javadoc/io/flutter/view/TextureRegistry.SurfaceProducer.html#handlesCropAndRotation()) to detect if the preview is already correctly rotated or not. This API was added exactly for that purpose. 3. Corrects the preview when needed using https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation (also subtracts rotation applied in `CameraPreview` widget). See flutter/flutter#154241 (comment) for slightly more context if desired.
…iented devices (flutter#9097) Fixes the incorrect camera preview rotation on landscape-oriented devices (like some tablets). Really, this PR generalizes the fix added in flutter#8629 for correcting the camera preview rotation by correcting the camera preview rotation according to the rotation from the natural device orientation instead of assuming the device is naturally oriented in portrait up. There are technically two different fixes in this PR; here's the breakdown for clarity: 1. For devices using the `SurfaceTexture` Impeller backend: Rotate the camera preview according to the current default display rotation (the rotation away from the natural device orientation) and ignore rotation added in `CameraPreview` widget completely (as it assumes the natural device orientation to be portrait up). 2. For devices using the `ImageReader` Impeller backend: Rotate the camera preview according to https://developer.android.com/media/camera/camera2/camera-preview#orientation_calculation, where `deviceOrientationDegrees` comes from [a call](https://developer.android.com/reference/android/view/Display?_gl=1*lysjx*_up*MQ..*_ga*NjY3Nzk0Mzg2LjE3NDU5NjY3ODc.*_ga_6HH9YJMN9M*MTc0NTk2Njc4Ni4xLjAuMTc0NTk2Njc4Ni4wLjAuMjEwMTI3NTg1Mg..#getRotation()) to get the current default display rotation (the rotation away from the natural device orientation) instead of assuming it to be the rotation degrees from portrait up. Also, fixes a potential regression that @bparrishMines discovered in the `DeviceOrientationManager`. The changes there essentially ensure that it resets its state every time it is "re-started" to listen into changes into device orientation changes. Fixes flutter/flutter#164493 *Seems to work on all physical devices I test, but not emulators (tried Pixel Tablet API 28, 35). I think that is an emulator bug? Otherwise tested on a new Pixel Tablet (API 34), Pixel Tablet C (API 12), Pixel 3A (phone, API 32). ## Pre-Review Checklist [^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
…rash Real device testing found the previous rotation fix (targetRotation set natively on Preview/ImageCapture) did not actually work, and a second, different crash in tap-to-focus. Root cause of the rotation bug, found via web research into the exact symptom against Flutter's own issue tracker: on API 29+, Flutter's ImageReader-backed SurfaceProducer texture path does not apply the camera sensor's rotation metadata itself (TextureRegistry.SurfaceProducer#handlesCropAndRotation reports false on this path) -- setting CameraX's own targetRotation natively cannot fix this, because the mismatch is in Flutter's own rendering pipeline, not in what CameraX tells the sensor to do. Confirmed against flutter/packages#8629, the PR that fixed the identical bug in Flutter's own official camera_android_camerax plugin: the correct fix applies rotation correction in Dart, via a RotatedBox, using CameraCharacteristics.SENSOR_ORIENTATION and the device's current orientation. Implemented the same pattern: - Pigeon contract: InitializeResult/SwitchCameraResult now report sensorOrientationDegrees, read via Camera2CameraInfo. getCameraCharacteristic(SENSOR_ORIENTATION) in CameraXSession. Removed the previous (ineffective) native targetRotation guess entirely. - CameraSessionInfo/CameraState carry sensorOrientationDegrees through to Dart. - New computeRotationQuarterTurns() in preview_transform.dart: pure function implementing the same formula as Flutter's own RotatedPreview widget -- (sensorOrientation - deviceOrientation * sign + 360) % 360 -- unit tested against the exact back-camera values confirmed on-device (sensor=90, needs 1 quarter turn) plus the front-camera case, sensor 0 degrees (no correction), and a range check. Device orientation is currently assumed portraitUp; live orientation-change tracking is out of Phase 2's scope. - CustomCameraPreview wraps the Texture in a RotatedBox using this value, with a FittedBox to rescale the now-correctly-oriented content into the cover/contain layout rect (a 90/270-degree rotation swaps the content's effective width/height for layout purposes). Verified on-device (Pixel 10): a handwritten "North" label, previously rotated 90 degrees and unreadable, now displays upright. Second, unrelated crash: CameraX's startFocusAndMetering() cancels an in-flight call when a newer one starts on the same camera -- completely normal for a tap-to-focus gesture (a new tap simply supersedes an older, still-focusing one), but the previous code let that cancellation propagate as an uncaught exception, crashing the controller into its error state. CameraXSession.setMeteringPoint now catches CameraControl.OperationCanceledException specifically and treats it as a no-op. Verified on-device across both spaced and rapid multi-tap sequences -- no crash, controls remain responsive. Also nudged the shutter button's bottom padding (24 -> 48) after feedback that the previous fix (moving it off being too high) had overcorrected to sitting too low, closer to how native Pixel/iOS camera apps keep it clear of the very bottom edge. 41/41 unit tests passing (4 new, covering the rotation formula), `flutter analyze` clean. Co-Authored-By: Claude Sonnet 5 <[email protected]>
* Implement Phase 2: live preview, capture, flash/zoom/focus/exposure, camera switch Extends the Pigeon contract to bind Preview + ImageCapture together with initialize() (CameraX binds all use cases in one bindToLifecycle() call, so there's no meaningful "initialized but no preview" state), and adds captureImage/setFlashMode/switchCamera/setZoom/setMeteringPoint/ setExposureCompensation. Dart: - CustomCameraController: capturing/switchingCamera transitions added to the state machine, each guarded the same way initializing/disposing are. State now carries textureId, previewSize, flashMode, zoomRatio, exposureCompensation. - CustomCameraPreview: Texture-backed live preview widget with cover/contain layout (preview_transform.dart, pure function, unit tested against 8 aspect-ratio/edge-case scenarios) and front-camera mirroring via Transform.flip. - ImageCaptureResult, FlashMode, NormalizedPoint as new immutable value types. - 17 new controller unit tests against FakeCameraPlatform covering capture/flash/switch/zoom/exposure race conditions and state transitions (37 total, all passing). Kotlin: - CameraXSession now binds real Preview + ImageCapture use cases, wires Preview's SurfaceProvider to a Flutter TextureRegistry.SurfaceProducer, and exposes zoom/focus-metering/exposure-compensation/flash controls. - ImageCaptureHandler wraps ImageCapture.takePicture() as a suspend call and reads back real dimensions + EXIF orientation. - Fixed a real async-timing bug found via on-device logging: CameraX invokes Preview's SurfaceProvider asynchronously, some milliseconds after bindToLifecycle() returns -- not synchronously as originally assumed. bind() now actually suspends until the surface request fires, via suspendCancellableCoroutine, instead of returning a resolution of 0x0. Combined setFocusPoint/setExposurePoint into one setMeteringPoint call (single FocusMeteringAction with FLAG_AF or FLAG_AE) after finding, via on-device testing, that CameraX's startFocusAndMetering() cancels an in-flight call when a second one starts on the same camera -- the example app's tap-to-focus gesture was issuing both calls back-to-back for the same tap, which reliably raced and pushed the controller into its error state on every tap. Verified end-to-end on a physical device (Pixel 10): - Live preview fills the screen correctly in cover mode, real content, correct focus -- confirmed via screenshot after chasing what first looked like a black-preview bug down to two real issues (the surface timing race above, and an unrelated test-scene framing coincidence that briefly looked like the same bug). - Tap-to-focus/expose no longer crashes the controller into error state. - captureImage() produced a real 4000x3000 JPEG at a real cache path, confirmed via the example app's on-screen capture summary. - switchCamera() correctly rebinds the session; preview and capability-driven UI (flash visibility) update to the new lens. Example app rewritten into a full custom UI (Stack of preview + tap-to- focus gesture layer + flash/switch/zoom/shutter controls), matching the brief's own composition pattern -- no bundled camera screen exists in d3_camera itself. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Fix preview rotation bug and rework example app UX per device feedback Real device testing on a Pixel 10 surfaced five issues with the Phase 2 work; this addresses all of them. Kotlin (real bug): - Preview and ImageCapture were bound with no explicit targetRotation, leaving CameraX to infer display rotation on its own for a session that isn't Activity-bound (see CameraXSession's own docs on why it binds to ProcessLifecycleOwner instead). Confirmed on-device this produced a rotated preview. Fixed by reading the real default display's rotation via DisplayManager and passing it explicitly to both Preview.Builder and ImageCapture.Builder. Example app (UX, not package API): - Shutter button was vertically centered against the whole control stack via one spaceBetween Column, landing well above where a phone's own camera app puts it. Split into two independent SafeArea regions: a top bar for flash/switch/zoom, and a bottom-pinned Align for the shutter alone. - Replaced the full-width Slider zoom control with a row of discrete zoom-level pills (0.6x/1x/2x/3x/5x/20x, generated from the device's actual reported capability range) -- closer to how real camera apps present zoom, and far easier to hit accurately by touch. - Capture produced a shutter sound but no visible result -- confirmed via logcat that ImageCapture.takePictureInternal was firing correctly, but the example app only showed a small text summary competing with the bottom controls. Replaced with a full-screen post-capture review (Image.file of the actual captured JPEG, a close button, and the dimensions/lens badge moved to the top instead of the bottom) -- the metadata-placement and missing-preview feedback are really the same UX gap. Verified on-device after the fixes: preview renders upright (text in the test scene reads correctly, not rotated), the zoom pills visibly change the field of view when tapped, the shutter sits at the bottom edge, and tapping capture opens the review screen showing the actual photo right-side up with the close button returning cleanly to a still- `ready` live camera. 37/37 unit tests still passing (fixes were UI-layer/native-binding only, no controller or platform-interface API changes); `flutter analyze` clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Fix real 90-degree preview rotation and focus-metering cancellation crash Real device testing found the previous rotation fix (targetRotation set natively on Preview/ImageCapture) did not actually work, and a second, different crash in tap-to-focus. Root cause of the rotation bug, found via web research into the exact symptom against Flutter's own issue tracker: on API 29+, Flutter's ImageReader-backed SurfaceProducer texture path does not apply the camera sensor's rotation metadata itself (TextureRegistry.SurfaceProducer#handlesCropAndRotation reports false on this path) -- setting CameraX's own targetRotation natively cannot fix this, because the mismatch is in Flutter's own rendering pipeline, not in what CameraX tells the sensor to do. Confirmed against flutter/packages#8629, the PR that fixed the identical bug in Flutter's own official camera_android_camerax plugin: the correct fix applies rotation correction in Dart, via a RotatedBox, using CameraCharacteristics.SENSOR_ORIENTATION and the device's current orientation. Implemented the same pattern: - Pigeon contract: InitializeResult/SwitchCameraResult now report sensorOrientationDegrees, read via Camera2CameraInfo. getCameraCharacteristic(SENSOR_ORIENTATION) in CameraXSession. Removed the previous (ineffective) native targetRotation guess entirely. - CameraSessionInfo/CameraState carry sensorOrientationDegrees through to Dart. - New computeRotationQuarterTurns() in preview_transform.dart: pure function implementing the same formula as Flutter's own RotatedPreview widget -- (sensorOrientation - deviceOrientation * sign + 360) % 360 -- unit tested against the exact back-camera values confirmed on-device (sensor=90, needs 1 quarter turn) plus the front-camera case, sensor 0 degrees (no correction), and a range check. Device orientation is currently assumed portraitUp; live orientation-change tracking is out of Phase 2's scope. - CustomCameraPreview wraps the Texture in a RotatedBox using this value, with a FittedBox to rescale the now-correctly-oriented content into the cover/contain layout rect (a 90/270-degree rotation swaps the content's effective width/height for layout purposes). Verified on-device (Pixel 10): a handwritten "North" label, previously rotated 90 degrees and unreadable, now displays upright. Second, unrelated crash: CameraX's startFocusAndMetering() cancels an in-flight call when a newer one starts on the same camera -- completely normal for a tap-to-focus gesture (a new tap simply supersedes an older, still-focusing one), but the previous code let that cancellation propagate as an uncaught exception, crashing the controller into its error state. CameraXSession.setMeteringPoint now catches CameraControl.OperationCanceledException specifically and treats it as a no-op. Verified on-device across both spaced and rapid multi-tap sequences -- no crash, controls remain responsive. Also nudged the shutter button's bottom padding (24 -> 48) after feedback that the previous fix (moving it off being too high) had overcorrected to sitting too low, closer to how native Pixel/iOS camera apps keep it clear of the very bottom edge. 41/41 unit tests passing (4 new, covering the rotation formula), `flutter analyze` clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Fix layout chrome, orientation lock, and capture/preview aspect+rotation mismatch Real device testing found five more issues; this addresses all of them. 1. Layout redesign (example app only): replaced the full-bleed Stack (feed behind floating controls) with the native-camera-app pattern -- a fixed, letterboxed preview rect (AspectRatio driven by the actual negotiated previewSize, not hardcoded) with a dedicated control panel below it. Zoom pills sit directly above the shutter, flash/switch directly below it, matching where Pixel Camera and iOS Camera place them -- previously all three floated on top of the live feed. 2. Orientation lock (package, not example app -- corrected mid-session after initially placing this in main(), which would have wrongly forced portrait on a consuming app that supports landscape elsewhere in its own UI): CustomCameraController.initialize() now locks the UI to portraitUp via SystemChrome.setPreferredOrientations for the lifetime of the camera session, restoring all four orientations in dispose(). This is scoped to exactly the controller's lifetime, not a global app setting, and it's what real camera apps do (verified via web research against Flutter's own camera plugin issue tracker and the pattern's general use). It also makes CustomCameraPreview's rotation-correction formula's fixed portraitUp assumption actually true rather than merely assumed. 3. Preview/capture aspect ratio mismatch: Preview and ImageCapture were each letting CameraX pick resolution independently, so the captured photo's aspect ratio could differ from what was actually framed live. Both use cases now share one ResolutionSelector with AspectRatioStrategy.RATIO_4_3_FALLBACK_AUTO_STRATEGY, so they crop to the same active-array aspect ratio. 4. Preview/capture rotation mismatch: the live preview (Dart-side RotatedBox, using SENSOR_ORIENTATION) displayed "North" upright, but the saved JPEG -- viewed via Image.file in the review screen, which does not honor EXIF orientation -- still showed it rotated. Root cause: CameraX's own ImageCapture-written EXIF tag (180 degrees) does not agree with the rotation value confirmed correct for the live preview (90 degrees) -- two different, unreconciled computations inside CameraX itself. Rather than trust the disagreeing value, added a single shared formula (RotationMath.kt's computeRotationDegrees, identical to preview_transform.dart's computeRotationQuarterTurns) and made ImageCaptureHandler physically rotate the captured bitmap's pixels by that value, resetting EXIF to normal afterward -- the file is now correct for any viewer (this package's own review UI, a share target, a native gallery app), not just contexts that happen to apply the same correction. Verified on-device (Pixel 10): the new layout matches native camera chrome exactly (letterboxed feed, zoom above/flash-switch below shutter); a captured "North"/"South" test photo displays upright in the review screen with correctly swapped 3000x4000 dimensions (previously 4000x3000, rotated and mismatched). 41/41 unit tests passing (added TestWidgetsFlutterBinding. ensureInitialized() to the controller test file, since initialize()/ dispose() now make real SystemChrome platform-channel calls); flutter analyze clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Add 4:3/16:9 aspect ratio toggle; fix review-screen framing mismatch Two more issues from continued device feedback. 1. Aspect ratio toggle: native camera apps (Pixel Camera, iOS Camera) default to 4:3 with a live 16:9 toggle. Added end-to-end support: - Pigeon: new AspectRatioPresetData enum; unified the three near-identical per-rebind result types (InitializeResult, SwitchCameraResult, and the new setAspectRatio's own result) into one CameraSessionResultData, since all three rebind the same CameraX use-case group and return the same shape. - CameraXSession.bind() now takes an aspect ratio parameter and picks the matching AspectRatioStrategy (4:3 or 16:9, both with FALLBACK_AUTO_STRATEGY); added setAspectRatio() (rebinds the current lens with a new ratio) and switchLens() (rebinds a new lens while preserving whatever ratio is currently set, so switching cameras doesn't silently reset a user's 16:9 choice). - CameraConfiguration.initialAspectRatio (default 4:3) sets the starting value; CustomCameraController.setAspectRatio() changes it live, transitioning through CameraStatus.switchingCamera the same way switchCamera() does (both rebind the whole use-case group). - Example app: a 4:3/16:9 text-label toggle button between flash and switch-camera. Verified on-device: toggling to 16:9 visibly changes the feed's letterboxed rect (measured 1080x607, ratio 1.779, matching 16:9's 1.778) and a capture taken in that mode produces a 2256x4000 file (ratio 0.564, matching 16:9) that displays correctly in the review screen. 2. Review-screen framing mismatch: even after the earlier aspect-ratio- and rotation-matching fixes, the captured photo shown in the review screen still visibly showed more vertical content than the live feed had framed. Root cause: the review screen's Image.file had no aspect-ratio constraint of its own, so BoxFit's default behavior let it fill the *entire* screen height (taller than the feed's own letterboxed AspectRatio box) rather than an equivalent contained box. Fixed by wrapping it in the same AspectRatio-driven layout the live feed uses (capture.width / capture.height, BoxFit.contain) -- confirmed on-device via precise before/after measurement (feed rect 1080x810 vs. the previously oversized review image) that the two now show the same framing. 46/46 unit tests passing (10 new, covering setAspectRatio's state transitions and initialize()'s pass-through of the configured starting ratio); flutter analyze clean. Co-Authored-By: Claude Sonnet 5 <[email protected]> * Fix reversed preview aspect ratio via CameraState.displayPreviewSize The live feed rendered 4:3 landscape while captureImage() returned a 3:4 portrait image -- the feed's proportions were the reverse of the captured image's. Measured on a Pixel 10: the preview rect was 1080x810 (4:3) for a session whose captures came back 3000x4000 (3:4). previewSize is CameraX's sensor-space resolution, which is landscape on a portrait-held phone (4000x3000) because the sensor is mounted 90 degrees rotated. CustomCameraPreview corrects that with a RotatedBox, so the *displayed* content is portrait -- but the axis swap was private to the widget, so anything sizing a box from previewSize got a landscape box around portrait content. The example app's AspectRatio did exactly that; so would any consumer's. Expose the swap as CameraState.displayPreviewSize and use it for every layout decision, in the widget and the example app alike, so the box a consumer lays out and the content drawn inside it can't disagree. Verified on-device: the feed rect is now 1080x1440 (3:4), matching the 3000x4000 capture, and the review screen frames identical content (same wall strip, arrows, and cropped "E"). The old landscape box had been cropping the "South" label out of the feed entirely. Co-Authored-By: Claude Opus 5 <[email protected]> * Anchor review-screen chrome to real edges instead of mid-screen The close button and the "3000x4000 (back)" metadata label both floated vertically centered over the middle of the photo. The Row holding them sat inside a StackFit.expand Stack, so it stretched to the full screen height and its default centered cross-axis alignment parked both children halfway down the screen -- anchored to nothing. Replace the Row with a Column that pins the close button to the top edge and the metadata caption to the bottom, so each sits in the letterbox rather than on the image. Verified on-device: photo occupies [0,547][1080,1987], close button ends at y=341, metadata starts at y=2259 -- no overlap with the image on either side. The black45 scrims behind both are now invisible against the letterbox, which is the right outcome: they existed only to keep the chrome legible while it sat on the photo. Co-Authored-By: Claude Opus 5 <[email protected]> * Make Android back gesture dismiss the review screen, not the app The review screen is swapped in by state (a conditional return in _CameraDemoScreen.build) rather than pushed as a route, so nothing sat on the navigator stack for Android's back gesture to pop -- back closed the entire app instead of returning to the capture screen. Wrap the review screen in a PopScope with canPop: false, routing the back intent to the same onDismiss callback the close button already uses, so gesture and button stay in agreement. Scoped to the review screen alone, leaving back on the capture screen to exit as usual. Verified on-device: back from the review screen returns to the capture screen with the app still foregrounded (zoom pills and ratio toggle present); back from the capture screen still exits to the launcher; the close button still dismisses. Co-Authored-By: Claude Opus 5 <[email protected]> * Add ready-made UI layer so consumers aren't forced to build their own The package was headless by design -- "fully custom UI composability" -- but that made the composable path the *only* path, so every consumer paid full cost even when they just wanted a camera. The example needed 550 lines to show basic use, which is a fair proxy for what integration actually cost. Add two layers above the primitives, without removing anything: D3CameraScreen -- a complete working camera screen (preview, shutter, zoom, flash, ratio toggle, switch, tap-to-focus, review). Its static show() returns the capture, so the common "go take a photo and come back with it" flow is one call. D3CameraScope -- owns the controller lifecycle (permission, initialize, listen, dispose) and hands over a ready controller, leaving all UI to the caller. The demo widgets move from example/ into lib/src/ui/ and are exported individually (D3ShutterButton, D3ZoomLevelBar, D3FlashButton, D3AspectRatioButton, D3CaptureReviewScreen), so custom UIs can mix built-in chrome with their own. Each layer is built from the one beneath it, so starting at the top costs nothing in flexibility later. This also promotes the on-device fixes from this branch -- rotation correction, displayPreviewSize framing, review/feed parity, and the back-gesture dismissal -- into package behavior rather than leaving them as example-only knowledge every consumer would have to rediscover. Permissions stay the consumer's choice: no permissions-plugin dependency is added; every entry point takes a callback instead. example/ drops 550 -> 201 lines while now demonstrating three usage patterns instead of one. Verified all three on-device: turnkey shows all controls at 4:3, embedded honors initialAspectRatio 16:9 (feed 912x1621), custom UI renders a correct feed with bespoke chrome, and capture works from the custom path. Co-Authored-By: Claude Opus 5 <[email protected]> * Keep the feed full-width at every ratio; cap zoom steps; add pinch Three fixes from on-device feedback. 1. 16:9 shrank the feed instead of growing it. The feed sat in an Expanded + Center, which fits the *available height*, so a taller ratio lost width: measured 912x1621 where the screen is 1080 wide. Anchor it to the top at full width instead and let height follow the ratio, so 4:3 -> 16:9 shows more rather than a smaller letterboxed rect. Now 1080x1440 and 1080x1920 respectively. 2. Controls now overlay the feed at every ratio. They were pinned to the screen bottom, which reads fine at 16:9 (where the feed runs past it) but stranded the zoom bar in the black band below a 4:3 feed. Hug the feed's bottom edge when it ends above the screen. 3. Zoom stopped at the device maximum -- 20x on this phone, mostly digital upscaling and noisy in the bar. Cap the discrete steps at 5x (configurable via D3ZoomLevelBar.maxLevel) and add pinch-to-zoom for the full hardware range. While pinching lands between steps the bar collapses to a live ratio readout instead of showing no selection. Adds the package's first widget tests (7), covering the layout geometry above and pinch behaviour. Pinch could not be verified on-device -- adb's `input` cannot produce simultaneous pointers and `sendevent` is blocked on this unrooted device -- so it is covered by real multi-touch gesture tests instead: spread zooms in, pinch zooms out, single-finger drag does not disturb zoom (protecting tap-to-focus), and none of it fires when disabled. This required a @VisibleForTesting platform seam on D3CameraScope and D3CameraScreen, mirroring CustomCameraController's own, since both previously hardcoded the real method-channel platform and so could not be widget-tested at all. 60/60 tests pass. Verified on-device at both ratios. Co-Authored-By: Claude Opus 5 <[email protected]> * Add close button; unclip zoom labels; re-anchor controls to the screen Three fixes from on-device feedback. 1. The camera had no explicit way out -- only the system back gesture, a dead end on any screen that isn't the app's root. Add a close button (top-left, `showCloseButton`) with an `onClose` callback; without one it pops its own route, which is what D3CameraScreen.show() relies on. It falls through silently rather than popping an app's only route. 2. Zoom pill labels were clipped vertically. The bar was forced into a 32px SizedBox while its own content -- container padding plus pill padding plus text -- needs 77px. Let it size to its content. 3. Controls were anchored to the feed's bottom edge, so they moved when the aspect ratio changed. Anchor them to the screen instead: a fixed position the user can rely on, and the native pattern. This also retires the trade-off flagged last commit, where 4:3 put the shutter on top of the subject. Note (3) reverses the feed-hugging behaviour added in 63d0125, so the widget test asserting it was replaced rather than kept -- it encoded the old intent. The replacement asserts what actually matters: control position is identical across ratios (verified on-device, zoom pill at [402,1833][489,1910] at both 4:3 and 16:9). 65/65 tests pass, including 5 new ones covering the close button's three paths and the two layout properties above. Co-Authored-By: Claude Opus 5 <[email protected]> * Close out Phase 2 verification; expose PreviewFit on D3CameraScreen Settles the three outstanding Definition-of-Done items. PreviewFit.contain re-verified after the rotation fix, via a deliberately square box in the example's custom demo: cover fills it and crops the feed's top/bottom, contain letterboxes and reveals what cover had cropped. That harness is needed because D3CameraScreen sizes its box to the feed's own ratio, making the two modes pixel-identical -- which also surfaced a real gap: PreviewFit was never threaded through D3CameraScreen, so turnkey consumers could not pick contain. Now exposed as `previewFit`. Front-camera mirroring is now covered by widget tests that read the horizontal scale off the render tree (negative front, positive back, and the flip must not disturb displayPreviewSize). Reading the render tree rather than matching widget types keeps them honest through a refactor of the preview's internals. Flash confirmed firing on capture against the same scene used for the ambient test shots: the hard vertical shadow is gone, lighting is even and warm with edge falloff, and the capture is still 3000x4000 and upright. One claim stays narrower than the checklist suggests, and the work item says so: the on-device front-camera pass confirmed orientation (which independently validates the sensor-270/sign+1 rotation case previously flagged as unverified), but the scene was too symmetric to judge mirroring by eye. Mirroring is verified in tests, not visually. 68/68 tests pass (up from 65). Co-Authored-By: Claude Opus 5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5 <[email protected]>
Fixes 90°-off preview rotation without locked capture orientation. Fixes flutter/flutter#154241.
The major changes in this PR:
buildPreviewto return aStatefulWidgetinstead of aTexturethat wouldn't pick up changes in device orientation.handlesCropAndRotationto detect if the preview is already correctly rotated or not. This API was added exactly for that purpose.CameraPreviewwidget).See flutter/flutter#154241 (comment) for slightly more context if desired.
Pre-launch Checklist
dart format.)[shared_preferences]pubspec.yamlwith an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.CHANGELOG.mdto add a description of the change, following repository CHANGELOG style, or this PR is exempt from CHANGELOG changes.///).