You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Android][Impeller]we need to be notified before destroying the surface #152839
I've read the migration guide and the migrated source of video_player.
Solution for the image loss problem in video_player:
In onSurfaceDestroyed, save and destroy the state of ExoPlayer, and then restore it in onSurfaceCreated. However, I believe this is a rather expensive operation, especially since the restoration process may cause noticeable pauses for the user. More importantly, it requires the video to stop playing, which may not be acceptable for some apps.
Therefore, I think a more efficient method should be used to solve this problem.
Proposal
Introduce an onSurfaceDestroying event.
In this event, we can remove the destroying surface from ExoPlayer to prevent errors. Then just call SetVideoSurface() in onSurfaceCreated (or even onSurfaceDestroyed). That should be able to avoid the steps of saving, destroying and restoring the state of ExoPlayer.
changed the title [-][Android][Impeller]we need to be notified before the destroying surface[/-][+][Android][Impeller]we need to be notified before destroying the surface[/+]on Aug 4, 2024
@jonahwilliams Thanks for reply
The difference is that in onSurfaceDestroying, you can do things that are not possible in onSurfaceDestroyed, such as saving the current frame. Additionally, since the surface has already been destroyed in onSurfaceDestroyed, ExoPlayer may encounter errors, meaning that the player instance also has to be destroyed. However, if you can receive a notification beforehand, you have the opportunity to avoid this issue.
It would not be any more expensive than creating the surface and plugin for the first time (in practice it should be less).
The cost indeed won’t be greater than the first time it was opened, but it still falls short of providing a seamless playback experience. This is because the media file may need to be reopened and buffered. Moreover, this issue only exists in Android.
What errors? Do you have an example of a reproduction where an error occurs as a result of the migration?
This is the error that occurs when I call clearVideoSurface in onSurfaceDestroyed. In fact, the error occurs whether or not clearVideoSurface is called. So, I believe that removing the Surface from ExoPlayer at this point might already be too late. To avoid the error, it might only be possible by saving, destroying, and restoring ExoPlayer’s state. I suspect this might be the reason why you did it this way.
To migrate to SurfaceProducer, users should avoid caching Surface instances and instead use the getSurface() method each time. However, this may not always be practical in real-world scenarios. This is because a Surface instance might need to retain user-set states, or libraries like ExoPlayer may inherently require caching the Surface instance. To maintain the usability of Texture widget, I believe it is still necessary to introduce an event that ensures users have the opportunity to prevent the following errors from occurring.
java.lang.IllegalStateException: releaseOutputBuffer() is valid only at Executing states; currently at Released state
at android.media.MediaCodec.releaseOutputBuffer(Native Method)
at android.media.MediaCodec.releaseOutputBufferInternal(MediaCodec.java:4173)
at android.media.MediaCodec.releaseOutputBuffer(MediaCodec.java:4147)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter.releaseOutputBuffer(AsynchronousMediaCodecAdapter.java:206)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.renderOutputBufferV21(MediaCodecVideoRenderer.java:1716)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.maybeReleaseFrame(MediaCodecVideoRenderer.java:1448)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.processOutputBuffer(MediaCodecVideoRenderer.java:1419)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:2070)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:873)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.render(MediaCodecVideoRenderer.java:1018)
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1136)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:561)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
2024-08-14 18:16:11.564 4369-4466 ExoPlayerImplInternal dev.xx.av_media_player_example E Playback error
androidx.media3.exoplayer.ExoPlaybackException: MediaCodecVideoRenderer error, index=0, format=Format(0, null, null, video/avc, avc1.64001F, -1, null, [848, 480, -1.0, ColorInfo(Unset color space, Unset color range, Unset color transfer, false, 8bit Luma, 8bit Chroma)], [-1, -1]), format_supported=YES
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:640)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
Caused by: androidx.media3.exoplayer.video.MediaCodecVideoDecoderException: Decoder failed: c2.goldfish.h264.decoder
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.createDecoderException(MediaCodecVideoRenderer.java:1982)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:893)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.render(MediaCodecVideoRenderer.java:1018)
at androidx.media3.exoplayer.ExoPlayerImplInternal.doSomeWork(ExoPlayerImplInternal.java:1136)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:561)
... 4 more
Caused by: java.lang.IllegalStateException: releaseOutputBuffer() is valid only at Executing states; currently at Released state
at android.media.MediaCodec.releaseOutputBuffer(Native Method)
at android.media.MediaCodec.releaseOutputBufferInternal(MediaCodec.java:4173)
at android.media.MediaCodec.releaseOutputBuffer(MediaCodec.java:4147)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter.releaseOutputBuffer(AsynchronousMediaCodecAdapter.java:206)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.renderOutputBufferV21(MediaCodecVideoRenderer.java:1716)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.maybeReleaseFrame(MediaCodecVideoRenderer.java:1448)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.processOutputBuffer(MediaCodecVideoRenderer.java:1419)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.drainOutputBuffer(MediaCodecRenderer.java:2070)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.render(MediaCodecRenderer.java:873)
... 7 more
2024-08-14 18:16:11.582 4369-4466 VideoFrameReleaseHelper dev.xx.av_media_player_example E Failed to call Surface.setFrameRate
java.lang.IllegalStateException: Surface has already been released.
at android.view.Surface.checkNotReleasedLocked(Surface.java:860)
at android.view.Surface.setFrameRate(Surface.java:1068)
at android.view.Surface.setFrameRate(Surface.java:1112)
at androidx.media3.exoplayer.video.VideoFrameReleaseHelper$Api30.setSurfaceFrameRate(VideoFrameReleaseHelper.java:440)
at androidx.media3.exoplayer.video.VideoFrameReleaseHelper.clearSurfaceFrameRate(VideoFrameReleaseHelper.java:385)
at androidx.media3.exoplayer.video.VideoFrameReleaseHelper.onStopped(VideoFrameReleaseHelper.java:235)
at androidx.media3.exoplayer.video.VideoFrameReleaseControl.onStopped(VideoFrameReleaseControl.java:231)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.onStopped(MediaCodecVideoRenderer.java:777)
at androidx.media3.exoplayer.BaseRenderer.stop(BaseRenderer.java:209)
at androidx.media3.exoplayer.ExoPlayerImplInternal.ensureStopped(ExoPlayerImplInternal.java:1836)
at androidx.media3.exoplayer.ExoPlayerImplInternal.disableRenderer(ExoPlayerImplInternal.java:1845)
at androidx.media3.exoplayer.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:1566)
at androidx.media3.exoplayer.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:1523)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:686)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
2024-08-14 18:16:11.582 4369-4490 MediaCodec dev.xx.av_media_player_example E flush() is valid only at Executing states; currently at Released state
2024-08-14 18:16:11.583 4369-4466 ExoPlayerImplInternal dev.xx.av_media_player_example E Disable failed.
java.lang.IllegalStateException: flush() is valid only at Executing states; currently at Released state
at android.media.MediaCodec.native_flush(Native Method)
at android.media.MediaCodec.flush(MediaCodec.java:2631)
at androidx.media3.exoplayer.mediacodec.AsynchronousMediaCodecAdapter.flush(AsynchronousMediaCodecAdapter.java:247)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.flushCodec(MediaCodecRenderer.java:960)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.flushOrReleaseCodec(MediaCodecRenderer.java:953)
at androidx.media3.exoplayer.mediacodec.MediaCodecRenderer.onDisabled(MediaCodecRenderer.java:780)
at androidx.media3.exoplayer.video.MediaCodecVideoRenderer.onDisabled(MediaCodecVideoRenderer.java:794)
at androidx.media3.exoplayer.BaseRenderer.disable(BaseRenderer.java:220)
at androidx.media3.exoplayer.ExoPlayerImplInternal.disableRenderer(ExoPlayerImplInternal.java:1846)
at androidx.media3.exoplayer.ExoPlayerImplInternal.resetInternal(ExoPlayerImplInternal.java:1566)
at androidx.media3.exoplayer.ExoPlayerImplInternal.stopInternal(ExoPlayerImplInternal.java:1523)
at androidx.media3.exoplayer.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:686)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.loopOnce(Looper.java:232)
at android.os.Looper.loop(Looper.java:317)
at android.os.HandlerThread.run(HandlerThread.java:85)
This is the error that occurs when I call clearVideoSurface in onSurfaceDestroyed. In fact, the error occurs whether or not clearVideoSurface is called.
Why are you calling this? AFAICT this is not in migrated PR (which also doesn't crash).
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
Activity
[-][Android][Impeller]we need to be notified before the destroying surface[/-][+][Android][Impeller]we need to be notified before destroying the surface[/+]jonahwilliams commentedon Aug 4, 2024
It's not really clear what you are proposing? What is the difference between onSurfaceDestroying and the existing lifecycle?
xxoo commentedon Aug 4, 2024
@jonahwilliams Thanks for reply
The difference is that in
onSurfaceDestroying, you can do things that are not possible inonSurfaceDestroyed, such as saving the current frame. Additionally, since the surface has already been destroyed inonSurfaceDestroyed,ExoPlayermay encounter errors, meaning that the player instance also has to be destroyed. However, if you can receive a notification beforehand, you have the opportunity to avoid this issue.xxoo commentedon Aug 4, 2024
@jonahwilliams I have updated the proposal to make it more clear. Please take another look thanks
8 remaining items
xxoo commentedon Aug 14, 2024
@matanlurey Thanks for reply
The cost indeed won’t be greater than the first time it was opened, but it still falls short of providing a seamless playback experience. This is because the media file may need to be reopened and buffered. Moreover, this issue only exists in Android.
This is the error that occurs when I call
clearVideoSurfaceinonSurfaceDestroyed. In fact, the error occurs whether or notclearVideoSurfaceis called. So, I believe that removing theSurfacefrom ExoPlayer at this point might already be too late. To avoid the error, it might only be possible by saving, destroying, and restoring ExoPlayer’s state. I suspect this might be the reason why you did it this way.To migrate to
SurfaceProducer, users should avoid cachingSurfaceinstances and instead use thegetSurface()method each time. However, this may not always be practical in real-world scenarios. This is because aSurfaceinstance might need to retain user-set states, or libraries like ExoPlayer may inherently require caching theSurfaceinstance. To maintain the usability ofTexturewidget, I believe it is still necessary to introduce an event that ensures users have the opportunity to prevent the following errors from occurring.matanlurey commentedon Aug 14, 2024
Why are you calling this? AFAICT this is not in migrated PR (which also doesn't crash).
matanlurey commentedon Aug 14, 2024
(At this point I need a specific code sample, i.e. "in this PR X, if I do Y, I observe a crash".)
xxoo commentedon Aug 15, 2024
@matanlurey
Because I tried to avoid saving and restoring the state of ExoPlayer
Please use this repo to reproduce the error
And visit here to see what was changed
I made the necessary changes on video_player_android 2.6.0
steps:
matanlurey commentedon Aug 15, 2024
That is not possible, and is the point of the PR.
At this point I am going to close this bug as WAI - when the PR merges, if there are problems, we can address them.
xxoo commentedon Aug 15, 2024
In the current api set, it is impossible indeed. And this is the reason I opened this proposal.
github-actions commentedon Aug 29, 2024
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.