Support Texture widget

We have ported the WebRTC plugin for iOS/Android, but there is no Texture widget in flutter-desktop-embedding, Texture is needed to support video rendering and video player development.

Yes, use C++ to write the plugins for all desktops.

@chinmaygarde

From a quick skim of the existing implementations it looks like Texture implementation requires access to engine APIs that aren't currently exposed via the embedding API.

@chinmaygarde Is that accurate?

@cloudwebrtc

Apologies for the late reply.

Support for external textures to embedders is definitely something on the roadmap. Currently, external textures are handled very differently on iOS (CoreVideo) and Android (GL_OES_EGL_image_external) and so exposing the same to embedders is not straightforward. It would be great if we could collaborate on figuring out a few things first. For example, should the embedder API try to create an abstraction over textures on the various platforms or support platform specific APIs? In the specific case of the WebRTC plugin linked, I suspect the easiest way for the platform to express its textures would be in the form of CoreVideo pixel buffers. OTOH, on Linux, I suspect EGL images would be better. Should the embedder API support both platform specific APIs? Or, should the engine only deal with plain old GL_TEXTURE_2D objects and have the embedder render into the same. That would require sharing of the OpenGL context used by Flutter with the embedder or creating another context within the same sharegroup.

@cloudwebrtc: I think exposing the current support for CoreVideo and EGL_Image_External as-is would be the easiest way to support your use case.

@gedw99: How are you going to generate and update the textures with the graphics you mentioned? Are you going to push new textures (from a pool) to Flutter each frame or do you want to update the same texture? Are there threading restrictions to the way you are going to be generating your textures? Can you provide a reduced test case for how you want to generate your textures? That would be extremely helpful to try to pin down an API that works for the broadest audience. Thanks!

Engine patch with example embedder implementation has been posted flutter-team-archive/engine#7087.

@chinmaygarde

@chinmaygarde I used your branch to create a webrtc plugin example that seems to work fine, thank you.
@stuartmorgan @gedw99
I modified some code:
1, FLETextureDelegate needs to be placed in the FLEViewController.h public declaration in order to provide it to the external framework to link it.
https://github.com/cloudwebrtc/flutter-desktop-embedding/blob/flutter-webrtc/library/macos/FLEViewController.h#L21
2. I added a FlutterEventChannel object to adapt the listen method of the dart layer EventChannel.
https://github.com/cloudwebrtc/flutter-desktop-embedding/blob/flutter-webrtc/plugins/flutter_webrtc/macos/FlutterEventChannel.h

/**
 * An event sink callback.
 *
 * @param event The event.
 */
typedef void (^FlutterEventSink)(id _Nullable event);

extern NSObject const* FlutterEndOfEventStream;

@protocol FlutterStreamHandler

- (FLEMethodError* _Nullable)onListenWithArguments:(id _Nullable)arguments
                                       eventSink:(FlutterEventSink)events;

- (FLEMethodError* _Nullable)onCancelWithArguments:(id _Nullable)arguments;

@end

@interface FlutterEventChannel : NSObject

+ (instancetype)eventChannelWithName:(NSString*)name
                     binaryMessenger:(NSObject<FLEBinaryMessenger>*)messenger;

- (void)setStreamHandler:(NSObject<FlutterStreamHandler>* _Nullable)handler;
@end

added a FlutterTextureRegistry class, adapted to the iOS-like Texture interface.
https://github.com/cloudwebrtc/flutter-desktop-embedding/blob/flutter-webrtc/plugins/flutter_webrtc/macos/FlutterTextureRegistry.h

@protocol FlutterTexture<NSObject>
- (CVPixelBufferRef _Nullable)copyPixelBuffer;
@end

@protocol FlutterTextureRegistry<NSObject>
- (int64_t)registerTexture:(NSObject<FlutterTexture> *)texture;
- (void)textureFrameAvailable:(int64_t)textureId;
- (void)unregisterTexture:(int64_t)textureId;
@end

NS_ASSUME_NONNULL_BEGIN

@interface FLETextureRegister : NSObject<FlutterTextureRegistry, FLETextureDelegate>
@property(nullable, weak) FLEViewController *controller;
-(instancetype) initWithController:(FLEViewController *)controller;
@end

NS_ASSUME_NONNULL_END

Should I add FlutterEventChannel and FlutterTextureRegistry to FlutterEmbedderMac.framework?

Thanks again, the Linux/Win plugin should be easier to port, just copy the FlutterOpenGLTexture object and provide the GL_RGBA8 buffer.

For testing please refer to:flutter-webrtc/flutter-webrtc#37 (comment)

screenshots

Should I add FlutterEventChannel and FlutterTextureRegistry to FlutterEmbedderMac.framework?

I'd be happy to accept a patch that adds FLEEventChannel (the naming convention I've been using for the macOS code in this project is to use FLE as a prefix, replacing Flutter when the iOS class starts with Flutter) and related classes to the framework!

It'll need to wait to land until the engine change lands though, and then the texture APIs will need to be added to the macOS framework (which will likely have some changes from what's in that branch), but feel free to start the PR whenever you like and it go through review in the meantime.

Okay, I created a PR for FELEventChannel. #174

Engine support landed in flutter-team-archive/engine#7087

New PR for FLETexture on [macOS] #263.

@cloudwebrtc

Moved to flutter/flutter#30717 and flutter/flutter#30718

@cloudwebrtc If you are still interested in moving forward with #263, 30717 would be the bug to associate a flutter/engine PR with.

How I understand for start work with flutter Texture widget, I must get TextureRegister from plugin, but for linux target platform in PluginRegistrar or FlutterDesktopPluginRegistrarRef I can't see method or ref like Registrar::texture() for get TextureRegistry

See my last comment; this bug is closed because the relevant code is in Flutter now, and the open bugs tracking the work are there now. The right place for your question is flutter/flutter#30718 (where there's a link to a patch with an implementation, which should answer your question).

Is texture widget supported on Windows at all?

No, see flutter/flutter#38601

Navigation Menu

Repository navigation

Support Texture widget #107

screenshots

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Support Texture widget #107

Description

Activity

cloudwebrtc commented on Jul 13, 2018

stuartmorgan-g commented on Jul 13, 2018

chinmaygarde commented on Sep 20, 2018

chinmaygarde commented on Dec 4, 2018

cloudwebrtc commented on Dec 16, 2018

screenshots

stuartmorgan-g commented on Dec 16, 2018

cloudwebrtc commented on Dec 17, 2018

chinmaygarde commented on Jan 9, 2019

cloudwebrtc commented on Feb 1, 2019

stuartmorgan-g commented on Apr 8, 2019

tttzof351 commented on May 3, 2019

stuartmorgan-g commented on May 4, 2019

chichid commented on Oct 6, 2019

stuartmorgan-g commented on Oct 6, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions