How to call flutter engine methods from another thread

Question

I'm using flutter desktop for linux. I'm calling a method called MarkTextureFrameAvailable that is supposed to mark a texture to be rerendered by the engine. Since I'm programming a video player, I need to call MarkTextureFrameAvailable from the player's thread. The problem is that the engine forces me to call MarkTextureFrameAvailable (and any other engine method) from the thread which created the engine.

You can see that all calls to the engine end up in the shell which always do checks to see if the calls are being made from the same thread that created the call:

task_runners_.GetPlatformTaskRunner()->RunsTasksOnCurrentThread()

(https://github.com/flutter/engine/blob/master/shell/common/shell.cc#L838)

This is how I'm creating the flutter engine:

int main(int argc, char **argv) {
  //..

  flutter::FlutterWindowController flutter_controller(icu_data_path);

  // Start the engine.
  if (!flutter_controller.CreateWindow(800, 600, "Flutter WebRTC Demo", assets_path,
                                       arguments)) {
    return EXIT_FAILURE;
  }

  // Register any native plugins.
  FlutterWebRTCPluginRegisterWithRegistrar(
      flutter_controller.GetRegistrarForPlugin("FlutterWebRTCPlugin"));

  // Run until the window is closed.
  flutter_controller.RunEventLoop();
  return EXIT_SUCCESS;
}

as you can see, the thread which creates the engine gets blocked by flutter_controller.RunEventLoop(); which is the only place I could put an event dispatcher that forced things to be executed from main's thread. I don't like this idea. Even though RunEventLoopWithTimeout exists, I need to put a timeout and keep checking in a queue for MarkTextureFrameAvailable calls. I don't think this is optimal.

So how should I call MarkTextureFrameAvailable from the main thread?

I found an example of usage of MarkTextureFrameAvailable here: https://github.com/cloudwebrtc/flutter-webrtc/blob/desktop/common/src/flutter_video_renderer.cc#L90 and it looks like it's another thread which calls it. How is it possible? When I do, I get a FATAL error, but he does and it works?

I spent two days trying to figure out which thread calls OnFrame on this example but couldn't find out because it uses https://github.com/flutter-webrtc/libwebrtc which uses google's webrtc: https://github.com/JumpingYang001/webrtc which is too big for me to find where OnFrame is called from. But it must me from a thread. How is it possible?

Solution

See my comment for the caveat to this answer. It appears that the example project you provided accomplishes this with a simple trick. They create a new class that inherits the flutter renderer class, overriding OnFrame among other things. When that override gets called it's in the context of the Flutter thread and so works as expected.

Comments

[PPP]

Thank you so much by your effort to help! However, the class from Flutter that it subclasses is Texture, which does not have onFrame method. That onFrame method is from RTCVideoRenderer which has nothing to do with Flutter.

[Pickle Rick]

See here. Looks like they do override Flutter, and possibly using a supported plugin system. My best advice would be to read Flutter documentation. I would be willing to bet if you placed a BP in OnFrame it would be in the context of the Flutter thread as expected.

[PPP]

I'm aleady using the plugin system and I've read the documentation, there's no onFrame method. The only thing from Flutter in FlutterVideoRenderer class is Texture, which has no onFrame method. Actually I found where onFrame is defined, and is a thing from WebRTC library as you can see here github.com/JumpingYang001/webrtc/…

[4LegsDrivenCat]

@LucasZanella Pickle Rick is right, FlutterVideoRenderer inherits from two classes - Texture (line 16) and RTCVideoRenderer<scoped_refptr<RTCVideoFrame>> (line 17) (C++ allows multiple inheritance) and then it overrides OnFrame method (line 24). When OnFrame event is fired, it is executed in the context of your main thread.

[PPP]

@4LegsDrivenCat but OnFrame is not from Flutter, it's from RTCVideoRenderer which is not from Flutter

[Josh Chiu]

has anyone solve this? I'm also encounter the same problem.