gRPC in Flutter crash when no internet

Question

I'm developing a Flutter app using gRPC and everything was working correctly until I decided to see what happens if there is no internet connection.

After doing that and making a request I get following error:

E/flutter (26480): gRPC Error (14, Error making call: Bad state: The http/2 connection is no longer active and can therefore not be used to make new streams.)

The problem is that even after re-enabling the connection, the error still occurs.
Do I have to recreate the clientChannel?

const String serverUrl = 'theaddress.com';
const int serverPort = 50051;

final ClientChannel defaultClientChannel = ClientChannel(
  serverUrl,
  port: serverPort,
  options: const ChannelOptions(
    credentials: const ChannelCredentials.insecure(),
  ),
);

I would simply like to throw some errors, but work properly once the internet connection comes back.

Solution 1

Based on @Ishaan's suggestion, I've used Connectivity package to create a client that reconnects when the internet is back up. So far it seems to be working.

import 'dart:async';

import 'package:connectivity/connectivity.dart';
import 'package:flutter_worker_app/generated/api.pbgrpc.dart';
import 'package:grpc/grpc.dart';
import 'package:rxdart/rxdart.dart';

class ConnectiveClient extends ApiClient {

  final CallOptions _options;
  final Connectivity _connectivity;
  ClientChannel _channel;
  bool hasRecentlyFailed = false;


  ConnectiveClient(this._connectivity, this._channel, {CallOptions options})
      : _options = options ?? CallOptions(),
        super(_channel) {
    //TODO: Cancel connectivity subscription
    _connectivity.onConnectivityChanged.listen((result) {
      if (hasRecentlyFailed && result != ConnectivityResult.none) {
        _restoreChannel();
      }
    });
  }

  ///Create new channel from original channel
  _restoreChannel() {
    _channel = ClientChannel(_channel.host,
        port: _channel.port, options: _channel.options);
    hasRecentlyFailed = false;
  }

  @override
  ClientCall<Q, R> $createCall<Q, R>(
      ClientMethod<Q, R> method, Stream<Q> requests,
      {CallOptions options}) {
    //create call
    BroadcastCall<Q, R> call = createChannelCall(
      method,
      requests,
      _options.mergedWith(options),
    );
    //listen if there was an error
    call.response.listen((_) {}, onError: (Object error) async {
      //Cannot connect - we assume it's internet problem
      if (error is GrpcError && error.code == StatusCode.unavailable) {
        //check connection
        _connectivity.checkConnectivity().then((result) {
          if (result != ConnectivityResult.none) {
            _restoreChannel();
          }
        });
        hasRecentlyFailed = true;
      }
    });
    //return original call
    return call;
  }

  /// Initiates a new RPC on this connection.
  /// This is copy of [ClientChannel.createCall]
  /// The only difference is that it creates [BroadcastCall] instead of [ClientCall]
  ClientCall<Q, R> createChannelCall<Q, R>(
      ClientMethod<Q, R> method, Stream<Q> requests, CallOptions options) {
    final call = new BroadcastCall(method, requests, options);
    _channel.getConnection().then((connection) {
      if (call.isCancelled) return;
      connection.dispatchCall(call);
    }, onError: call.onConnectionError);
    return call;
  }
}

///A ClientCall that can be listened multiple times
class BroadcastCall<Q, R> extends ClientCall<Q, R> {
  ///I wanted to use super.response.asBroadcastStream(), but it didn't work.
  ///I don't know why...
  BehaviorSubject<R> subject = BehaviorSubject<R>();

  BroadcastCall(
      ClientMethod<Q, R> method, Stream<Q> requests, CallOptions options)
      : super(method, requests, options) {
    super.response.listen(
          (data) => subject.add(data),
          onError: (error) => subject.addError(error),
          onDone: () => subject.close(),
        );
  }

  @override
  Stream<R> get response => subject.stream;
}

Solution 2

I guess you're one of the few people attempting it.

GRPC connections take a bit of time to create a new connection, not just in dart, but all other languages. If you want, you can put a catch listener on the error code 14 and manually kill the connection and re-connect. There's also idleTimeout channel option that might be of help to you, the default for which is 5 mins in grpc-dart

There was a fix for the unexpted crash issue https://github.com/grpc/grpc-dart/issues/131 , so do try to update your dependencies (grpc-dart) which will prevent the crash, but the problem of reconnection on network might still remain.

After this fix the crashes have stopped, but the stale connection issue does remain for me too. I've resorted to showing snackbars with sentences like "Cannot connect to servers, please try again in a few minutes".

Solution 3

I haven't been using gRPC until today.

Since I've taken time to try to simulate this error I'll post here my answer but all my intel has been lead by @ishann answer that I've upvoted and that should be the accepted one.

I've just tried dart hello world example.

I've the server running on my machine and the client as a Flutter application.

When I don't run the server I get the error

gRPC Error (14, Error connecting: SocketException:

But as soon as the server goes up, all start working as expected, but then I've realized that I was recreating the channel every time, so that's not the OP scenario.

That's my fist Flutter code:

void _foo() async {
  final channel = new ClientChannel('192.168.xxx.xxx',
      port: 50051,
      options: const ChannelOptions(
          credentials: const ChannelCredentials.insecure()));
  final stub = new GreeterClient(channel);

  final name = 'world';

  var _waitHelloMessage = true;
  while (_waitHelloMessage) {
    try {
      final response = await stub.sayHello(new HelloRequest()..name = name);
      print('Greeter client received: ${response.message}');
      _waitHelloMessage = false;
    } catch (e) {
      print('Caught error: $e');
      sleep(Duration(seconds: 1));
    }
  }
  print('exiting');
  await channel.shutdown();
}

Same behaviour if I put device in airplain mode and than switch back to normal wifi/lte connection.

With this other playground project instead I've reproduced either

Caught error: gRPC Error (14, Error making call: Bad state: The http/2 connection is no longer active and can therefore not be used to make new streams.)

From which you cannot come up without recreating the channel, and

Caught error: gRPC Error (14, Error connecting: SocketException: OS Error: Connection refused, errno = 111, address = 192.168.1.58, port = 38120)

(for example shut down the server) from which instead you can get up again without recreating the channel.

The former error code it's not so easy to get because it seems that the channel throttle between wifi and lte connection.

import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter_app_test_grpc/grpc/generated/helloworld.pbgrpc.dart';
import 'package:grpc/grpc.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;
  ClientChannel _channel;

  @override
  void dispose() {
    _shutdown();
    super.dispose();
  }

  void _shutdown() async {
    if (null != _channel) {
      print('shutting down...');
      await _channel.shutdown();
      print('shut down');
      _channel = null;
    } else {
      print ('connect first');
    }
  }

  void _connect() {
    print('connecting...');
    _channel = new ClientChannel('192.168.xxx.xxx',
        port: 50051,
        options: const ChannelOptions(
            credentials: const ChannelCredentials.insecure()));
    print('connected');
  }

  void _sayHello() async {
    if (_channel != null) {
      final stub = new GreeterClient(_channel);

      final name = 'world';

      try {
        final response = await stub.sayHello(new HelloRequest()..name = name);
        print('Greeter client received: ${response.message}');
      } catch (e) {
        print('Caught error: $e');
        //sleep(Duration(seconds: 2));
      }

      //print('exiting');
      //await channel.shutdown();
    } else {
      print('connect first!');
    }
  }

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.display1,
            ),
          ],
        ),
      ),
      floatingActionButton: Padding(
        padding: const EdgeInsets.only(left: 36.0),
        child: Row(
          children: <Widget>[
            Padding(
              padding: const EdgeInsets.all(8.0),
              child: FloatingActionButton(
                onPressed: _connect,
                tooltip: 'Increment',
                child: Icon(Icons.wifi),
              ),
            ),
            Padding(
              padding: const EdgeInsets.all(8.0),
              child: FloatingActionButton(
                onPressed: _sayHello,
                tooltip: 'Increment',
                child: Icon(Icons.send),
              ),
            ),
            Padding(
              padding: const EdgeInsets.all(8.0),
              child: FloatingActionButton(
                onPressed: _shutdown,
                tooltip: 'Increment',
                child: Icon(Icons.close),
              ),
            ),
          ],
        ),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

That's my flutter doctor -v if could be of any help:

$ flutter doctor -v
[✓] Flutter (Channel beta, v1.0.0, on Mac OS X 10.14.1 18B75, locale en-IT)
    • Flutter version 1.0.0 at /Users/shadowsheep/flutter/flutter
    • Framework revision 5391447fae (6 weeks ago), 2018-11-29 19:41:26 -0800
    • Engine revision 7375a0f414
    • Dart version 2.1.0 (build 2.1.0-dev.9.4 f9ebf21297)

[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
    • Android SDK at /Users/shadowsheep/Library/Android/sdk
    • Android NDK location not configured (optional; useful for native profiling support)
    • Platform android-28, build-tools 28.0.3
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)
    • All Android licenses accepted.

[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    • ios-deploy 1.9.4
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 31.3.3
    • Dart plugin version 182.5124
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1248-b01)

[✓] VS Code (version 1.30.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.21.1

[✓] Connected device (1 available)
    [...]

• No issues found!

Comments

[Marcin Szałek]

And when do you restore the connection?

[Marcin Szałek]

But it still doesn't fix the problem on how to automatically restore the connection, right?

[shadowsheep]

@MarcinSzałek yep, right! I do confirm here that when you get the error 14 with http/2 connection is no longer active you have to recreate the channel. In this actual scenario you cannot reconnect automatically. With the socket error instead you are able to reconnect automatically.

[ishaan]

I'd use this: pub.dartlang.org/packages/connectivity to listen to network availability and then try to reconnet. You can design a small lib that can automatically do this for GRPC. So, when the network isn't available, just show a snackbar that network isn't available or something similar. And when it is, you'll always have a valid network connection. I haven't done this yet, its in my pipeline, will post some sample code once I'm done with it! :)

[ishaan]

I'm glad it worked out for you! Maybe drop a pub package for it? ;)

[Marcin Szałek]

Can't do because this class needs to extend your generated Client class :(

[etzuk]

Great solution, at the beginning I tried to use super.response.asBroadcastStream() as well but with the same error :( I try to handle StatusCode.unauthenticated in one place to be able to refresh the token every time I received this error. If you successfully managed this without BroadcaseCall please update:)

[Şükriye]

@ishaan hi, did you have success with reconnecting with GRPC using flutter/dart?

[man of knowledge]

Connectivity package does not check if there is an internet connection, it uses MethodChannel to determine which type of connection the phone uses like Wifi or Cellular data. if the phone is connected to the internet through the router and the router does not have access to the internet(ISP cut you off because of the bill), connectivity package will say that you are connected to the internet because you are still using Wifi service on your phone, when in reality you can't send any request outside of the router.