Flutter : Overlaying the graphics on the barcode which is detected by firebase vision

Question

I am trying to implement a barcode scanner app in Flutter where the camera can be embedded as a widget rather than making it fullscreen.

So I went with the flutter_qr_mobile_vision package as we can use the native camera features and also restricting the camera preview size.

But the above package gives only the barcode string which is detected by the MLKit and not the barcode details like bounding box, barcode type, and other information, so I took the source code of the above package and made some changes so that I get the bounding box which can be used to overlay a graphic on the detected barcode(s).

The code for what im trying to do can be found here

Now I'm facing two problems while using these changes (Android implementation only as of now)

1. When I'm trying to overlay the bounding box on top of the barcode, results are inconsistent across multiple devices.

For example:

Works fine on oneplus in fullscreen

Same code output on Redmi 5

I have kept the camera resolution by default as (1280 x 720) for all the devices and scaling the output as well before overlaying

I'm trying to understand what is causing this on different devices

2. Now If I try to resize the widget height of the camera preview, the results are inconsistent on oneplus as well

I feel this is something to do with the scaling part but im not sure.

Below code is for camera preview and overlay graphic

import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';
import 'package:qr_mobile_vision/qr_camera.dart';
import 'package:qr_mobile_vision/qr_barcode.dart';
import 'package:vision_demo/barcode_detctor_painter.dart';
import 'package:vision_demo/overlay.dart';

void main() {
  debugPaintSizeEnabled = false;
  runApp(new HomePage());
}

class HomePage extends StatefulWidget {
  @override
  HomeState createState() => new HomeState();
}

class HomeState extends State<HomePage> {
  @override
  Widget build(BuildContext context) {
    return new MaterialApp(home: new MyApp());
  }
}

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => new _MyAppState();
}

class _MyAppState extends State<MyApp> {
  bool camState = false;
  List<Barcode> barcode = List<Barcode>();

  @override
  initState() {
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return new Scaffold(
      appBar: new AppBar(
        title: new Text('Demo app'),
      ),
      body: new Center(
        child: new Column(
          crossAxisAlignment: CrossAxisAlignment.center,
          mainAxisAlignment: MainAxisAlignment.start,
          children: <Widget>[
            camState
                ? new SizedBox(
                    width: MediaQuery.of(context).size.width,
                    height: MediaQuery.of(context).size.height - 300 - AppBar().preferredSize.height,
                    child: Stack(
                      children: [
                        new QrCamera(
                          onError: (context, error) => Text(
                            error.toString(),
                            style: TextStyle(color: Colors.red),
                          ),
                          qrCodeCallback: (code) {
                            setState(() {
                              barcode = code;
                            });
                          },
                          child: new Container(
                            decoration: new BoxDecoration(
                              color: Colors.transparent,
                            ),
                          ),
                        ),
                        Container(
                            constraints: const BoxConstraints.expand(),
                            decoration: ShapeDecoration(
                              shape: QrScannerOverlayShape(cutOutSize: MediaQuery.of(context).size.width - 160, borderColor: Colors.white),
                            )),
                        LayoutBuilder(builder: (BuildContext context, BoxConstraints constraints) {
                          return _buildResults(constraints);
                        })
                      ],
                    ),
                  )
                : Expanded(child: new Center(child: new Text("Camera inactive"))),
          ],
        ),
      ),
      floatingActionButton: new FloatingActionButton(
          child: new Text(
            "press me",
            textAlign: TextAlign.center,
          ),
          onPressed: () {
            setState(() {
              camState = !camState;
            });
          }),
    );
  }

  Widget _buildResults(BoxConstraints constraints) {
    const Text noResultsText = Text('No results!');

    if (barcode == null) {
      return noResultsText;
    }

    CustomPainter painter;

    final Size imageSize = Size(720.0, 1280.0);

    if (barcode is! List<Barcode>) return noResultsText;
    painter = BarcodeDetectorPainter(imageSize, barcode);

    return CustomPaint(
      size: Size(double.maxFinite, double.maxFinite),
      painter: painter,
    );
  }
}

Below is the code im using to resize for overlaying the graphics.

import 'package:flutter/material.dart';
import 'package:qr_mobile_vision/qr_barcode.dart';

class BarcodeDetectorPainter extends CustomPainter {
  BarcodeDetectorPainter(this.absoluteImageSize, this.barcodeLocations); // absoluteImageSize will always be (1280 x 720) 

  final List<Barcode> barcodeLocations;
  final Size absoluteImageSize;

  @override
  void paint(Canvas canvas, Size size) {
    final double scaleX = size.width / absoluteImageSize.width;
    final double scaleY = size.height / absoluteImageSize.height;

    Rect scaleRect(Barcode barcode) {
      return Rect.fromLTRB(
        barcode.boundingBox.left * scaleX,
        barcode.boundingBox.top * scaleY,
        barcode.boundingBox.right * scaleX,
        barcode.boundingBox.bottom * scaleY,
      );
    }

    final Paint paint = Paint()
      ..style = PaintingStyle.fill
      ..strokeWidth = 2.0;

    for (Barcode barcode in barcodeLocations) {
      paint.color = Colors.green;

      canvas.drawRect(scaleRect(barcode), paint);
    }
  }

  @override
  bool shouldRepaint(BarcodeDetectorPainter oldDelegate) {
    return oldDelegate.absoluteImageSize != absoluteImageSize || oldDelegate.barcodeLocations != barcodeLocations;
  }
}

Resolving these 2 issue is important because after this i want to restrict the scanning area so that i can capture only the barcode which is aligned on the cutout size that I have added.

Any sort of help is appreciated.

Solution

I was able to solve this issue, the problem is having a constant resolution throughout multiple devices, some devices might not have the resolution that I have set, so programmatically setting the best minimum resolution based on the screen size has resolved my problem.