Integrating a custom AutoML tflite model with flutter app
1,136
@Shubham It appears that the exceptions exist anyhow, even I use the method:
Uint8List imageToByteListFloat32(img.Image image, int inputSize, double mean, double std) {
var convertedBytes = Float32List(1 * inputSize * inputSize * 3 );
var buffer = Float32List.view(convertedBytes.buffer);
int pixelIndex = 0;
for (var i = 0; i < inputSize; i++) {
for (var j = 0; j < inputSize; j++) {
var pixel = image.getPixel(j, i);
buffer[pixelIndex++] = ((img.getRed(pixel) - mean) / std).toDouble();
buffer[pixelIndex++] = ((img.getGreen(pixel) - mean) / std).toDouble();
buffer[pixelIndex++] = ((img.getBlue(pixel) - mean) / std).toDouble();
}
}
return convertedBytes.buffer.asUint8List();
}
Author by
Hazel Wang
Updated on December 17, 2022Comments
-
Hazel Wang over 1 year
I am new to Flutter, basically, I followed a tutorial online to train a custom image labeling model with Google's AutoML API then downloaded the model as three files(dict.txt, manifest.json, model.tflite), and now I am trying to integrate it with my flutter application.
Here is my code to load and run the TFlite model:
Future loadModel() async { try{ res = await Tflite.loadModel( model: "assets/models/model.tflite", labels: "assets/models/dict.txt", ); print("loading tf model..."); print(res); }on PlatformException{ print ("Failed to load model"); } } Future recognizeImageBinary(File image) async { var imageBytes = await image.readAsBytesSync(); var bytes = imageBytes.buffer.asUint8List(); img.Image oriImage = img.decodeJpg(bytes); img.Image resizedImage = img.copyResize(oriImage, height: 112, width: 112); var recognitions = await Tflite.runModelOnBinary( binary: imageToByteListUint8(resizedImage, 112), numResults: 2, threshold: 0.4, asynch: true ); setState(() { _recognitions = recognitions; }); }
According to the tutorial, AutoML custom trained model is with the type Uint8, so I used the function below to convert it:
Uint8List imageToByteListUint8(img.Image image, int inputSize) { var convertedBytes = Uint8List(4 * inputSize * inputSize * 3); var buffer = Uint8List.view(convertedBytes.buffer); int pixelIndex = 0; for (var i = 0; i < inputSize; i++) { for (var j = 0; j < inputSize; j++) { var pixel = image.getPixel(j, i); buffer[pixelIndex++] = img.getRed(pixel); buffer[pixelIndex++] = img.getGreen(pixel); buffer[pixelIndex++] = img.getBlue(pixel); } } return convertedBytes.buffer.asUint8List(); }
And I got exceptions like this:
E/AndroidRuntime( 6372): FATAL EXCEPTION: AsyncTask #2 E/AndroidRuntime( 6372): Process: com.soton.gca_app, PID: 6372 E/AndroidRuntime( 6372): java.lang.RuntimeException: An error occurred while executing doInBackground() E/AndroidRuntime( 6372): at android.os.AsyncTask$3.done(AsyncTask.java:318) E/AndroidRuntime( 6372): at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:354) E/AndroidRuntime( 6372): at java.util.concurrent.FutureTask.setException(FutureTask.java:223) E/AndroidRuntime( 6372): at java.util.concurrent.FutureTask.run(FutureTask.java:242) E/AndroidRuntime( 6372): at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:243) E/AndroidRuntime( 6372): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133) E/AndroidRuntime( 6372): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607) E/AndroidRuntime( 6372): at java.lang.Thread.run(Thread.java:760) E/AndroidRuntime( 6372): Caused by: java.lang.IllegalArgumentException: Cannot convert between a TensorFlowLite tensor with type UINT8 and a Java object of type [[F (which is compatible with the TensorFlowLite type FLOAT32). E/AndroidRuntime( 6372): at org.tensorflow.lite.Tensor.throwIfTypeIsIncompatible(Tensor.java:316) E/AndroidRuntime( 6372): at org.tensorflow.lite.Tensor.throwIfDataIsIncompatible(Tensor.java:304) E/AndroidRuntime( 6372): at org.tensorflow.lite.Tensor.copyTo(Tensor.java:183) E/AndroidRuntime( 6372): at org.tensorflow.lite.NativeInterpreterWrapper.run(NativeInterpreterWrapper.java:166) E/AndroidRuntime( 6372): at org.tensorflow.lite.Interpreter.runForMultipleInputsOutputs(Interpreter.java:311) E/AndroidRuntime( 6372): at org.tensorflow.lite.Interpreter.run(Interpreter.java:272) E/AndroidRuntime( 6372): at sq.flutter.tflite.TflitePlugin$RunModelOnBinary.runTflite(TflitePlugin.java:478) E/AndroidRuntime( 6372): at sq.flutter.tflite.TflitePlugin$TfliteTask.doInBackground(TflitePlugin.java:419) E/AndroidRuntime( 6372): at sq.flutter.tflite.TflitePlugin$TfliteTask.doInBackground(TflitePlugin.java:393) E/AndroidRuntime( 6372): at android.os.AsyncTask$2.call(AsyncTask.java:304) E/AndroidRuntime( 6372): at java.util.concurrent.FutureTask.run(FutureTask.java:237) E/AndroidRuntime( 6372): ... 4 more
I got really confused now, anyone can please help here?