Use Flutter to send a http Post-Request (containing an image) to a Flask API

Question

I've trained a CNN on the CIFAR10 dataset (placeholder, will be replaced with a different model later) and integrated the model into a flask API. The API is hosted on Heroku, and I would now like to use Flutter / Dart to take pictures on my phone, send them to the Flask API, run my trained model on them and return the prediction.

Using python, I can easily make a post request to my API and return the predictions. Here is my simple python code for this:

import requests
import json

img = open('some-picture.jpg', 'rb')

files = {'image': img}
response = requests.post("url_to_api", files=files)

print(response.text)

I haven't been using Flutter / Dart for very long, and I gather that the process of making htpp requests is a little more complex than in python. Could someone give me some pointers or perhaps code that allows me to take a picture with my camera, upload it to my API, and store the response in a variable? Here's my (simplified) python code for the flask API:

from flask import Flask, request
import os
import numpy as np
from PIL import Image
from tensorflow import keras

app = Flask(__name__)
app.config["DEBUG"] = True

model = keras.models.load_model('cifar10_cnn.h5')
labels = ["Airplane", "Automobile", "Bird", "Cat", "Deer", "Dog", "Frog", "Horse", "Ship", "Truck"]


@app.route('/', methods=["POST"])
def predict():
    
    # stuff not relevant to question, left out for conciseness #
    
    file = request.files['image']

    image = Image.open(file).resize((32, 32))
    image = np.array(image)
    image = image / 255
    image = image.reshape(-1, 32, 32, 3)

    predictions = model.predict([image])
    index = np.argmax(predictions)
    results = {'Prediction:': labels[index]}

    return results

if __name__ == '__main__':
    app.run()

So far I know that Multipart files seem like the way to go, and that the Dio package might be worth looking into. If further tips or code could be provided I would be grateful.

Solution 1

You may already know how to choose an image from the gallery/camera (e.g. using image_picker library). You fill a class field like File image; with the result of that picker. This could be as easy as:

import 'dart:io';
import 'package:image_picker/image_picker.dart';
class _MyHomePageState extends State<MyHomePage> {
  File image;
  final picker = ImagePicker();

  pickImageFromGallery(ImageSource source) async {
    final image = await picker.getImage(source: source);

    setState(() {
      this.image = File(image.path);
    });
  }
}

(Change the ImageSource source to match your desire: camera or gallery)

Then you can upload that file to your api. Using http library:

import 'package:http/http.dart' as http;
import 'package:http_parser/http_parser.dart';

class _MyHomePageState extends State<MyHomePage> {
doUpload(){
    var request = http.MultipartRequest(
      'POST',
      Uri.parse("url_to_api"),
    );
    Map<String, String> headers = {"Content-type": "multipart/form-data"};
    request.files.add(
      http.MultipartFile(
        'image',
        image.readAsBytes().asStream(),
        image.lengthSync(),
        filename: "filename",
        contentType: MediaType('image', 'jpeg'),
      ),
    );
    request.headers.addAll(headers);
    print("request: " + request.toString());
    request.send().then((value) => print(value.statusCode));
}
}

For each one of these libraries, you have to add them as dependency inside pubspec.yaml in your flutter project:

cupertino_icons: ^0.1.3
http: ^0.12.2
image_picker: ^0.6.7

Solution 2

I have done similar work with Django and Flutter. I used image_picker to select image and used dio to upload image.

This is upload function:

  _upLoadImage(File image) async {
    setState(() {
      loadingdone = false;
    });
    String path = image.path;
    var name = path.substring(path.lastIndexOf("/") + 1, path.length);
    var suffix = name.substring(name.lastIndexOf(".") + 1, name.length);
    FormData formData = FormData.fromMap({
        "img": await MultipartFile.fromFile(path,filename: name)
    });
    

    Dio dio = new Dio();
    var respone = await dio.post<String>("http://192.168.1.104:8000/uploadImg/", data: formData);
    if (respone.statusCode == 200) {
      Fluttertoast.showToast(
          msg: 'Done!',
          gravity: ToastGravity.BOTTOM,
          textColor: Colors.grey);
      setState(() {
          _label = jsonDecode(respone.data.toString())['label'];
          _score = jsonDecode(respone.data.toString())['score'];
          loadingdone = true;
      });
      
    }
  }

Run upload function after select image:

  Future getImage() async {
    var image = await ImagePicker.pickImage(source: ImageSource.gallery);
    _upLoadImage(image);
    setState(() {
      _image = image;
    });
  }