How to send an image directly from flask server to html?

Question

I am new to flask and am trying to make an app such an image is taken by the html and js from the webcam and then it is sent to the server with ajax request. I got this part. Then some processing is done on the image and it has to be sent back to the frontend. I know how to send data normally in flask, as in

@app.route('/')
def function():
    return render_template("index.html", data = data)

But in python images are in form of numpy arrays and js cannot read numpy arrays and convert it to images( atleast I don't know of any way to do that). so what is the way this can be done?

Solution

This shows how to convert numpy array to PIL.Image and then use it with io.BytesIO to create file PNG in memory.

And then you can use send_file() to send PNG to client.

from flask import Flask, send_file
from PIL import Image
import numpy as np
import io

app = Flask(__name__)

raw_data = [
    [[255,255,255],[0,0,0],[255,255,255]],
    [[0,0,1],[255,255,255],[0,0,0]],
    [[255,255,255],[0,0,0],[255,255,255]],
]

@app.route('/image.png')
def image():
    # my numpy array 
    arr = np.array(raw_data)

    # convert numpy array to PIL Image
    img = Image.fromarray(arr.astype('uint8'))

    # create file-object in memory
    file_object = io.BytesIO()

    # write PNG in file-object
    img.save(file_object, 'PNG')

    # move to beginning of file so `send_file()` it will read from start    
    file_object.seek(0)

    return send_file(file_object, mimetype='image/PNG')


app.run()

The same way you can send it as GIF or JPG.

Comments

[Shantanu Shinde]

I got that but how do I display the image on front end?

[furas]

if you put <img src="/image.png"> in HTML - you can use JavaScript to insert it - then browser will automatically download and display it.

[furas]

eventually Ajax can receive data from /image.png, encode to base64 and put all as string in `'<img src="data:image/png;base64,image_encoded_base64'" />

[Shantanu Shinde]

but how do I know the processing at backend has been completed, so I can insert the html? also which of the html should I insert, one in first comment or second?

[furas]

I think you could do: ajax sends data to server and waits for answer, server processes data and uses send_file() to send image, ajax receives image and uses base64 to put as string in '<img src="data:image/png;base64,image_data_encoded_to_base64'" />

[Shantanu Shinde]

but how do I get the base64 in the js? I am new to flask, so don't know the details.

[Shantanu Shinde]

if you can, please give me an eg html code for how this works

[furas]

I would encode it in Python and send already base64 to client

[furas]

using Google I found that JavaScript has built-in function btoa(str) which create base64.

[Shantanu Shinde]

no, my question is how do I receive the base64 at the client? should I render the html again and pass it as data?

[furas]

when you use ajax to send data then it receives data from server. So server may use send_file() to send string base64. And it insert <img> to HTML in browser. I don't remeber but with jQuery it could be $(where_to_insert).insert("<img src=...>")

[Shantanu Shinde]

I think you did not get my question. How do you receive the data? like, my ajax request is :$.ajax({ type:"POST", url: $SCRIPT_ROOT + '/apply_filter', data: { /*data*/ }}).done(function(){ console.log("sent");}); so how do I get the base64?

[furas]

you have response from server as data in .done(function(data){...})

[Shantanu Shinde]

one last thing, will the data variable have the bas64 of the image? and also can the variable directly used or is it an array or dictionary?

[Shantanu Shinde]

I tried your way, but I am getting an error Failed to load resource: net::ERR_INVALID_URL in html

[Shantanu Shinde]

here is my done function .done(function(data){ var image = document.createElement("img") image.src = "data:image/png;base64," + data; document.body.appendChild(image); });