FastAPI and Python threads

Question

I have a little issue with FastAPI and additional threads spawning. Let's say i have an application that serves two endpoints.

1. One of them to /create_user to create user in some database
2. Other is just /ping. Ping is made simply because my app is running in Kubernetes and it's continuously checking if my app is alive by sending GET request and receive response_code 200.
3. Additionally i have a separate process made with threading.Thread, that receive some key from external service. Key have TTL, so it needs to be renewed from time to time.

The problem is when i load data by first endpoint, the database i'm loading into, is pretty slow and can answer up to 10 seconds. In that moment all other endpoints(including /ping) is locked. So k8s is thinking that my app is dead and try to rollback it.

I could simply try to increase the number of workers, that serve the application with command uvicorn main:app --workers 4 BUT additional thread is also spawning with each worker and output in logs looks smth like that

INFO:     Application startup complete.
Hello from additional thread
INFO:     Started server process [88030]
Hello from additional thread 
INFO:     Waiting for application startup
Hello from additional thread Hello from additional thread
INFO:     Application startup complete. Hello from additional thread

My question is it possible to spawn only one additional thread with multiple gunicorn workers?

Here is code snippet from my main.py

@app.post("/api/v1/create_user")
async def create_user() -> JSONResponse:
    """Some creation magic here"""
    return JSONResponse(status_code=status.HTTP_201_CREATED, content={"Success": True,                                                                     "Username":raw_credentials["user"]})
    
    
@app.get("/ping", status_code=status.HTTP_200_OK)
async def dummy_response():
    return

# Special treads lunching for some jobs that need to be repeated during app lifecycle.
t1 = Thread(target=renew_api_token)
t1.start()

Solution

I believe the main issue is that you're probably not using an async compatible lib to access your database.

That's the reason you're seeing all other endpoints being locked when your application is waiting for the database.

There's two solutions to this problem.

You can find an async lib to access you database.

Or you can use def create_user() instead of async def create_user(), that way FastAPI will run that function inside a ThreadPool for you.

Comments

[Stasello Boldirev]

Thanks a lot! I forget that with normal def instead of async def FastAPI run function inside ThreadPool. Search for async db lib i think is overkill, cause service will be running with not so big load

[BRad]

I came across this question when implementing API for ML. It is not possible to implement PyTorch inference on CPU using async approach. Probably most of tools for ML in python don't support async inference. To summarize, this is not a viable solution for ML.