How can i resolve HTTPSConnectionPool(host='www.googleapis.com', port=443) Max retries exceeded with url (Google cloud storage)

Question

I have created API using Django Rest Framework. API communicates with GCP cloud storage to store profile Image(around 1MB/pic). While performing load testing (around 1000 request/s) to that server. I have encountered the following error. I seem to be a GCP cloud storage max request issue, but unable to figure out the solution of it.

Exception Type: SSLError at /api/v1/users

Exception Value: HTTPSConnectionPool(host='www.googleapis.com', port=443): Max retries exceeded with url: /storage/v1/b/<gcp-bucket-name>?projection=noAcl (Caused by SSLError(SSLError("bad handshake: SysCallError(-1, 'Unexpected EOF')",),))

Solution

Looks like you have the answer to your question here:

• "...buckets have an initial IO capacity of around 1000 write requests per second...As the request rate for a given bucket grows, Cloud Storage automatically increases the IO capacity for that bucket"

Therefore it automatically Auto-Scale. The only thing is that you need to increase the requests/s gradually as described here:

• "If your request rate is expected to go over these thresholds, you should start with a request rate below or near the thresholds and then double the request rate no faster than every 20 minutes"

Looks like your bucket should get an increase of I/O capacity that will work in the future.

You are actually right in the edge (1000 req/s), but I guess this is what is causing your error.