How do I upload to Google Cloud Storage with Curl and an API key?

http curl post google-cloud-storage

10,755

API keys do not provide authentication. They associate a call with a project for purposes of quota and a few other things, but you cannot use them to access resources that require anything beyond anonymous access.

What you'll need is an "Access Key", which can be acquired in a variety of ways, usually via an OAuth exchange. If you have the gcloud command installed, the easiest way to grab a quick access key to use with cURL is to run gcloud auth print-access-token. The following should work:

$> curl -v --upload-file my-file.txt \
    -H "Authorization: Bearer `gcloud auth print-access-token`" \ 
    'https://storage.googleapis.com/my-bucket/my-file.txt'

10,755

Dee Choksi

System developer #SOreadytohelp

Updated on September 15, 2022

Comments

Dee Choksi over 1 year

I am trying to upload to google cloud storage with Curl and an API key, without success. The error message seems to indicate that I lack the Content-length header, which I don't. Any ideas?

$ curl -v -T ./myfile -X POST https://storage.googleapis.com/my-app/my-bucket/myfile?key=<my-api-token>
> Host: storage.googleapis.com
> User-Agent: curl/7.51.0
> Accept: */*
> Content-Length: 4
> Expect: 100-continue
> 
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 411 Length Required
< Date: Thu, 23 Feb 2017 13:46:59 GMT
< Content-Type: text/html; charset=UTF-8
< Server: UploadServer
< Content-Length: 1564
< Alt-Svc: quic=":443"; ma=2592000; v="36,35,34"
< Connection: close
< 
<!DOCTYPE html>
<html lang=en>
  <meta charset=utf-8>
  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
  <title>Error 411 (Length Required)!!1</title>
  <style>
    [snip snip]
  </style>
  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>
  <p><b>411.</b> <ins>That’s an error.</ins>
  <p>POST requests require a <code>Content-length</code> header.  <ins>That’s all we know.</ins>
* Curl_http_done: called premature == 0
* Closing connection 0

Dee Choksi about 7 years

Thanks! Now I get a different error: <?xml version='1.0' encoding='UTF-8'?><Error><Code>InvalidArgument</Code><Messag‌e>Invalid argument.</Message><Details>POST object expects Content-Type multipart/form-data</Details></Error>
ShanEllis about 7 years

What curl command are you using now? Looks like you're doing a POST instead of a PUT.
Dee Choksi about 7 years

That was it! I somehow still had -X POST in the command. Thanks.
Rob Haswell about 2 years

I could not get this to work with this URL. Instead I needed to use the form: https://{bucket}.storage.googleapis.com/{object}
nroose about 2 years

I mean, if we have gcloud to get the auth access token, then it's easier to use gsutil than curl. So... The real problem is that gcloud is a half a gig and I don't want that on my container.