use curl to POST multipart/form-data, file and lots of key-value pairs

Question

As part of simulating what the front-end of an application will do while I work on the backend, I have been running various curl commands. Was easy to get just a file to be sent as Content-Type:application/octet-stream or just json with Content-Type:application/json

I sent json inline with this:

curl -X POST -H "Content-Type: application/json" -d '{"username":"xyz","password":"xyz"}' http://127.0.0.1:8088

Pulled the json out of a file and sent it like this:

curl -X POST -H "Content-Type: application/json" -H 'Accept: application/json' --data-binary @test.json http://127.0.0.1:8088

(any ideas what the 'Accept' does? does not work without it.., solution from here)

Sent just a file by itself like this:

curl --request POST -H "Content-Type:application/octet-stream"  --data-binary "@photo.jpg"  http://127.0.0.1:8088

Multipart with json inline and a picture goes very nicely like this:

curl --verbose --request POST --header "Content-Type:multipart/form-data" --form key1=value1  --form [email protected]   http://127.0.0.1:8088

(solution from here)

The trouble starts when I try to pull both the key-value pairs from a file and the photo, or to paste json into the curl command which also uploads a file. Can curl be convinced to send "Content-Type:multipart/form-data" with key value pairs coming from a file and a file attachment coming from a file?

john.json

{
  "surname" : "Doe",
  "name" : "John",
  "city" : "Manchester",
  "address" : "5 Main Street",
  "hobbies" : ["painting","lawnbowls"]
}

and john.jpg

I have tried some things but it just gives me error messages:

Tried inline, pasting in json:

$ curl --verbose --request POST --header "Content-Type:multipart/form-data" --data '{"surname" : "Doe","name" : "John","city" : "Manchester","address" : "5 Main Street", "hobbies" : ["painting","lawnbowls"]}'  --form [email protected]   http://127.0.0.1:8088
Warning: You can only select one HTTP request method! You asked for both POST 
Warning: (-d, --data) and multipart formpost (-F, --form).

Then I tried to get them both from a file, but it didn't like that either:

$ curl --verbose --request POST --header "Content-Type:multipart/form-data" --form [email protected]  --form [email protected]   http://127.0.0.1:8088
Warning: Illegally formatted input field!
curl: option --form: is badly used here
curl: try 'curl --help' or 'curl --manual' for more information

Any ideas how to make this work?

Solution 1

I think you're on the right track, but taking a look at the curl manual page might get you further.

Some key take aways from the --form option documentation:

The difference between @ and < is then that @ makes a  file  get
attached in the post as a file upload, while the < makes a text 
field and just get the contents for that text field from a file.

So for the JSON use <, but for the picture use @.

I think also you should specify the content type of each section to help the web server know how to parse each section. Although it probably has expectations for each field.

You can also tell curl what Content-Type to use by using 'type=', in a manner similar to:

curl -F "[email protected];type=text/html" example.com

You'll have to look up the MIME types for JSON and jpegs.

And then the final piece to keep in mind: This option can be used multiple times.

Most of this is just an echo of what @Tanaike is saying above, but with more explanation from the documentation. I would recommend reading it in further detail.

---

I think the largest complain curl has with your command is that each part of the form has the key upload. That's sort of ambiguous. Is there one key for JSON and one key for the picture?

It's also really important to know what the webserver expects from each field in the form. There's a wide difference between a file upload and a text field.

Solution 2

With the help from @Breedly and @Tanaike got the following command to work, someone may find it useful one day:

curl --verbose --request POST --header "Content-Type:multipart/form-data" --form "upload1=<john.json" --form "[email protected]"  http://127.0.0.1:8088

It is happy to have "Content-Type:multipart/form-data" just once to cover both of them in this case. It really however wants the "<" not a "@" for the json.

The following also works:

curl --verbose --request POST  --form "upload1=<john.json;type=application/json" --form "[email protected];type=multipart/form-data"  http://127.0.0.1:8088

Comments

[Tanaike]

In your question, --form key1=value1 --form [email protected] works. So how about this --form "upload1=<john.json" --form "[email protected]"? And also in your situation, --header "Content-Type:multipart/form-data" might be not required. Because I couldn't test this and I'm not sure whether this is the solution for your situation, so I commented here. If this didn't work, I'm sorry.

[cardamom]

@Tanaike that didn't quite work, thought you may have been onto something with the angle bracket, haven't seen that before. Am playing with the solution in this comment and this answer but haven't worked it out yet.

[Tanaike]

I'm really sorry I couldn't help.

[cardamom]

Yes, it is working now. If I change the one "@" to a "<" and then make the two keys different it works. A little bit more fiddling, will post now the command which worked..