How to POST file contents using cURL?

Question

I want to do an HTTP POST of the contents (as a string) of a local file located at path/to/my-file.txt to a URL endpoint at http://example.com/.

For example, I might want to do the following:

1. Extract the contents of the file my-file.txt as a string.
2. URL encode the string.
3. Store the encoded string as a variable named foo.

Then do something like this:

curl -d "data=foo" http://example.com/

(I don't actually need the foo variable. It's just a convenient way to describe my question.)

What are the commands I would need to execute this? Do I need to write a shell script? If so, how might it look?

Solution 1

According to the last section of -d in man curl:

If you start the data with the letter @, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin. Multiple files can also be specified. Posting data from a file named foobar would thus be done with --data @foobar. When --data is told to read from a file like that, carriage returns and newlines will be stripped out.

That is you don't have to do anything fancy just prepend your filename with a @.

Solution 2

As mentioned in this related question if you want the file uploaded without any changes (curl defaults to stripping carriage-return/line-feed characters) then you may want to use the --data-binary option:

curl -X POST --data-binary @path/to/my-file.txt http://example.com/

Solution 3

To be explicitly clear, the accepted answer suggests:

curl -d "data=@path/to/my-file.txt" http://example.com/

The manual reference is here.

Also see this SE answer and this one also for multi-parts.

Comments

[Admin]

See stackoverflow.com/questions/5143915/…

[Cameron Kerr]

And if you want some useful shell escaping (eg. to use jq to preprocess some JSON, try: curl ... -d @<(jq '{"schema": . | tostring }' myschema.avsc') Use-case: wrap a JSON document into a string of another JSON document to pass it to a REST API. (Confluent Schema Registry)

[ahmet alp balkan]

Also keep in mind that if you're uploading binary files, you should use --data-binary.

[Jus12]

how can we post data from a url?

[Čamo]

PLEASE How can I stop stripping line endings from the file input? I need it because of elasticsearch bulk format needs to ended with new line.

[Čamo]

But I need to send application/json but with line endings

[Čamo]

I need it because of Elasticsearch _bulk api and the solution is to send header Content-Type: application/x-ndjson

[Steve Storck]

Not really. The way you state it can be confusing, and people might not know what you mean. The original question wanted to have data=foo sent to the server. The file called my-file.txt would contain only data=foo. Then the curl syntax would be curl -X POST -d @path/to/my-file.txt http://example.com. Your syntax is a bit non-standard and confusing.

[Yuki]

Does not upload the file for me.

[gkephorus]

@Čamo See the comment of Ahmet

[Čamo]

I found it already.