URL encoding a string in bash script

Question

I am writing a bash script in where I am trying to submit a post variable, however wget is treating it as multiple URLS I believe because it is not URLENCODED... here is my basic thought

MESSAGE='I am trying to post this information'
wget -O test.txt http://xxxxxxxxx.com/alert.php --post-data 'key=xxxx&message='$MESSAGE''

I am getting errors and the alert.php is not getting the post variable plus it pretty mush is saying

can't resolve I can't resolve am can't resolve trying .. and so on.

My example above is a simple kinda sudo example but I believe if I can url encode it, it would pass, I even tried php like:

MESSAGE='I am trying to post this information'
MESSAGE=$(php -r 'echo urlencode("'$MESSAGE'");')

but php errors out.. any ideas? How can i pass the variable in $MESSAGE without php executing it?

Solution 1

You want $MESSAGE to be in double-quotes, so the shell won't split it into separate words, then pass it to PHP as an argument:

ENCODEDMESSAGE="$(php -r 'echo rawurlencode($argv[1]);' -- "$MESSAGE")"

Solution 2

On CentOS, no extra package needed:

python -c "import urllib;print urllib.quote(raw_input())" <<< "$message"

Solution 3

Extending Rockallite's very helpful answer for Python 3 and multiline input from a file (this time on Ubuntu, but that shouldn't matter):

cat any.txt | python3 -c "import urllib.parse, sys; print(urllib.parse.quote(sys.stdin.read()))"

This will result in all lines from the file concatenated into a single URL, the newlines being replaced by %0A.

Comments

[Darkman]

Look for my answer here.

[Leo Gallucci]

This solution is python 2 only

[Jason R. Coombs]

Python 3 version looks like python3 -c "import urllib.parse; print(urllib.parse.quote(input())".

[Anton K]

@JasonR.Coombs you forgot one closing brace :)

[Jason R. Coombs]

A correct Python 3 version looks like python3 -c "import urllib.parse; print(urllib.parse.quote(input()))" (tested this time).

[Mike]

Python version agnostic: python -c $'try: import urllib.request as urllib\nexcept: import urllib\nimport sys\nsys.stdout.write(urllib.quote(input()))' <<< "$message" and adds no \n

[Sudip Bhattarai]

No this doesn't work. Replace sys.stdin.read() with input() instead. Using the read() adds %0A which is file termination character.

[Murphy]

Using input() would only return the first line, but my intention was to get all lines from the file, encoded in one URL. You could easily convert %0A to newline again if necessary: [...] | sed "s/%0A/\n/g".

[iainH]

Works for me as posted by @Murphy (28/01/2020)