Curl a string with whitespaces and characters in it?

Question

I'm trying to create a "simple" script that I'm going to use to create some documentation for Unix/Mac machines. I've successfully received the information I need, but it seems like I can't send it over to a webserver I've created with CURL because it gives some whitespaces and characters in the information output. I've tried with very many methods but nothing seems to work correctly..

This is the URL output that the script will send over to curl:

"ip.ip.ip.ip/documentation/index.php?avdelning=XX&user=XXX&hostname=XXX&model=ModelName:MacminiModelIdentifier:Macmini4,1&RAM=Memory:8GB&CPU=ProcessorName:IntelCore2DuoProcessorSpeed:2.4GHzNumberofProcessors:1&CPUCORES=TotalNumberofCores:2&MACADDRESS=etherxx:xx:xx:xx:xx:xx&SERIALNUMBER=SerialNumber(system):XXXXXXXX&LASTLOGGEDIN=XXXXX

Using only the following string will work if I do it directly with CURL and will not give me any errors and I receive the information on the webserver.

But when I use the script it does not work. The script:

#!/bin/bash

STR_RAM=$(/usr/sbin/system_profiler SPHardwareDataType | grep Memory: | sed 's/ //g');
STR_MODEL=$(/usr/sbin/system_profiler SPHardwareDataType | grep Model | sed 's/ //g');
STR_CPUNAME=$(/usr/sbin/system_profiler SPHardwareDataType | grep Processor | sed 's/ //g');
STR_CPUCORES=$(/usr/sbin/system_profiler SPHardwareDataType | grep Cores: | sed 's/ //g');
STR_SERIAL=$(/usr/sbin/system_profiler SPHardwareDataType | grep Serial | sed 's/ //g');
STR_MACADDR=$(ifconfig | grep en0 -A3 | grep ether | sed 's/ //g');
STR_LOGGEDIN=$(finger | tail -1 | sed 's/ //g');
STR_HOSTNAME=$(hostname | sed 's/ //g');

STR_URL=ip.ip.ip.ip/documentation/index.php?avdelning=$1\&user=$2\&hostname=$STR_HOSTNAME\&model=$STR_MODEL\&RAM=$STR_RAM\&CPU=$STR_CPUNAME\&CPUCORES=$STR_CPUCORES\&MACADDRESS=$STR_MACADDR\&SERIALNUMBER=$STR_SERIAL\&LASTLOGGEDIN=$STR_LOGGEDIN;


STR_URL2=$(echo $STR_URL | sed 's/ //g' | awk '{printf("\"%s\"\n", $0);}');
curl $STR_URL2;

As you can see I've added a bunch of probably unnecessary editing with sed and awk but nothing seems to work.

I get these error messages: After I added the awk to add double quotes before and after the URL string I get:

Could not resolve "ip.ip.ip.ip

Which is very strange since I can run the curl command directly in terminal without that error. And If I do not remove the spaces etc I receive multiple lines that the URL can not be resolved. I've tried with urlencoding in perl and bash but that does not seem to fix the problem either.

I'm running this on a Mac now. Usage: sudo sh script.sh randomstring randomstring

EDIT: Output of new script

sudo sh script2.sh IT Denniz
curl: (3) Illegal characters found in URL
xx.xx.xx.xx/documentation/index.php?avdelning=XX&user=Denniz&hostname=admins-Mac-mini-2.local&model=ModelName:Macmini
ModelIdentifier:Macmini4,1&RAM=Memory:8GB&CPU=ProcessorName:IntelCore2Duo
ProcessorSpeed:2.4GHz
NumberofProcessors:1&CPUCORES=TotalNumberofCores:2&MACADDRESS=  etherXX:XX:XX:XX:XX&SERIALNUMBER=SerialNumber(system):XXXXXXXXXXX&LASTLOGGEDIN=adminadmins008Jun10:21

Solution

You don't want to insert literal quotes into the string. You just want to put quotes around the variable. I would do this:

#!/bin/bash

hw_data=$(/usr/sbin/system_profiler SPHardwareDataType)

ram=$(grep Memory: <<<"$hw_data")
model=$(grep Model <<<"$hw_data")
cpuname=$(grep Processor <<<"$hw_data")
cpucores=$(grep Cores: <<<"$hw_data")
serial=$(grep Serial <<<"$hw_data")

macaddr=$(ifconfig | grep en0 -A3 | grep ether)
loggedin=$(finger | tail -1)
hostname=$(hostname)

avdelning=$1
user=$2

url="ip.ip.ip.ip/documentation/index.php"
url+="?avdelning=${avdelning}"
url+="&user=${user}"
url+="&hostname=${hostname}"
url+="&model=${model}"
url+="&RAM=${ram}"
url+="&CPU=${cpuname}"
url+="&CPUCORES=${cpucores}"
url+="&MACADDRESS=${macaddr}"
url+="&SERIALNUMBER=${serial}"
url+="&LASTLOGGEDIN=${loggedin}"

curl "${url// /}"
#    ^          ^   <- these are the critical quotes

Read this for more about how critical quoting is: Security implications of forgetting to quote a variable in bash/POSIX shells

Comments

[Denniz]

Exactly what I thought, that's why I added awk '{printf("\"%s\"\n", $0);} which did not work unforunatley, then I receive the error message that it can not resolve "ip.ip.ip.ip

[Angel Todorov]

Stronger than that: always quote your variables unless you know exactly what side effects you want by leaving them unquoted (and those side effects can be very subtle)

[eyoung100]

Is there a link you can provide for the different types of string expansion? i.e. single quotes vs double vs none.

[eyoung100]

Perfect... Now the proof is there for other readers.

[Denniz]

Thank you it seems more promising now but still it does not work! Now it gives me this error: Curl (3): Illegal characters found in URL. Tried with --globoff without succes.. Could it be (system) that's illegal? Or commas? I've tried to google which characters are disallowed without succes..

[Angel Todorov]

In your question, so you can properly format, show the output of echo "${url// /}"

[Denniz]

Alright, edited the post with the new output! Thank you so much for helping me!

[Angel Todorov]

You need to filter the hw_data better. grep Processor is returning more than one line so you have newlines in the url