Javascript login using cURL in bash script

Question

I am trying to write a bash script in which I need to login to a website which uses javascript in its form. I have never used cURL so any help would be appreciated. I know I need to use cookies and I have the http headers, but I don't know what I need to do with these.

The headers are

Response Headers

Cache-Control   no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Content-Length  0
Content-Type    text/html;charset=UTF-8
Date    Thu, 17 May 2012 11:25:15 GMT
Expires Tue, 01 Jan 1980 00:00:00 GMT
Last-Modified   Thu, 17 May 2012 11:25:16 GMT
Pragma  no-cache
Server  Apache-Coyote/1.1
X-Powered-By    Servlet 2.4; JBoss-4.2.3.GA (build: SVNTag=JBoss_4_2_3_GA  date=200807181417)/JBossWeb-2.0

Request Headers    
Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection  keep-alive
Content-Type    application/x-www-form-urlencoded; charset=UTF-8
Cookie  SMLOGOUT=true; JSESSIONID=8D5757001A594D5CBB07C9250D1CB2B7; JSESSIONIDSSO=A0569CD1D6C981989F0FE691E9AFC314
Host    https:/example.com
Referer https://example.com
User-Agent  Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
X-WCF-Fragment  true

Any help or pointing me in the right direction would be appreciated. Thanks

Solution 1

From the request header it can be easily seen that you are sending some post data. But you didn't provide it. I give you a simple example on how a http request can be converted to curl command.

Suppose you have this request where you are posting 2 form variables var1 and var2 get POSTed to http://www.example.com/form.url. The request would look like this,

POST /form.url HTTP/1.1
Host: www.example.com
Accept:  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=UTF-8

var1=val1&var2=val2

When its converted to curl its like,

curl -d 'var1=val1&var2=val2' \
--header 'Accept:  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' \
--header 'Accept-Encoding gzip, deflate' \
--header 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' \
'http://www.example.com/form.url'

Note:

1. This way you can add as much as header you want. But its better you add only the headers which is necessary. Because curl will pass most headers for you (e.g. Host, User-Agent, Content-Type, Accept etc).

2. If you want to manage cookie add a file cookie.txt in the current directory and -b cookie.txt -c cookie.txt command switches to curl. Means,

   -b/--cookie <name=string/file> Cookie string or file to read cookies from (H)
   -c/--cookie-jar <file> Write cookies to this file after operation (H) 
   

3. -d switch stands for the data string that will be passed in the request body.

   -d/--data <data>   HTTP POST data (H)
   

Now I hope you can build your command.

Solution 2

In http, there are two ways of sending data to a URL:

• POST
• GET

In GET, the data is sent as part of the URL. You may see a URL that looks like this:

http://foo.com/bar/barfoo/script.js?param1=this&param2=that&param3=the_other

This URL is sending data to a JavaScript at http://foo.com/bar/barfoo/script.js. It is sending to this script the following parameters:

• param1 = this
• param2 = that
• param3 = the_other

In a POST operation, the data is sent differently and is not encoded in the URL itself. THis happens a lot when using web forms (like what you are trying to do). You can use the --form parameter in curl to pass POST data much like you word in a HTTP form.

From the CURL manage:

-F/--form <name=content>
    (HTTP) This lets curl emulate a filled-in form in which a user has
    pressed the submit  button.  This  causes  curl  to  POST  data using
    the Content-Type multipart/form-data according to RFC 2388. This
    enables uploading of binary files etc. To force the 'content' part to
    be  a  file, prefix the file name with an @ sign. To just get the
    content part from a file, prefix the file name with the symbol <. The
    difference between @ and <  is  then  that  @  makes  a  file  get
    attached  in  the  post as a file upload, while the < makes a text
    field and just get the contents for that text field from a file.


Example, to send your password file to the server, where 'password' is the
name of  the  form- field to which /etc/passwd will be the input:

    curl -F password=@/etc/passwd www.mypasswords.com

To  read content from stdin instead of a file, use - as the filename. This
goes for both @ and < constructs.

You can also tell curl what Content-Type to use by using 'type=', in a
manner similar to:

    curl -F "[email protected];type=text/html" url.com

  or

    curl -F "name=daniel;type=text/foo" url.com

You can also explicitly change the name field of a file upload part by
setting filename=, like this:

    curl -F "file=@localfile;filename=nameinpost" url.com

Hope this helps a bit.

Solution 3

Sorry but curl doesn't do javascript.

consider

• manually deciphering the javascript and replicating the behavior in a perl/ruby/python script
• employing selenium

Comments

[Shiplu Mokaddim]

Are you only these headers are sent? Wht are the data?

[sineil]

I'm not too sure what you mean?

[Shiplu Mokaddim]

In a HTTP response there are other parts. specially body, request method, type etc.

[sineil]

Edited to include all headers