How to download this webpage with Wget?

Question

I want to download the webpage http://forum.ubuntu-it.org/, but it requires a username and password. So I have used this:

wget --save-cookies cookies.txt --post-data 'user=goyamy&passwrd=mypassword' http://forum.ubuntu-it.org/

But it does not work! Why?

Solution 1

That's possibly because the server uses session cookies to track authentication. Add the option --save-cookies alongside to force the cookie to be saved. So your commmand looks like this:

wget --keep-session-cookies --save-cookies cookies.txt --post-data 'user=goyamy&passwrd=mypassword' http://forum.ubuntu-it.org/

I haven't tested it though.

Solution 2

Here is an example script which will dump cookies from Chrome (v19).

#!/bin/bash -e
#
# Quick and dirty script which dumps all Chrome cookies in 
# the specified SQLite database to stdout in Netscape format.

COOKIE_FILE='~/.config/google-chrome/Default/Cookies'

echo -e '.mode tabs \n select host_key, httponly, path, secure, ' \
  'expires_utc/10000000, name, value from cookies;' |
  sqlite3 $COOKIE_FILE |
  sed -e 's/\t0\t/\tFALSE\t/g ' -e 's/\t1\t/\tTRUE\t/g'

Comments

[naitoon]

It worked after replacing the ~ in COOKIE_FILE by its particular value in my configuration. I suppose there's a lacking export or the like in my system. The error that I received before making this change was unable to open database file.

[ankostis]

Please "quote" the variable in sqlite $COOKIE_FILE.