passing credentials in PHP cURL help

Question

I am trying to pass credentials to a website so I can use file_get_contents on it to extract some data but it is not working, I am getting a blank page so any idea what is wrong here?

<?php


$username="[email protected]";
$password="Koin";

$url="confluence.rogersdigitalmedia.com";


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);

$str= file_get_contents("confluence.rogersdigitalmedia.com/display/prodsupport/Team+Calendar");
echo $str;
?>

Here is the new code it is still not working stuck at login screen when I do get contents....

<?php
$username="[email protected]";
$password="Koin";

$url="confluence.rogersdigitalmedia.com";


$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);


//Replaced due to special chars in url for username and pass
//curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_USERPWD, urlencode($username) . ':' . urlencode($password));
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);

echo file_get_contents('http://confluence.rogersdigitalmedia.com/exportword?pageId=1114407');
?>

New code: I know $url is the URL for which I have to login, but what do I put in $data? I know it's my login info, but how do I put it (e.g., <username> space <password>)?

<?php
function do_post_request($url, $data, $optional_headers = null)
{
  $params = array('http' => array(
              'method' => 'POST',
              'content' => $data
            ));
  if ($optional_headers !== null) {
    $params['http']['header'] = $optional_headers;
  }
  $ctx = stream_context_create($params);
  $fp = @fopen($url, 'rb', false, $ctx);
  if (!$fp) {
    throw new Exception("Problem with $url, $php_errormsg");
  }
  $response = @stream_get_contents($fp);
  if ($response === false) {
    throw new Exception("Problem reading data from $url, $php_errormsg");
  }
  return $response;
}

Solution

You probably need to escape the @ in the username:

curl_setopt($ch, CURLOPT_USERPWD, urlencode($username) . ':' . urlencode($password));

This is because some characters (such as @ and :) have special meanings in URLs. You need to escape them so that the server will treat them as the characters, rather than as indicating something about the HTTP request.

Scrap all that. Your edit shows that you have a fundamental misapprehension about how HTTP works. You can't use CURL to pass HTTP login credentials to a site that expects a web form to be filled out and then expect to be able to log in with file_get_contents (which is part of a completely separate API).

In terms of how you actually do this... First, find out whether the site has an API. If it does not, find out whether they actually want you to be screen-scraping. It would be very bad form to screen-scrape a website that doesn't allow it. It may well in fact be a breach of copyright law.

Second, program to the API. This means logging in in the way that the site enforces. This will be with form values (which you will need to send in CURLOPT_POSTFIELDS, probably) and cookies (which you will need to use in every request, probably using CURLOPT_COOKIEFILE).

Work at understanding how the client-server relationship works, study the API and don't just chuck code at the problem and expect it to work.

Comments

[Bulvak]

I cant login without the @ because unless I type the whole @ring.gil.com after my name it wont let me login so those are needed...what can I do? EDIT: Thanks Ill try it out.

[lonesomeday]

You need to escape the value, so that the server realises that the @ character is part of the username. If you don't encode it, the server will think that the username is okh and the server is ring.gil.com:[email protected].

[Bulvak]

How is it now? I edited it like the way you said (or atleast I think I did if I am mistaken please correct me) and I still get same login screen

[Bulvak]

Itsn ot copy right law, the site belongs to the company I am at but it is externally hosted and requires a login authenticated by the company which I have all I need is to read data from a calendar such as a name....I am stuck on this from 4 weeks I just want to get it over with..thanks yes the site has an api but its not useful for data extraction api is called REST api.

[lonesomeday]

It won't be a breach of copyright law, in that case. But you'll need to work out what data you need to log into the other server. I can't help you with that. Presumably whoever designed the other site will be able to. And there's no reason that an API designed to REST principles wouldn't be able to do data extraction...

[Bulvak]

do you mean this thing I found? Honestly there is a very easy way to do this with google calendar api but instead I am stuck with this...it could be finished in 1 week as opposed to this thing...im really stuck and confused....never worked so advanced into php REST API Login This is a helper for login that allows clients to use Drupal session logins through the /=/login URL, which is sometimes convenient. This allows clients to fetch a session cookie that can be used as an authentication token for subsequent requests.