file_get_contents failed to open stream: HTTP request failed! HTTP/1.1 500 Internal > Server Error in

Question

I see many questions matching to my issue. But none of them is providing any workable solution. so instead of making complex functionality i need simple solution. Please I dont' want cURL. I've already done a lot with this code.

I'm working on a scraping project and need to load the contents of product revisions from ebay.

I am using PHP function

file_get_contents();

Here is the URL i'm troubling with.

https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183

this file_get_contents(); is working perfectly fine on different URLs but don't know why not on this.

This is the code i'm using.

<?php
$html = file_get_contents('https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183');
echo $html;
?>

but i'm getting this error..

Warning: file_get_contents(https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&&item=272908801183): failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in /Users/samz/Documents/projects/scraping/test.php on line 3

Solution

file_get_contents (read the tip on blue rectangle) can be easily blocked on server side through php.ini avoid using it. When you want to get data from an other site use curl instead. http://php.net/manual/en/book.curl.php, there are plenty of options that you can use with curl, by playing a bit the following code can work with your url.

$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch,CURLOPT_URL,"https://cgi.ebay.com/ws/eBayISAPI.dll?ViewItemRevisionDetails&item=272908801183");
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/0.A.B.C Safari/525.13");
$data = curl_exec($ch);
curl_close($ch);

and by echoing the $data varible you can see the whole page.

echo $data;

you can try parsing the data from the page by utilizing php DOM Methods and convert them to the data type you want (object class, array etc).

Comments

[wclark]

please include more of test.php. What is on and around line 3?

[MFarooqi]

on line 3 is ... echo $html; the script i wrote above. you can copy and paste this script in any php file and hit hinter. You will get the same error. this was the simplest method to re-create error for you gurus. BTW it should be line 2. I've an extra line on my test.php file

[Admin]

why are you not using the ebay API ?

[MFarooqi]

I didn't know that.. @nogad .. thanks for this tip.. I will check out that a well.. but it was doing through multiple site...

[MFarooqi]

@nogad I've tested their basic tutorial on developer.ebay.com/DevZone/finding/HowTo/… and i'm getting the same error "failed to open stream: HTTP request failed! HTTP/1.0 500 Internal Server Error in...."