How to connect a socket to an http server through proxy?

Question

Recently I wrote a program using sockets in C, to connect to an HTTP server running locally and thereby to do requests to that.

That worked fine for me. After that I tried the same code to connect to another server on the web (e.g. www.google.com), but I was not able to connect and was getting another html response from the proxy server in my network.

• My local IP is: 10.0.2.58
• The proxy IP is: 10.0.0.1

This is the response I got :

HTTP/1.1 302 Found
Expires: Fri, 10 Feb 2012 12:47:35 GMT
Expires: 0
Cache-Control: max-age=180000
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Connection: close
Location: http://10.0.0.1:8000/index.php?redirurl=http%3A%2F%2F10.0.2.58%2F
Content-type: text/html
Content-Length: 0
Date: Wed, 08 Feb 2012 10:47:35 GMT
Server: lighttpd/1.4.29

How can I bypass this proxy to connect to external servers?

---

Response got when tried with CONNECT

HTTP/1.1 302 Found
Expires: Fri, 10 Feb 2012 13:37:58 GMT
Expires: 0
Cache-Control: max-age=180000
Cache-Control: no-store, no-cache, must-revalidate
Cache-Control: post-check=0, pre-check=0
Pragma: no-cache
Connection: close
Location: http://10.0.0.1:8000/index.php?redirurl=http%3A%2F%2F10.0.2.58http%3A%2F%2Fwww.google.com%2F
Content-type: text/html
Content-Length: 0
Date: Wed, 08 Feb 2012 11:37:58 GMT
Server: lighttpd/1.4.29

---

Working code which connect's to my local apache

#include<unistd.h>
#include<stdio.h>
#include<sys/types.h>
#include<sys/socket.h>
#include<netinet/in.h>
#include<arpa/inet.h>
#include<netdb.h>
#include<string.h>

#define MAX_BUFFER_SIZE 1024

int main(int argc,char *argv[])
{
  int clsd,ssd,status;
  char buffer[1024];
  char request[]="GET / HTTP/1.1\r\nHost:10.0.2.58\r\n\r\n";
  struct sockaddr_in srvr_addr;

  struct addrinfo hints,*res;

  srvr_addr.sin_family=AF_INET;
  srvr_addr.sin_port=htons(80);
  srvr_addr.sin_addr.s_addr=inet_addr("10.0.2.58");//Local server

  clsd =socket(AF_INET,SOCK_STREAM,IPPROTO_TCP);
  if(clsd<=0)
  {
        perror("Socket init failed..\n");return 1;
  }
  ssd=connect(clsd,(struct sockaddr *)&srvr_addr,(socklen_t)(sizeof srvr_addr));
  if(clsd<=0)
  {
        perror("Socket connect failed..\n");return 1;
  }
  write(clsd,request,strlen(request));
  memset((void *)&request,0x00,strlen(request));
  memset(&buffer,0x00,MAX_BUFFER_SIZE);

 do
 {
  status=read(clsd,&buffer,MAX_BUFFER_SIZE);
  write(1,&buffer,status);
 memset((void *)&request,0x00,strlen(request));
  memset(&buffer,0x00,MAX_BUFFER_SIZE);

 do
 {
  status=read(clsd,&buffer,MAX_BUFFER_SIZE);
  write(1,&buffer,status);
 }while(status>0);
 close(clsd); 
 return 0;
}

Solution 1

To use connections via proxy (or if they are implicitly proxy-fied), first you should connect to proxy, send a 'CONNECT' message with target host; proxy will establish connection and return you data.

Here is in steps:

1. open socket to proxy host
2. send 'CONNECT http://www.google.com:80 HTTP/1.0\r\n\r\n' string
3. wait for recv

You must specify protocol (in our case is HTTP 1.0, non-chunked) with ending newline characters, so proxy knows how to communicate with end point.

You can find details about CONNECT method at http://www.ietf.org/rfc/rfc2817.txt

Solution 2

If you're specifically trying to bypass the proxy, you should talk to whoever administers your network to find out if that's even possible. If your first block of output is an attempt to connect to Google then it appears to me that there's some kind of transparent proxy on your network that you'll have to take special (and network-specific) steps to bypass.

Of course, if you're just interested in getting data, you could try following the redirect...

Comments

[Akhil Thayyil]

Ya the proxy in the network is transparent, i guess so .. How can i bypass that ? I can fetch pages using firefox and all , like that how can i fetch the page using my own code ?

[dennis]

Good :) Now you must redirect. "HTTP/1.1 302 Found" is response from server where it instructs you to read 'Location:' part from response, and go to it.

[Akhil Thayyil]

But when i copy the redirect url and try it in browser , its redirecting me to proxy admin's page

[dennis]

Hm... it is a bit hard to guess until I see some code. Did you tried with telnet? Check this: anta.net/misc/telnet-troubleshooting/http-proxy.shtml, except port 25, you should use 80.

[Akhil Thayyil]

dennis , the proxy is a transparent one , so am not getting telnet response , its always showing trying to connect

[dennis]

Well, if proxy is transparent then all packages must go through it, without sender knowledge and connection to google.com should go without problems. I checked your code (assuming you are aware you are not connecting to google, but to some internal machine) and it should work (after filling missing code due incomplete paste).

[Akhil Thayyil]

dennis , the code i pasted is connecting to my local apache , that works fine , but to any external sites say google its causing problem.

[dennis]

Are you able to browse net via firefox? Did your admin had to setup something before you are able to do so?

[Akhil Thayyil]

dennis , i think that is the problem of my network setup , i was running linux in vmware , so the linux is only able to connect to internal n/w . Any ways thanks for you help and support ( Plus one for that)

[nickgrim]

If the proxy is capturing all outgoing traffic on port 80 (and possibly more), then you'll need to talk to whoever runs the proxy - they'll be able to tell you (1) if you can bypass it (2) how you can bypass it (3) what you should do if you can't.