Is there any command-line, generic HTTP proxy (like Squid)?

Question

I can easily use Netcat (or, Socat) to capture traffic between my browser and a specific host:port.

But for Linux, does there exist any command-line counterpart of a Squid-like HTTP proxy that I can use to capture traffic between my HTTP client (either browser or command-line program) and any arbitrary host:port?

Solution 1

Both Perl and Python (and probably Ruby as well) have simple kits that you can use to quickly build simple HTTP proxies.

In Perl, use HTTP::Proxy. Here's the 3-line example from the documentation. Add filters to filter, log or rewrite requests or responses; see the documentation for examples.

use HTTP::Proxy;
my $proxy = HTTP::Proxy->new( port => 3128 );
$proxy->start;

In Python, use SimpleHTTPServer. Here's some sample code lightly adapted from effbot. Adapt the do_GET method (or others) to filter, log or rewrite requests or responses.

import SocketServer
import SimpleHTTPServer
import urllib
class Proxy(SimpleHTTPServer.SimpleHTTPRequestHandler):
    def do_GET(self):
        self.copyfile(urllib.urlopen(self.path), self.wfile)
httpd = SocketServer.ForkingTCPServer(('', 3128), Proxy)
httpd.serve_forever()

Solution 2

This may not be the best solution, but if you use any proxy then it will have a specific host:port so the netcat solution with still work, albeit you'll have to pick apart the proxy meta-data to make sense of it.

The easiest way to do this might be to use any random anonymization proxy out there and just channel all the traffic through netcat. (I.e., set your browser proxy to localhost:port and then forward the data to the real proxy.)

If you want to have a local proxy then a SOCKS5 proxy with ssh -D <port> localhost is probably your easiest option. Obviously, you need to tell your browser to use a "socks" proxy rather than an "http" proxy.

So, something like this (assuming your local machine accepts incoming ssh connections):

ssh -fN -D 8000 localhost
nc -l 8080 | tee capturefile | nc localhost 8000

Naturally, that'll only work for one browser connection attempt, and then exit, and I have not attempted to forward the return data to the browser, so you'll need your full netcat solution.

Solution 3

https://mitmproxy.org/

An example:

mitmdump  --ssl-insecure --mode reverse:google.com

Comments

[Piotr Dobrogost]

Related: Linux, simple http proxy

[Harry]

Thanks. Do you have any name suggestions for anonymization proxies for Linux? I would prefer ones that are not as heavyweight as Squid (in terms of size, memory, startup speed), are startable by a non-root user, etc.

[Harry]

Also, I didn't follow your SOCKS5 proxy example. Could you please explain what is going on with the 2 invocations above? I, e.g. know that ssh -fN -D ... starts a SOCKS5 proxy on localhost port 8000, but what does nc -l ... do? And, to which port should I make my browser point to: 8000 or 8080?

[sunnysideup]

@harry for an anomyzation proxy have a look at privoxy

[Harry]

I believe this is exactly the kind of solution I was looking for. Thanks.

[ams]

When I say "anonymization proxy" I don't mean something you set up yourself: I mean an existing proxy out there on the net. @UlrichDangel suggests privoxy and I'm sure there are others too.

[ams]

The nc commands accept all traffic from port 8080 and forward it to 8000. I did not provide a full netcat solution because your post suggested you already knew how to do that part.

[Russell Silva]

The Python one worked for me for HTTP requests but it doesn't seem to support HTTPS requests.

[Gilles 'SO- stop being evil']

@RussellSilva Try piware.de/2011/01/creating-an-https-server-in-python or gist.github.com/dergachev/7028596