How to intercept HTTP requests and responses of a client machine (like fiddler does)

Question

I thought that the only way to intercept a request is to use a proxy, but fiddler somehow can intercept HTTP requests and responses without configuring anything on any browsers.

What's going on under the hood ?

And do you know any library to do that ? (In any languages)

Solution 1

Fiddler actually does use a proxy. I believe the installer automatically configures IE to use Fiddler's proxy. You can also configure other browsers to go through the same proxy, so Fiddler will profile their network traffic too.

More info here

Solution 2

Fiddler is a proxy, written in C# and wrapping basic sockets.

It registers with WinINET using the appropriate API call while running, and detaches in the same way. Most browsers automatically detect the WinINET proxy setting and use it. Firefox does not, which is why current versions of Fiddler install a Firefox addon.

-EricLaw [MSFT]

Solution 3

I don't know how fiddler is doing it, but it can be done via a Layered Service Provider on Windows.

From Wikipedia:

"A Layered Service Provider (LSP) is a feature of the Microsoft Windows Winsock 2 Service Provider Interface (SPI). A Layered Service Provider is a DLL that uses Winsock APIs to insert itself into the TCP/IP stack. Once in the stack, a Layered Service Provider can intercept and modify inbound and outbound Internet traffic. It allows processing all the TCP/IP traffic taking place between the Internet and the applications that are accessing the Internet (such as a web browser, the email client, etc). "

Solution 4

May be running the network interface in promiscuous mode. This is how WireShark is able to monitor network traffic and display it.

More Info: http://en.wikipedia.org/wiki/Promiscuous_mode

Solution 5

From the MSDN notes on extending fiddler

Fiddler supports a JScript .NET event-handling engine that allows the user to automatically modify the HTTP request or response. The engine can modify the visual appearance of the session in the Fiddler user interface (UI), to draw attention to errors or to remove uninteresting sessions from the list altogether.

View more solutions

Comments

[Nicolas Dorier]

There is nothing configured in options of my browsers, and it works even if I've installed chrome or firefox after fiddler. If fiddler is turned off, everything works fine. I can't imagine than programmers of fiddler have thought about all these use cases, it would be too hard.

[KOTJMF]

Huh, I've never actually gotten Fiddler to profile any other browser without using configuring them to use Fiddler as a proxy. What version of Fiddler are you using?

[Nicolas Dorier]

It's version 2.2.2.0, I've never had to configure anything in my browsers to make fiddler work.

[Jordan S. Jones]

Fiddler changes the System Proxy configuration which is automatically used by IE, Safari, Chrome. Recent versions of Fiddler also install a Firefox plugin that will automatically set the proxy configuration when it is running/actively capturing packets. As far as Opera, and other browsers go, your mileage may vary.

[KOTJMF]

I just opened Fiddler (which has been prompting me to update for the last couple of months :P), and as of 2.2.1.4 (released 3/31/09), Fiddler includes the FiddlerHook Firefox extension that Jordan mentioned. This explains why it doesn't catch traffic from other browsers on my machine, I'll have to install the update!

[Nicolas Dorier]

I'd like to accept this answer too, because that's the response to the 2nd part of my question >_<, Thanks !

[Peter Štibraný]

Isn't it great when core developers send answers to questions about their own software? Fiddler is cool, I don't know what I would be doing without it :-) (Latest Eclipse 3.5 also configures itself to use proxy from IE, so when I start Fiddler, Eclipse now uses it too!)

[Lothar]

Nice but it will not help you with HTTPS