how to Capture https with fiddler, in java

Question

I am running the following java program in the Eclipse IDE:

import java.net.*;
import java.io.*;

public class HH
{
    public static void main(String[] args) throws Exception
    {
        //if i comment out the system properties, and don't set any jvm arguments, the program runs and prints out the html fine.
        System.setProperty("http.proxyHost", "localhost"); 
        System.setProperty("http.proxyPort", "8888"); 
        System.setProperty("https.proxyHost", "localhost"); 
        System.setProperty("https.proxyPort", "8888"); 

        URL x = new URL("https://www.google.com");
        HttpURLConnection hc = (HttpURLConnection)x.openConnection();

        hc.setRequestProperty("User-Agent","Mozilla/5.0 (Windows NT 6.0)
        AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2");

        InputStream is = hc.getInputStream();

        int u = 0;
        byte[] kj = new byte[1024];
        while((u = is.read(kj)) != -1)
        {
            System.out.write(kj,0,u);
        }
        is.close();
    }
}

This produces the following exception, if fiddler is RUNNING, both while capturing, and not capturing:

Exception in thread "main" javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(Unknown Source)
    at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
    at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(Unknown Source)
    at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(Unknown Source)
    at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
    at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown ...

If I close fiddler, the program runs fine without any exceptions, producing the html on the url i am connecting to.

alternatively, if i specify System.setProperty("https.proxyPort", "443");, instead of: System.setProperty("https.proxyPort", "8888");, it runs and prints out all html, without exceptions, even while fiddler is open, in capturing mode, but there is still no capturing from fiddler at all.

Then if I set these system properties through eclipse's jvm arguments like: -DproxySet=true -DproxyHost=127.0.0.1 -DproxyPort=8888, the same exact exception happens again, so long as the fiddler app is running, both in capturing and non capturing mode. If i close fiddler, the program will run perfectly fine.

If i use: System.setProperty("http.proxyHost", "127.0.0.1"); instead of: System.setProperty("http.proxyHost", "localhost");, it runs fine with fiddler application running, both cap-/non capturing mode, but also NO captured traffic.

Is anyone out there, able to capture their own https traffic with fiddler, NOT through a web browser, but through a java program? What are the jvm arguments, how do you set it up to do this? thanks

Solution 1

Create a keystore containing the Fiddler certificate. Use this keystore as the truststore for the JVM along with the proxy settings.

Here's how to do that:

• Export Fiddler's root certificate

Tools -> Fiddler Options... -> HTTPS -> Export Root Certificate to Desktop

• Create a keystore with this certificate

Open command line as administrator (keytool doesn't work otherwise)

<JDK_Home>\bin\keytool.exe -import -file C:\Users\<Username>\Desktop\FiddlerRoot.cer -keystore FiddlerKeystore -alias Fiddler

Enter a password when prompted. This should create a file called FiddlerKeystore.

• Now start the JVM with Fiddler as the proxy and this keystore as the truststore. You'll need these vmargs:

-DproxySet=true

-DproxyHost=127.0.0.1

-DproxyPort=8888

-Djavax.net.ssl.trustStore=<path\to\FiddlerKeystore>

-Djavax.net.ssl.trustStorePassword=<Keystore Password>

Use these vmargs in your eclipse run configuration and you should be good to go.

I'm able to capture HTTPS requests made from the JVM without any issues with this setup.

Solution 2

You can also import the Fiddler key into the Java trusted certificates store (as long as you are aware that this is not secure and you don't do this on any non-development environment):

1. Export Fiddler's root certificate from within Fiddler:

Tools → Fiddler Options... → HTTPS → Actions → Export Root Certificate to Desktop

2. Start an elevated Command Prompt and use the following command to import the certificate. Replace the jdk1.7.0_79 part with your appropriate JDK/JRE version. If you have multiple JDK/JRE's installed, you'll need to perform this action per environment.

"keytool.exe" -import -noprompt -trustcacerts -alias FiddlerRoot -file c:\work\FiddlerRoot.cer  -keystore "C:\Program Files\Java\jdk1.7.0_79\jre\lib\security\cacerts"  -storepass changeit

I also had a problem with decrypting HTTPS traffic using the Google API Client in combination with Fiddler. The problem was that by default, the client uses it's own cert store:

InputStream keyStoreStream = GoogleUtils.class.getResourceAsStream("google.jks");
SecurityUtils.loadKeyStore(certTrustStore, keyStoreStream, "notasecret");

And this is how i fixed this:

HttpTransport transport = new NetHttpTransport() 
//instead of transport = GoogleNetHttpTransport.newTrustedTransport();

Solution 3

Create a keystore containing the fiddler certificate and use it:

java -DproxySet=true -DproxyHost=127.0.0.1 -DproxyPort=8888 -Dhttps.proxyPort=8888 -Dhttps.proxyHost=127.0.0.1 -Djavax.net.ssl.trustStore=<path to FiddlerKeystore> -Djavax.net.ssl.trustStorePassword=<password> -jar test.jar

If you use third party HTTP libraries, you need to set the connection proxies. Example with Apache Commons HttpClient:

HttpClient httpClient = new HttpClient();
httpClient.getHostConfiguration().setProxy("localhost", 8888);

UPDATE:

if you are using Apache HttpClient 4.5.5 or newer, you need to do it like this:

HttpHost proxy = new HttpHost("localhost", 8888, "http");
DefaultProxyRoutePlanner routePlanner = new DefaultProxyRoutePlanner(proxy);
CloseableHttpClient httpclient = HttpClients.custom()
                .setRoutePlanner(routePlanner)
                .build();

Solution 4

I found that I also required the following java command line options

-Dhttps.proxyPort=8888 
-Dhttps.proxyHost=127.0.0.1

View more solutions

Comments

[CodeMangler]

I started with those vmargs too, but later realized that they were redundant. i.e. I could capture just fine without them, and using just these two, instead of the proxyPort and proxyHost args wouldn't work. Not sure if it's VM/environment dependent. --EDIT-- I was capturing from Oracle JDK 6 on Windows 7 btw.

[Tharsan Sivakumar]

This is how I did. Great thanks :) techie-mixture.blogspot.com/2016/07/…

[rnrneverdies]

This solution also works by using System.setProperty(<prop>, <value>); instead of -D<prop>=<value>. Thanks.

[timo]

Im developing a webapp and use jboss / wildfly 18 as appserver. I tried this and a few other similar options to get fiddler to work, but had no luck yet.. "unable to find valid certification path to requested target" Your solution should work for all java apps right?

[StevenV]

With Fiddler 5, exporting root certificate: Tools -> Fiddler Options... -> HTTPS -> Actions -> Export Root Certificate to Desktop