Handling an invalid security certificate using MATLAB's urlread command

Question

I'm accessing an internal database using MATLAB's urlread command, everything was working fine until the service was moved to a secure server (i.e. with an HTTPS address rather than an HTTP address). Now urlread no longer successfully retrieves results. It gives an error:

Error downloading URL. Your network connection may be down or your proxy settings improperly configured.

I believe the problem is that the service is using an invalid digital certificate since if I try to access the resource directly in a web browser I get "untrusted connection" warning which I am able to pass through by adding the site to an Exception list. urlread doesn't have an obvious way of handling this problem.

Under the hood urlread is using Java to access web resources, and the error is thrown at this line:

inputStream = urlConnection.getInputStream;

where urlConnection is a Java object: sun.net.www.protocol.https.HttpsURLConnectionImpl.

Anyone suggest a workaround for this problem?

Solution

Consider the following Java class. I used this page as reference:

Disabling Certificate Validation in an HTTPS Connection

C:\MATLAB\MyJavaClasses\com\stackoverflow\Downloader.java

package com.stackoverflow;

import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.security.cert.X509Certificate;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import javax.net.ssl.HostnameVerifier;

public class Downloader {
    public static String getData(String address) throws Exception {
        // Create a trust manager that does not validate certificate chains
        TrustManager[] trustAllCerts = new TrustManager[] {
            new X509TrustManager() {
                public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                    return null;
                }
                public void checkClientTrusted(X509Certificate[] certs, String authType) {
                }
                public void checkServerTrusted(X509Certificate[] certs, String authType) {
                }
            }
        };

        // Create a host name verifier that always passes
        HostnameVerifier allHostsValid = new HostnameVerifier() {
            public boolean verify(String hostname, SSLSession session) {
                return true;
            }
        };

        // Install the all-trusting trust manager
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

        // Install the all-trusting host verifier
        HttpsURLConnection.setDefaultHostnameVerifier(allHostsValid);

        // open connection
        URL page = new URL(address);
        HttpURLConnection conn = (HttpURLConnection) page.openConnection();
        BufferedReader buff = new BufferedReader(new InputStreamReader(conn.getInputStream()));

        // read text
        String line;
        StringBuffer text = new StringBuffer();
        while ( (line = buff.readLine()) != null ) {
            //System.out.println(line);
            text.append(line + "\n");
        }
        buff.close();

        return text.toString();
    }

    public static void main(String[] argv) throws Exception {
        String str = getData("https://expired.badssl.com/");
        System.out.println(str);
    }
}

MATLAB

First we compile the Java class (we must use a JDK version compatible with MATLAB):

>> version -java
>> system('javac C:\MATLAB\MyJavaClasses\com\stackoverflow\Downloader.java');

Next we instantiate and use it MATLAB as:

javaaddpath('C:\MATLAB\MyJavaClasses')
dl = com.stackoverflow.Downloader;
str = char(dl.getData('https://expired.badssl.com/'));
web(['text://' str], '-new')

Here are a few URLs with bad SSL certificates to test:

urls = {
    'https://expired.badssl.com/'       % expired
    'https://wrong.host.badssl.com/'    % wrong host
    'https://self-signed.badssl.com/'   % self-signed
    'https://revoked.grc.com/'          % revoked
};

---

UPDATE: I should mention that starting with R2014b, MATLAB has a new function webread that supersedes urlread.

Comments

[Ian Hopkinson]

Thanks Amro - that appears to be working for a test case with an invalid certificate! My Java competence is a bit minimal and I ended up taking the package statement out to get it to work, possibly if I put stuff into the right directory structure I wouldn't need to do that.

[Amro]

@amadeus: thanks for fixing the link. FWIW, here's the old page as captured by the Wayback Machine

[Amro]

An alternate solution is to import the website certificate in question as trusted into MATLAB's JRE keystore. See here for instructions: mathworks.com/matlabcentral/answers/…

[dwm8]

How would I go about implementing this code if I am running R2015a on OS X? This appears to be a solution to an exact problem I'm having, but I have almost no Java experience and am not sure how to put this solution to use. Thanks for the help

[Amro]

@dwm8: you would need the Java JDK (must be version 1.7.x) to compile the above Java class javac Downloader.java, add the directory to the MATLAB Java classpath as above (mind the package layout), and use it the class from inside MATLAB. So let's say you have ~/dev/com/stackoverflow/Downloader.java compiling it produces ~/dev/com/stackoverflow/Downloader.class. Then in MATLAB you add javaaddpath('~/dev') and use it as d = com.stackoverflow.Downloader()

[Amro]

@dwm8 I updated the answer with more instructions. Also worth checking out the new webread function in MATLAB