SSL for devices in local network

Question

Initial question

We make devices which run a webserver and the user can control some functionality of the device by browsing directly to the IP of the device. This can be a fixed IP when a direct WiFi or ethernet connection is used but in most cases this is the IP that the device has received from a DHCP server in the network.

More and more HTTPS is required to access some of the more advanced functionality of a browser. For example to access cache (https://developer.mozilla.org/en-US/docs/Web/API/Cache), to allow the webcam to be used (https://blog.mozilla.org/webrtc/camera-microphone-require-https-in-firefox-68/), Service Workers (https://www.digicert.com/dc/blog/https-only-features-in-browsers/), ... The list keeps growing every day.

I'm all pro to have secure systems but I think there is one major issue. The way HTTPS (TLS) is set up a certificate is only marked as valid if the domain name matches the one in the certificate and the certificate authority is accepted by the client's browser, the chain of trust as it is called. This works beautifully on the web where fixed hostnames are used.

However when users are not using the internet but their local network the hostname is not known beforehand. Sometimes users can use local DNS, mDNS but this is not always the case. Many times users just use the internal IPv4 address. This is where the trouble begins because there are two options with using the devices we make:

1. The user does not use HTTPS (we do not enforce it, read on to see why). The major browsers at this time do not give an explicit warning but mark the page as 'Not secure' in light grey. Most users don't even notice it and are very happy.
2. The user uses HTTPS on the same device. Altough this makes there connection more secure the browsers are now telling them explicitly to use the device with extreme caution and that the connection is probably hacked and private data could be stolen. The site is now marked 'insecure' in red and the user must press 2 or 3 buttons to allow for a certificate exception.

Option number 2 is the cause that we do not force the devices to be accessed by HTTPS because it simply alarms to many users and floods customer service. Five years ago this was not really an issue because everything could be done without HTTPS. With more and more API's now only working in a 'Secure Context' this is really becoming a problem for us.

Therefore I think the need is becoming very big to come up with a system to use HTTPS without the hostname system, strictly in internal networks. I could imagine that the private IPv4 ranges could be excluded from the warnings or something more clever. This brings me to my question, do you face the same problems and how can this be solved?

Update 1

As pointed out in the first comment the now proposed solution is to use a wildcard certificate and to configure a DNS entry for the device on a public domain. This however has the issue that the client still requires an active internet connection. This is certainly not always the case in these kind of setups.

Update 2

I also found this article on Let's encrypt which talks about the same subject without giving a solution: https://letsencrypt.org/docs/certificates-for-localhost/

Update 3: hypothetical solution idea

After reading the below answers and comments I was thinking of a possible secure solution for the problem. Would the below setup (if it would be allowed) secure?

1. Request an intermediate CA certificate from a trusted root CA which has Name Constraints which only allows it to create multiple intermediate CA's which can only create certificates for a single fixed hostname '*.mydevice.local' or something similar and which allows all private IPv4 addresses to be used in the SAN.
2. Every deployed device would be factory installed with a unique intermediate CA created by the intermediate CA I was talking about in step 1. This on-device CA would than be name constrained on '.mydevice.local'.
3. Every time that the device changes IP-address (boot, DHCP change, ...) it would than be able to generate a certificate with it's on-device intermediate CA.

I think this would solve the problem completely and have the following advantages:

• No browser warnings because the chain of trust relays back to the trusted root CA.
• Every device would have a unique certificate.
• Compromise of a single intermediate CA would not be that big of an issue because it can only be used to create trusted certificate's for the device's specific fixed hostname.

Please comment if I overlook something.

Update 4:

I want to thank everyone for all the help and thinking along. The conclusion for me is that the whole idea behind certificates and the trust chain behind it doesn't allow what I want. This is because there is simply no way for a CA to be sure that the internal IP address I'm pointing to is uniquely owned by the device that I want to reach. An internal IP, for example 192.168.0.10, is owned by thousands devices and thus it is not possible to grant a certificate which allows browsers to show no warning display.

The only option is to do the certificate validation by manual intervention (installing the device certificate, pushing your own device's CA to the user, and the various more complex options as proposed in the answers). This is simply something I need to live with.

Nevertheless I think I'm going to open a ticket with Firefox and Chrome. Because I think that for internal IP-addresses a simple grey non-secure warning, as with HTTP, is more than enough of a warning. The red warnings should only be shown when making use of HTTPS in the use case it was designed for.

Update 5:

I have filed a bug report at Bugzilla: https://bugzilla.mozilla.org/show_bug.cgi?id=1705543 I'm posting this link as a reference so anyone can follow the issue.

Solution 1

Step 1: Make the device generate a self-signed certificate for its IP address and/or hostname on first setup or when the IP address is changed unless there is a customer-provided certificate in place. DO NOT sell devices with a common certificate (see the Let's Encrypt article you linked).

Step 2: Make your documentation describe how to recognize your specific self-signed certificates and accept them in the common browsers.

Step 3: Make your documentation describe how to replace the self-signed certificate with another one generated by the organization's own Certificate Authority or by a public CA.

Solution 2

I deal with this issue extensively for my current job - including high emphasis on the works-offline issues. I'm coming at the question with the precondition that enterprise-ish elements like adding a custom root CA are not appropriate or impractical.

First thing to remember is that if they do not have a DNS server available when offline - TLS for private (RFC 1918) IP addresses without manual modification and poking doesn't work. If this is a real concern that cannot be worked around - documentation and user training are your only real options.

If this is deployed beyond your physical control - wildcard certs are a scary proposition. A hacked-and-extracted wildcard cert can be used by a malicious party to do a lot of damage.

Also remember that more browsers do not accept long certificate expiration windows (see this for a recent Apple/iOS one - limiting a certs lifetime to a year-and-a-bit)

I'd recommend investing in infrastructure to automate deployments of certificates. We also have the luxury of controlling a small dnsmasq server on the device that provides the DNS and DHCP aspects required - again controlled by deployment automation that can ensure that it is up to date. Without knowing more about your solution and sales model its hard to tell if that would work for you. ie: our solution is not static and has a small 4G gateway - so internet losses are fine so long as it gets to talk once every few months.

IMO Puppet is good for internet constrained devices (as opposed the current dev-ops darling Ansible) as the agent aspect allows the device to attempt to deploy itself when it has an opportunity - and works well behind NAT'ted IPv4 gateways.

Solution 3

In one of our projects we faced a problem, that we wanted to connect to websocket on a local network from webpage, that was served from the Internet. In order to use secure web socket connection (wss://) we needed the SSL certificate, but you can't get one for local IP's. The bottomline was that we could not change client machines configuration (no /etc/hosts, no importing of own CA's).

Looking for the solution we got inspired by the service xip.io and came up with the idea, that we could use wildcard SSL certificate and slightly change the xip.io's method to put the local IP without dots into the URL - to get the subdomain that is complaint with the wildcard SSL. So we started our custom DNS service, which works like this:

dig 10-0-0-1.my.local-ip.co

;; ANSWER SECTION:
10-0-0-1.my.local-ip.co. 299    IN  A   10.0.0.1

And we provide SSL certificates for domain *.my.local-ip.co for free on the website http://local-ip.co

The only downside is you still need internet connection to have it running, but on the other hand you don't have to install or configure anything to get it running...

Cheers, Marcin

Solution 4

You're over thinking this. All you need to do is make HTTPS an available feature to be switched on should customers require it. In doing so, HTTP is disabled or better still, connections to the HTTP port of the device will be redirected to the HTTPS port.

To make HTTPS an available feature on your device it needs to be able to handle a certificate file or certificate chain and serve HTTPS with that cert. You don't need to generate your own certificates or handle anything in this area really. Just allow customers to be able to upload their own certificates to your device and reconfigure the device accordingly, then you're basically done. The DNS side of it is NOT your concern.

HTTPS (TLS) requires a DNS infrastructure to work properly, or as a minimum a correctly configured host file on every computer that connects to your device by HTTPS. Again, that's not your concern as every network is configured differently and that aspect of it is outside your control.

Just provide ample documentation on how to set it up if required. It would never be switched on by default.

This solution is exactly what we do for our device. And it works. In practice very few of our customers use it.

If you want to do some more research into how other devices handle it, look no further than the online help pages of various routers and managed switches.

Solution 5

I'm not sure about your environment, so will make some assumptions, please correct me if I am wrong.

Standard ways of doing this are pushing out a new root certificate with active directory, you can sign as many certs as you want, including for ip addresses, for a public example of a HTTPS IP check out https://1.1.1.1/
That is achievable locally.

If you are running that sort of set up, pushing DNS changes (so people use app.company.com instead) would be easy too, your DNS server should be able to be configured to respond to what ever query you like. Running a local DNS server which only returns internal DNS with no internet access is another part of infrastructure that you would need to maintain going forwards though (assuming its not already there, DCHP should provide you a DNS server, so please look at that first)

If people are using this to develop on, use https://github.com/FiloSottile/mkcert its really easy to use and gives trusted certificates (again by inserting them into all trust stores) - This is an automated version of the first option.

To answer the part about why private ips are excluded, its easier on a private network to re route traffic, if an attacker gets in, they can arp spoof the server, get all traffic to it and then control all inputs to the server (which may contain user logins), without the end user ever knowing. Having a valid cert is much harder to spoof, while the attacker could get the traffic, all users would be shown a "this is not secure" page instead as the TLS step would have failed. Local networks do have requirements to be secure too.

View more solutions

Comments

[Criggie]

Daan - what is the expected service life of the gear? Will you provide firmware/software updates to the user? Will it be mandatory for the users to apply updates?

[Gizmo]

Why not plop down a device that can act as a reverse proxy (from HTTPS to HTTP)? Keep that (wilcard) cert up-to-date automatically, the device / one device secure, and be done with it.

[marcelm]

On update 3: are you sure you can issue such intermediates? I'm not sure CAs will give you intermediates, and I also have a vague memory of such name constraints not being enforced (which would explain why you couldn't obtain such intermediates). But perhaps my memory is betraying me.

[Daan Pape]

@marcelm I'm also quite sure such kind of intermediate CA is unobtanium at this point in time. Therefore I marked my solution as hypothetical just to see if it would be secure or not. Off course if the name constraints are not enforced this would be a big issue for my hypothetical solution. So if anyone could elaborate on this?

[jcaron]

What's the context of use of those devices? Home? Office? Corporate? How many users access them? Can you have them run an installer on their computers, or use a custom app to access your device, or install a trusted certificate?

[Daan Pape]

The context is home use. I think your suggestion of deploying a custom app which does the certificate pinning for the customer is the only option at this point in time.

[jcaron]

@DaanPape I'm a bit surprised by the combination of "home use", "local network" and "possibly no internet"? In corporate environments yes, but for home use, are you not taking into account a very very small portions of your user base? One easy solution if you have Internet access is to run everything via a public ("cloud") server. Though of course there may be performance implications if you are streaming stuff and whatnot.

[Braiam]

You should really evaluate your target market. Maybe try not to use "advanced browser features" and stick to pure GET/POST and maybe websockets. You could of course implement the other characteristics, but your product should be able to work with plain HTTP.

[Ginnungagap]

Your hypothetical won't work because obtaining such an intermediate for a WebPKI trusted CA will require guarantees you cannot provide; mainly the unicity of the key, its safe storage in a HSM and an auditable log of every certificate it generated. Even IF you managed to have all the normal requirements waived, you'd at least need to prove ownership of the domain you're NC to. Also regarding a previous comment, name constraints are enforced by most major browsers ie. Firefox, Safari and Chrome (and thus Edge, Brave and many other rebrandings), not sure about IE.

[Brian]

There are even more flaws with OP's proposed solution than Ginnungagap listed. However, that's the sort of thing that needs its own question. Regardless: It's impossible to actually work-around this issue; OPs constraints are in direct conflict with the constraints that PKI operates under.

[Brian]

Regarding update 4: I think opening a ticket with Firefox or Chrome is futile. You're correct that the whole idea behind certificates doesn't allow what you want, but this is an intended feature with known workarounds. These workarounds are disruptive to users, but that's by design. If anything, we're moving towards making stuff like this harder. E.g., TLS 1.3 made some of this stuff worse (i.e., worse in terms of turnkey implementation but better in terms of security), despite complaints made by affected companies.

[Daan Pape]

I don't know about others but I think making encrypted communication possible without being disruptive to users and without the use of internet should remain possible I think.

[Patrick Mevzek]

HTTPS and IP addresses just do not mix well. Yes you can have certificates on IP addresses and even some public CA allow that now. But obviously only for public IP addresses on which you can have some proof of ownership (but the whole model is still fragile, as are DV certificates: right after certificate issuance, the IP address can change ownership, not even taking into account plays at the BGP level). Which just means that you may need to try looking at the problem on a different angle instead of trying to pursue with https://192.0.2.42/ type of URLs.

[Wyck]

Wanted to draw attention to the Android issue Add .local mDNS resolving to Android and the Chromium issue Consider bypassing Android resolution to allow .local via mDNS which could potentially unblock using "powerful features" (API's requiring secure context) on local networks air-gapped from the internet.

[Tomek]

@DaanPape does my post resolve anything regarding this? serverfault.com/a/1095654/957829

[Aerik]

You may want to check out this issue with the W3C discussing pretty much the same topic github.com/w3c/webappsec-secure-contexts/issues/60

[Daan Pape]

This is not an option for most 'normal' consumers. There should be a better solution, currently not supported by any browser, which allows the above scenario to work without any user interaction.

[Mikael H]

Until IPv6 (and firewall knowledge) is common among regular users that is not likely to change. Making your documentation show why a self-signed certificate isn't necessarily bad (provided you can verify it in some way), is probably the best workaround for non-technical consumers. Another workaround is to provide a 'VPN' over HTTPS initiated by the device, via a central portal you control. In this case you take on a lot of responsibility for your own security measures, though.

[Daan Pape]

Hello, thank you for your extensive answer on the subject. Unfortunately I do not have any control about the destination network or setup. So updating trust stores from clients is not an option.

[Daan Pape]

Thank you for your comment. I see you are facing the same issue as we have and hoping that end users read the manual is the only solution at this point in time. I have updated my initial question with some idea I have.

[exussum]

Is there just 1 of these devices on the network ? If so you could use MultiCast DNS and auto register the domain names for local users ?

[Daan Pape]

There could be multiple devices on the network. mDNS could be a possible solution but it doesn't always work. Sometimes IP multicasting is disabled on networks.

[Brian]

@DaanPape: Not only is there not a better solution, but there should not be a better solution. Fundamentally, the signing infrastructure is designed such that self-signing should NOT be automatically trusted. Of course, there are many options for how this trust can be enabled by the user (e.g., browser exception, shared via active directory, added to the user's trust store by an installation program, etc.).

[MSalters]

@Brian: There definitely should be a better solution, because the current browser stupidity is teaching users that an HTTPS warning/error is due to some arcane technical mumbo-jumbo, and not a sign of a real problem. Good UI is fundamental to good security.

[ddm-j]

There is no better solution that wouldn't compromise someone else's security. The best I could come up with would be inventing a dedicated certificate type that would give a yellow address bar and show the serial number of the device, and getting all browser vendors to support that, but that is an effort of several years.

[Bergi]

Interesting. Could you please describe the approach in your answer itself, for if the local-ip.co website goes down?

[djdomi]

@MarcinD and using a public cert seems to have no effort, useless answer in my mind

[Marcin D]

@djdomi - with this method you can use fully valid SSL certificate and get no warning from the browser itself; it is AFAIK the only way to get SSL on LAN without changing clients configuration

[djdomi]

@MarcinD after adding the new Answer, my Previous comment is obsolete, so yes NOW it make sense what you wrote :)

[dave_thompson_085]

TLS definitely does work without DNS or any hostnames, for example in FTPS LDAPS and some VPNs. Even HTTPS can use IPaddresses, see RFC 2818. But publicly-trusted CAs can't issue a cert for a private (RFC 1918 or site/link-local) address, which is what you usually need for LAN. Also the 398-day limit is for CAs subject to CABforum rules, or as the Apple page puts it, preinstalled Root CAs, and "will not affect certificates issued from user-added or administrator-added Root CAs").

[dave_thompson_085]

HTTPS does not 'require' hostnames; see RFC 2818 or as noted in #1060320 https://1.1.1.1 (Cloudflare DNS)

[AndrewL]

@dave_thompson_085 Didn't actually know about RFC 2818 - good to know, I'll update. That said - I think you're missing the intention here. We are dealing with off-the-shelf consumer devices in an unknown local network, not enterprise managed devices with an IT support team. Its trying to make things work with whatever the big phone/browser vendors provide, without mucking around with elements like Root CA's. As these are concepts that no company should want to have to expose their Regular Joe users to.

[Aerik]

Sure - but HTTPS does require a certificate, and a certificate requires a unique address, does it now? Which means that is extremely cumbersome to set up HTTPS for a local address for use by multiple end-users.

[Admin]

Sadly the local-ip.co approach doesn't really work due to DNS rebind attack protection. The most recommended fix to thwart DNS rebind attacks is to block name resolutions by external servers that result in local IPs. Dnsmasq has a flag for that. OpenWRT sets this flag by default. I would assume many enterprise DNS configs set it too. This completely kills the local-ip.co approach :-(

[Admin]

I have tried using this. It seems to work so far, but it's terribly slow. I guess this is redirecting traffic trough the local-ip.co server?

[Admin]

@tuket no, the IP address returned is in LAN, so no traffic is redirected by our servers... please check your LAN DNS configuration and TvE comment on that

[Admin]

@MarcinD Thanks, what is TvE?