Is "HTTPS Everywhere" still relevant?

Question

HTTPS Everywhere is a browser extension, a collaboration between The Tor Project and the Electronic Frontier Foundation, that automates rewriting requests for HTTP URLs to the secure HTTPS alternative if available. It's apparently been around for roughly a decade but was never on my radar until someone recently asked about it. Trying to research it has produced a mixed bag of information.

1. Regardless of need, it isn't clear how useful it is "out-of-the-box". Various articles refer to the need to supplement defaults with whitelists and rules to get the full benefits. So implementing it appears to not be a trivial task.

2. At least at one time, a substantial portion of web sites were HTTP-only, so using such software could provide only limited benefit. It appears that sites dealing with sensitive personal data have pretty much moved to HTTPS-only. Google implemented various measures to incentivize web sites to convert to HTTPS. It isn't clear how big of a problem HTTP still is (or if it still is, whether the problem is quickly disappearing).

   It also isn't clear whether sites converting to HTTPS are retaining the HTTP links just for legacy visitors, and automatically redirecting to their HTTPS site.

3. The major browsers appear to all have either incorporated logic to prefer HTTPS sites when available, or are well into the process of implementing it. At least Google (haven't seen anything about other search engines), has a program by the same name (not clear if it is actually the same product), to automatically attempt an HTTPS connection on searches.

4. Three years ago or so, there were articles about "why you need to install HTTPS Everywhere". A number of more recent articles have suggested that people should stop suggesting that people install this software. The gist seems to relate to browsers already duplicating the functionality.

So it isn't clear whether HTTP is still a substantive problem needing a solution, and if so, whether software that tries HTTPS links first can solve what's left of it. Has this whole issue been overtaken by events?

I'm looking for context rather than opinion (i.e., the facts describing the current situation rather than opinion about how good or bad it is, or whether I need the software). For example, do the major browsers now provide the remedy that HTTPS Everywhere was developed for? Is HTTP now virtually limited to sites where there's no personal data? Is there government or industry regulation that is intended to render this a non-issue? In other words, the kinds of objective information that will allow me (and others) to understand the current state of affairs in order to form my own opinion and determine relevance for myself.

Solution 1

HTTPS Everywhere certainly used to be more necessary during the days of mixed content and half-hearted website configurations. The web is certainly more mature nowadays, with technologies like HSTS which can be used by any site, and public key pinning for the bigger players (now deprecated in favor of Certificate Transparency - thanks to Justin for informing me).

So, whether the extension is useful depends heavily on your individual use case. Making custom rules for websites that serve both HTTP and HTTPS is something the extension excels at, and I'm not aware of any others that do a similar job. Even in situations where a website doesn't support HTTPS, the extension will ensure that any references to third-party domains such as CDNs will be upgraded to HTTPS, even if the original reference was protocol-neutral.

Solution 2

Speaking as a previous ruleset contributor to HTTPS Everywhere, I have the following to offer.

• The HTTPS Everywhere project periodically tests all of their rewriting rules and disables those which fail for any reason. This ensures a relatively quick response to changing website configurations, but can lead to a significant portion of the rulesets being disabled unless significant maintenance effort is expended. Suggestions that the central rulesets should be supplemented mainly arise from ignorance that these central rulesets can and should be corrected. It's a matter of volunteer availability.

• Significant progress has been made in moving the web to HTTPS-only, but many sites are still misconfigured and many more have not implemented the crucial HSTS preload protection needed to prevent first-connection attacks. Sites which implement this protection are shortly thereafter removed from HTTPS Everywhere's rulesets.

• Web browser technology is very useful, but anything they do beyond the HSTS preload list is only nice to have. HTTPS Everywhere provides a stopgap for sites which have not enabled HSTS through the browser and essentially need a custom community-maintained HSTS configuration.

In summary, it doesn't harm to keep it installed. Bear with it for a few more years and hopefully all this will become redundant.

Solution 3

While improved awareness of HTTPS and HSTS have certainly brought security standards forward, there is still use for the HTTPS Everywhere extension:

HSTS is great at protecting against HTTP downgrade attacks but one thing to notice is that it is based on a trust on first use model. This means your first connection to the site must be through HTTPS or else the HSTS protection can be compromised (for example a HTTP to HTTPS 301 redirect is a window of opportunity for an attack).

HSTS normally protects against this with the HTST preload list, a list of domains built into the browser which forces the first connection to use only HTTPS for those sites. However getting onto the list (and waiting for the change to be applied in browsers) takes some time and not every site bothers to register itself. This is where the browser extension helps out by ensuring all first connections are through HTTPS only.

Another smaller case is when the website's HTTPS is located on a different path from the usual. For example a website might have http://www.example while having their secure site on https://secure.example. HTTPS Everywhere keeps a database of domains to ensure you are going to the correct URL for HTTPS.

Footnote: public key pinning also helps, but even Chrome decided to remove it for low adoption rates and the potential for being a foot-gun.

Solution 4

I have noticed there are still a couple of websites around which have https support but do not redirect http traffic to https. The extension isn't nearly as useful as it used to be however. A few years back websites like youtube, wikipedia and reddit had https support but defaulted to http. HTTPS everywhere solved that and is still solving the problem for the small handful of websites that still default to http but have https support.

View more solutions

Comments

[Admin]

I'm not sure how this plays into this, if at all, but in some cases, HTTPS is not supported by a browser or OS, in which case HTTP is required. I've gotten complaints when my website automatically rewrote HTTPS to HTTP, because this made my site entirely inaccessible to some people. Now my site defaults to HTTP, although it does support HTTP. Ideally, in the future I'll have it default to HTTPS, but not rewrite HTTP to HTTPS, which should never be done as a general best practice.

[Admin]

@InterLinked there are ways for your server to automatically upgrade to HTTPS only if it is correctly supported (and continue using HTTP for archaic noncompliant clients) - see scotthelme.co.uk/we-dont-do-https-for-backwards-compatibilit‌​y for example

[Admin]

@MatijaNalis Thanks for the link, I'll look into it. I'm curious if it works for browsers that claim to support SSL but don't do it properly. For example, support for SSL in Chrome 49 on Windows XP is basically nonexistent - you just get SSL certificate errors. Pages work perfectly in Internet Explorer and Firefox. HTTP also works, but many (maybe not all) HTTPS sites do not work. I'd need a solution that would handle this kind of situation, too.

[Admin]

@InterLinked Some of them should work - for example https image one. Browser will try to get 1x1 pixel image over HTTPS, and only if it succeeds it will receive HSTS header which will force all future connections to that domain to use HTTPS. If TLS failed for any reason, the browser would not receive HSTS headers over it, and would continue using website via HTTP (only without showing 1x1 https image). If you want quick test, I've implemented some (but not this 1x1 image) of the upgrade methods here

[Admin]

@InterLinked When you say rewrite, do you also mean redirect? Because this is what I'm doing: Redirect permanent / https://${subname}.${name}.${tld}/, does this not qualify as a general best practice?

[Admin]

Well, I've tried both rewrite using htaccess and other methods of redirecting. It's not a general best practice to redirect unconditionally because it won't work for all clients.

[Admin]

As of 2021 Aug Firefox has baked it into their browser and Chrome will have it in the next few months. In firefox you need to go into its preferences to activate it (its off by default). So you no longer need this extension. But you should probably look into extensions like ublock origin and trace for better protection.

[fixer1234]

So it sounds like the major browsers don't fully handle this yet, and the Internet is improved but not "cured"? "whether the extension is useful depends heavily on your individual use case": If that's the case, that implies that the need hasn't completely gone away. Thanks. +1. Let me give this some time for other answers before accepting.

[briantist]

@fixer1234 Brave (a Chromium-based privacy-focused browser that can even use Chrome extensions) does have this functionality built in. If its counter is correct, then it certainly has upgraded a lot of my connections to HTTPS which seems useful to me, regardless of whether you get it via built-in functionality or an extension. I agree with this answer's take: the state of things is vastly better than it was, but mistakes still happen, and smaller websites have a hard time getting things right so it will never be "cured" as long as the default protocol is unsecured.

[BrainStone]

I was gonna say that yeah by now it is pretty useless, a tiny improvement at best since browsers try to get HTTPS sites if possible. (or soon will be) But I haven't thought about it upgrading static resources (of CDNs). And since it is whitelist based and doesn't need to check if the domain also serves HTTPS it can upgrade directly which certainly seems to be beneficial. (Even if the target resource gets redirected to the HTTPS variation. Because with HTTPS Everywhere the HTTP request never even gets sent)

[IllusiveBrian]

Plenty of sites don't use HSTS headers/preload because they want to support the widest array of people they can. I just checked Amazon.com and it doesn't appear to return a Strict-Transport-Security header, for example. HTTPS Everywhere would protect against a malicious link to Amazon that used http to exploit the connection.

[BoffinBrain]

Yes, it certainly does, because the extension's list of domains is larger than the list of pinned keys that are shipped with the browser. I'm still not sure why a website wouldn't use HSTS, though, since older devices that only support SSL etc. will not care about that header anyway.

[IllusiveBrian]

@BoffinBrain TLS 1.0 and 1.1 are supposed to be deprecated but some browsers still only support them and not TLS 1.2, so sites may want to give those users the option to downgrade to http rather than not be able to use their site when they stop supporting 1.0 and 1.1. Eventually some vulnerability will be found with 1.2 or it will just be considered too weak compared to the then modern TLS1.5 and sites will stop supporting it as well, and a whole bunch of software made without support for TLS1.3 will be forced to drop to http. Not saying I support that thinking entirely, but it has merit.

[BoffinBrain]

That sounds like a terrible idea! Using SSL or old TLS is still better than HTTP. Nobody should be encouraging a downgrade to no encryption at all.

[IllusiveBrian]

@BoffinBrain Not supporting deprecated transport security versions prevents version downgrade attacks. More users use software the supports both TLS1.2 and lower versions than that don't support TLS1.2, so there is also the argument that they are protecting people keeping their software reasonably up to date over people not making that effort, given that the former pool is larger and downgrade attacks are hard for a user to spot.

[IllusiveBrian]

Also, I was wrong about Amazon, they just have www.amazon.com listed, but not amazon.com (which is just a redirect to www.amazon.com).

[BoffinBrain]

Hmm. Yep, I guess downgrades are bad too. I just had a peek at Amazon and thankfully there is a HSTS after the redirects (I think the header is disallowed on a HTTP response, which makes sense).

[Jonathan]

I'm a hobbyist site maintainer, and implemented HTTPS+HSTS on my site. Getting on the HSTS preload list took me about 20 minutes, 10 of which was reading what to do. I do agree that some wouldn't bother to do a complete job in this though.

[AlphaD]

@Jonathan Are you sure it only took you 20 mins? It might take only that much to apply to join, but it normally requires longer for the new version of the browser to be pushed out with the updated list. It is supposed to be embedded into the software.

[AlphaD]

@Jonathan Also clarified the answer a bit to address your point. Thanks.

[James_pic]

@Jonathan it's often easier to make this sort of stuff work on small or hobbyist sites than on big widely used sites. nickcraver.com/blog/2017/05/22/https-on-stack-overflow is informative.

[Jonathan]

@Alphad: Indeed 20 min to apply and verify that the application went through. I never tracked when the list actually got distributed to browsers - the perks of being a hobbyist...

[Jonathan]

@James_pic: Indeed. I did have some other troubles outside of this list - for example, getting WordPress to generate links correctly - the whole job was ~2 hours, and that with the certificate handled automatically by my hosting provider and Let's Encrypt.

[Martijn Heemels]

@Jonathan A big problem for domains with large number of hostnames and subdomains is that the preload feature requires includeSubdomains to be enabled. It's usually quite easy to enable HSTS on the www-subdomain but suddenly having it enabled everywhere is risky, especially since it's irreversible. Making an inventory and HTTPS-enabling every service with no way back is a lot of work for many enterprises.

[Justin]

Note that HTTP Public Key Pinning was deprecated in 2018 and was replaced by Certificate Transparency

[MechMK1]

Another reason is also that some websites used to serve different content, based on whether they were called via HTTP or HTTPS. Since this is now mostly recognized as an anti-patter, HSTS can now be deployed more widely.

[Admin]

Nothing that wrong with serving content files like JQuery over HTTP rather than HTTPS, it's a generic file that doesn't contain anything sensitive and it's marginally faster too.

[user11153]

@LTPCGO It's EVERYTHING WRONG to let anyone inject any javascript into your "secure" page by performing MITM.

[Admin]

@user11153 true for script content...my point is still valid for e.g. images. Most browsers seem to have some sort of warning though if a page mixes content.

[fixer1234]

Thanks for weighing in. I'm pretty unfamiliar with all this stuff, which is why I posted the question in the first place. Much of what's in this answer just went over my head. Also, there's liberal use of sarcasm, which I can appreciate when I understand it, but it makes things confusing when I don't know the subject. Any chance you could add a "for dummies" summary with some context/explanation of the key points?

[Michael]

@user11153 You can use subresource integrity to make sure that the CDN shows the same file that you expected.

[phyrfox]

@fixer1234 I wouldn't worry about this answer too much. It kind of reads like they believe you may be wearing a tin foil hat... or even that HTTPS itself is some sort of tin foil hat. Security and privacy is more important than this answer suggests, even if you "have nothing to hide." ... tl;dr All those "conspiracies" about government and hackers spying are false or overblown, and using HTTPS makes your traffic look suspect (so, don't encrypt anything at all?)

[Damon]

@phyrfox: On the contrary, the tinfoil hat is explicitly being put on your head (both by actual tinfoil-hat wearers, and people who have a financial interest). While encryption (and authentication) is good and necessary where it's necessary, it is by no means necessary when reading an "ordinary" website. It isn't even desirable (desirable for you) because it prevents easy caching/reusing of content. Of course, caching is undesirable for other parties (e.g. content providers) since you are to get your personalized content, which is an awful euphemism for something very, very bad.

[Damon]

It's a real thing. People who speak of "fake news" are usually ridiculed, but if you are being honest, this is, sadly, exactly what today's "personalized content" world looks like. While the news per se are not fake (well, sometimes they are), the selection that you are being shown differs (try loading the same news page together with a coworker for example, you'll be surprised). Information should be truthful, complete, and unbiased, but it's not. Encryption helps doing that because content is "automatically" individual.

[Shautieh]

@Michael I don't think it's a good enough reason to avoid https, especially since a good chunk of your visitors will use a browser that does not implement that.