What events caused mass migration to HTTPS?

Question

For several years I see that Google, Facebook, etc start serving (and even redirect to) content via HTTPS.

Serving sites that prompts passwords in insecure HTTP was wrong even in 1999 but it considered acceptable even in 2010.

But nowadays even public pages (like queries from Bing/Google) is served via HTTPS.

What events caused mass migration to HTTPS? Wikileaks scandal, US / EU law enforcement, reduced cost of SSL/TSL handshake with generally dropped cost of server time, growing up IT culture level in management?

Even public efforts like https://letsencrypt.org/ started not so long ago...

@briantist As I also maintain hobby sites and interested in cheap/effortless SSL/TLS solution. For VPS (which starts from 5$/month) I recently evaluated Let's encrypt with certbot (other bots available) in webroot operation mode. This provides me with valid SAN certificate for 3 month (and it is in cron job - renewal performed a month before expiration date):

certbot certonly -n --expand --webroot \
        -w /srv/www/base/ -d example.com \
        -w /srv/www/blog/ -d blog.example.com

Solution 1

There are a lot of factors that went into it including:

• Browser and server technology for security with virtual hosts. You used to need a dedicated IP address per secure site, but that is not the case anymore using SNI.
• Lower cost and free security certificates. Let's Encrypt now issues about half of all certificates for free. Ten years ago I was looking at $300/year for a wildcard domain, but now even paid wildcard certificates can be as little as $70/year.
• Overhead of HTTPS dropped significantly. It used to require extra server resources, but now the overhead is negligible. It is even often built into load balancers that can talk HTTP to backend servers.
• Ad networks such as AdSense started supporting HTTPS. A few years ago, it was not possible to monetize an HTTPS website with most ad networks.
• Google announcing HTTPS as a ranking factor.
• Big companies like Facebook and Google that moved to HTTPS for everything normalized the practice.
• Browsers are starting to warn about HTTP being insecure.

For big companies like Google that could always afford to move to HTTPS I think there were a couple of things that pushed them to implement it:

• Leaking of competitive intelligence data over HTTP. I believe that Google moved to HTTPS in large part because so many ISPs and competitors were looking at what users were searching over HTTP. Keeping search engine queries under wraps was a big motivation for Google.
• Rise of malware targeting sites like Google and Facebook. HTTPS makes it harder for malware to intercept browser requests and inject ads or redirect users.

There are also some reasons that you are seeing HTTPS more often in cases where both work:

• Google is preferring to index the HTTPS version when the HTTP version also works
• Many folks have the HTTPS Everywhere browser plugin that automatically has them use HTTPS sites when available. That means that those users also create new links to HTTPS sites
• More sites are redirecting to HTTPS because of security and privacy concerns.

Solution 2

The answers thus far talk about various pull and push reasons as to why HTTPS is becoming more and more popular.

However, there are 2 major wake-up calls from around 2010 and 2011 that showed how important HTTPS actually is: Firesheep allowing session hijacking, and the Tunisian government intercepting Facebook logins to steal credentials.

Firesheep was a Firefox plugin from October 2010 created by Eric Butler, which allowed anyone with the plugin installed to intercept other requests on public WiFi channels and use the cookies from those requests to impersonate the users making those requests. It was free, easy to use and above all, it didn't need specialist knowledge. you just click a button to harvest cookies, and then another to start a new session using any of the harvested cookies.

Within days, copycats with more flexibility appeared, and within weeks, many major sites started supporting HTTPS. Then a few months later, a second event occurred that sent another ripple of awareness through the Internet.

In December 2010, the Arab Spring started in Tunisia. The Tunisian Government, like many others of the region, tried to suppress the revolt. One of the ways they tried this was by hindering Social Media, including Facebook. During the revolt, it became clear that the Tunisian ISPs, who were largely controlled by the Tunisian government, were secretly injecting password harvesting code into the Facebook login page. Facebook quickly acted against this once they noticed what was happening, switching the entire country to HTTPS and requiring those affected to confirm their identity.

Solution 3

There was what became to be called Operation Aurora which (allegedly) was Chinese crackers breaking into USA computers like Google's.

Google went public with Operation Aurora in 2010. Seems they decided to convert the loss into value by showing efforts of securing their products. So instead of losers they show up as leaders. They needed real efforts otherwise they would have been publicly ridiculed by those who understand.

Google is an internet company so it was critcal for them to reinstall trust in their users about the communication. The plan worked and other corps needed to follow or face their users migrate to google.

In 2013 what came to be called Global surveillance disclosures prominently by Snowden happened. People lost trust in corps.

Made a lot of people consider to go indie and use HTTPS which then caused the recent migration. He and who he worked with gave explicit calls to use encryption explaining that survellience needs to be expensive.

strong encryption * critically high volume of users = expensive survellience.

It was 2013. That said, more recently Snowden told that this is probably not enough any more and you should spend money on people who work on legally strengthening your rights for you, too, so the tax money goes away from the survellience industry.

Nevertheless for the avarage Joe webmaster the long-standing issue with HTTPS was that getting a certificate costed money. But you need certs for HTTPS. It was solved in late 2015 when Let's Encrypt beta became available to the general public. It gives you free certs for HTTPS automatically via the ACME protocol. ACME is an Internet draft which means to people that you can sort-of rely on it.

Solution 4

Encrypting transmissions over the internet is more secure against nefarious agents intercepting or scanning this data and inserting themselves in the middle, faking you into thinking they are the real web page. Successful intercepts such as this only encourage more and others to follow.

Now that it is more affordable, and the technology more accessible, it is easier to push for everyone to do more secure things that protects us all. Being more secure reduces costs and expenses of those affected by data breeches.

When the work involved in breaking encryption becomes difficult and expensive, it will keep the level of activity down and restricted only to those willing to invest the time and money involved. Like locks on your house doors, it will keep most people out and free the police up to concentrate on higher level criminal activity.

Solution 5

One other thing I didn't see mentioned, on Sep 29, 2014, CloudFlare (a very popular proxying CDN because most sites of moderate size can effectively use them for free with simple DNS changes), announced the offering of free SSL for all sites they proxy.

Essentially, anyone proxying through them could automatically and immediately hit their site with https:// and it just worked; no changes needed on the backends, nothing to pay for or renew.

For me personally and for many other people in the same boat, this is tipped the scale for me. My sites are all basically personal/hobby sites that I would have liked to use SSL for, but couldn't justify the cost and maintenance time. Often the cost was more about having to use a more expensive hosting plan (or start paying instead of using free options) as opposed to the cost of the cert itself.

View more solutions

Comments

[dan]

This is a pretty broad, opinion-based question that will likely lead to a list of different factors, versus a single definitive one, so it's being converted to a Community Wiki so others can easily edit and contribute.

[DocRoot]

The "internet" is safer for the end user if everything is SSL.

[Dhrumil Munshi]

Something, something, SOPA, paranoia, tin-foil hattery, and increased safety for users.

[kasperd]

Is it really a mass migration though? As you note in the question the process has taken a long time. Could it be that we are just now seeing the steepest part of a logistic growth curve? If the process was actually accelerated recently, I would attribute that to Snowden.

[Kamal Chanda]

This is what finally did it for us, nobody wants a bright red "not secure" on an ecommerce site..

[n611x007]

letsencrypt started in 2012. announced 2014, public beta late 2015, public in 2016.

[Maciej Piechotka]

@MonkeyZeus I think FireSheep in 2010 changed it from tin-foil hattery to 'bored high schoolers sitting next to you in Starbacks'. Well before 2010 it was still possible but you need to know how.

[user253751]

Pretty sure Firesheep came out right before it started.

[JLL]

Two words: Ed Snowden. Suddenly ordinary people became aware of the need for encryption to protect privacy, not only against criminals but also world governments.

[sylbru]

blog.mozilla.org/security/2017/01/20/…

[wb9688]

Don't forget HTTP/2, which is currently only implemented for HTTPS, also don't forget that Google ranks HTTPS sites (slightly) higher than HTTP sites…

[Martijn]

I suggest a change in order. I think it was a privacy problem, which is now fixable because of the technical advances. I dont think people TLS because 'they can now'. :)

[Stephen Ostermiller]

There has always been the privacy problem and everybody always knew it. Yes, privacy was the driving concern for a few large companies, but for the mass of smaller websites, the ease and cost were bigger factors. I say that from personal experience. I've always wanted to secure my personal websites, but it only became cheap and easy enough recently.

[rluks]

What about Jeff Atwood's blog? His post made me switch. A lot of professionals might be reading Coding Horror.

[Derecho]

You misspelled 1% overhead.

[Stephen Ostermiller]

@MichaelHampton -- Good point. I updated the answer

[Chris Hayes]

I'm guessing the Firesheep one should be 2010, or Arab Spring should be 2011. Otherwise the "a few months later" bit doesn't make any sense.

[Nzall]

@ChrisHayes oops, Firesheep was 2010, not 2011. Fixed. Also, we didn't know about the Tunisian Government stealing Facebook credentials until January 2011.

[gavenkoa]

See my update to question. Details is available in my Setting Let’s Encrypt on Lighttpd blog post blog.defun.work/post-72b3f008-e28e-11e6-bad9-485b39c42d0f.ht‌​ml

[briantist]

@gavenkoa that's cool, but if I'm at the point where I have a VPS and I'm maintaining the OS, that's already way beyond the amount of effort I want to expend (these days, I mean; I used to run a web host). At that point, I'd have no issue even if I were manually updating the certs (though I certainly wouldn't if I didn't have to). I would typically be using shared hosting nowadays, or in the case of my current site, github pages proxied through CloudFlare. But yeah certbot seems great if you've already got the environment where you could run it.

[gavenkoa]

I've read old article scotthelme.co.uk/tls-conundrum-and-leaving-cloudflare Don't know if they still allow man-in-the-middle attack for today offers but their SSL protections have issues in the past (begging 2014)...

[briantist]

@gavenkoa I'm aware of those concerns, though CF is pretty open about the configuration options and what they mean; if one wants to use them one should also be aware of the details. I wouldn't exactly call them issues, but in any case that's a bit beyond the scope of this question. Their offering was a one-click (often) zero-effort, free of charge way to switch a site to https, so even with the configuration of http from CF to your backend, to browsers and search engines it looks the same and I believe it was a large source of small site conversions.