SSL routines:tls_process_server_certificate:certificate verify failed

Question

Since last night, several of my scripts (on different servers) using file_get_contents("https://...") and curl functions stopped working.
Example request that fails:

file_get_contents("https://domain.tld/script.php");

---

Error:

PHP Warning:  file_get_contents(): SSL operation failed with code 1. OpenSSL Error messages:
error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed in /home/domain/public_html/script.php on line 19

I already "fixed" the problem using:

$arrContextOptions=array(
    "ssl"=>array(
       "verify_peer"=>false,
       "verify_peer_name"=>false,
    ),
); 

file_get_contents("https://domain.tld/path/script.php", false, stream_context_create($arrContextOptions));

The "fix" is far from ideal since I'm not verifying the authenticity of the connection, but until I understand the origin of the problem and how to prevent it from happening again, I'll be forced to use it.

---

Notes:

• PHP scripts with Curl also stopped working and the fix is similar: curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0); curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);;
• The SSL certificate is issued by Let's Encrypt and it was renewed last night ("not valid before 2020/12/24");
• All servers have the same timezone;
• I'm using CentOS 7/Ubuntu 18 and Virtualmin;
• If I open "https://domain.tld/script.php" on Firefox/Chrome, no SSL warnings are shown and the certificate is valid;
• I've tried to update the CA certificates (yum install ca-certificates.noarch), but the latest version is already installed;

---

I understand what's wrong, what I cannot figure out is why it started happening and how to fix it (the real fix).

---

Question:

How to fix and prevent it from happening again?

Solution

The problem was an outdated CA certificate and I found the solution on a Let's Encrypt community thread :

Manual Solution:

• Replace the contents of /home/[domain]/ssl.ca with lets-encrypt-r3-cross-signed.pem
• restart apache/nginx

Virtualmin Solution:

• Go to Virtualmin -> Server Configuration -> SSL Certificate -> CA Certificate

  • Option 1: Choose upload file and use lets-encrypt-r3-cross-signed.pem
  • Option 2: Paste the contents of lets-encrypt-r3-cross-signed.pem using the Pasted certificate text option.

• Press "Save Certificate"

---

Note:

This issue was fixed on webmin 1.970, so make sure you've the latest version installed, which wasn't my case due to the webmin repo not being enabled. If that's also your case, just enable or add the webmin repo and run yum update.

Comments

[ArSeN]

Suddenly appearing issues sound like one (or multiple) of the certificates in the chain expired. Have you double checked the lets encrypt certs are renewed and their chain is valid as well? Browsers tend to be a bit more "forgiving" when it comes to verification since they often have different root-certs than long-standing tools like programming languages.

[Pedro Lobito]

@ArSeN The Certificate is valid on all browsers and devices I've tested, but after using https://www.digicert.com/helpit gives me an error: "TLS Certificate is not trusted : The certificate is not signed by a trusted authority (checking against Mozilla's root store). If you bought the certificate from a trusted authority, you probably just need to install one or more Intermediate certificates. Contact your certificate provider for assistance doing this for your server platform." - The certificates are normally renewed automatically and I never add this error before.

[Pedro Lobito]

Let's encrypt "Chain of Trust" was updated on the last 8 of December. Can this be related?

[ArSeN]

Let's encrypt announced some new root and intermediate certs back in september. Its possible your 3-month period was up now and its hitting you. However, the root cause is that the certificate is not valid, I'd recommend generating a new one. Edit: Yes! ;)

[Pedro Lobito]

@ArSeN Thanks. The certificate was renewed last night. Are you suggesting that I try to force renew ti again?

[ArSeN]

You can try that, but its highly speculative. It might also be required that you update PHP and/or curl and/or your operating system, since those are all places where trust certificates might or might not be stored, which is highly depending on your setup. After all, CentOS 7 is (kinda) EoL which might very well be the root cause (no pun intended) here.

[Pedro Lobito]

I've found a thread on let's encrypt website matching the same exact problem. I'll try the solution presented there and post the results after. Thank you very much and Merry Christmas.

[Pedro Lobito]

The thread I mentioned before IS the solution, which is updating the CA certificate on the problematic domain with letsencrypt.org/certs/lets-encrypt-r3-cross-signed.pem