Apache fails to start, ssl issue

Question

For some reason Apache does not start. I tried rebooting the server but to no avail. It's about 2am and my brain is fried. Here is the apache log. I did not touch any configs, certs, or keys to cause this so I don't know what happened. There is the first line here but this is bogus but the key is in the exact place that I specified the virtualhosts file to look at.

[Wed Mar 04 00:13:51.898515 2015] [ssl:error] [pid 4307] AH02203: Init: Private key not     found
[Wed Mar 04 00:13:51.899365 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag
[Wed Mar 04 00:13:51.899381 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error
[Wed Mar 04 00:13:51.899391 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag
[Wed Mar 04 00:13:51.899402 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error (Type=RSA)
[Wed Mar 04 00:13:51.899413 2015] [ssl:error] [pid 4307] SSL Library Error: error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib
[Wed Mar 04 00:13:51.899422 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag
[Wed Mar 04 00:13:51.899432 2015] [ssl:error] [pid 4307] SSL Library Error: error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error (Type=PKCS8_PRIV_KEY_INFO)

Virtual hosts:

# Optionally load the headers module:
#LoadModule headers_module modules/mod_headers.so

LoadModule ssl_module /usr/lib/apache2/modules/mod_ssl.so
SSLCertificateFile /etc/ssl/certs/foobar.org.ssl.crt
SSLCertificateKeyFile /etc/ssl/private/foobar.org.ssl.key
SSLCertificateChainFile /etc/ssl/certs/sub.class1.server.ca.pem
SSLCACertificateFile /etc/ssl/certs/ca.pem
SSLCompression Off
SSLHonorCipherOrder On
SSLSessionCache shmcb:/etc/apache2/logs/ssl_gcache_data(512000)
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS:!AES256
SSLUseStapling on
SSLStaplingCache "shmcb:logs/stapling-cache(150000)"
SSLSessionCacheTimeout  300

<VirtualHost *:443>
ServerAdmin [email protected]
ServerName foobar.org
#  ServerAlias foobar.org
DirectoryIndex index.html index.php
DocumentRoot /var/www/foobar.org/public_html/
LogLevel warn
ErrorDocument 404 /404.php
#  ErrorDocument 500 /500.html
ErrorLog /var/www/foobar.org/log/error.log
CustomLog /var/www/foobar.org/log/access.log combined
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
Header always set Strict-Transport-Security "max-age=15768000;includeSubDomains"
</VirtualHost>

ls command output:

-rw-rw-r-- 1 brandon brandon  1767 Mar  4 00:02 foobar.org.ssl.key
-rw-r----- 1 root    ssl-cert 1704 Apr 20  2013 ssl-cert-snakeoil.key

Solution

There are a few critical issues here. The errors that Apache is spitting out are telling you that the private key is failing to be parsed as valid ASN.1. This would suggest that you have extraneous data somewhere in the key such as a space. Secondly, your permissions are much too lax. The private key should be owned by root, and have the permissions similar to 640 or -rw-r----.

Comments

[user9517]

You should just g to sleep. I'm sure this will take just a couple of minutes to solve when your head is clearer.

[brandon]

I don't understand what the error is. I don't understand what the heck apache is bellyaching about.

[Dan]

Tried to reboot?

[brandon]

@Dan: Yes I have. Many times in fact.

[Martin]

@brandon can you post the virtualHosts config and the output of ls -l /path/to/private.key please

[Dan]

Sound like something is really wrong with the private key, whatever is in that file is not recognized as a private key

[brandon]

Added virtualhosts and ls output.

[Snowlockk]

Your key should NOT be world readable.

[brandon]

That's... weird. I swear I didn't set it as world.

[user9517]

It probably shouldn't be owned my you either - now honestly stop wasting your and our time and go to bed.

[brandon]

Well I'm not coming back here for help again.