Squid Proxy Server self compiled installation - ssl_crtd helpers are crashing

Question

I am trying to setup Squid-Proxy Server 3.5.26 from squid-cache.org with SSL-Dump configuration. Using the standard apt-get approach to installing Squid-Proxy did not include OpenSSL modules that are needed. I'm following this guide: SSL Dump Guide.

To compile Squid with the OpenSSL modules, I referred to the documentation and managed to get Squid compiled and installed. In order to compile/install Squid, I also included the Debian/Ubuntu additions near the bottom of the guide set the correct file locations.

Following the install I completed the Init Script section, downloading the init file from the provided link to /etc/init.d/squid. Then set the permissions of the file to 755 and assigned ownership to root (this matched the other files in init.d).

If I use the default squid configuration file /etc/squid/squid.conf with http_port 3128 the service starts fine. However, when I update this to the following:

http_port 3128 ssl-bump \
cert=/etc/squid/ssl_cert/myCA.pem \
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB

# For squid 3.5.x
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M 4MB

acl step1 at_step SslBump1

ssl_bump peek step1
ssl_bump bump all

The squid service will no longer start and /var/log/syslog contains the following error

squid[2062] Squid Parent: (squid-1) Process 2128 started
squid(squid-1) The ssl_crtd helpers are crashing to rapidly, need help!
squid[2062] Squid Parent: (squid-1) Process 2128 exited with status 1
squid[2062] Squid Parent (squid-1) process 2128 will not be restarted due to repeated, 
frequent failures.

Any suggestions on how I can get ssl-dump working would be appreciated :)

Update

Found a little more detail about the crash in the cache.log file. I noticed that the ssl_db has not been initialised. But running the suggested command (but with Ubuntu relevant path) /lib/squid/ssl_crtd -c -2 returns Can not create directory.

2017/07/11 14:38:20 kid1| Set Current Directory to /var/cache/squid
2017/07/11 14:38:20 kid1| Starting Squid Cache version 3.5.26 for x86_64-pc-
linux-gnu...
2017/07/11 14:38:20 kid1| Service Name: squid
2017/07/11 14:38:20 kid1| Process ID 2128
2017/07/11 14:38:20 kid1| Process Roles: worker
2017/07/11 14:38:20 kid1| With 65535 file descriptors available
2017/07/11 14:38:20 kid1| Initializing IP Cache...
2017/07/11 14:38:20 kid1| DNS Socket created at [::], FD 6
2017/07/11 14:38:20 kid1| DNS Socket created at 0.0.0.0, FD 7
2017/07/11 14:38:20 kid1| Adding nameserver 192.168.1.254 from 
/etc/resolv.conf
2017/07/11 14:38:20 kid1| Adding nameserver 8.8.8.8 from /etc/resolv.conf
2017/07/11 14:38:20 kid1| helperOpenServers: Starting 5/32 'ssl_crtd' 
processes
(ssl_crtd): Uninitialized SSL certificate database directory: 
/var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".
(ssl_crtd): Uninitialized SSL certificate database directory: 
/var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".
2017/07/11 14:38:20 kid1| Logfile: opening log 
daemon:/var/log/squid/access.log
2017/07/11 14:38:20 kid1| Logfile Daemon: opening log 
/var/log/squid/access.log
(ssl_crtd): Uninitialized SSL certificate database directory: 
/var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".
(ssl_crtd): Uninitialized SSL certificate database directory: 
/var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".
(ssl_crtd): Uninitialized SSL certificate database directory: 
/var/lib/ssl_db. To initialize, run "ssl_crtd -c -s /var/lib/ssl_db".
2017/07/11 14:38:20 kid1| Unlinkd pipe opened on FD 25
2017/07/11 14:38:20 kid1| Store logging disabled
2017/07/11 14:38:20 kid1| Swap maxSize 102400 + 262144 KB, estimated 28041 
objects
2017/07/11 14:38:20 kid1| Target number of buckets: 1402
2017/07/11 14:38:20 kid1| Using 8192 Store buckets
2017/07/11 14:38:20 kid1| Max Mem  size: 262144 KB
2017/07/11 14:38:20 kid1| Max Swap size: 102400 KB
2017/07/11 14:38:20 kid1| Rebuilding storage in /var/cache/squid (clean log)
2017/07/11 14:38:20 kid1| Using Least Load store dir selection
2017/07/11 14:38:20 kid1| Set Current Directory to /var/cache/squid
2017/07/11 14:38:20 kid1| Finished loading MIME types and icons.
2017/07/11 14:38:20 kid1| HTCP Disabled.
2017/07/11 14:38:20 kid1| Squid plugin modules loaded: 0
2017/07/11 14:38:20 kid1| Adaptation support is off.
2017/07/11 14:38:20 kid1| Accepting SSL bumped HTTP Socket connections at 
local=[::]:3128 remote=[::] FD 28 flags=9
2017/07/11 14:38:20 kid1| Done reading /var/cache/squid swaplog (0 entries)
2017/07/11 14:38:20 kid1| Store rebuilding is 0.00% complete
2017/07/11 14:38:20 kid1| Finished rebuilding storage from disk.
2017/07/11 14:38:20 kid1|         0 Entries scanned
2017/07/11 14:38:20 kid1|         0 Invalid entries.
2017/07/11 14:38:20 kid1|         0 With invalid flags.
2017/07/11 14:38:20 kid1|         0 Objects loaded.
2017/07/11 14:38:20 kid1|         0 Objects expired.
2017/07/11 14:38:20 kid1|         0 Objects cancelled.
2017/07/11 14:38:20 kid1|         0 Duplicate URLs purged.
2017/07/11 14:38:20 kid1|         0 Swapfile clashes avoided.
2017/07/11 14:38:20 kid1|   Took 0.01 seconds (  0.00 objects/sec).
2017/07/11 14:38:20 kid1| Beginning Validation Procedure
2017/07/11 14:38:20 kid1|   Completed Validation Procedure
2017/07/11 14:38:20 kid1|   Validated 0 Entries
2017/07/11 14:38:20 kid1|   store_swap_size = 0.00 KB
2017/07/11 14:38:20 kid1| WARNING: ssl_crtd #Hlpr1 exited
2017/07/11 14:38:20 kid1| Too few ssl_crtd processes are running (need 1/32)
2017/07/11 14:38:20 kid1| Closing HTTP port [::]:3128
2017/07/11 14:38:20 kid1| storeDirWriteCleanLogs: Starting...
2017/07/11 14:38:20 kid1|   Finished.  Wrote 0 entries.
2017/07/11 14:38:20 kid1|   Took 0.00 seconds (  0.00 entries/sec).
FATAL: The ssl_crtd helpers are crashing too rapidly, need help!

Squid Cache (Version 3.5.26): Terminated abnormally.
CPU Usage: 0.044 seconds = 0.028 user + 0.016 sys
Maximum Resident Size: 98768 KB
Page faults with physical i/o: 0
2017/07/12 10:32:53| Set Current Directory to /var/cache/squid

Update 2

Removing the folder /var/lib/ssl_db then rerunning /lib/squid/ssl_crtd -c -2 succeeded. The cache log is then clean of errors but the service still does not start :(.

Update 3

After a reboot, the service starts fine. I will post this as the answer.

Solution

Checking the /var/log/squid/cache.log file revealed that ssl_crtd was crashing because the SSL_db was not initializing properly. The process to fix this is:

1. Remove ssl_db folder, location shown in log file
2. Run /lib/squid/ssl_crtd -c -s
3. Reboot the server
4. Start service

Comments

[Panther]

We need a lot more information to help debug this =) See serverfault.com/questions/785034/…

[maxhap]

Thanks for the quick response bodhi.zazen. What information would you like? I have already come across the post you liked but the question/issue it relates to is not the same, I'm not even up to setting the proxy on a client machine haha. I have found some related entries in the cache.log. I will update my question with them.