fail2ban apache-noscript not working

Question

My debian server has come under attack recently and I've been poring over the logs to see what I can do to harden it. I've been running fail2ban for a while but I've noticed the apache-noscript jail doesn't seem to be working. The fail2ban package has recently been updated and the contents of this particular jail has changed but I don't know if it ever worked before. This is the jail as it is now:

# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
# $Revision: 728 $
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = apache-common.conf

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failure messages in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values:  TEXT
#
failregex = ^%(_apache_error_client)s (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)\s*$
            ^%(_apache_error_client)s script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat\s*$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =

It now refers to apache-common.conf which looks like this:

# Generic configuration items (to be used as interpolations) in other
# apache filters
#
# Author: Yaroslav Halchenko
#
#

[INCLUDES]

# Load customizations if any available
after = apache-common.local


[DEFAULT]

# Common prefix for [error] apache messages which also would include <HOST>
_apache_error_client = \[[^]]+\] \[error\] \[client <HOST>\]

This is my jail.local:

# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
# $Revision: 728 $
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# common.local
before = apache-common.conf

[Definition]

# Option:  failregex
# Notes.:  regex to match the password failure messages in the logfile. The
#          host must be matched by a group named "host". The tag "<HOST>" can
#          be used for standard IP/hostname matching and is only an alias for
#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
# Values:  TEXT
#
failregex = ^%(_apache_error_client)s (File does not exist|script not found or unable to stat): /\S*(\.php|\.asp|\.exe|\.pl)\s*$
            ^%(_apache_error_client)s script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat\s*$

# Option:  ignoreregex
# Notes.:  regex to ignore. If this regex matches, the line is ignored.
# Values:  TEXT
#
ignoreregex =
root@serverb3:/etc/fail2ban/filter.d# cat ../jail.local
# Fail2Ban configuration file.
#
# This file was composed for Debian systems from the original one
#  provided now under /usr/share/doc/fail2ban/examples/jail.conf
#  for additional examples.
#
# To avoid merges during upgrades DO NOT MODIFY THIS FILE
# and rather provide your changes in /etc/fail2ban/jail.local
#
# Author: Yaroslav O. Halchenko <[email protected]>
#
# $Revision$
#

# The DEFAULT allows a global definition of the options. They can be overridden
# in each jail afterwards.

[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host
ignoreip = 127.0.0.1/8 62.84.178.107 217.204.106.228 192.168.10.0/24 192.168.0.1
#ignoreip = 127.0.0.1/8
bantime  = 600
maxretry = 3

# "backend" specifies the backend used to get files modification. Available
# options are "gamin", "polling" and "auto".
# yoh: For some reason Debian shipped python-gamin didn't work as expected
#      This issue left ToDo, so polling is default backend for now
backend = auto

#
# Destination email address used solely for the interpolations in
# jail.{conf,local} configuration files.
destemail = root@localhost

#
# ACTIONS
#

# Default banning action (e.g. iptables, iptables-new,
# iptables-multiport, shorewall, etc) It is used to define
# action_* variables. Can be overridden globally or per
# section within jail.local file
banaction = iptables-multiport

# email action. Since 0.8.1 upstream fail2ban uses sendmail
# MTA for the mailing. Change mta configuration parameter to mail
# if you want to revert to conventional 'mail'.
mta = sendmail

# Default protocol
protocol = tcp

# Specify chain where jumps would need to be added in iptables-* actions
chain = INPUT

#
# Action shortcuts. To be used to define action parameter

# The simplest action to take: ban only
action_ = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]

# ban & send an e-mail with whois report to the destemail.
action_mw = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
              %(mta)s-whois[name=%(__name__)s, dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"]

# ban & send an e-mail with whois report and relevant log lines
# to the destemail.
action_mwl = %(banaction)s[name=%(__name__)s, port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
               %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]

# Choose default action.  To change, just override value of 'action' with the
# interpolation to the chosen action shortcut (e.g.  action_mw, action_mwl, etc) in jail.local
# globally (section [DEFAULT]) or per specific section
action = %(action_)s

#
# JAILS
#

# Next jails corresponds to the standard configuration in Fail2ban 0.6 which
# was shipped in Debian. Enable any defined here jail by including
#
# [SECTION_NAME]
# enabled = true

#
# in /etc/fail2ban/jail.local.
#
# Optionally you may override any other parameter (e.g. banaction,
# action, port, logpath, etc) in that section within jail.local

[ssh]

enabled  = true
port     = 666
filter   = sshd
logpath  = /var/log/auth.log

[dropbear]

enabled  = false
port     = ssh
filter   = sshd
logpath  = /var/log/dropbear
maxretry = 6

# Generic filter for pam. Has to be used with action which bans all ports
# such as iptables-allports, shorewall
[pam-generic]

enabled  = true
# pam-generic filter can be customized to monitor specific subset of 'tty's
filter   = pam-generic
# port actually must be irrelevant but lets leave it all for some possible uses
port     = all
banaction = iptables-allports
port     = anyport
logpath  = /var/log/auth.log

[xinetd-fail]

enabled   = true
filter    = xinetd-fail
port      = all
banaction = iptables-multiport-log
logpath   = /var/log/daemon.log
maxretry  = 2


[ssh-ddos]

enabled  = true
port     = 666
filter   = sshd-ddos
logpath  = /var/log/auth.log

#
# HTTP servers
#

[apache]

enabled  = true
port     = http,https
filter   = apache-auth
logpath  = /var/log/apache*/*error.log

# default action is now multiport, so apache-multiport jail was left
# for compatibility with previous (<0.7.6-2) releases
[apache-multiport]

enabled   = false
port      = http,https
filter    = apache-auth
logpath   = /var/log/apache*/*error.log
maxretry  = 6

[apache-noscript]

enabled  = true
port     = http,https
filter   = apache-noscript
logpath  = /var/log/apache*/*error.log
maxretry = 0
bantime = 2419200

[apache-overflows]

enabled  = true
port     = http,https
filter   = apache-overflows
logpath  = /var/log/apache*/*error.log
maxretry = 2

[apache-nohome]

enabled = true
port = http,https
filter = apache-nohome
logpath = /var/log/apache*/*error.log
findtime = 600

[apache-404]

enabled = true
port = http,https
filter = apache-404
logpath = /var/log/apache*/*access.log
findtime = 600
maxretry = 5
bantime = 86400

[apache-400]

enabled = true
port = http,https
filter = apache-400
logpath = /var/log/apache*/*.log
maxretry = 0
bantime = 2419200

[web-admin]

enabled = true
port = http,https
filter = web-admin
logpath = /var/log/auth.log

[apache-w00t]

enabled = true
port = http,https
filter = apache-w00t
logpath = /var/log/apache*/*access.log
maxretry = 0
bantime = 2419200

[apache-vtigercrm]

enabled = true
port = http,https
filter = apache-vtigercrm
logpath = /var/log/apache*/*access.log
maxretry = 0
bantime = 2419200

[apache-vicidial]

enabled = true
port = http,https
filter = apache-vicidial
logpath = /var/log/apache*/*access.log
maxretry = 0
bantime = 2419200

[apache-phpmyadmin]

enabled = true
port = http,https
filter = apache-phpmyadmin
logpath = /var/log/apache*/*access.log
maxretry = 0
bantime = 2419200


#
# FTP servers
#

[vsftpd]

enabled  = false
port     = ftp,ftp-data,ftps,ftps-data
filter   = vsftpd
logpath  = /var/log/vsftpd.log
# or overwrite it in jails.local to be
# logpath = /var/log/auth.log
# if you want to rely on PAM failed login attempts
# vsftpd's failregex should match both of those formats
maxretry = 6


[proftpd]

enabled  = true
port     = ftp,ftp-data,ftps,ftps-data
filter   = proftpd
logpath  = /var/log/proftpd/proftpd.log


[pure-ftpd]

enabled  = false
port     = ftp,ftp-data,ftps,ftps-data
filter   = pure-ftpd
logpath  = /var/log/auth.log
maxretry = 6


[wuftpd]

enabled  = false
port     = ftp,ftp-data,ftps,ftps-data
filter   = wuftpd
logpath  = /var/log/auth.log
maxretry = 6


#
# Mail servers
#

[postfix]

enabled  = true
port     = smtp,ssmtp
filter   = postfix
logpath  = /var/log/mail.log
maxretry = 5
bantime = 86400

[couriersmtp]

enabled  = false
port     = smtp,ssmtp
filter   = couriersmtp
logpath  = /var/log/mail.log

[roundcube]
enabled  = true
port     = http,https
filter   = roundcube
logpath  = /home/web/roundcube*/logs/errors
findtime = 600

#
# Mail servers authenticators: might be used for smtp,ftp,imap servers, so
# all relevant ports get banned
#

[courierauth]

enabled  = false
port     = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter   = courierlogin
logpath  = /var/log/mail.log


[sasl]

enabled  = false
port     = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter   = sasl
maxretry = 6
# You might consider monitoring /var/log/mail.warn instead if you are
# running postfix since it would provide the same log lines at the
# "warn" level but overall at the smaller filesize.
logpath  = /var/log/mail.log

[dovecot]

enabled = true
port    = smtp,ssmtp,imap2,imap3,imaps,pop3,pop3s
filter  = dovecot
logpath = /var/log/mail.log
findtime = 600

# DNS Servers


# These jails block attacks against named (bind9). By default, logging is off
# with bind9 installation. You will need something like this:
#
# logging {
#     channel security_file {
#         file "/var/log/named/security.log" versions 3 size 30m;
#         severity dynamic;
#         print-time yes;
#     };
#     category security {
#         security_file;
#     };
# };
#
# in your named.conf to provide proper logging

# !!! WARNING !!!
#   Since UDP is connection-less protocol, spoofing of IP and imitation
#   of illegal actions is way too simple.  Thus enabling of this filter
#   might provide an easy way for implementing a DoS against a chosen
#   victim. See
#    http://nion.modprobe.de/blog/archives/690-fail2ban-+-dns-fail.html
#   Please DO NOT USE this jail unless you know what you are doing.
#[named-refused-udp]
#
#enabled  = false
#port     = domain,953
#protocol = udp
#filter   = named-refused
#logpath  = /var/log/named/security.log

[named-refused-tcp]

enabled  = false
port     = domain,953
protocol = tcp
filter   = named-refused
logpath  = /var/log/named/security.log

All other jails are working as far as I can see.

If I run fail2ban-regex "/var/log/apache2/error.log" /etc/fail2ban/filter.d/apache-noscript.conf it gets two hits. One of the IP addresses it finds is banned because it also triggered another jail but the other IP address isn't banned at all (i.e. grepping the fail2ban logs for "noscript" gets no hits and nothing in "iptables -L -n"). Here are the related entries from the apache error.log:

[Wed Nov 13 02:11:06 2013] [error] [client 91.226.212.41] script '/home/web/remote.php' not found or unable to stat

[Fri Nov 15 00:20:34 2013] [error] [client 85.114.135.126] script '/home/web/wp-login.php' not found or unable to stat

It looks to me like both these entries should have triggered the jail. Any ideas why they didn't?

TIA.

Solution 1

I know it's an old thread, but still on top of google searches on this topic. I've just run into a very similar issue after upgrading Debian to Stretch (and such Apache to 2.4.25)

Fail2Ban worked OK, but some apache-related jails were not match its filters' regex patterns. After a few hours of trial and error I discovered, that fail2ban uses the tag to identify the remote host, but it is replacing only the IP address/host name, and since newer Apache log is created with adding the port after host IP, it will not match.

So the original regex line in fail2ban's apache-noscript.conf:

[[]client <HOST>[]] script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat*$

did not match a corresponding line in error log, like:

[client 213.97.42.29:50067] script '/home/www/vhost/webroot/wp-login.php' not found or unable to stat

After added the optional :port pattern to the regex, it worked like charm.

Note that the tag precedes the optional port number regex pattern, so this must be added between and the following [ character, like this:

<HOST>(:\d{1,5})?

So the same line looks like this after modification:

[[]client <HOST>(:\d{1,5})?[]] script '/\S*(\.php|\.asp|\.exe|\.pl)\S*' not found or unable to stat*$

I hope it will help someone in the future.

Solution 2

I also noticed apache-noscript.conf couldn't catch entries like script not found or unable to stat: /usr/lib/cgi-bin/php and its variants in /var/log/apache2/error.log on my debian wheezy laptop server.

[Fri Dec 20 20:09:34 2013] [error] [client 89.248.160.192] script not found or unable to  stat: /usr/lib/cgi-bin/php
[Fri Dec 20 20:09:34 2013] [error] [client 89.248.160.192] script not found or unable to stat: /usr/lib/cgi-bin/php4
[Fri Dec 20 20:09:35 2013] [error] [client 89.248.160.192] script not found or unable to stat: /usr/lib/cgi-bin/php5
[Fri Dec 20 20:09:35 2013] [error] [client 89.248.160.192] script not found or unable to stat: /usr/lib/cgi-bin/php-cgi
[Fri Dec 20 20:09:36 2013] [error] [client 89.248.160.192] script not found or unable to stat: /usr/lib/cgi-bin/php.cgi

After closely examining the filter /etc/fail2ban/filter.d/apache-noscript.conf, I realised that the lack of php and its variants without a leading period in the failregex expression is what was making the filter to fail.

After modifying the failregex expression by adding \php|\php4|\php5|\php-cgi|\php.cgi as below,

failregex = ^%(_apache_error_client)s (File does not exist|script not found or unable to stat: /\S*(\php|\php4|\php5|\php-cgi|\php.cgi|\.php|\.asp|\.exe|\.pl)\s*$

and testing the filter by running

fail2ban-regex "/var/log/apache2/error.log" /etc/fail2ban/filter.d/apache-noscript.conf

the filter caught all the script not found or unable to stat: /usr/lib/cgi-bin/php entries and its variants! Issue resolved. I hope this helps somebody else.

Comments

[user102469]

Emailed same question to fail2ban mailing list. Will post back here if I get a reply.

[user102469]

Thanks for the reply. I think there's something wrong with your regex though as fail2ban-regex throws a wobbler when I use it - "Unable to compile regular expression..."

[Alexis Wilke]

In a regex, if you have \. and want to remove the period, then remove both characters: the backslash and the dot. So \.php becomes php. I suppose the \p does not hurt, but some escaped character have a meaning such as the \S and \s which we see in the same regex.

[Darren]

Thanks for the answer. I am the original OP but lost that old account info. I can’t check the format of my apache logs at the moment but I do know the noscript jail is working ok for me. I suggest you try and file this as a bug with the fail2ban developers.