Missing X-Spam-Status header

Question

I recently upgraded to Ubuntu 14.04.1 LTS (trusty) and have followed the directions in https://help.ubuntu.com/14.04/serverguide/mail-filtering.html and am sending and receiving mail just fine.

While I do see X-Virus-Scanned headers in my messages, which suggests mail is indeed being processed, I do not see any X-Spam-Level or X-Spam-Score headers being added to messages. This makes downstream procmailrc and client-side filtering ...more difficult.

While having $final_spam_destiny = D_DISCARD in /etc/amavis/conf.d/20-debian_defaults does greatly reduce spam to my inbox, I had concerns of false-positives prior to tuning and didn't know were there going, so have set it to D_PASS for the time being. This exposed the problem.

I'm not sure where to look to start diagnosing the problem (otherwise I'd post a suspect configuration file).

/etc/amavis/conf.d/15-content_filter_mode has the lines uncommented to enable virus and spam checks, and virus checking appears to be working according to the headers.

Spam Assassin certainly seems to be starting just fine, too.

SpamAssassin debug facilities: info
SA info: zoom: able to use 360/360 'body_0' compiled rules (100%)
SpamAssassin loaded plugins: AskDNS, AutoLearnThreshold, Bayes, BodyEval, Check, DKIM, DNSEval, FreeMail, HTMLEval, HTTPSMismatch, Hashcash, HeaderEval, ImageInfo, MIMEEval, MIMEHeader, Pyzor, Razor2, RelayEval, ReplaceTags, Rule2XSBody, SPF, SpamCop, URIDNSBL, URIDetail, URIEval, VBounce, WLBLEval, WhiteListSubject
SpamControl: init_pre_fork on SpamAssassin done

I've also set $log_level = 2; in /etc/amavis/conf.d/50-user and don't see any obvious errors rolling by in the logs.

Q: Any recommendations of what to try next?

UPDATE (it appears that I have the right setting already):

/etc/amavis/conf.d$ grep sa_tag_level_deflt *
20-debian_defaults:# $sa_tag_level_deflt  = 2.0;  # add spam info headers if at, or above that level
20-debian_defaults:$sa_tag_level_deflt  = -999;  # add spam info headers if at, or above that level

Solution 1

So, problem solved, and it was a conglomeration of things, which I'll detail here for others finding themselves down the same path.

First of all, spamassassin appears to have issues (in some environments) running with ipv6. Dunno why. Edit /etc/default/spamassassin, or /etc/init.d/spamassassin if you don't have one, and add --ipv4 to the OPTIONS list. Then use service spamassassin restart to bounce the service, and service spamassassin status to check that spamd is running. This step may not be necessary for your configuration.

Next, make sure that you have an /etc/procmailrc file, that it's permissions are world readable, and that it contains something like this:

DROPPRIVS=yes
:0fw
| /usr/bin/spamc
:0
* ^X-Spam-Status: Yes
$HOME/mail/possible-spam

Next, if you're running webmin you'll want to check the procmail service, and make sure it's not issuing you a warning that postfix is not configured to use procmail. In /etc/postfix/main.cf, you'll want to comment out the default mailbox_command to run deliver and replace it with procmail. You'll need to run service postfix restart after making the edit.

#mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/dovecot.conf -m "${EXTENSION}"
mailbox_command = procmail -a "$EXTENSION"

If you've been manually moving your spam messages into a mail folder called spam, then you want to train spam assassin like this:

sa-learn --progress  --dbpath ~/.spamassassin --spam --mbox ~/mail/spam

Note that some further tuning can be done, and it will require doing a service spam assassin restart afterward, but this can help.

One, in /etc/spamassassin/local.cf, set the required_score to a slightly lower value if you get lots of spam:

#   Set the threshold at which a message is considered spam (default: 5.0)
#
required_score 4.0

In the same file, /etc/spamassassin/local.cf, it is also possible to filter by spam-heavy countries by adding this at the end. Here's Russia and China, two very heavy contributors historically speaking based on some email analysis.

# Filter Russia
header RELAYCOUNTRY_RU X-Relay-Countries =~ /RU/
describe RELAYCOUNTRY_RU Relayed through Russia
score RELAYCOUNTRY_RU 8.0

# Filter China
header RELAYCOUNTRY_CN X-Relay-Countries =~ /CN/
describe RELAYCOUNTRY_CN Relayed through China
score RELAYCOUNTRY_CN 8.0

report_header 1
fold_headers 1
detailed_phrase_score 1

The above not only solved the missing header, which I suspect was the procmail not getting invoked.

Solution 2

The output of command head -n 1 /etc/mailname must be your domain, like example.com.

If the output is something like mail.example.com, that's the bug.

So, you can fix this by changing the file /etc/amavis/conf.d/05-domain_id

Solution #1

Inside /etc/amavis/conf.d/05-domain_id:

Change:

    chomp($mydomain = `head -n 1 /etc/mailname`);

To:

    chomp($mydomain = `hostname -d`);

PS: The output of hostname -d must be your domain, like example.com.

Simplest solution than ever:

Inside /etc/amavis/conf.d/05-domain_id:

Change:

    chomp($mydomain = `head -n 1 /etc/mailname`);

To:

    $mydomain = "example.com";

Solution 3

/etc/amavis/conf.d/20-debian_defaults http://pastebin.com/5xuvNVtD

Change line 36 to something like this:

$sa_tag_level_deflt = -999; # add spam info headers if at, or above that level

After saving and restarting of the service or reloading the configuration, all the emails should have the x-spam-status header.

Comments

[Anders F. U. Kiær]

@WaltStoneburner Which command did you use?

[Walt Stoneburner]

Bounced the a mavis, dovecot, and postfix services... just because. I got a message tagged as spam and it had the header -- so now I'm convinced it's working (thank you), and my new problem is that it's just not catching a whole lot yet. Is there a training phase like spamprobe has?

[Anders F. U. Kiær]

maybe this can lead you in the right direction xyzzyxyzzy.net/2012/09/15/…

[Walt Stoneburner]

I've been trying the edit the last comment (if not to fix the typo). I used $ sudo service [...] restart, which worked as expected. I'm currently looking at the sa-learn command, so seeing this resource helps me feel like I'm very much on the right track. Thanks all.

[Walt Stoneburner]

Discovered I had a very dated .spamassassin directory (it got neglected when I switch to a different technology), now that I'm back, it couldn't use it. Used sa-learn, sa-update, and bounced the spam assassin service.

[Walt Stoneburner]

Great tip! My own /etc/mailname file is producing the fully qualified machine name and the hostname -d was correctly producing the domain, so don't know if that was it for my system. But great advice.