Postfix + Amavisd-New Speed Up

Postfix + Amavisd-New Speed Up


So recently, I’ve begun to filter a lot more email than I used to. Sometime ago, I set up a standard CentOS server with postfix, postgrey, amavisd-new, spamassassin and clamav to filter email for my own domain. It was fine because I get hardly any email and frankly, RBL checks in postfix and postgrey stops 99% of the SPAM, anyway. However, overtime, as you do, I started filtering mail for other people, as well. Somehow, I managed to get to the point where I’m filtering 10’s of millions of emails a year (seriously).

And lately, the email queues have been growing a little too long for my liking. Postfix is pretty rock solid and does its job well – as do all the OSS components mentioned above. The VMs (plural) doing the filtering had pretty reasonable CPU usage (nothing too high) and free memory- yet mail queues were getting into the dozens of minutes of delay. Not good enough. qshape queries had become depressingly long. Dozens of lines and records into the 60 minute column.

So with ok CPU usage, yet an inability to process enough email at once, it’s obvious we’re not running multiple sessions properly. So we need to look at the $max_servers variable in the amavisd config file (/etc/amavisd.conf). In the image below, I’ve changed the value to 30.

$max_servers = 30;            # num of pre-forked children (2..30 is common), -m
$daemon_user  = "amavis";     # (no default;  customary: vscan or amavis), -u
$daemon_group = "amavis";     # (no default;  customary: vscan or amavis), -g

The default value for this is 2, so I tried changing it up 10. No difference. Email still sat in the queue no matter how many times I flushed it. Larger or smaller values didn’t help. So I thought perhaps it was a postfix issue or a clamavd issue, etc. and tried all of those angles. Nothing made any difference.

Anyway, it turns out (I didn’t know this, however many of you may have) that the maximum number of sessions of filtering amavisd-new can do is controlled in two places – not just in /etc/amavisd.conf. You also need to edit it /etc/postfix/ I probably should have thought of this faster – especially as I did change session values in there in my testing (for postfix, not for amavisd) but of course a long time passed since I set this up and had to revisit it and I wasn’t thinking clearly.

In the postfix master file, the value for the amavisd sessions must be identical to the amavisd.conf file in /etc/.

amavisfeed unix    -       -       n        -      30     lmtp
    -o lmtp_data_done_timeout=1200
    -o lmtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

Once these two values were the same, I was able to restart both services (amavisd and postfix) and the email queue flew like I’ve never seen. A queue with some 400 emails in it emptied in about a minute, instead of taking half an hour.

qshape now looks like this:

                                          T  5 10 20 40 80 160 320 640 1280 1280+
                                 TOTAL  0  0  0  0  0  0   0   0   0    0     0

a much happier sight!

Part Two

The next thing to notice is that on CentOS 6, the default (rpm based) install for amavisd and clamd is mismatched. So if we’re going to be throwing 30 sessions of virus scanning at once at the CPU, we need to be as efficient as possible.

The socket configured in /etc/clamd.conf is:

LocalSocket /var/run/clamav/clamd.sock

While in the /etc/amavisd.conf the setting is:

   \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd"],
   qr/\bOK$/m, qr/\bFOUND$/m,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],

And it should be noted that clamd is disabled by default (above lines are commented out), so CentOS uses clamscan instead of clamd. Clamscan is not a daemon and it’s far, far more processor intensive.

You need to change the socket above in /etc/amavisd.conf to match the socket in /etc/clamd.conf and then restart amavisd.

   \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.sock"],
   qr/\bOK$/m, qr/\bFOUND$/m,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],

You should now see that clamd is handling your virus scanning, not clamscan. The difference in CPU usage cannot be overstated. It’s simply enormous.

Using the above steps, I’ve managed to get CPU usage under control and email flowing on a much larger scale than previously possible.

About the Author

RodneyI'm a veteran of way too many years of IT (although I still love it) and I currently head up the techincal work over at Host One (major sponsor of this site), where I'm also a partner. Feel free to ask me anything about Cloud Computing and I'll try to be helpful, in a non-salesy kind of way.View all posts by Rodney →

Leave a Reply

Time limit is exhausted. Please reload CAPTCHA.