Skip to main content

virus scanning using QHPSI

A large fraction of today’s emails is infected by a virus or a worm. It is necessary to recognize those malicious emails as soon as possible already in the DATA phase of the SMTP conversation and to reject them.
When you use IndiMail, it is ultimately qmail-queue which is responsible for queueing your messages. qmail-queue stores the message component of queued mails (captured duing DATA phase of the SMTP conversation) under the mess subdirectory.
Files under the mess subdirectory are named after their i-node number. Let us look at a typical log sequence for a message received on the local system.
@400000004b9da2f03b424bb4 new msg 660188
@400000004b9da2f03b426324 info msg 660188: bytes 2794 from fogcreek_xxx@response.whatcounts.com qp 3223 uid 555
@400000004b9da2f03b42c0e4 starting delivery 6: msg 660188 to local mailstore@indimail.org
@400000004b9da2f03b42dc3c status: local 1/10 remote 0/20
@400000004b9da2f106a1e234 delivery 6: success: did_1+0+0/
@400000004b9da2f1091e676c status: local 0/10 remote 0/20
@400000004b9da2f1091fa3d4 end msg 660188


The above lines indicates that qmail has received a new message, and its queue ID is 660188. What this means is that is qmail-queue has created a file named /var/indimail/queue/mess/NN/660188. The i-node number of the file is 660188. This is the queue file that contains the message. The queue ID is guaranteed to be unique as long as the message remains in the queue (you can't have two files with the same i-node in a filesystem).
To perform virus scanning, it would be trivial to do virus scanning on the mess file above in qmail-queue itself. That is exactly what IndiMail does by using a feature called Qmail High Performance Virus Scanner (QHPSI). QHPSI was conceptualized by Erwin Hoffman. You can read here for more details.
IndiMail takes QHPSI forward by adding the ability to add plugins.
The QHPSI extension for qmail-queue allows to call an arbitary virus scanner directly, scanning the incoming data-stream on STDIN. Alternatively, it allows plugins to be loaded from the /var/indimail/plugins directory. This directory can be changed by defining PLUGINDIR environment variable. QHPSI can be advised to pass multiple arguments to the virus scanner for customization. To run external scanner or load scanner plugins, qmail-queue calls qhpsi, a program setuid to qscand. By default, qhpsi looks for the symbol virusscan to invoke the scanner. The symbol can be changed by setting the environment variable QUEUE_PLUGIN to the desired symbol.
Today’s virus scanner -- in particluar Clam AV -- work in resource efficient client/server mode (clamd/clamdscan) and include the feature to detect virii/worms in the base64 encoded data stream. Thus, there is no necessity to call additional programs (like reformime or ripmime) except for the virus scanner itself.
To enable virus scanning in IndiMail during the SMTP data phase, you can implement either of the two methods below

1. Using tcprules

Define QHPSI in tcp.smtp and rebuild tcp.smtp.cdb using tcprules.
:allow,QHPSI=’/usr/bin/clamdscan %s --quiet --no-summary’

2. Using envdir for SMTP service under supervise(8)

Define QHPSI in SMTP service's variable directory
# echo /usr/bin/clamdscan %s --quiet --no-summary > /service/qmail-smtpd.25/variables/QHPSI


If you have installed IndiMail using RPM available here or here, QHPSI is enabled by default by defining it in the qmail-smtpd.25 variables directory. If you have clamd, clamav already installed on your server, the rpm installation also installs two services under supervise.
  • freshclam - service to update the clamd virus databases
  • clamd - service to run the clamd scanner
You may need to disable clamd, freshclam startup by your system boot process and enable the startup under indimail. Do have the clamd, freshclam service started up by indimail, remove the down file. i.e.
# rm /service/freshclam/down /service/clamd/down
# /usr/bin/svc -u /service/clamd /service/freshclam


% tail -f /var/log/indimail/freshclam/current
@400000004b9da034170f6394 cdiff_apply: Parsed 17 lines and executed 17 commands
@400000004b9da03417103e54 Retrieving http://database.clamav.net/daily-10574.cdiff
@400000004b9da0342261b83c Trying to download http://database.clamav.net/daily-10574.cdiff (IP: 130.59.10.36)
Downloading daily-10574.cdiff [100%]g daily-10574.cdiff [ 13%]
@400000004b9da03509c39c64 cdiff_apply: Parsed 436 lines and executed 436 commands
@400000004b9da03510c3485c daily.cld updated (version: 10574, sigs: 24611, f-level: 44, builder: ccordes)
@400000004b9da03510c4d2e4 bytecode.cvd version from DNS: 2
@400000004b9da03510c4de9c bytecode.cvd is up to date (version: 2, sigs: 2, f-level: 44, builder: nervous)
@400000004b9da03510c82e44 Database updated (729340 signatures) from database.clamav.net (IP: 130.59.10.36)


% cat /var/log/indimail/clamd/current
@400000004b9da0260d6c1a94 Limits: Global size limit set to 104857600 bytes.
@400000004b9da0260d6c264c Limits: File size limit set to 26214400 bytes.
@400000004b9da0260d6c3204 Limits: Recursion level limit set to 16.
@400000004b9da0260d6c3dbc Limits: Files limit set to 10000.
@400000004b9da0260d6c4974 Archive support enabled.
@400000004b9da0260d6c5144 Algorithmic detection enabled.
@400000004b9da0260d6c5cfc Portable Executable support enabled.
@400000004b9da0260d6c68b4 ELF support enabled.
@400000004b9da0260d6c7084 Detection of broken executables enabled.
@400000004b9da0260e7abfbc Mail files support enabled.
@400000004b9da0260e7acb74 OLE2 support enabled.
@400000004b9da0260e7ad344 PDF support enabled.
@400000004b9da0260e7adefc HTML support enabled.
@400000004b9da0260e7ae6cc Self checking every 600 seconds.
@400000004b9da2a3116a177c No stats for Database check - forcing reload
@400000004b9da2a3206deb04 Reading databases from /var/indimail/share/clamd
@400000004b9da2a70489facc Database correctly reloaded (728651 signatures)
@400000004b9da2a7061e372c /var/indimail/queue/queue2/mess/16/660188: OK


Once you have QHPSI enabled, qmail-queue will add the header X-QHPSI in the mail. You will have the following header

X-QHPSI: clean

in case the email is clean and the following header if a virus is found

X-QHPSI: virus found

The default configuration of IndiMail will allow these emails to be delivered to the inbox. This is because some sites have have legislations like SOX, etc to enforce archiving of all emails that come into the system. In case you want to reject the email at SMTP you can do the following

# echo 1 > /service/qmail.smtpd/variables/REJECTVIRUS
# svc -d /service/qmail-smtpd.25
# svc -u /service/qmail-smtpd.25

One can also create a vfilter to deliver such email to the quarantine folder

/usr/bin/vcfilter -i -t virusFilter -c 0 -k "virus found" -f Quarantine -b 0 -h 28 prefilt@$1


If you implement different method, than explained above, let me know.



3 comments

Popular posts from this blog

IndiMail Installation for newbies in < 10 steps

Installing Indimail using YUM/APT Repository Install OS
OpenSUSE
openSUSE Leap 42.3
openSUSE Leap 42.2
openSUSE 13.2
openSUSE 13.1
SUSE Linux Enterprise 12 SP2
SUSE Linux Enterprise 12 SP1
SUSE Linux Enterprise 12
Red Hat
Feodra 27
Fedora 26
Red Hat Enterprise Linux 7
Red Hat Enterprise Linux 6
CentOS 7
CentOS 6
Debian
Debian 8.0
Debian 7.0
Ubuntu 17.04
Ubuntu 16.10
Ubuntu 16.04
Ubuntu 14.04
Ubuntu 12.04
Click the below URL for Install Instructions
https://software.opensuse.org/download.html?project=home%3Aindimail&package=indimail Shutdown MySQL if already running and disable MySQL from being started up by the system % /etc/init.d/mysqld stop % sudo chkconfig mysqld off % /bin/rm -f /service/mysql.3306/down
Start IndiMail
% sudo service indimail start Check Servicess
% sudo /usr/bin/svstat /service/* /service/clamd: up (pid 1014) 2985 seconds /service/dnscache: up (pid 1021) 2985 seconds /service/fetchmail: down 2985 seconds /service/freshclam: up (pid 1020) 2…

Using Docker Engine to Run IndiMail / IndiMail-MTA

IndiMail now has docker images. You can read about installing Dockerhere. Once you have installed docker-engine, you need to start it. Typically it would be
$ sudo service docker start
To avoid having to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes the ownership of the Unix socket read/writable by the docker group.
Warning: The docker group is equivalent to the root user; For details on how this impacts security in your system, see Docker Daemon Attack Surface for details.$ sudo groupadd docker $ sudo usermod -aG docker your_username
Log out and login again to ensure your user is running with the correct permissions. You can run the unix id command to confirm that you have the docker group privileges. e.g.
$ id -a uid=1000(mbhangui) gid=1000(mbhangui) groups=1000(mbhangui),10(wheel),545(docker) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Now we need to pull the docker image for I…

Writing Filters for IndiMail

IndiMail provides multiple methods by which you can intercept an email in transit and modify the email headers or the email body. A filter is a simple program that expects the raw email on standard input and outputs the message text back on standard output. The program /bin/cat can be used as a filter which simply copies the standard input to standard output without modifying anything. Some methods can be used before the mail gets queued and some methods can be used before the execution of local / remote delivery.

It is not necessary for a filter to modify the email. You can have a filter just to extract the headers or body and use that information for some purpose. IndiMail also provides the following programs - 822addr(1), 822headerfilter(1), 822bodyfilter(1), 822field(1), 822fields(1), 822header(1), 822body(1), 822headerok(1), 822received(1), 822date(1), 822fields(1) to help in processing emails.

Let us say that we have written a script /usr/local/bin/myfilter. The myfilter program …