Saturday, October 31, 2009

Using spamassassin program with IndiMail

Just few days back a user asked me whether spamassassin can be used with IndiMail.

IndiMail uses environment variables SPAMFILTER, SPAMEXITCODE to configure any spam filter to be used. All that is required for the spam filter is to read a mail message on stdin, output the message back on stdout and exit with a number which indicates whether the message is ham or spam.

The default installation of IndiMail creates a configuration where mails get scanned by bogofilter for spam filtering. bogofilter exits with value '0' in case the message is spam and with value '1' when message is ham. The settings for SPAMFILTER, SPAMEXITCODE is as below

SPAMFILTER="/usr/bin/bogofilter -p -u -d /etc/indimail"
SPAMEXITCODE=0

Assuming that you have installed, setup and trained spamassassin, you can follow the instructions below to have IndiMail use spamassassin.

spamassasin has a client spamc which exits 1 when message is spam and exits 0 if the message is ham. To use spamassassin, just use the following for SPAMFILTER, SPAMEXITCODE


SPAMFILTER="path_to_spamc_program -E-d host -p port -u user"
SPAMEXITCODE=1

(see the documentation on spamc for description of arguments to spamc program). You an also use -U socket_path, to use unix domain socket instead of -d host, which uses tcp/ip

Since IndiMail uses envdir program to set environment variable, a simple way would be to set SPAMFILTER, SPAMEXITCODE is to do the following
% su
# echo "spamcPath -E -d host -p port -u user" > /service/qmail-smtpd.25/variables/SPAMFILTER
# echo 1 > /service/qmail-smtpd.25/variables/SPAMEXITCODE

What if you want to use both bogofilter and spamasssin. You can use a simple script like below as the SPAMFILTER program
#!/bin/bash
#
# you can -U option in spamc, pointing to a unix domain path instead of -d
#
DESTHOST=x.x.x.x

#
# pass the output of bogofilter to spamc and passthrough spamc output to stdout
# store the exit status of bogofilter in status1 and spamc in status2
#
/usr/bin/bogofilter -p -d /etc/indimail | /usr/bin/spamc -E -d $DESTHOST -p 783
STATUS=("${PIPESTATUS[@]}")
status1=${STATUS[0]}
status2=${STATUS[1]}

# bogofilter returned error
if [ $status1 -eq 2 ] ; then
exit 2
fi
# spamc returned error see the man page for spamc
if [ $status2 -ge 64 -a $status2 -le 78 ] ; then
exit 2
fi

#
# message is spam
# bogofilter returns 0 on spam, spamc returns 1 on spam
#
if [ $status1 -eq 0 -o $status2 -eq 1 ] ; then
exit 0
fi
exit 1

Let us call the above script as bogospamc and let us place it in /usr/bin
% su
# echo /usr/bin/bogospamc > /service/qmail-smtpd.25/variables/SPAMFILTER
# echo 0 > /service/qmail-smtpd.25/variables/SPAMEXITCODE

Saturday, October 24, 2009

Troublesome MySQL Configuration

Quite a few of users who attempt to install IndiMail first time, face the biggest issue with MySQL installation and configuration. Most of the issues relate to the following
  1. MySQL version less than 5.1: MySQL fails to startup under supervise
  2. MySQL version less than 5.1: svctool fails to create a default database for IndiMail during rpm installation.
  3. /etc/my.cnf uses socket=/var/lib/mysql/mysql.sock. IndiMail uses /tmp/mysql.sock. Connection to MySQL fails if you use mysql(1) (without -S opton).
  4. You get MySQL syntax error when running indimail programs
The solution to the first problem is to delete the lines --general-log-file and the line --slow-query-log from the file /service/mysql.3306/run. After deleting the lines you can issue the command

/var/indimail/bin/svc -u /service/mysql.3306

to start up MySQL.

The solution to the second problem is to create a blank MySQL database by running the following command

/var/indimail/bin/svc -d /service/mysql.3306
/bin/rm -r /var/indimail/mysqldb/data
/usr/bin/mysql_install_db --user=mysql --datadir=/var/indimail/mysqldb/data

chown -R mysql:mysql /var/indimail/mysqldb/data
/var/indimail/bin/svc -u /service/mysql.3306
mysql -u root -p
mysql> use mysql;
mysql> update user set password=PASSWORD('some_pass') where user='root';
mysql> CREATE USER indimail identified by 'ssh-1.5-';
mysql> CREATE USER mysql identified by '4-57343-';
mysql> CREATE USER admin identified by 'benhur20';
mysql> CREATE USER repl identified by 'slaveserver';
mysql> GRANT SELECT,CREATE,ALTER,INDEX,INSERT,UPDATE,DELETE, \
mysql> CREATE TEMPORARY TABLES, \
mysql> LOCK TABLES ON indimail.* to 'indimail';
mysql> GRANT RELOAD,SHUTDOWN,PROCESS,SUPER on *.* to admin;
mysql> GRANT REPLICATION SLAVE on *.* to repl;


The third problem can be solved by replacing socket=/var/lib/mysql/mysql.sock with socket=/tmp/mysql.sock in /etc/my.cnf or equivalent file. You can also copy /var/indimail/etc/indimail.cnf as .indimail.cnf in your home directory. i.e.

cp /var/indimail/etc/indimail.cnf $HOME/.indimail.cnf


To avoid the above problems, it is recommended to use one of the below MySQL versions
  • mysql-6.0.3-alpha
  • mysql-6.0.9-alpha
  • mysql-5.0.26
  • mysql-5.0.77
  • mysql-5.1.40
  • mysql-5.4.3-beta
  • mysql-5.5.0-m2
  • MariaDB 5.1.42
Let me know if you are using any version of MySQL other than the above. Run the following command to get the MySQL version

% mysql_config --version

The fourth problem relates to a workaround made in IndiMail to prevent MySQL injection. What is needed is to set NO_BACKSLASH_ESCAPES in the MySQL server. You can use either of the two methods below
  1. This SQL mode also can be enabled automatically when the server starts by using the command-line option
    `--sql-mode=NO_BACKSLASH_ESCAPES' or by setting
  2. Set `sql-mode=NO_BACKSLASH_ESCAPES' in the server option file (for
    example, `my.cnf' or `my.ini', depending on your system).


Thursday, October 15, 2009

Cost of writing IndiMail

IndiMail is Free Software / Open Source Software

What would it have costed to code IndiMail if it wasn't free and if there wasn't any FS/OSS ?

Using David A. Wheeler's 'SLOCCount' gives the following result. The cost has been arrived by using $12000/year as the average salary of an Indian programmer. Sloccount uses COCOMO Software Cost Estimation Model.

Total Estimated Cost to Develop = $ 3,011,907

SLOC Directory SLOC-by-Language (Sorted)

106466 clamav-0.95.2 ansic=94967,sh=11300,perl=199
73173 qmail-1.03 ansic=68645,perl=2361,sh=2167
58207 indimail-1.6.2 ansic=51563,sh=6644
45914 indium-1.0 tcl=39628,sh=5604,ansic=682
41943 bogofilter-1.2.1 ansic=34807,sh=4629,perl=1842,lex=475,lisp=179
14670 flash-0.9.4 ansic=8346,sh=6324
9762 ucspi-tcp-0.88 ansic=9581,sh=181
9362 altermime-0.3.10 ansic=5963,sh=3399
9317 mpack-1.6 ansic=9292,perl=25
8131 ripmime-1.4.0.9 ansic=8116,sh=15
4256 pam-multi-1.0 ansic=3053,sh=1203
3318 libdkim-1.3 cpp=2316,ansic=924,sh=78
1833 nssd-1.0 ansic=1833
1772 fortune-1.1 ansic=1698,sh=74


Totals grouped by language (dominant language first):
ansic: 299470 (77.16%)
sh: 41618 (10.72%)
tcl: 39628 (10.21%)
perl: 4427 (1.14%)
cpp: 2316 (0.60%)
lex: 475 (0.12%)
lisp: 179 (0.05%)


Total Physical Source Lines of Code (SLOC) = 388,124
Development Effort Estimate, Person-Years (Person-Months) = 104.58 (1,254.96)
(Basic COCOMO model, Person-Months = 2.4 * (KSLOC**1.05))
Schedule Estimate, Years (Months) = 3.14 (37.62)
(Basic COCOMO model, Months = 2.5 * (person-months**0.38))
Estimated Average Number of Developers (Effort/Schedule) = 33.36
Total Estimated Cost to Develop = $ 3,011,907
(average salary = $12,000/year, overhead = 2.40).


"generated using David A. Wheeler's 'SLOCCount'."

IndiMail Queue Mechanism

Indimail has the ability of configuring multiple local and remote queues. A queue is a location on your hard disk where email are deposited ...