Tuesday, April 26, 2016

Using Docker Engine to Run IndiMail / IndiMail-MTA

IndiMail now has docker images. You can read about installing Docker here. Once you have installed docker-engine, you need to start it. Typically it would be
$ sudo service docker start

To avoid having to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes the ownership of the Unix socket read/writable by the docker group.
Warning: The docker group is equivalent to the root user; For details on how this impacts security in your system, see Docker Daemon Attack Surface for details.
$ sudo groupadd docker 
$ sudo usermod -aG docker your_username

Log out and login again to ensure your user is running with the correct permissions. You can run the unix id command to confirm that you have the docker group privileges. e.g.
$ id -a
uid=1000(mbhangui) gid=1000(mbhangui) groups=1000(mbhangui),10(wheel),545(docker) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Now we need to pull the docker image for IndiMail. use the docker pull command. The values for tag can be fedora-23, centos7, debian8, ubuntu-15.10, ubuntu-14.03. If your favourite OS is missing, let me know. You can find the list of all images here.
$ docker pull cprogrammer/indimail:tag

(for indimail-mta image, execute docker pull cprogrammer/indimail-mta:tag

You can now list the docker image by executing the docker images command.

$ docker images
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
cprogrammer/indimail       fedora-23           a02e6014a67b        53 minutes ago      1.774 GB

Now let us run a container with this image using the image id a02e6014a67b listed above by running the docker run command. The --privileged flag gives all capabilities to the container, and it also lifts all the limitations enforced by the device cgroup controller. In other words, the container can then do almost everything that the host can do. This flag exists to allow special use-cases, like running Docker within Docker. In our case, I want the systemctl command to work and the container run like a normal host.

$ docker run -d -h indimail.org --privileged a02e6014a67b /sbin/init

I have now figured out the you don't require the --privileged flag. This flag gives the container access to the host's systemd. A better way is to add SYS_ADMIN capability

$ docker run -ti --cap-add=SYS_ADMIN -e "container-docker" -v /sys/fs/cgroup:/sys/fs/cgroup:ro a02e6014a67b /sbin/init

The above will start a fully functional Fedora 23 OS with IndiMail, MySQL, sshd, httpd services up and running.

We can list the running container by running the docker ps command

$ docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
fd09c7ca75be        a02e6014a67b        "/sbin/init"        38 seconds ago      Up 37 seconds                           desperate_jones 

We now have a running container and can attach to it and use it like any functional host. Run the docker exec command. The -ti option attaches a pseudo terminal and makes the session interactive.
$ docker exec -ti fd09c7ca75be /bin/bash --login
#
# /var/indimail/bin/svstat /service/*
/service/fetchmail: down 32 seconds
/service/greylist.1999: up (pid 203) 32 seconds
/service/indisrvr.4000: up (pid 178) 32 seconds
/service/inlookup.infifo: up (pid 192) 32 seconds
/service/mysql.3306: up (pid 181) 32 seconds
/service/proxy-imapd.4143: up (pid 191) 32 seconds
/service/proxy-imapd-ssl.9143: up (pid 188) 32 seconds
/service/proxy-pop3d.4110: up (pid 197) 32 seconds
/service/proxy-pop3d-ssl.9110: up (pid 179) 32 seconds
/service/pwdlookup: up (pid 195) 32 seconds
/service/qmail-imapd.143: up (pid 222) 32 seconds
/service/qmail-imapd-ssl.993: up (pid 200) 32 seconds
/service/qmail-pop3d.110: up (pid 212) 32 seconds
/service/qmail-pop3d-ssl.995: up (pid 184) 32 seconds
/service/qmail-poppass.106: up (pid 216) 32 seconds
/service/qmail-qmqpd.628: down 32 seconds
/service/qmail-qmtpd.209: up (pid 153) 32 seconds
/service/qmail-send.25: up (pid 182) 32 seconds
/service/qmail-smtpd.25: up (pid 187) 32 seconds
/service/qmail-smtpd.366: up (pid 208) 32 seconds
/service/qmail-smtpd.465: up (pid 194) 32 seconds
/service/qmail-smtpd.587: up (pid 196) 32 seconds
/service/qmail-spamlog: up (pid 221) 32 seconds
/service/qscanq: up (pid 213) 32 seconds
/service/udplogger.3000: up (pid 211) 32 seconds
You now have a fully functional mail server with a pre-configured virtual domain indimail.org and a pre-configured virtual user testuser01@indimail.org. You can use IMAP/POP3/SMTP to your heart's content. If not satisfied, try out the ssl enabled services IMAPS/POP3S/SMTPS or STARTTLS command. If still not satisfied, read the man pages in /var/indimail/man/* :)

You can stop the container by executing the docker stop command.

$ docker stop fd09c7ca75be

You can make your changes to the container and commit changes by using the docker commit command. Learning how to use docker is not difficult. Just follow the Docker Documentation. If you are lazy like me, just read the Getting Started guide.

I am also a newbie as far as docker is concerned. Do let me know your experience with network settings and other advanced docker topics, that you may be familiar with. Do send few bottles of beer my way if you can.

NOTE: There are few defaults for the indimail docker container image
  • root password is passxxx@xxx
  • mysql user, password for indimail is indimail, ssh-1.5-
  • mysql privileged user, password is mysql, 4-57343-
  • password for postmaster@indimail.org virtual imap/pop3 account is passxxx
  • password for testuser01@indimail.org virtual imap/pop3 account is passxxx

27 comments:

Unknown said...

Hello, thanks for the tutorial! After installing the server I'd ask how can I send/receive emails/msg with this server, and preferably with QMTP protocol? Is there another blog on this topic? Thank you!

cprogrammer said...

You can enable QMTP service by doing the following

# rm /service/qmail-qmtpd.209/down
# svc -u /service/qmail-qmtpd.209

qmail-remote will automatically use QMTP if it sees mail exchanger records for QMTP. See this
https://cr.yp.to/im/mxps.html

You can also setup /etc/indimail/control/qmtproutes for artificial qmtp routes. See the man page for qmail-remote

Unknown said...

Hi @cprogrammer thank you for the kind reply!

However I still have difficulties understanding everthing. For example the synopsis for sending mail with qmail-remote is qmail-remote host sender recip right? If I want to send a mail/msg from this docker container to user@gmail.com, the command I use is
qmail-remote gmail.com root@indimail.org user@gmail.com
And I got this error msg: DI (qmail-remote) was invoked improperly. (#5.3.5)

Do you have any idea about this? Thank you!

cprogrammer said...

indimail has almost everyting of qmail modified. Let us say your message file is /tmp/mail.txt. Let us say the file size of /tmp/mail.txt is 55 butes. To use qmail-remote

/usr/sbin/qmail-remote gmail.com root@yourdomain "" 55 user@gmail.com < /tmp/mail.txt

The syntax for qmail-remote is
qmail-remote domain sender QQEH size recipient

Where QQEH is a string of extra headers to be inserted in the email. It can be an empty string

Unknown said...

OK thank you again for the reply.

And again a problem is I can't even send msg to myself with comand
/usr/sbin/qmail-remote indimail.org root@indimail.org "" 55 root@indimail.org < /tmp/mail.txt
cuz the error "ZSorry, I wasn't able to establish a SMTP connection for indimail.org to...".
I assume it was because I don't have any MX records do I? I don't quite understand this post https://cr.yp.to/im/mxps.html. Do you how could I have the records, or saying how can I at least send msg to myself?

Thank you!

cprogrammer said...

That is because indimail.org is my domain and the mx record will be used. But you can create an artificial route in /etc/indimail/control/smtproutes

indimail.org:127.0.0.1:25

If you have this, qmail-remote will connect to 127.0.0.1 for indimail.org

Unknown said...

Hello! Again thanks for the guide.

And again I failed to send mail from root@indimail.org to root@indimail.org with the error msg "DConnected to 127.0.0.1 but sender was rejected. Remote host said: 503 Polite people say hello first (#5.5.4)".

So basically in order to send mails I need set routes in smtproutes/qmtproutes, in the format of "indimail.org:127.0.0.1:25" in smtproutes and "indimail.org:127.0.0.1:209" in qmtproutes, correct? And precisely how can I set them if I want to send mail to myself(root@indimail.org) or to gmail?

Thank you!

cprogrammer said...

"DConnected to 127.0.0.1 but sender was rejected. Remote host said: 503 Polite people say hello first (#5.5.4)"."

This is an error from qmail-stmpd. What is the hostname that you have set?

Unknown said...

I haven't set any hostname. What I've done is install this docker image, and then created a smtproutes file as you indicated before. I think the default host name is exactly indimail.org in this image isn't it?
Anyway what I'm trying to do is to send a mail from this docker image in SMTP/QMTP, and that brings us to my initial question: what is the procedure I shall follow in order to that based on this docker image? Do I need firstly to create a host different from indimail, and then what? Any idea?

Thank you!

cprogrammer said...

You can run the command
uname -n
or
hostname

The hostname depends on the podman or the docker command you used. What is the exact command you used to start the image?

cprogrammer said...

Also check the smtp log /var/log/svc/smtpd.25/current

ls -l /service/qmail-smtpd.25/variables/BADHELO*

Unknown said...

Here it is:
indimail.org:(root) / >hostname
indimail.org
indimail.org:(root) / >uname -a
Linux indimail.org 5.14.8 #1 SMP Mon Sep 27 11:39:33 CEST 2021 x86_64 x86_64 x86_64 GNU/Linux
indimail.org:(root) / >ls -l /service/qmail-smtpd.25/variables/BADHELO*
-rw-r--r-- 1 root root 0 Sep 12 14:11 /service/qmail-smtpd.25/variables/BADHELOCHECK

cprogrammer said...

Just do the following

echo "something.indimail.org" > /etc/indimail/control/helohost

That will solve your issue of using indimail.org on 127.0.0.1


indimail.org:(root) / >echo test|qmail-remote indimail.org root@indimail.org "" 44 root@indimail.org
DConnected to 127.0.0.1 but sender was rejected.
Remote host said: 503 Polite people say hello first (#5.5.4)
indimail.org:(root) / >echo apollo.indimail.org > /etc/indimail/control/helohost
indimail.org:(root) / >echo test|qmail-remote indimail.org root@indimail.org "" 44 root@indimail.org
rFrom: RCPT: K127.0.0.1 accepted message - Protocol SMTP.
Remote host said: 250 ok 1645837203 qp 1026

Unknown said...

Thank you again for the solution here! Appreciate it really a lot!

Then if I may I have 2 more questions regarding solving my problems for meeting my expectation:
1. As I mentioned I need to be able to send mails via QMTP.Creating a qmtproutes file with

indimail.org:127.0.0.1:25

doesn't allow a qmtp communication since running
echo test|qmail-remote indimail.org root@indimail.org "" 44 root@indimail.org
I got the error msg "Zrecipient did not talk proper QMTP (#4.3.0)"

2. An extension of question 1: my final purpose would be sending a qmtp mail to another server. I was thinking to create 2 the above docker images in 2 virtual machines. So in this case do you know how can I set up the environment (the domains and the routes etc)

Thank you!

cprogrammer said...

You are getting error because you are trying qmtp with a SMTP server. QMTP protocol is served by qmail-qmtpd on port 209. By default the docker installation has QMTP disabled. But you can enable it. You have to run the below command only once

1. rm /service/qmail-qmtpd.209/down - This will allow qmail-qmtpd to be started on every reboot
2. svc -u /service/qmail-qmtpd.209 - This is required only first time after deleting the above file

Now if you try sending the mail, it will still fail because of access list.
So you have to edit the file /etc/indimail/tcp/tcp.qmtp and add the line in the first line of the file
127.:allow

After you add the above line, you have to rebuild the cdb by executing the command

# qmailctl cdb

First do the above, then we will talk about the rest.

cprogrammer said...

also the qmtproutes file should be

indimail.org:127.0.0.1:209

Unknown said...

Perfect that works!

indimail.org:(root) /etc/indimail/control >echo test|qmail-remote indimail.org root@indimail.org "" 44 root@indimail.org
rRemote host said: ok 1646052430 qp 1275
K127.0.0.1 accepted message - Protocol QMTP

So in order to send mail to another host(preferably to another indimail host supporting qmtp like this one) how should I set up the environment? Any idea?

Ps: my purpose is to be able to capture qmtp traffic with tcpdump/wireshark. Since sending mail to the server itself doesn't generate traffic passing eth interface, that's why I'm thinking to create another docker image of indimail and make them send mails. If you have other better ideas pls let me know. Thank you!

cprogrammer said...

1. Are you running podman or docker?
2. What is the exact docker or podman comand you have used to start the container?

Unknown said...

1. I'm running docker image for ubuntu focal;
2. I use this one to start the container:
docker run -d -h indimail.org --privileged a02e6014a67b /sbin/init

cprogrammer said...

Ok. You are using docker and not podman. I don't have much idea of how networking works in containers. But this is what I found

https://stackoverflow.com/questions/39362730/how-to-capture-packets-for-single-docker-container

cprogrammer said...

I tried the stack overflow. This is what i did

On terminal 1
# docker run -d -h indimail.org --privileged --name svscan a02e6014a67b
# docker run -it --rm --net container:svscan nicolaka/netshoot tcpdump -vvv -i any 'tcp port 209'

Now on another terminal
# docker exec -ti svscan bash

Now you are connected to the container. Run the command
# telnet 0 209

Now on terminal 1 you should see
docker run -it --rm --net container:svscan nicolaka/netshoot tcpdump -vvv -i any 'tcp port 209'
tcpdump: data link type LINUX_SLL2
tcpdump: listening on any, link-type LINUX_SLL2 (Linux cooked v2), snapshot length 262144 bytes
17:00:10.143242 lo In IP (tos 0x10, ttl 64, id 36699, offset 0, flags [DF], proto TCP (6), length 60)
localhost.41668 > localhost.209: Flags [S], cksum 0xfe30 (incorrect -> 0xf8f9), seq 3590271574, win 65495, options [mss 65495,sackOK,TS val 1436964214 ecr 0,nop,wscale 7], length 0
17:00:10.143265 lo In IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
localhost.209 > localhost.41668: Flags [S.], cksum 0xfe30 (incorrect -> 0x35d3), seq 4120126064, ack 3590271575, win 65483, options [mss 65495,sackOK,TS val 1436964214 ecr 1436964214,nop,wscale 7], length 0
17:00:10.143289 lo In IP (tos 0x10, ttl 64, id 36700, offset 0, flags [DF], proto TCP (6), length 52)
localhost.41668 > localhost.209: Flags [.], cksum 0xfe28 (incorrect -> 0x5c8f), seq 1, ack 1, win 512, options [nop,nop,TS val 1436964214 ecr 1436964214], length 0
17:00:10.144615 lo In IP (tos 0x0, ttl 64, id 28374, offset 0, flags [DF], proto TCP (6), length 52)
localhost.209 > localhost.41668: Flags [F.], cksum 0xfe28 (incorrect -> 0x5c8c), seq 1, ack 1, win 512, options [nop,nop,TS val 1436964216 ecr 1436964214], length 0
17:00:10.144709 lo In IP (tos 0x10, ttl 64, id 36701, offset 0, flags [DF], proto TCP (6), length 52)
localhost.41668 > localhost.209: Flags [F.], cksum 0xfe28 (incorrect -> 0x5c89), seq 1, ack 2, win 512, options [nop,nop,TS val 1436964216 ecr 1436964216], length 0
17:00:10.144748 lo In IP (tos 0x0, ttl 64, id 28375, offset 0, flags [DF], proto TCP (6), length 52)
localhost.209 > localhost.41668: Flags [.], cksum 0xfe28 (incorrect -> 0x5c89), seq 2, ack 2, win 512, options [nop,nop,TS val 1436964216 ecr 1436964216], length 0
^C
6 packets captured
12 packets received by filter
0 packets dropped by kernel

cprogrammer said...

This also works. You don't require the stack overflow solution

on the container install tcpdump using apt-get, yum, dnf

and run
tcpdump -vvv -i any tcp port 209

Unknown said...

Thank you for the reply but that's not what I need actually.
For me capturing pkt is fine, but generating pkts is the problem. If I have another indimail host(e.g. with IP 172.17.0.1), in my qmtproutes file I shall add an entry like indimail.org:172.17.0.1:209 in order to send mails, correct?

cprogrammer said...

Yes. Correct. As long as the IP 172.17.0.1 is reachable from the docker container.

cprogrammer said...

and also don't forget to add an entry in /etc/indimail/tcp/tcp.qmtp of 172.17.0.1 the IP address of the docker host. If you change tcp.qmtp, you have to re-generate the cdb file by using the command

# qmailctl cdb

Unknown said...

OK made it! Thank you so much man you saved me really!
Just a less important follow-up question: do you also have an idea how to check the msg/mail received?

cprogrammer said...

You can use imap / imaps / pop3 / pop3s to check mails received

IndiMail Queue Mechanism

Indimail has the ability of configuring multiple local and remote queues. A queue is a location on your hard disk where email are deposited ...