the djb way

serialmail


qmail in the bush

Imagine our favorite international NGO again, Example Org. They want to setup relief operations in some "coalition" devastated region. So they put up a headquarters office in, say, Kabul, with three field offices up-country.

We are a long way from $19.95 dial-ups with unlimited access. The indiscriminate use of cluster bombs and "daisy-cutters" has laid the country to waste. The latest puppet regime is just the barest window-dressing for the chaos and exploitation that will reign over the impoverished masses for decades to come.

As a result, Internet and telecommunication services in the country are next to nil. What services do exist are tres cher, very expensive. Example gets a persistent connection only for its Kabul office, via satellite link-up. Otherwise, full-time connectivity for the field offices is either unavailable and/or cost prohibitive.

However, they do work out the means to communicate periodically with their field offices. This can be done through their own HF radios, or satellite telecommunications, or maybe some of the offices even have access to a workable land-line. Maybe it's a PPP link over Example's own equipment, a connection "borrowed" from a wealthier NGO in town, or a service contracted through a third-party provider. Whatever. The point is, even though the field offices don't have full-time, persistent connectivity, intermittent data communications are feasible --and affordable-- through some means or other.

qmail can be used in this scenario, to provide regular Internet mail to and from all field offices, taking advantage of the single Internet connection in Kabul for all inbound and outbout mail.

Here's the sketch:

"mazar"
  +---+
  |   |
  |   |
  |   |\
  +---+ \
         \
          \
           \
            \    "kabul"
             \ aka "mailhub" 
              +---+
"herat"       |   |
+---+         |   |-------------> internet
|   |---------|   |
|   |         +---+
|   |          |
+---+          |
               |
               |
               |
             +---+
             |   |
             |   |
             |   |
             +---+
            "farah"

The headquarters office in Kabul, with its persistent connection, will serve as the hub for email communications. The three field offices, denoted by the servers "mazar", "herat", and "farah", will then connect periodically with "kabul" to transmit and receive mail.

By "periodically" we mean anything from once or twice per hour, to once or twice per day. No surfing in this scenario, "kabul" won't be a gateway for streaming video or chat for the field offices, either. This is just the basic store-and-forward email system, much like your great-great-grandfather used to use, long long ago.

"herat"

First we'll describe the use of serialmail in the field, using "herat" as an example. The server in "herat" has a full qmail installation, including fastforward, a qmail-qmtpd service, and the serialmail package. It is also configured to handle mail to/from the headquarters office as described in fastforward internationale.

The server "herat" has these entries in both control/locals and control/rcpthosts:

example.org
herat.example.org

So mail address to recipient@example.org and recipient@herat.example.org will be delivered locally.

The server "herat" also has this single entry in control/virtualdomains:

:qrelay

This entry is a "wildcard" that matches everything. The effect is that any and all mail not local will be prepended with qrelay- and delivered to the user "qrelay". So mail destined for <myfriend@yahoo.com> will be rewritten to <qrelay-myfriend@yahoo.com> and delivered to the user "qrelay".

"qrelay" is a user we made up. It should be added to /etc/passwd, member group "nofiles", disabled login shell. We'll give it a home directory of /var/qmail/qrelay:

# mkdir /var/qmail/qrelay
# chown qrelay /var/qmail/qrelay

Create a maildir in the home directory of "qrelay":

# maildirmake  /var/qmail/qrelay/MAILDIR
# echo "./MAILDIR/" > /var/qmail/qrelay/.qmail-default
# chown -R qrelay /var/qmail/qrelay/*
# chown -R qrelay /var/qmail/qrelay/.*

Now all non-local mail --that is, all mail heading somewhere the hell out of "herat"-- will in effect be queued in the Maildir /var/qmail/qrelay/MAILDIR.

Whenever "herat" is connected to "kabul", the collected outbound mail can be sent with the serialmail utility, maildirqmtp:

# maildirqmtp /var/qmail/qrelay/MAILDIR 1.2.3.4 qrelay-

This sends the collected mail, via QMTP, to the server at IP address "1.2.3.4", stripping the prepended "qrelay-" from the address on each envelope. Just substitute the real IP address for the link to "kabul" and you're good to go.

"kabul"

Back in "kabul", the mailhub server is configured with /etc/tcprules/qmtp.rules that set RELAYCLIENT="" for connections from field offices. This allows field offices to use "kabul" as a mailhub for their outbound mail.

"kabul" is also setup with a fastforward configuration as described in fastforward internationale. That is, mail destined for up-country offices is rewritten to recipient@mazar.example.org, recipient@herat.example.org, etc.

"kabul" is also setup with a "qrelay" user, and a control/virtualdomains file like this:

mazar.example.org:qrelay-mazar
herat.example.org:qrelay-herat
farah.example.org:qrelay-farah

Now any mail destined for "mazar.example.org" will be prepended with qrelay-mazar- and delivered locally to user "qrelay"; mail for "herat.example.org" will be prepended with qrelay-herat- and delivered locally to user "qrelay"; and so on. So incoming mail received for <holly@example.org>, who is stationed in Herat, will be:

  1. Rewritten to <holly@herat.example.org> by fastforward;
  2. Rewritten to <qrelay-herat-holly@herat.example.org> by control/virtualdomains;
  3. Delivered to user "qrelay".

"qrelay" is a user we made up again. Just like in the field offices, member group "nofiles", disabled login shell, home directory in /var/qmail/qrelay:

# mkdir /var/qmail/qrelay
# chown qrelay /var/qmail/qrelay

Now here is where "kabul" is a little different from the up-country configuration. We setup a separate maildir "queue" for each of the field offices:

# cd /var/qmail/qrelay
# maildirmake MAILDIR.MAZAR
# maildirmake MAILDIR.HERAT
# maildirmake MAILDIR.FARAH
# echo "./MAILDIR.MAZAR/" > .qmail-mazar-default
# echo "./MAILDIR.HERAT/" > .qmail-herat-default
# echo "./MAILDIR.FARAH/" > .qmail-farah-default
# chown -R qrelay /var/qmail/qrelay/*
# chown -R qrelay /var/qmail/qrelay/.*

Now whenever a field office connects with "kabul", arrange to run maildirqmtp on the queue designated for that office. For example, when "herat" connects, run:

# maildirqmtp /var/qmail/qrelay/MAILDIR.HERAT 7.8.7.8 qrelay-herat-

This sends the mail collected in MAILDIR.HERAT, via QMTP, to the IP address for "herat", stripping the prepended qmail-herat- from each address. All the staff in Herat will be grateful for this lifeline to the world.

gloss & glue

The above description provides the basic framework for using qmail and serialmail utilities, to provide email services for poorly connected hosts. Such applications are not necessarily limited to impoverished countries, either, and we were kidding about your great-great grandfather. This is the system we use ourselves in a variety of situations today.

The procedures we have outlined here have glossed over some of the particulars, though. These are the details that will vary from place to place: type of connection; bandwidth available; static vs. dynamic IP address assignment; internal vs. external network; client/server roles of "peers" in links; competencies of field personnel; etc.

Where bandwidth is challenged, you will need to take additional measures to conserve resources. These may include:

The serialmail configuration described above may be further adapted to any number of circumstances and environments. You can use QMTP/serialmail for some transfers, POP/fetchmail for others, --even UUCP may get some play. (By avoiding UUCP, though, your configuration will be plug-and-play compatible with conventional TCP/IP services when better bandwidth options become available.)

Your field office server may be installed on a portable computer, solar charged. An older laptop will well serve a field office with a number of staff and a local network of several workstations. Once or twice a day, you can actually carry this "server" if you need to, on over to the guy in town who's lucky enough to have some bandwidth and willing to share it for a few beers every now and then. With a small additional investment, a couple of 802.11b bridges and directional antennas could make the trip wirelessly.

Most of the world faces more than "the last mile" problem. It's more like "the last 500 kilometer" problem. What is needed is infrastructure. Before you can move trucks and trains, you gotta have roads and rails.

Unix and qmail are basic resources for building network roads in bandwidth-challenged environments, providing effective, reliable, appropriate, sustainable and affordable infrastructure. Depending on the circumstances, you can easily add your own user-interfaces and/or automation to the simple setup described here --from shell scripts to GUIs to web interfaces-- tailored to fit the requirements of the installation. Here are the tools and techniques; the rest of the glue is up to you.


Copyright © 2004, Wayne Marshall.
All rights reserved.

Last edit 2004.02.17, wcm.