User Tools

Site Tools


build:lists

This is an old revision of the document!


Mailing Lists

We originally considered using Majordomo, but then decided to go with Mailman.

Mailman

Choosing Mailman

  1. Newer
  2. Better docs
  3. Works out of the box with postfix
  4. Web management interface
  5. Tool for member migration from majordomo
  6. htdig patch

The initial installed version was mailman 2.1.9 installed on 11/18/06 by Jeff Muse.

Install Notes

http://www.gnu.org/software/mailman/index.html

http://www.list.org/mailman-install/

The list.org site was the primary reference used.

We also used a patch from http://www.openinfo.co.uk/mm/patches/444884/index.html to enable searching list archives via htdig.

Preface

We got mailman up and running on bud after the two of you left. Search functionality is working as well. Right now, mailman is only handling mail lists for bud.sluug.org. We'll need to update it to handle lists for all of sluug.org before moving it into production.

Unfortunately, mailman setup required editing a number of config files, so there's some missing detail in my .bash_history. Also, there was a fair amount of backtracking and undoing, so I'm just going to summarize what I did for documentation purposes.

Jefff Muse to Carl Fitch

Obtain Source

Download mailman-2.1.9 from http://sourceforge.net/projects/mailman/, verifying with gnupg. Also download the patch to allow htdig functionality from http://www.openinfo.co.uk/mm/patches/444884/index.html. Put both in /usr/local/src.

Create User and Group

sudo groupadd mailman
sudo useradd -c "Gnu Mailman" -s /no/shell -d /no/home -g mailman mailman

Install Dependencies

sudo apt-get install htdig htdig-doc gawk libdb2 lockfile-progs gettext
sudo apt-get install python2.4-dev

Create system link from python to python2.4

sudo update-alternatives /usr/bin/python python /usr/bin/python2.4 1

Rebuild: We did not need to install libdb2. Debian 4.0 now uses python-2.4.4 as the default, so we did not do the update-alternatives step.

Pre-installation Steps

As root, create /usr/local/mailman-2.1.9, make it owned by group mailman, and chgrp 2775 /usr/local/mailman.

Extract and Prepare Source

The Debian Mailman package did not include search capibility. To enable search Mailman has to be compiled with htdig. Extract the source file into /usr/local/src/mailman-2.1.9 and move the patch into that directory. gunzip the patch.

Disable most language support. This is an ugly hack to get around a problem with Japanese-language support in python on Debian Etch. Comment out the following line in messages/Makefile.in:

#LANGUAGES=     ar ca cs da de en es et eu fi fr hr hu ia it ja ko lt nl \
#                               no pl pt pt_BR ro ru sl sr sv tr uk vi zh_CN zh_TW

and replace it with:

LANGUAGES=     de es

In templates/Makefile.in, also comment out the original LANGUAGES line.

#LANGUAGES=     ar ca cs da de en es et eu fi fr hr hu ia it ja ko lt nl \
 #                               no pl pt pt_BR ro ru sl sr sv tr uk vi zh_CN zh_TW

and replace it with:

LANGUAGES=     de en es

In misc/paths.py.in, comment out the following lines:

import japanese
import korean
import korean.aliases

Apply the Patch and Build

patch -p1 < htdig-2.1.9.0.1.patch
./configure --prefix=/usr/local/mailman-2.1.9 \
--with-mailhost=localhost --with-urlhost=bud.sluug.org
make
sudo make install
cd /usr/local/
sudo ln -s /usr/local/mailman-2.1.9/ mailman
cd /usr/local/mailman
sudo chgrp mailman .
sudo chmod a+rx,g+ws .
cd bin
sudo ./check_perms
sudo ./check_perms -f
cd archives
sudo chown www-data private
sudo chmod o-x private
cd ..
sudo chown root mailmain-2.1.9

Apache

Then edit /etc/apache2/sites-available/www.sluug.org and add:

        ScriptAlias /mailman/ /usr/local/mailman/cgi-bin/
		    Alias /pipermail/ /usr/local/mailman/archives/public/
		    <Directory /usr/local/mailman/archives/public/>
		       Options FollowSymlinks
					 AuthType Basic
           AuthName "SLUUG Discussion Archive Access"
           AuthUserFile /etc/apache2/passwords
					 Require valid-user
		    </Directory>

Protect with Password

cd /etc/apache2/
sudo htpasswd -c passwords discuss
sudo chown root:www-data passwords
sudo chmod 640 passwords
sudo /etc/init.d/apache2 reload

Configure Mailman

Then edit /usr/local/mailman/Mailman/mm_cfg.py and add:

IMAGE_LOGOS='/images/'
MTA='Postfix'
USE_HTDIG='true'
HTDIG_HTSEARCH_PATH = '/usr/lib/cgi-bin/htsearch'
HTDIG_RUNDIG_PATH = '/usr/bin/rundig' 

Start Mailman on Boot

cd /usr/local/mailman
sudo cp scripts/mailman /etc/init.d/mailman
sudo /usr/sbin/update-rc.d mailman defaults
cd ../bin
sudo ./genaliases''

Postfix

main.cf

Edit /etc/postfix/main.cf:

alias_maps = hash:/etc/aliases,hash:/usr/local/mailman/data/aliases
unknown_local_recipient_reject_code = 550''

maps

sudo postfix reload
cd /usr/local/mailman/data
sudo chown mailman:mailman aliases*
sudo chmod g+w aliases*
cd ../bin
sudo ./newlist mailman

– Jeff, What does this do? –

sudo config_list -i data/sitelist.cfg mailman

Mailman uses a site-wide mailing list called "mailman". This list is where system-generated mails appear to come from. This command applies a generic template defined in sitelist.cfg to the mailman list.

cron

cd ../cron
sudo crontab -u mailman crontab.in

Copy Apache Images to Mailman

sudo ../bin/mailmanctl start
su
cd /usr/local/mailman/archives/private
cp *png *jpg /home/web/www.sluug.org/images/

Initiallize htdig

I'm not sure if these next steps were absolutely necessary

rundig
htfuzzy -v synonyms
htfuzzy -v endings
htmerge

This is absolutely necessary: create a list via the web page (see below) and send some email to it. So is the next step.

/usr/bin/python -S /usr/local/mailman/cron/nightly_htdig -v

Everything appears to be working now. The search engine updates once a day at 2:19 via mailman's cron.


Importation of SLUUG Archives

Expand list to all of SLUUG

First, edit /usr/local/mailman/Mailman/Defaults.py:

Change

DEFAULT_EMAIL_HOST = 'bud. sluug.org' to
DEFAULT_EMAIL_HOST = 'sluug.org'

This will let us handle lists for all of sluug.org instead of just bud. The bud_test list remains functional.

Restart mailman:

$sudo /etc/init.d/mailman restart

Create Lists

Create the discuss and steercom lists:

$cd /usr/local/mailman/bin
$./newlist discuss
$./newlist steercom

Import lists

Import the lists:

$gunzip /home/jmuseinst/discuss*gz
$gunzip /home/jmuseinst/steercom*gz
$cd /usr/local/mailman/bin
$for file in /home/jmuseinst/discuss*
> do sudo ./arch dicsuss $file
> done
$for file in /home/jmuseinst/steercom*
> do sudo ./arch steercom $file
> done

I also imported the subscriber lists from majordomo on michelob into mailman on bud. This, too, was trivial:

scp /opt/majordomo/lists/discuss  and /opt/majordomo/lists/steercom over to bud.

Then on bud:

$cd /usr/local/mailman/bin
$ sudo ./add_members -r /home/jmuseinst/discuss -w n discuss
$ sudo ./add_members -r /home/jmuseinst/steercom -w n steercom

Mailman can only base the archive on the timestamp in the mail, so we've got a few mails archived from 2021 and 2013. Now we know who had problems with keeping their clocks set accurately :-(

Generate Search index

Regenerate the search index:

$ sudo /usr/bin/python -S /usr/local/mailman-2.1.9//cron/nightly_htdig -v

Also, comment out all the majordomo entries in /etc/aliases and run "newaliases".

newaliases

There's a cron job that gzip's the monthly archives for all the lists.

Create List Descriptions

In the administrative interface for discuss (http://bud.sluug.org/mailman/admin/discuss), add the following under "terse description":

SLUUG general discussion

Under "introductory description", add:

This is where we discuss Unix and Unix-like operating systems. We also discuss applications that run on these operating systems, programming in these environments, and even a little bit of hardware.

IIn the administrative interface for steercom (http://bud.sluug.org/mailman/admin/steercom), add the following under "terse description":

SLUUG Steering Committee discussion
This is the place to discuss SLUUG business matters, planning and related items.

In both administrative interfaces, where it says "Where are replies to list messages directed? Poster is strongly recommended for most mailing lists.", check "This list".

Archive Locations

Discuss archives are at http://bud.sluug.org/pipermail/discuss/. List info is at http://bud.sluug.org/mailman/listinfo/discuss.

Steercom list info is at http://bud.sluug.org/mailman/listinfo/steercom. Archives are at http://bud.sluug.org/pipermail/steercom/

I haven't moved sysadmin over, because it looks like it has a different password protection scheme than the other lists.


Create new mailing lists at http://bud.sluug.org/mailman/create

Get information about lists on bud at http://bud.sluug.org/mailman/listinfo

Get on the test list at http://bud.sluug.org/mailman/listinfo/bud_test

Look at the test list archives at http://bud.sluug.org/pipermail/bud_test/

Configure the bud_test list at http://bud.sluug.org/mailman/admin/bud_test/

Mailman TODO:

  • We should probably SSL-require the administrative pages for mailman.
  • We need to test mailman with some of the majordomo archives from

michelob.

  • We need to set up mailman to handle lists for all of sluug.org
  • /usr/local/mailman/bin/add_members should be run on a list of current

subscribers to ANNOUNCE, DISCUSS, SYSADMIN, and STEERCOM. I'm not quite sure how passwords, if any, will be handled.

  • We need to copy the archives from michelob to bud and index them via

htdig. I'm not sure how htdig and/or apache will handle the compressed files as currently configured.

GENERAL MAIL TODO:

  • We need to get virtual users set up in some way. Craig and I discussed

this, and two options are postfix maps and a mysql database. We didn't make any decisions. Two particular challenges will be copying existing passwords for POP3/IMAP access and mail filtering (procmail/maildrop/whatever). Once we get users set up, we'll need to migrate their mail spools.

  • We need to get spamassassin working. We particularly need to take a

look at the performance impact of scanning list mail. This should probably be done incoming list mail only.

  • We need to get some form of webmail up and running. I'm partial to

Horde becauses it has a powerful interface and a ton of cool modules. I haven't used the password module, but it might be particularly useful for us. See http://www.horde.org/accounts/screenshots/accounts.png. If we use horde, we'll be using mysql, so that might be the way to go for virtual users.

That should do it for now - enjoy the rest of the weekend.

Jeff Muse


Notes

Logs are in /usr/local/mailman/logs/. The log files are used as follows:

  • post - successful posts to a list
  • subscribe - subscription additions, deletions, auto-unsubscribes (due to bounces), and pending
  • vette - results of admin actions to deferred posts
  • bounce - tracks bounces, so failing members can automatically be purged

TODO: We should move these to /var/log and put them under log rotation.

TODO

  • We need to get virtual users set up in some way. Craig and I discussed

this, and two options are postfix maps and a mysql database. We didn't make any decisions.

Two particular challenges will be copying existing passwords for POP3/IMAP access and mail filtering (procmail/maildrop/whatever). Once we get users set up, we'll need to migrate their mail spools.

  • We need to get spamassassin working. We particularly need to take a

look at the performance impact of scanning list mail. This should probably be done incoming list mail only.

  • We need to get some form of webmail up and running. I'm partial to

Horde becauses it has a powerful interface and a ton of cool modules. I haven't used the password module, but it might be particularly useful for us. See http://www.horde.org/accounts/screenshots/accounts.png. If we use horde, we'll be using mysql, so that might be the way to go for virtual users.

Majordomo

NOTE: This was the original attempt, but is no longer being used.

Majordomo was installed to use the same config & archives as current system

Installed Majordomo from source:

Source distribution (gzip'd) of current version (1.94.5) of Majordomo
http://www.greatcircle.com/majordomo/1.94.5/majordomo-1.94.5.tar.gz
Source distribution (compressed) of current version (1.94.5) of Majordomo
http://www.greatcircle.com/majordomo/1.94.5/majordomo-1.94.5.tar.Z
Source distribution (uuencoded) of current version (1.94.5) of Majordomo
http://www.greatcircle.com/majordomo/1.94.5/majordomo-1.94.5.tar.Z.uu

http://www.greatcircle.com/majordomo/1.94.5/majordomo-1.94.5.tar.gz
#tar xzf majordomo-1.94.5.tar.gz
#cd majordomo-1.94.5

Create majordomo user and group using the following commands
#groupadd majordomo
#useradd majordomo -g majordomo
#cat /etc/passwd | grep majordomo

Create installation directory for majordomo
#mkdir /usr/local/majordomo

Edit the makefile
#vi Makefile

Change the line with PERL variable definition to
PERL = /usr/bin/perl
Change the line with W_HOME variable definition to
W_HOME = /usr/local/majordomo
Change the lines with W_USER and W_GROUP variables definition to

W_USER = 1008
W_GROUP = 1008
set TMPDIR = /tmp

Edit the configuration file
#cp sample.cf majordomo.cf
#vi majordomo.cf

Set $whereami variable to your host name and that is enough
$whereami = "your-host.com";

Install the majordomo
#make wrapper
#make install
#make install-wrapper

Check the installation
#cd /usr/local/majordomo; ./wrapper config-test

Installed MHonArc from package

Tarr'd & gzip'd /opt/majordomo directory on Michelob

scp'd to bud & extracted to /usr/local/majordomo

Instlled MHonArc on Bud (to use same tool as Michelob)

Credits

- Jeff Muse - Initial install of Mailman and providing notes from the install.

Comments

build/lists.1185166108.txt.gz · Last modified: 2007/07/22 23:48 by 24.207.253.154