====== Replace Mailman 2 ======
This document will be used to coordinate the effort associated with replacing Mailman 2.
====== Assorted technical notes: ======
https://www.sluug.org/pipermail/sysadmin/2023-August/010383.html
------------------------------------------------------------------------
Date: Thu, 17 Aug 2023 11:31:08 -0500
From: Ken Johnson
Subject: Mailman 3
mailman and mailman3 can co-exist under Debian 11. The cli commands are
completely different, for example. I did not convert until on Debian 11.
There can be python issues, of course -- the longer you wait to dump
python2, the more that can be messy, though I did not have very many issues
personally. In some ways it may be easier to convert after the update,
because the version of mailman3 you are moving to is closer to current, so
it is easier to find help and information online. Also, for SLUUG the
mailing list is pretty exposed to the internet, so best to avoid delay in
that conversion, since no updates for mailman will be available.
As you look into this, you will probably see that Xapian is the recommended
search option, but it is not the default. Once I got it working, Xapian
was fine for me. If I recall correctly,it took me a while to figure out
the configuration, because I had a hard time finding examples.
--
First of all, my message of 17 August 2023 contains a critical error. _Xapian_ is the recommended search option, but
_Whoosh_ is the default. Replace 'Whoosh' with 'Xapian' in my message of 17 August.
===
This message on the Debian Users mailing list was very helpful to me when I was struggling to configure mailman3.
https://lists.debian.org/debian-user/2022/07/msg00528.html
===
This message thread on the mailman3 mailing list may be helpful:
https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/ZSSVDKKVKZYNQFKLTLWC4V6IIWL7Y3RP/
===
This fragment from /etc/mailman3/mailman-web.py may be helpful as an example:
## 12 Jul 2023 - KLJ - Configure Xapian instead of default Whoosh
#
# Full-text search engine
#
HAYSTACK_CONNECTIONS = {
'default': {
# You can also use the Xapian engine, it's faster and more accurate,
# but requires another library.
# http://django-haystack.readthedocs.io/en/v2.4.1/installing_search_engines.html#xapian
# Example configuration for Xapian:
'ENGINE': 'xapian_backend.XapianEngine',
'PATH': '/var/lib/mailman3/web/fulltext_index',
},
}
--
Password management: Mailman3 and Postgresql (don't use sqlite for mailman3!) will require the installer to specify some passwords during the setup process. Be prepared, and have a plan in advance for where and how to store and communicate those passwords.
====== Planning ======
Will use Debian packages.
No need to create mailman database and user before installing packages.
Package mailman3 depends on any of 4 dbconfig-* packages.
Currently installed is "dbconfig-common", which is not acceptable.
Testing shows installation of "mailman3" package without specifying any
"dbconfig-*" will also install "dbconfig-sqlite3" by default.
Specifying "dbconfig-mysql" will install that and not "dbconfig-sqlite3".
What packages to install?
* mailman3 mailman3-web
* python3-mailman-hyperkitty
* python3-django-postorius
* python3-django-mailman3
* listadmin
* dbconfig-mysql
* Skipping xapian (not Whoosh, but mailman3-web depends on python3-whoosh)
* And -doc for any package that has it, except prereq libraries: mailman3-doc xapian-doc
Since the Debian package of "mailman3-web" depends on "python3-whoosh",
will not install "xapian" initially, as it just complicates the procedure.
If desired, can install and use "xapian" later.
====== Installation ======
===== Package installation =====
export DEBIAN_PRIORITY=low
export DEBIAN_FRONTEND=readline
export PROD_LIST="mailman3 mailman3-web python3-mailman-hyperkitty \
python3-django-postorius python3-django-mailman3 listadmin \
mailman3-doc dbconfig-mysql python3-pymysql python3-mysqldb"
script -a /var/tmp/mailman3-install-log.txt
TERM=dumb apt-get install $PROD_LIST
Answer the installation dialog questions, see below.
# End the script before continuing
==== Responses to installation dialog: ====
^ Prompt ^ Response ^
^ Configuring mailman3 ^^
| Configure database for mailman3 with dbconfig-common? | yes |
| Database type to be used by mailman3: | 2 |
| Connection method for MySQL database of mailman3: | 1 |
| Authentication plugin for MySQL database: | 1 |
| MySQL database name for mailman3: | mm3db |
| MySQL username for mailman3: | mm3dbuser |
| MySQL application password for mailman3: | (Hidden) |
| Password confirmation: | (hidden) |
| Add the HyperKitty configuration to mailman.cfg? | yes |
^ Configuring mailman3-web ^^
| Configure database for mailman3-web with dbconfig-common? | yes |
| Database type to be used by mailman3-web: | 2 |
| Connection method for MySQL database of mailman3-web: | 1 |
| Authentication plugin for MySQL database: | 1 |
| MySQL database name for mailman3-web: | mm3webdb |
| MySQL username for mailman3-web: | mm3webuser |
| MySQL application password for mailman3-web: | (Hidden) |
| Password confirmation: | (hidden) |
| Domain name for sender email addresses: | sluug.org |
| Username of the Postorius superuser: | postoriusroot |
> Email address of the Postorius superuser: | mailing-lists@sluug.org |
> Password for the Postorius superuser: | (Hidden) |
> Web server(s) to configure automatically: | 2 |
> Should the webserver(s) be restarted now? | yes |
^ Configuring libpaper1 ^^
> System's default paper size: | 1 |
^ It now starts installing and setting up all the other packages ^^
==== Correcting mismatched archiver key ====
The Hyperkitty key in "/etc/mailman3/mailman-web.py" ("MAILMAN_ARCHIVER_KEY")
and "/etc/mailman3/mailman-hyperkitty.cfg" ("api_key") need to match.
In testing, "MAILMAN_ARCHIVER_KEY" was set to "SecretArchiverAPIKey".
export DEBIAN_PRIORITY=low
export DEBIAN_FRONTEND=readline
dpkg-reconfigure python3-mailman-hyperkitty
This didn't ask any questions, exiting with message:
"Replacing config file /etc/mailman3/mailman-hyperkitty.cfg with new version".
Now both files have the same key.
===== Initial configuration =====
As documented in "/usr/share/doc/mailman3/README.Debian",
edit /etc/postfix/main.cf to "add the following settings".
Also verify there were none of those settings already in it.
Verify the hash files are in "/var/lib/mailman3/data/"
However, not readable by user/group, except "list".
As of installation, both are empty, except for comments.
Restart postfix with "postfix reload".
=== Configuring Apache is not needed. ===
* The installation configuration scripts created a sym link from "/etc/apache2/conf-available/mailman3.conf" to "/etc/mailman3/apache.conf".
* And a sym link from "/etc/apache2/conf-enabled/mailman3.conf" to "../conf-available/mailman3.conf".
* The installation also created sym links in "/etc/apache2/mods-enabled/" to existing files in "../mods-available/" for 3 "proxy*".
* So this is already done.
__**Everything below has not been tested yet and is for planning**__
==== Change other settings in /etc/ files not configured by "debconf". ====
=== Edit "/etc/mailman3/mailman-web.py": ===
* Change "TIME_ZONE" from "UTC" to "America/Chicago"
* Also reference "/usr/share/mailman3-web/settings.py".
=== Edit "/etc/mailman3/mailman.cfg": ===
* Change "site_owner" from "changeme@example.com" to TBD.
=== Update Mailman config for required HTTPS: ===
* mailman-hyperkitty.cfg: "base_url"
* mailman-web.py: "POSTORIUS_TEMPLATE_BASE_URL"
* mailman.cfg" "use_https"
=== Update logrotate configuration ===
* Longer log retention.
* Switch to multiple log files?
====== Additional configuration and testing ======
==== Create a new test list ====
mailman create --language no --owner postmaster@___ -no-notify ____@lists.__
Verify Postfix hashes are accessible and automatically populated.
Test subscriptions.
Test posting.
* ASCII postings are delivered and archived as plain text ASCII. Not converted to "quoted-printable" or "base64".
* Can footers be added without this happening?
Test archives.
==== Test import of an existing test list ====
Check archive accuracy.
Configure to delete attachments from the archive during import?
Test list deletion.
Does creating a list stop the errors about "gatenews"?
Will "/usr/local/bin/mailman_queue_check.sh" work?
====== Things to change before import of production lists ======
Protect archives from address scraping and secure private lists.
Test new configuration options and enhancements to make now.
Especially any global options that could impact all production lists.
* "header_checks" are now global?
* "[dmarc]" and/or "[ARC]".
====== Import Mailman 2 lists ======
====== Other changes to make later ======
Index mailing list archives.
Delete or deflate HTML postings and delete attachments on all lists.
====== Cleanup for deletion of Mailman 2 ======