User Tools

Site Tools


replace_mailman_2

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
replace_mailman_2 [2024/01/06 06:39]
SLUUG Administration created
replace_mailman_2 [2025/11/18 02:13] (current)
SLUUG Administration Progress update
Line 2: Line 2:
  
 This document will be used to coordinate the effort associated with replacing Mailman 2. This document will be used to coordinate the effort associated with replacing Mailman 2.
 +
 +====== Assorted technical notes: ======
 +
 +https://​www.sluug.org/​pipermail/​sysadmin/​2023-August/​010383.html
 +------------------------------------------------------------------------
 +  Date: Thu, 17 Aug 2023 11:31:08 -0500
 +  From: Ken Johnson
 +  Subject: ​ Mailman 3
 +
 +mailman and mailman3 can co-exist under Debian 11.  The cli commands are
 +completely different, for example. ​ I did not convert until on Debian 11.
 +There can be python issues, of course -- the longer you wait to dump
 +python2, the more that can be messy, though I did not have very many issues
 +personally. ​ In some ways it may be easier to convert after the update,
 +because the version of mailman3 you are moving to is closer to current, so
 +it is easier to find help and information online. ​ Also, for SLUUG the
 +mailing list is pretty exposed to the internet, so best to avoid delay in
 +that conversion, since no updates for mailman will be available.
 +
 +As you look into this, you will probably see that Xapian is the recommended
 +search option, but it is not the default. ​ Once I got it working, Xapian
 +was fine for me.  If I recall correctly,​it took me a while to figure out
 +the configuration,​ because I had a hard time finding examples.
 +
 +--
 +
 +First of all, my message of 17 August 2023 contains a critical error. ​ _Xapian_ is the recommended search option, but
 +_Whoosh_ is the default. ​ Replace '​Whoosh'​ with '​Xapian'​ in my message of 17 August.
 +
 +===
 +
 +This message on the Debian Users mailing list was very helpful to me when I was struggling to configure mailman3.
 +
 +https://​lists.debian.org/​debian-user/​2022/​07/​msg00528.html
 +
 +===
 +
 +This message thread on the mailman3 mailing list may be helpful:
 +
 +https://​lists.mailman3.org/​archives/​list/​mailman-users@mailman3.org/​thread/​ZSSVDKKVKZYNQFKLTLWC4V6IIWL7Y3RP/​
 +
 +===
 +This fragment from /​etc/​mailman3/​mailman-web.py may be helpful as an example:
 +
 +## 12 Jul 2023 - KLJ - Configure Xapian instead of default Whoosh
 +#
 +# Full-text search engine
 +#
 +HAYSTACK_CONNECTIONS = {
 +    '​default':​ {
 +        # You can also use the Xapian engine, it's faster and more accurate,
 +        # but requires another library.
 +        # http://​django-haystack.readthedocs.io/​en/​v2.4.1/​installing_search_engines.html#​xapian
 +        # Example configuration for Xapian:
 +        '​ENGINE':​ '​xapian_backend.XapianEngine',​
 +        '​PATH':​ '/​var/​lib/​mailman3/​web/​fulltext_index',​
 +    },
 +}
 +
 +--
 +
 +Password management: ​ Mailman3 and Postgresql (don't use sqlite for mailman3!) will require the installer to specify some passwords during the setup process. ​ Be prepared, and have a plan in advance for where and how to store and communicate those passwords.
 +
 +----
 +
 +====== Planning ======
 +
 +Will use Debian packages.
 +
 +No need to create mailman database and user before installing packages.
 +
 +Package mailman3 depends on any of 4 dbconfig-* packages.
 +Currently installed is "​dbconfig-common",​ which is not acceptable.
 +Testing shows installation of "​mailman3"​ package without specifying any
 +"​dbconfig-*"​ will also install "​dbconfig-sqlite3"​ by default.
 +Specifying "​dbconfig-mysql"​ will install that and not "​dbconfig-sqlite3"​.
 +
 +What packages to install?
 +
 +      * mailman3 mailman3-web
 +      * python3-mailman-hyperkitty
 +      * python3-django-postorius
 +      * python3-django-mailman3
 +      * listadmin
 +      * dbconfig-mysql
 +      * Skipping xapian (not Whoosh, but mailman3-web depends on python3-whoosh)
 +      * And -doc for any package that has it, except prereq libraries: mailman3-doc xapian-doc
 +
 +Since the Debian package of "​mailman3-web" ​ depends on "​python3-whoosh",​
 +will not install "​xapian"​ initially, as it just complicates the procedure.
 +If desired, can install and use "​xapian"​ later.
 +After installation and testing, search works without any extra configuration.
 +
 +====== Installation ======
 +
 +===== Package installation =====
 +
 +<​code>​
 +export DEBIAN_PRIORITY=low
 +export DEBIAN_FRONTEND=readline
 +export PROD_LIST="​mailman3 mailman3-web python3-mailman-hyperkitty \
 +        python3-django-postorius python3-django-mailman3 listadmin \
 +        mailman3-doc dbconfig-mysql ​ python3-pymysql python3-mysqldb"​
 +script -a /​var/​tmp/​mailman3-install-log.txt
 +TERM=dumb apt-get install $PROD_LIST
 +Answer the installation dialog questions, see below.
 +# End the script before continuing
 +</​code>​
 +
 +==== Responses to installation dialog: ====
 +
 +^ Prompt ​                                                   ^ Response ​       ^
 +^  Configuring mailman3 ​ ^^
 +| Configure database for mailman3 with dbconfig-common? ​    | yes             |
 +| Database type to be used by mailman3: ​                    | 2               |
 +| Connection method for MySQL database of mailman3: ​        | 1               |
 +| Authentication plugin for MySQL database: ​                | 1               |
 +| MySQL database name for mailman3: ​                        | mm3db           |
 +| MySQL username for mailman3: ​                             | mm3dbuser ​      |
 +| MySQL application password for mailman3: ​                 | (Hidden) ​       |
 +| Password confirmation: ​                                   | (hidden) ​       |
 +| Add the HyperKitty configuration to mailman.cfg? ​         | yes             |
 +^  Configuring mailman3-web ​ ^^
 +| Configure database for mailman3-web with dbconfig-common?​ | yes             |
 +| Database type to be used by mailman3-web: ​                | 2               |
 +| Connection method for MySQL database of mailman3-web: ​    | 1               |
 +| Authentication plugin for MySQL database: ​                | 1               |
 +| MySQL database name for mailman3-web: ​                    | mm3webdb ​       |
 +| MySQL username for mailman3-web: ​                         | mm3webuser ​     |
 +| MySQL application password for mailman3-web: ​             | (Hidden) ​       |
 +| Password confirmation: ​                                   | (hidden) ​       |
 +| Domain name for sender email addresses: ​                  | sluug.org ​      |
 +| Username of the Postorius superuser: ​                     | postoriusroot ​  |
 +| Email address of the Postorius superuser: ​                | mailing-lists@sluug.org |
 +| Password for the Postorius superuser: ​                    | (Hidden) ​       |
 +| Web server(s) to configure automatically: ​                | 2               |
 +| Should the webserver(s) be restarted now?                 | yes             |
 +^  Configuring libpaper1 ​ ^^
 +| System'​s default paper size:                              | 1               |
 +^  It now starts installing and setting up all the other packages ​ ^^
 +
 +==== Correcting mismatched archiver key ====
 +
 +The Hyperkitty key in "/​etc/​mailman3/​mailman-web.py"​ ("​MAILMAN_ARCHIVER_KEY"​)
 +and "/​etc/​mailman3/​mailman-hyperkitty.cfg"​ ("​api_key"​) need to match.
 +In testing, "​MAILMAN_ARCHIVER_KEY"​ was set to "​SecretArchiverAPIKey"​.
 +
 +<​code>​
 +export DEBIAN_PRIORITY=low
 +export DEBIAN_FRONTEND=readline
 +dpkg-reconfigure python3-mailman-hyperkitty
 +</​code>​
 +
 +This didn't ask any questions, exiting with message:
 +"​Replacing config file /​etc/​mailman3/​mailman-hyperkitty.cfg with new version"​.
 +Now both files have the same key.
 +
 +===== Initial configuration =====
 +
 +As documented in "/​usr/​share/​doc/​mailman3/​README.Debian",​
 +edit /​etc/​postfix/​main.cf to "add the following settings"​.
 +Also verify there were none of those settings already in it.
 +
 +Verify the hash files are in "/​var/​lib/​mailman3/​data/"​
 +However, not readable by user/group, except "​list"​.
 +As of installation,​ both are empty, except for comments.
 +
 +Restart postfix with "​postfix reload"​.
 +
 +=== Configuring Apache is not needed. ===
 +
 +  * The installation configuration scripts created a sym link from "/​etc/​apache2/​conf-available/​mailman3.conf"​ to "/​etc/​mailman3/​apache.conf"​.
 +  * And a sym link from "/​etc/​apache2/​conf-enabled/​mailman3.conf"​ to "​../​conf-available/​mailman3.conf"​.
 +  * The installation also created sym links in "/​etc/​apache2/​mods-enabled/"​ to existing files in "​../​mods-available/"​ for 3 "​proxy*"​.
 +  * So this is already done.
 +
 +==== Change other settings in /etc/ files not configured by "​debconf"​. ====
 +
 +=== Edit "/​etc/​mailman3/​mailman-web.py":​ ===
 +
 +  * Change "​TIME_ZONE"​ from "​UTC"​ to "​America/​Chicago"​
 +  * In the "​INSTALLED_APPS"​ values, comment '​django_mailman3.lib.auth.fedora'​. ​ Apparently you can't comment '​allauth.socialaccount'​ or it will fail.  Commenting the fedora entry will remove the "​Please sign in with one of your existing third party accounts. Or ...".
 +  * The builtin Gravatar support can be disabled by adding "​HYPERKITTY_ENABLE_GRAVATAR = False"​. ​ However, this didn't seem to work in testing. ​ Neither did "​GRAVATAR_SECURE_URL = ''"​.
 +  * Also review "/​usr/​share/​mailman3-web/​settings.py"​ for other settings.
 +
 +=== Edit "/​etc/​mailman3/​mailman.cfg":​ ===
 +
 +  * Change "​site_owner"​ from "​changeme@example.com"​ to the same used above for "​Postorius superuser"​.
 +
 +=== Update Mailman config for required HTTPS: ===
 +
 +  * mailman-hyperkitty.cfg:​ "​base_url"​ - no change, since "​localhost"​.
 +  * mailman-web.py:​ "​POSTORIUS_TEMPLATE_BASE_URL"​ - no change, since "​localhost"​.
 +  * mailman.cfg"​ "​use_https"​ - leave at "​no",​ this is for internal use?
 +
 +=== Update logrotate configuration ===
 +
 +Update "/​etc/​logrotate.conf"​ to add before the "​include"​ line:
 +<​code>​
 +tabooext + .bak .off .out .save
 +taboopat + *[-.][0-9][0-9][0-9][0-9][0-9][0-9]
 +taboopat + *[-.][0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]
 +</​code>​
 +
 +Update "/​etc/​logrotate.d"​ files "​mailman3"​ and "​mailman3-web"​ to use
 +monthly rotation, longer retention, file name format, permissions,​ etc,
 +Also using a wild card for multiple log files.
 +
 +Update "/​etc/​mailman3/​mailman.cfg"​ to add some separate log files, to match
 +some in Mailman 2.  After the "​[logging.debian]"​ section, add new subsections:​
 +<​code>​
 +[logging.bounce]
 +path: bounce.log
 +propagate: yes
 +[logging.vette]
 +path: vette.log
 +[logging.mischief]
 +path: mischief.log
 +[logging.subscribe]
 +path: subscribe.log
 +[logging.error]
 +path: error.log
 +propagate: yes
 +[logging.archiver]
 +path: archiver.log
 +</​code>​
 +
 +Change existing logs to make readable.
 +
 +Restart both "​mailman3"​ and "​mailman3-web"​.
 +<​code>​
 +date;​systemctl restart mailman3;​date
 +date;​systemctl restart mailman3-web;​date
 +</​code>​
 +
 +====== Additional configuration and testing ======
 +
 +===== Customize =====
 +
 +Create and edit "/​var/​lib/​mailman3/​templates/​hyperkitty/​top.html"​
 +Copy the SLUUG logo section of "/​srv/​www/​www.sluug.org/​public/​header.html"​
 +into it.
 +Make sure it is owned by user:group "​list:​list"​.
 +
 +Create and edit "/​var/​lib/​mailman3/​templates/​postorius/"​
 +<​code>​
 +cp -ip /​usr/​lib/​python3/​dist-packages/​postorius/​templates/​postorius/​base.html /​var/​lib/​mailman3/​templates/​postorius/​
 +vi /​var/​lib/​mailman3/​templates/​postorius/​base.html
 +</​code>​
 +Copy the SLUUG logo section of "/​srv/​www/​www.sluug.org/​public/​header.html"​
 +into it.
 +Make sure it is world readable.
 +
 +Restart "​mailman3-web"​.
 +
 +===== Create a new test list =====
 +
 +<​code>​
 +"​mailman-wrapper create --language no --owner postmaster@___ -no-notify ____@sluug.org
 +</​code>​
 +
 +Set the list description via "​Django site admin"​.
 +
 +Verify Postfix hashes are accessible and automatically populated.
 +
 +Test subscriptions.
 +
 +Test posting.
 +
 +Test archives.
 +
 +Do archives hide the address of the poster? ​ No.
 +
 +==== Encoding of postings ====
 +
 +The acceptable operation is that
 +ASCII postings are delivered and archived as plain text ASCII.
 +Not converted to "​quoted-printable"​ or "​base64"​.
 +As discovered with Mailman 2, when the default footers are added,
 +the charset is "​utf-8",​ and the Content-Transfer-Encoding:​ is "​base64"​.
 +There might also be a concern with added content breaking DKIM signatures.
 +
 +<​code>​
 +mkdir -p /​var/​lib/​mailman3/​templates/​site/​en/​
 +touch    /​var/​lib/​mailman3/​templates/​site/​en/​list:​member:​generic:​footer.txt
 +chown -R list:list /​var/​lib/​mailman3/​templates/​site/​
 +</​code>​
 +
 +This worked, and the postings are now delivered without being converted.
 +
 +Or do it via the web interface at URL
 +.../​mailman3/​postorius/​lists/​testmm3.bock2.sluug.org/​templates
 +That saves it in the database instead of as a file.
 +Always use one or the other, don't mix them.
 +This is only for templates of things added to mail, not changing web pages.
 +Since this change was done for Mailman 2 via the "​config_list"​ command to
 +update each list in their database, the same should be done for Mailman 3.
 +It should be done this way for existing lists when they are imported.
 +
 +What about "​list:​member:​digest:​footer"​ for a digest message?
 +In the directory of template files, there is no separate
 +footer for regular and digest, only "​generic:​footer"​.
 +In the web pages for creating templates, there is no "​generic:​footer",​ but
 +there are "​regular:​footer"​ and "​digest:​footer"​.
 +
 +===== Change example.com again =====
 +
 +Many places are still referring to "​example.com"​.
 +  * "​Subject:​ [example.com]"​
 +  * "... account on example.com."​
 +  * "Thank you for using example.com!"​
 +
 +This is in database "​mm3webdb"​ table "​django_site"​.
 +Change by logging in to the web site as admin and going to URL
 +"​.../​mailman3/​admin/​sites/​site/​1/​change/"​.
 +Change the "​Domain name" value to "​sluug.org"​ and the
 +"​Display name" value to "SLUUG mailing lists"​.
 +
 +In /​var/​log/​syslog,​ every 15 minutes:
 +<​code>​
 +mariadbd[____]:​ ... [Warning] Aborted connection ____ to db: '​mm3db'​ user: '​mm3dbuser'​ host: '​localhost'​ (Got an error reading communication packets)
 +</​code>​
 +
 +Does creating a list stop the errors about "​gatenews"?​
 +No.  Still seen in mariadb logs.
 +Commented the "​gatenews"​ entry in "/​etc/​cron.d/​mailman3"​.
 +This stopped the warning every 15 minutes, but there are still two each day,
 +at the time of the other cron entries.
 +Searching online found this is a common problem and is just an annoyance with
 +nothing wrong. ​ It might be a problem with "​SQLAlchemy"​ not issuing "​quit"​
 +to properly close the connection.
 +
 +Will "​mailman_queue_check.sh"​ work?
 +Mailman 3 might be compatible with minor changes.
 +Mailman 2 uses "/​var/​lib/​mailman/​qfiles/"​.
 +Mailman 3 uses "/​var/​lib/​mailman3/​queue/"​.
 +
 +__**Everything below has not been tested yet and is for planning**__
 +
 +===== Test import of an existing test list =====
 +
 +Check archive accuracy.
 +
 +Configure to delete attachments from the archive during import?
 +Initial testing shows about half the attachments from the MM2 archives are
 +imported to MM3, others aren'​t. ​ The difference hasn't been determined.
 +
 +Test list deletion:
 +  * Delete the list - ????
 +  * Delete the archive - ????
 +
 +====== Things to change before import of production lists ======
 +
 +Protect archives from address scraping and secure private lists.
 +The protection method used for Mailman 2 doesn'​t apply to Mailman 3.
 +Instead, make the list archive private, forcing subscribers (Or owners
 +and moderators) to log in with their personal credentials to see the archive.
 +This means there is no configuration at this time, unless ​
 +we decide to make all archives public, and use the old protection method.
 +
 +Test new configuration options and enhancements to make now.
 +Especially any global options that could impact all production lists.
 +
 +MM3 restricts the list "​info"​ field to under 255 characters.
 +Change any lists that are longer, to avoid truncation during import.
 +Most MM2 production lists have much longer text.
 +They also contain information that will be obsolete, such as how to
 +access the archives. ​ The text could be changed before or after import.
 +Even if changed after import, planning what the text should be can speed the process.
 +
 +Messages posted with certain non-ASCII characters may fail to be imported
 +to the MM3 archives. ​ Most non-ASCII characters cause no problems.
 +
 +TBD settings:
 +  * "​header_checks"​ are now global?
 +  * "​[dmarc]"​ and/or "​[ARC]"​.
 +  * "​charset"​. ​ Leaving as "​us-ascii"​.
 +
 +==== Mailman 3 fails if SQL not ready ====
 +
 +After a reboot or restarting multiple processes, Mailman will often try
 +to connect to the SQL database server before that server has completed
 +startup to the point it can accept connections and perform operations.
 +This makes the Mailman service fail and terminate.
 +This is a well known problem.
 +There is no systemd configuration present to state Mailman depends on the
 +SQL service. ​ This has been corrected in later Debian Mailman releases.
 +
 +Correction is by a systemd override to establish the dependency.
 +
 +Detailed actions are to be documented here. ----
 +
 +====== Import Mailman 2 lists ======
 +
 +To check the archives for potential errors before attempting import:
 +  * Use cleanarch or cleanarch3 to check for bad From and Date headers.
 +  * Use check_hk_archive (replaced by check_hk_import) to check for messages that would fail import.
 +
 +While cleanarch is a standard part of MM2, cleanarch3 is not in the
 +currently installed Debian python3-django-hyperkitty package.
 +It is in the more recent 1.3.12-1 (1.3.8 and later) release.
 +The same for check_hk_archive (replaced by check_hk_import).
 +Download the latest package, extact the two scripts to a temporary location.
 +
 +===== Extra steps for 3 lists =====
 +
 +For 3 lists, that existed before the 2007 move from Majordomo, the
 +"/​var/​lib/​mailman/​archives/​private/​LISTNAME.mbox/​LISTNAME.mbox"​ file
 +doesn'​t go back as far as the original Majordomo files imported into mailman.
 +The monthly text files created during the import to generate the HTML
 +do have all the mail.
 +In "​var/​lib/​mailman/​archives/​private/​LISTNAME/"​
 +are the "​YYYY-MMMMM.txt"​ and "​YYYY-MMMMM.txt.gz"​ files for each month
 +since 2000.  Should just need to concatenate them in the proper order.
 +Since these are not being updated, this preparation work can be done
 +anytime before the archive import.
 +
 +<​code>​
 +months='​January
 +February
 +March
 +April
 +May
 +June
 +July
 +August
 +September
 +October
 +November
 +December'​
 +years='​2000 2001 2002 2003 2004 2005 2006 2007'
 +
 +archive_path=/​var/​lib/​mailman/​archives/​private
 +export listname=____
 +
 +for year in $years
 +  do
 +    for month in $months
 +      do
 +        file="​${archive_path}/​${listname}/​$year-$month.txt"​
 +        #echo "​$file"​
 +        if [ -f "​$file"​ ]
 +          then
 +            cat "​$file"​
 +          fi
 +      done
 +  done > /​tmp/​${listname}-rebuilt.txt
 +
 +Edit /​tmp/​${listname}-rebuilt.txt and delete everything starting with the
 +first message in the mbox file until the end.
 +Except the original monthly text file for the initial import from Majordomo
 +has sections that go back in time, and some are repeated.
 +This makes it more complicated to determine what to delete or keep.
 +
 +Also have to change the "From " lines that were massaged to obscure the sender.
 +Before that, search for any that are incorrect "From " lines within a message.
 +These might have been archived that way.  In that case, leave them alone.
 +<​code>​
 +1G/^\CFrom .*@/ Any within a message to fix
 +:%g/^\CFrom .* at .*200\d$/s/ at /_at_/ " Make normal mail standard line
 +</​code>​
 +
 +Run the two checking tools, the same as below.
 +The "From " will probably change for a small number of messages.
 +
 +<​code>​
 +/​tmp/​cleanarch3 \
 +    /​tmp/​${listname}-rebuilt.txt \
 +    > /​tmp/​${listname}-rebuilt.mbox
 +diff \
 +    /​tmp/​${listname}-rebuilt.txt \
 +    /​tmp/​${listname}-rebuilt.mbox
 +
 +/​tmp/​check_hk_import /​tmp/​${listname}-rebuilt.mbox
 +</​code>​
 +
 +===== Standard steps for all lists =====
 +
 +Stop postfix, to prevent any incoming mail going to the mailman 2 queue
 +while in the process of transitioning. ​ Not needed for low volume lists
 +where no postings can be expected during this period.
 +
 +Since the import and indexing of a large archive can take three hours,
 +use a technique to continue running if an interactive session is lost.
 +Don't attempt during a storm.
 +
 +Stop mailman 2 ...
 +
 +Must create an empty list before import.
 +For each list:
 +<​code>​
 +export config_path=/​var/​lib/​mailman/​lists
 +export archive_path=/​var/​lib/​mailman/​archives/​private
 +export listname=____
 +export listfull=${listname}@sluug.org
 +
 +/​tmp/​cleanarch3 \
 +    ${archive_path}/​${listname}.mbox/​${listname}.mbox \
 +    > /​tmp/​${listname}-cleaned.mbox
 +diff \
 +    ${archive_path}/​${listname}.mbox/​${listname}.mbox \
 +    /​tmp/​${listname}-cleaned.mbox
 +
 +/​tmp/​check_hk_import ​ /​tmp/​${listname}-cleaned.mbox
 +
 +Check the number of archive messages is reasonable.
 +The MM2 list archives imported from Majordomo have many duplicate HTML files.
 +ls ${archive_path}/​${listname}/​[12]???​-* | grep -c 0......html
 +
 +mailman-wrapper create ​  ​${listfull}
 +((The only output is that the list was created))
 +
 +mailman-wrapper import21 ${listfull} ${config_path}/​${listname}/​config.pck
 +((If there are no problems, there might be no output))
 +((The only output might be errors and "​Importing members"​ progress))
 +
 +chmod a+r /​tmp/​${listname}-*.mbox # Since run as "​www-data"​ user
 +
 +((Include "​rebuilt.mbox"​ only if it was created.))
 +date ; time mailman-web hyperkitty_import --no-sync-mailman \
 +    --verbosity 3 --no-color \
 +    --list-address ${listfull} \
 +    /​tmp/​${listname}-rebuilt.mbox \
 +    /​tmp/​${listname}-cleaned.mbox \
 +    >> /​tmp/​import-${listname}-log.txt 2>&1 ; date
 +((It lists each message as it is processed.))
 +
 +date ; time mailman-web update_index_one_list --verbosity 2 --no-color \
 +    ${listfull} \
 +    >> /​tmp/​index-${listname}-log.txt 2>&1 ; date
 +((There is no output during the first phase, a quick summary of
 +the number indexed, then nothing during last phase))
 +</​code>​
 +
 +^  Mailing list and list archive settings to review or set  ^^
 +^ Setting ​                  ^ Action ​         ^
 +^ List settings ​ ^^
 +| Show list on index page   | Probably OK               |
 +| Description ​              | Probably OK               |
 +| Information ​              | Update for new situation ​ |
 +| Display name              | Probably OK               |
 +| Subject prefix ​           | Probably OK               |
 +| Other "​Settings" ​         | Probably OK               |
 +^ Other list setting pages  ^^
 +| Header Filters ​           | Verify they were imported |
 +| Alter Messages ​           | Verify they were imported |
 +| DMARC Mitigations ​        | Verify they were imported |
 +| Digest ​                   | Verify they were imported |
 +| Message Acceptance ​       | Verify they were imported |
 +| Archiving ​                | Probably need to set this |
 +| Member Policy ​            | Verify they were imported |
 +| Bounce Processing ​        | Verify they were imported |
 +^ Archive settings ​ ^^
 +| Display name              | Probably OK               |
 +| Description ​              | Probably blank, must set  |
 +| Subject prefix ​           | Probably OK               |
 +| Archive policy ​           | Defaults to "​public",​ must set this to "​private"​ for most lists |
 +| Owners ​                   | Review, need at least one selected ​ |
 +| Moderators ​               | Review, need at least one selected ​ |
 +
 +Test by posting and web access?
 +
 +Delete the list from the Mailman 2 configuration by moving the config out of
 +the production directory.
 +<​code>​
 +mv -iv ${config_path}/​${listname} ${config_path}-disabled/​
 +</​code>​
 +
 +Delete Mailman 2 list aliases and create Mailman 3 aliases? ​ No.
 +Mailman 2 uses "/​var/​lib/​mailman/​data/​aliases"​ for "​alias_maps"​.
 +Mailman 3 uses "/​var/​lib/​mailman3/​data/​postfix_lmtp"​ for "​local_recipient_maps"​.
 +No action is needed.
 +The new list aliases are created when the lists are imported.  ​
 +The "​local_recipient_maps"​ takes precedence over "​alias_maps"​.
 +
 +Start mailman 2 ...
 +
 +Start postfix if it was stopped.
 +
 +There is a postfix configuration conflict that results in warning
 +"do not list domain ____ in BOTH virtual_mailbox_domains and relay_domains"​.
 +This is a well known problem.
 +See https://​docs.mailman3.org/​projects/​mailman/​en/​latest/​src/​mailman/​docs/​mta.html#​unusual-postfix-configuration
 +
 +Check all configuration web pages on the old and new systems,
 +to make sure no options, settings, or values were lost.
 +
 +Change the list "​info"​ field for each list.
 +
 +Check the list archive HyperKitty Description and Archive policy.
 +Might need to set Owners and Moderators.
 +
 +Test ...
 +
 +====== Other changes to make later ======
 +
 +Delete or deflate HTML postings and delete attachments on all lists.
 +The "​Collapse alternatives"​ and "​Convert html to plaintext"​ options.
 +
 +Change the "​Action to take on messages which have no content"​ setting
 +from "​Discard"​ to "​Reject"​.
 +
 +Create "/​etc/​cron.d/​mailman3-local"​ that runs some check that all
 +expected processes are running.
 +  * 15 "^list .*/​usr/​lib/​mailman3/​bin/"​ for "​mailman3",​
 +  * 2 "​www-data .*uwsgi"​ for "​mailman3.web",​
 +  * 6-7 "​manage.py qcluster"​ for "​mailman3.web",​
 +  * Perhaps also check the output of the "​mailman-wrapper status"​ command for "​running"​.
 +
 +Is there a way to disable "​Download"​ of HyperKitty archive as a "​.mbox"​ file?
 +
 +====== Cleanup for deletion of Mailman 2 ======
 +
 +Stop and disable the Mailman 2 service.
 +
 +Edit "/​etc/​postfix/​main.cf"​ and remove "​hash:/​var/​lib/​mailman/​data/​aliases"​
 +from "​alias_maps"​. ​
 +
 +Save the monthly text files of the Mailman 2 archives.
 +
 +Delete "/​etc/​cron.d/​mailman-local"​ that runs "​mailman_queue_check.sh"​.
 +Delete "/​usr/​local/​bin/​mailman_queue_check.sh"​.
 +
 +Delete Mailman 2 from the Apache site configuration.
 +
 +====== Administration ======
 +
 +===== Command line =====
 +
 +Since the services are started by systemd, it is probably better to
 +do start/​stop/​restart using the systemctl command. ​ Using the direct
 +commands might not update systemd on the service status.
 +
 +Never use the "​mailman"​ command,
 +only "​mailman-wrapper"​ that runs as user "​list"​.
 +
 +Useful "​mailman-wrapper"​ commands:
 +  * "​mailman-wrapper --help"​ - display the available commands.
 +  * "​mailman-wrapper COMMAND --help"​ - display help for one command.
 +  * "​mailman-wrapper lists" - list the available lists.
 +  * "​mailman-wrapper members FULL_LIST_NAME_WITH_HOSTNAME | wc -l" - count of members to one list.  In a future release, this might be handled by the "​--count-only"​ option.
 +  * "​mailman-wrapper findmember '​REGEXP'"​ - Display all memberships for matching users.
 +  * "​mailman-wrapper restart"​ - Stop and restart the Mailman runner subprocesses. ​ See above about using the systemctl command.
 +
 +"​mailman-web"​ - Django'​s command-line utility for administrative tasks.
 +Runs the "​mailman3-web"​ "​manage.py"​ command as user "​www-data"​.
 +
 +"​django-admin"​ - 
 +Similar to "​mailman-web",​ but without the proper settings.
 +
 +===== URLs =====
 +
 +==== URLs for general users ====
 +
 +  * https://​WEBSERVER/​mailman3 ​    / - General list access and user login. ​ What should be announced to humans.
 +  * https://​WEBSERVER/​mailman3/​postorius/​lists/​ - What just specifying "​mailman3"​ redirects to.  What should be used in links from the main web site.
 +  * https://​docs.mailman3.org/​en/​latest/​userguide.html - List Member Manual
 +
 +==== Alternate URLs to ignore ====
 +
 +  * https://​WEBSERVER/​mailman3/​hyperkitty/​ - long list of "​Available lists",​ with "Sign In" for regular list members.
 +  * https://​WEBSERVER/​mailman3/​postorius/​ - short list of "​Mailing Lists"​.
 +  * https://​WEBSERVER/​mailman3/​postorius/​lists/​ - the same as just "​postorius/"​.
 +
 +==== URLs for Administration ====
 +
 +  * https://​WEBSERVER/​mailman3/​admin/​ - Administration login, only "for a staff account"​.
 +  * https://​WEBSERVER/​mailman3/​LISTNAME.HOSTNAME/​ - Access private lists not published on the index of lists. ​ You will still have to log in or directly subscribe to do anything more than see the list description. ​ This uses a "​."​ instead of "​@"​.
 +  * https://​WEBSERVER/​mailman3/​hyperkitty/​list/​LISTNAME@HOSTNAME/​ - Access private archives. ​ You will still have to log in to see the private archive. ​ This uses a "​@"​.
 +
 +You can use the general URL for lists, then login and you will see private
 +"lists that you are owner, moderator or subscriber for".
 +
 +To approve or reject a held message:
 +  * Log into the admin URL
 +  * Go to "View site".
 +  * Select the list name from the table of lists.
 +  * Go to "Held messages"​.
 +  * Select the check box for the messages to be handled.
 +  * Select the action to perform.
 +
 +==== Usage notes ====
 +
 +Trying to log into the general URL with the admin account might fail with a
 +500 error.
 +  * Logging into the general URL with your personal account/​password used for list subscription doesn'​t give access to list administration,​ only changing your account and relationships to lists.
 +  * This is even if your account is the list owner.
 +  * Use this with your personal credentials to login and control "lists that you are owner, moderator or subscriber for".
 +
 +Use the admin URL, then log in, then either select
 +  * For the mailing list, select "View site" in the heading,
 +    * "​Mailman settings"​ to change subscription settings for this address.
 +    * A list to change "List Settings",​ handle pending actions, etc.
 +  * For list archives, select "​Mailing lists" under the "​HyperKitty",​ then a list to change description,​ etc.
 +    * There is no command line interface for most HyperKitty actions.
 +    * To remove the archive for a list, you must use the admin web interface and the "​HyperKitty > Mailing lists" (admin/​hyperkitty/​mailinglist/​) web page.  This only affects the list archive, not the base list.
 +
 +When logging out of the web interface,
 +watch for the logout confirmation web page.
 +
 +===== Processes =====
 +
 +There are separate services and processes for the mailing list and web.
 +  * The "​mailman3-web"​ service controls the "​uwsgi"​ processes. and includes "​python3 manage.py"​ processes.
 +  * The "​mailman3"​ service is Mailman 3, for processing mail.  For the "/​usr/​lib/​mailman3/​bin/"​ processes.
 +  * The "​mailman"​ service is Mailman 2, not 3.
 +
 +If you update a template file, you may need to restart the matching service
 +in order for the changed file to be used.
 +
 +===== Logs =====
 +
 +The logs are in "/​var/​log/​mailman3/",​ with web actions in the
 +"​web/"​ subdirectory.
 +
 +The structure of log files was changed to be closer to the
 +separate log files of Mailman 2.
  
replace_mailman_2.1704544778.txt.gz · Last modified: 2024/01/06 06:39 by SLUUG Administration