This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | |||
|
replace_mailman_2 [2025/11/12 04:44] SLUUG Administration Progress update |
replace_mailman_2 [2025/11/18 02:13] (current) SLUUG Administration Progress update |
||
|---|---|---|---|
| Line 346: | Line 346: | ||
| Configure to delete attachments from the archive during import? | Configure to delete attachments from the archive during import? | ||
| + | Initial testing shows about half the attachments from the MM2 archives are | ||
| + | imported to MM3, others aren't. The difference hasn't been determined. | ||
| Test list deletion: | Test list deletion: | ||
| Line 370: | Line 372: | ||
| Even if changed after import, planning what the text should be can speed the process. | Even if changed after import, planning what the text should be can speed the process. | ||
| - | Messages posted with non-ASCII characters may fail to be imported to the | + | Messages posted with certain non-ASCII characters may fail to be imported |
| - | MM3 archives. Temporarily changing the charset to "utf-8" seems to solve this. | + | to the MM3 archives. Most non-ASCII characters cause no problems. |
| - | There is no need to restart the server processes, since this is being done | + | |
| - | for the command line archive import. | + | |
| TBD settings: | TBD settings: | ||
| Line 405: | Line 405: | ||
| The same for check_hk_archive (replaced by check_hk_import). | The same for check_hk_archive (replaced by check_hk_import). | ||
| Download the latest package, extact the two scripts to a temporary location. | Download the latest package, extact the two scripts to a temporary location. | ||
| + | |||
| + | ===== Extra steps for 3 lists ===== | ||
| For 3 lists, that existed before the 2007 move from Majordomo, the | For 3 lists, that existed before the 2007 move from Majordomo, the | ||
| Line 433: | Line 435: | ||
| archive_path=/var/lib/mailman/archives/private | archive_path=/var/lib/mailman/archives/private | ||
| + | export listname=____ | ||
| for year in $years | for year in $years | ||
| Line 449: | Line 452: | ||
| Edit /tmp/${listname}-rebuilt.txt and delete everything starting with the | Edit /tmp/${listname}-rebuilt.txt and delete everything starting with the | ||
| first message in the mbox file until the end. | first message in the mbox file until the end. | ||
| + | Except the original monthly text file for the initial import from Majordomo | ||
| + | has sections that go back in time, and some are repeated. | ||
| + | This makes it more complicated to determine what to delete or keep. | ||
| + | |||
| Also have to change the "From " lines that were massaged to obscure the sender. | Also have to change the "From " lines that were massaged to obscure the sender. | ||
| Before that, search for any that are incorrect "From " lines within a message. | Before that, search for any that are incorrect "From " lines within a message. | ||
| Line 470: | Line 477: | ||
| /tmp/check_hk_import /tmp/${listname}-rebuilt.mbox | /tmp/check_hk_import /tmp/${listname}-rebuilt.mbox | ||
| </code> | </code> | ||
| + | |||
| + | ===== Standard steps for all lists ===== | ||
| Stop postfix, to prevent any incoming mail going to the mailman 2 queue | Stop postfix, to prevent any incoming mail going to the mailman 2 queue | ||
| while in the process of transitioning. Not needed for low volume lists | while in the process of transitioning. Not needed for low volume lists | ||
| where no postings can be expected during this period. | where no postings can be expected during this period. | ||
| + | |||
| + | Since the import and indexing of a large archive can take three hours, | ||
| + | use a technique to continue running if an interactive session is lost. | ||
| + | Don't attempt during a storm. | ||
| Stop mailman 2 ... | Stop mailman 2 ... | ||
| Line 504: | Line 517: | ||
| ((If there are no problems, there might be no output)) | ((If there are no problems, there might be no output)) | ||
| ((The only output might be errors and "Importing members" progress)) | ((The only output might be errors and "Importing members" progress)) | ||
| - | |||
| - | Edit "/etc/mailman3/mailman.cfg" to temporarily add or uncomment: | ||
| - | [language.en] | ||
| - | charset: utf-8 | ||
| chmod a+r /tmp/${listname}-*.mbox # Since run as "www-data" user | chmod a+r /tmp/${listname}-*.mbox # Since run as "www-data" user | ||
| + | ((Include "rebuilt.mbox" only if it was created.)) | ||
| date ; time mailman-web hyperkitty_import --no-sync-mailman \ | date ; time mailman-web hyperkitty_import --no-sync-mailman \ | ||
| - | --verbosity 1 --no-color \ | + | --verbosity 3 --no-color \ |
| --list-address ${listfull} \ | --list-address ${listfull} \ | ||
| /tmp/${listname}-rebuilt.mbox \ | /tmp/${listname}-rebuilt.mbox \ | ||
| /tmp/${listname}-cleaned.mbox \ | /tmp/${listname}-cleaned.mbox \ | ||
| - | 2>&1 | tee -a /tmp/import-${listname}-log.txt ; date | + | >> /tmp/import-${listname}-log.txt 2>&1 ; date |
| - | ((There is a percentage of progress and summary of actions)) | + | ((It lists each message as it is processed.)) |
| date ; time mailman-web update_index_one_list --verbosity 2 --no-color \ | date ; time mailman-web update_index_one_list --verbosity 2 --no-color \ | ||
| ${listfull} \ | ${listfull} \ | ||
| - | 2>&1 | tee -a /tmp/index-${listname}-log.txt ; date | + | >> /tmp/index-${listname}-log.txt 2>&1 ; date |
| - | ((There is a summary of the number indexed)) | + | ((There is no output during the first phase, a quick summary of |
| + | the number indexed, then nothing during last phase)) | ||
| </code> | </code> | ||
| - | |||
| - | Edit "/etc/mailman3/mailman.cfg" to delete or comment what was added above. | ||
| ^ Mailing list and list archive settings to review or set ^^ | ^ Mailing list and list archive settings to review or set ^^ | ||
| Line 591: | Line 600: | ||
| Delete or deflate HTML postings and delete attachments on all lists. | Delete or deflate HTML postings and delete attachments on all lists. | ||
| The "Collapse alternatives" and "Convert html to plaintext" options. | The "Collapse alternatives" and "Convert html to plaintext" options. | ||
| + | |||
| + | Change the "Action to take on messages which have no content" setting | ||
| + | from "Discard" to "Reject". | ||
| Create "/etc/cron.d/mailman3-local" that runs some check that all | Create "/etc/cron.d/mailman3-local" that runs some check that all | ||
| Line 657: | Line 669: | ||
| * https://WEBSERVER/mailman3/admin/ - Administration login, only "for a staff account". | * https://WEBSERVER/mailman3/admin/ - Administration login, only "for a staff account". | ||
| - | * https://WEBSERVER/mailman3/LISTNAME.HOSTNAME/ - Access private lists not published on the index of lists. You will still have to log in or directly subscribe to do anything more than see the list description. | + | * https://WEBSERVER/mailman3/LISTNAME.HOSTNAME/ - Access private lists not published on the index of lists. You will still have to log in or directly subscribe to do anything more than see the list description. This uses a "." instead of "@". |
| - | * https://WEBSERVER/mailman3/hyperkitty/list/LISTNAME@HOSTNAME/ - Access private archives. You will still have to log in to see the private archive. | + | * https://WEBSERVER/mailman3/hyperkitty/list/LISTNAME@HOSTNAME/ - Access private archives. You will still have to log in to see the private archive. This uses a "@". |
| You can use the general URL for lists, then login and you will see private | You can use the general URL for lists, then login and you will see private | ||
| Line 685: | Line 697: | ||
| * For list archives, select "Mailing lists" under the "HyperKitty", then a list to change description, etc. | * For list archives, select "Mailing lists" under the "HyperKitty", then a list to change description, etc. | ||
| * There is no command line interface for most HyperKitty actions. | * There is no command line interface for most HyperKitty actions. | ||
| - | * To remove the archive for a list, you must use the admin web interface and the "HyperKitty > Mailing lists" (admin/hyperkitty/mailinglist/) web page. | + | * To remove the archive for a list, you must use the admin web interface and the "HyperKitty > Mailing lists" (admin/hyperkitty/mailinglist/) web page. This only affects the list archive, not the base list. |
| When logging out of the web interface, | When logging out of the web interface, | ||