User Tools

Site Tools


build:lists

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
build:lists [2008/03/05 19:28]
4.245.78.137
build:lists [2018/05/28 03:34] (current)
SLUUG Administration [Logs]
Line 55: Line 55:
  
  
-NOTE: MailMan requires at least Python 2.4. Debian 4.0 meets this requirement. Debian 3.1 did not, so we had to install ​Pyton 2.4 and use update-alternatives to make it the default version. Debian 3.1 also required installing libdb2.+NOTE: MailMan requires at least Python 2.4. Debian 4.0 meets this requirement. Debian 3.1 did not, so we had to install ​Python ​2.4 and use update-alternatives to make it the default version. Debian 3.1 also required installing libdb2. 
  
 =====Pre-installation Steps===== =====Pre-installation Steps=====
-As root, create /​usr/​local/​mailman-2.1.9, make it owned by group mailman, and chgrp 2775 /​usr/​local/​mailman.+As root, create ​the Mailman directory, set its ownership and permissions:​ 
 +<code rootshell>​ 
 +mkdir /​usr/​local/​mailman-2.1.9 
 +chown mailman ​/​usr/​local/​mailman-2.1.9 
 +chmod 2775 /​usr/​local/​mailman-2.1.9 
 +ln -s mailman-2.1.9 /​usr/​local/​mailman 
 +</​code>​
  
 =====Extract and Prepare Source===== =====Extract and Prepare Source=====
-The Debian Mailman package did not include search ​capibility. To enable search Mailman has to be compiled with htdig. ​ Extract the source file into /​usr/​local/​src/​mailman-2.1.9 and move the patch into that directory. gunzip the patch. ​+The Debian Mailman package did not include search ​capability. To enable search Mailman has to be compiled with htdig. ​ Extract the source file into /​usr/​local/​src/​mailman-2.1.9 and move the patch into that directory. gunzip the patch. ​
  
 Disable most language support. This is an ugly hack to get around a problem with Japanese-language support in python on Debian Etch. Comment out the following line in messages/​Makefile.in:​ Disable most language support. This is an ugly hack to get around a problem with Japanese-language support in python on Debian Etch. Comment out the following line in messages/​Makefile.in:​
Line 117: Line 124:
  
 =====Apache configuration for all lists===== =====Apache configuration for all lists=====
-Then edit /​etc/​apache2/​sites-available/​www.sluug.org and add:+Then edit /​etc/​apache2/​sites-available/​000-www.sluug.org and add:
  
 <code root> <code root>
Line 144: Line 151:
  
 ====Tell Apache to use password protection==== ====Tell Apache to use password protection====
-Edit /​etc/​apache2/​sites-available/​000-www.sluug.org and add a new directory section, filling in the directory where the archive is stored, the name of the list for the AuthName, and the file with the password.+Edit /​etc/​apache2/​sites-available/​000-www.sluug.org and <del>add a new directory section</​del>​, filling in the directory where the archive is stored, the name of the list for the AuthName, and the file with the password.
 <code root> <code root>
         # Define password protection for this list's archives         # Define password protection for this list's archives
Line 156: Line 163:
 </​code>​ </​code>​
  
 +===Security holes===
 +
 +In 2018, on amber, it was found there were two major holes that allowed bots
 +to access all mailing list contents:
 +  - Each archived posting has two URLs, ''/​pipermail/''​ and  ''/​mailman/​htdig/'',​ however only the first is protected.
 +  - The sluug.org and stlwebdev.org archives are defined in separate config files, and only protect the lists defined in each, but all lists are accessible through either domain.
 +
 +Therefore, the above ''<​Directory>''​ sections were replaced with
 +''<​LocationMatch>''​ sections that first denied all access to the two URLs,
 +then allowed password protected access to to selected lists available through
 +that domain. ​ Also unlimited access to two lists used for announcements.
 +Any list not overridden by a subsequent section will be blocked by the
 +first global section.
 +Using wildcards allowed protecting both URL paths without duplicating all the
 +password statements.
 +
 +<​code>​
 +    # Heavy use of symbolic links in Mailman configuration
 +    <​Directory /​usr/​local/​mailman/​archives/​public>​
 +        Options FollowSymlinks
 +    </​Directory>​
 +
 +    # Block all access to other lists' archives
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig|/​pipermail)">​
 +        Order allow,deny
 +        Deny from all
 +    </​LocationMatch>​
 +
 +    # Define password protection for this list's archives
 +    # For all these lists: discuss steercom jobs test*
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​(discuss|steercom|jobs|test)|/​pipermail/​(discuss|steercom|jobs|test))">​
 +        Allow from all
 +        AuthType Basic
 +        AuthName "SLUUG Discussion Archive Access"​
 +        AuthUserFile /​etc/​apache2/​discuss-passwords
 +        Require valid-user
 +    </​LocationMatch>​
 +
 +    # Define password protection for this list's archives
 +    # For sysadmin list only
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​sysadmin|/​pipermail/​sysadmin)">​
 +        Allow from all
 +        AuthType Basic
 +        AuthName "SLUUG Sysadmin Archive Access"​
 +        AuthUserFile /​etc/​apache2/​sysadmin-passwords
 +        Require valid-user
 +    </​LocationMatch>​
 +
 +    # No password protection for this list's archives
 +    # For all these lists: announce users
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​(announce|users)|/​pipermail/​(announce|users))">​
 +        Allow from all
 +    </​LocationMatch>​
 +</​code>​
 +
 +All common parts of the port 80 and port 443 ''<​VirtualHost>''​ definitions
 +were moved to a common file to eliminate complete duplication.
 +
 +Make similar changes to the stlwebdev web site definition.
 ====Create the  password file==== ====Create the  password file====
 The name of the file matches the AuthUserFile configuration statement. ​ The username for the htpasswd command is whatever is used for that password file.  You will be prompted for the password by the htpasswd command. The name of the file matches the AuthUserFile configuration statement. ​ The username for the htpasswd command is whatever is used for that password file.  You will be prompted for the password by the htpasswd command.
Line 223: Line 293:
 </​code>​ </​code>​
  
 +Edit ''​crontab''​ before or after installation to:
 +  * Comment the ''​gate_news''​ entry. ​ We are not using any usenet gateway.
 =====Copy Apache Images to Mailman===== =====Copy Apache Images to Mailman=====
 <code root> <code root>
Line 406: Line 478:
 Configure the bud_test list at [[http://​bud.sluug.org/​mailman/​admin/​bud_test/​]] Configure the bud_test list at [[http://​bud.sluug.org/​mailman/​admin/​bud_test/​]]
  
-=====Mailman TODO:=====+=====Creating lists for virtual domains=====
  
-  ​We should probably SSL-require the administrative pages for mailman.+Make sure your domain'​s MX records point to bud.sluug.org
  
-  ​We need to test mailman with some of the majordomo archives from +Make sure your domain is included in virtual_alias_maps in /​etc/​postfix/​main.cf
-michelob.+
  
 +* In /​etc/​postfix/​virtual,​ add a new entry for your list. The LHS should be the address to which users send their posts, and the RHS should be the name of the list you are creating (no domain part, just the list name).
 +
 +* Run 
 +
 +<code root>
 +# postmap /​etc/​postfix/​virtual
 +</​code>​
 +
 +* In /​usr/​local/​mailman/​Mailman/​mm_cfg.py,​ add your domain to POSTFIX_STYLE_VIRTUAL_DOMAINS. If there needs to be more than one domain set in POSTFIX_STYLE_VIRTUAL_DOMAINS,​ this variable probably needs to be set in python list syntax. ​
 +
 +* Restart mailman
 +
 +<code root>
 +# /​etc/​init.d/​mailman restart
 +</​code>​
 +
 +* Create a new vhost in /​etc/​apache2/​sites-available for your virtual domain if it doesn'​t already exist. Using /​etc/​apache2/​sites-available/​www.sluug.org as an example, copy the mailman <​directory>​ containers into your vhost, making sure to update passwords, descriptions,​ and archive paths as necessary.
 +
 +* Restart apache
 +
 +<code root>
 +# apache2ctl graceful
 +</​code>​
 +
 +* Create your list
 +
 +<code root>
 +#/​usr/​local/​mailman/​bin/​newlist list@domain
 +</​code>​
 +
 +You will be prompted for the email address of a list admin and for an administrative password for the list. Check that email account for a welcome mail. Go to the admin URL in that mail and select "​Privacy Options"​->"​Recipient Filters"​. Add your list's email address to the box labeled "Alias names (regexps) which qualify as explicit to or cc destination names for this list." If you don't, all posts to your new list will be held for admin approval with a message about implicit destinations.
 +
 +* Send some test mails to verify that the list is working as expected.
 +
 +Currently we are hosting announce@stlwebdev.org and discuss@stlwebdev.org.
 +
 +
 +=====Mailman TODO:=====
 +
 +  * We should probably SSL-require the administrative pages for mailman.
 +  * We need to test mailman with some of the majordomo archives from michelob.
   * We need to set up mailman to handle lists for all of sluug.org   * We need to set up mailman to handle lists for all of sluug.org
 +  * /​usr/​local/​mailman/​bin/​add_members should be run on a list of current subscribers to ANNOUNCE, DISCUSS, SYSADMIN, and STEERCOM. I'm not quite sure how passwords, if any, will be handled.
 +  * We need to copy the archives from michelob to bud and index them via htdig. I'm not sure how htdig and/or apache will handle the compressed files as currently configured.
  
-  * /​usr/​local/​mailman/​bin/​add_members should be run on a list of current +=== Spam Subscriptions ===
-subscribers to ANNOUNCE, DISCUSS, SYSADMIN, and STEERCOM. I'm not quite +
-sure how passwords, if any, will be handled.+
  
-  * We need to copy the archives from michelob ​to bud and index them via +In 2018, it was discovered that one or more criminals were using a 
-htdigI'm not sure how htdig and/or apache will handle ​the compressed files as currently configured.+bot network ​to make repeated subscription requests to multiple lists, 
 +with the intent of SLUUG sending thousands of subscription confirmation 
 +e-mails ​to one address, that would eventually change to the next target. 
 +Looking in the logs, this had been happening for years, with SLUUG sending 
 +over 80k spam confirmations. 
 +This also taxed the SLUUG mail server when hundreds of e-mails were 
 +rejected ​and sitting in the mail queue for retry.
  
 +In response, a local mod was made to ''​subscribe.py''​
 +<code python>
 +sluug_sub_mod1_value = cgidata.getvalue('​sluug_sub_mod1',​ ''​)
 +    if not sluug_sub_mod1_value:​
 +        syslog('​mischief',​ '​Subscribe w/o local mod form field as: %s: %s', email, remote)
 +        results.append(_('​Subscription failed due to internal error!'​))
 +    elif sluug_sub_mod1_value != '​sluug_sub_mod1-20180517':​
 +        syslog('​mischief',​ '​Subscribe w/ wrong local mod form field value %s as: %s: %s', sluug_sub_mod1_value,​ email, remote)
 +        results.append(_('​Subscription failed due to internal error!'​))
 +</​code>​
  
-GENERAL MAIL TODO:+Also modified the generic ''​listinfo.html''​ template and the customized 
 +version for the announce list (no other customized versions needed changes) 
 +to add: 
 +<code html> 
 +<input type="​hidden"​ name="​sluug_sub_mod1"​ value="​sluug_sub_mod1-20180517">​ 
 +</​code>​
  
-  * We need to get virtual users set up in some way. Craig and I discussed +The permanent fix to stop all spam subscriptions is to upgrade ​to a current 
-this, and two options are postfix maps and mysql database. We didn't +release of mailman that includes ​''​SUBSCRIBE_FORM_SECRET''​ and probably other 
-make any decisions. Two particular challenges will be copying existing  +new features ​to combat the bots.
-passwords for POP3/IMAP access and mail filtering (procmail/​maildrop/​whatever). Once we get users set up, we'll need to migrate their mail spools.+
  
-  * We need to get spamassassin working. We particularly need to take a +=== GENERAL MAIL TODO: ===
-look at the performance impact of scanning list mail. This should +
-probably be done incoming list mail only.+
  
-  * We need to get some form of webmail up and running. I'm partial to +  ​* We need to get virtual users set up in some way. Craig and I discussed this, and two options are postfix maps and a mysql database. We didn't make any decisions. Two particular challenges will be copying existing passwords for POP3/IMAP access and mail filtering (procmail/​maildrop/​whatever). Once we get users set up, we'll need to migrate their mail spools. 
-Horde becauses it has a powerful interface and a ton of cool modules. I +  * We need to get spamassassin working. We particularly need to take a look at the performance impact of scanning list mail. This should probably be done incoming list mail only. 
-haven'​t used the password module, but it might be particularly useful +  ​* We need to get some form of webmail up and running. I'm partial to Horde becauses it has a powerful interface and a ton of cool modules. I haven'​t used the password module, but it might be particularly useful for us. See http://​www.horde.org/​accounts/​screenshots/​accounts.png. If we use horde, we'll be using mysql, so that might be the way to go for virtual users.
-for us. See http://​www.horde.org/​accounts/​screenshots/​accounts.png. If +
-we use horde, we'll be using mysql, so that might be the way to go for +
-virtual users.+
  
 That should do it for now - enjoy the rest of the weekend. That should do it for now - enjoy the rest of the weekend.
Line 447: Line 574:
  
 ---- ----
- 
 ====== Notes ====== ====== Notes ======
  
Line 457: Line 583:
   * vette - results of admin actions to deferred posts   * vette - results of admin actions to deferred posts
   * bounce - tracks bounces, so failing members can automatically be purged   * bounce - tracks bounces, so failing members can automatically be purged
 +  * mischief - Detected attempts to use mailman that failed, including subscriptions by bots that were blocked.
 +  * error - Problems.
 +  * smtp - All mail activities?
 +  * smtp-failure - All mail problems?
  
 TODO: We should move these to /var/log and put them under log rotation. TODO: We should move these to /var/log and put them under log rotation.
 +See ''​contrib/​*redhat_fhs.patch''​ for a source modification to change
 +log and data file locations.
  
 Important information about problems might also be in the Apache server logs.  Currently in ''/​var/​log/​apache2/''​. Important information about problems might also be in the Apache server logs.  Currently in ''/​var/​log/​apache2/''​.
- 
 ===== Problems seen ===== ===== Problems seen =====
  
build/lists.1204766929.txt.gz · Last modified: 2008/03/05 19:28 (external edit)