User Tools

Site Tools


build:lists

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
build:lists [2018/05/28 02:25]
SLUUG Administration [cron]
build:lists [2018/05/28 03:34] (current)
SLUUG Administration [Logs]
Line 151: Line 151:
  
 ====Tell Apache to use password protection==== ====Tell Apache to use password protection====
-Edit /​etc/​apache2/​sites-available/​000-www.sluug.org and add a new directory section, filling in the directory where the archive is stored, the name of the list for the AuthName, and the file with the password.+Edit /​etc/​apache2/​sites-available/​000-www.sluug.org and <del>add a new directory section</​del>​, filling in the directory where the archive is stored, the name of the list for the AuthName, and the file with the password.
 <code root> <code root>
         # Define password protection for this list's archives         # Define password protection for this list's archives
Line 163: Line 163:
 </​code>​ </​code>​
  
 +===Security holes===
 +
 +In 2018, on amber, it was found there were two major holes that allowed bots
 +to access all mailing list contents:
 +  - Each archived posting has two URLs, ''/​pipermail/''​ and  ''/​mailman/​htdig/'',​ however only the first is protected.
 +  - The sluug.org and stlwebdev.org archives are defined in separate config files, and only protect the lists defined in each, but all lists are accessible through either domain.
 +
 +Therefore, the above ''<​Directory>''​ sections were replaced with
 +''<​LocationMatch>''​ sections that first denied all access to the two URLs,
 +then allowed password protected access to to selected lists available through
 +that domain. ​ Also unlimited access to two lists used for announcements.
 +Any list not overridden by a subsequent section will be blocked by the
 +first global section.
 +Using wildcards allowed protecting both URL paths without duplicating all the
 +password statements.
 +
 +<​code>​
 +    # Heavy use of symbolic links in Mailman configuration
 +    <​Directory /​usr/​local/​mailman/​archives/​public>​
 +        Options FollowSymlinks
 +    </​Directory>​
 +
 +    # Block all access to other lists' archives
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig|/​pipermail)">​
 +        Order allow,deny
 +        Deny from all
 +    </​LocationMatch>​
 +
 +    # Define password protection for this list's archives
 +    # For all these lists: discuss steercom jobs test*
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​(discuss|steercom|jobs|test)|/​pipermail/​(discuss|steercom|jobs|test))">​
 +        Allow from all
 +        AuthType Basic
 +        AuthName "SLUUG Discussion Archive Access"​
 +        AuthUserFile /​etc/​apache2/​discuss-passwords
 +        Require valid-user
 +    </​LocationMatch>​
 +
 +    # Define password protection for this list's archives
 +    # For sysadmin list only
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​sysadmin|/​pipermail/​sysadmin)">​
 +        Allow from all
 +        AuthType Basic
 +        AuthName "SLUUG Sysadmin Archive Access"​
 +        AuthUserFile /​etc/​apache2/​sysadmin-passwords
 +        Require valid-user
 +    </​LocationMatch>​
 +
 +    # No password protection for this list's archives
 +    # For all these lists: announce users
 +    # Alternate path to same files via htdig search results
 +    <​LocationMatch "​^(/​mailman/​htdig/​(announce|users)|/​pipermail/​(announce|users))">​
 +        Allow from all
 +    </​LocationMatch>​
 +</​code>​
 +
 +All common parts of the port 80 and port 443 ''<​VirtualHost>''​ definitions
 +were moved to a common file to eliminate complete duplication.
 +
 +Make similar changes to the stlwebdev web site definition.
 ====Create the  password file==== ====Create the  password file====
 The name of the file matches the AuthUserFile configuration statement. ​ The username for the htpasswd command is whatever is used for that password file.  You will be prompted for the password by the htpasswd command. The name of the file matches the AuthUserFile configuration statement. ​ The username for the htpasswd command is whatever is used for that password file.  You will be prompted for the password by the htpasswd command.
Line 461: Line 524:
  
   * We should probably SSL-require the administrative pages for mailman.   * We should probably SSL-require the administrative pages for mailman.
- +  ​* We need to test mailman with some of the majordomo archives from michelob.
-  ​* We need to test mailman with some of the majordomo archives from +
-michelob. +
   * We need to set up mailman to handle lists for all of sluug.org   * We need to set up mailman to handle lists for all of sluug.org
 +  * /​usr/​local/​mailman/​bin/​add_members should be run on a list of current subscribers to ANNOUNCE, DISCUSS, SYSADMIN, and STEERCOM. I'm not quite sure how passwords, if any, will be handled.
 +  * We need to copy the archives from michelob to bud and index them via htdig. I'm not sure how htdig and/or apache will handle the compressed files as currently configured.
  
-  * /​usr/​local/​mailman/​bin/​add_members should be run on a list of current +=== Spam Subscriptions ===
-subscribers to ANNOUNCE, DISCUSS, SYSADMIN, and STEERCOM. I'm not quite +
-sure how passwords, if any, will be handled.+
  
-  * We need to copy the archives from michelob ​to bud and index them via +In 2018, it was discovered that one or more criminals were using a 
-htdigI'm not sure how htdig and/or apache will handle ​the compressed files as currently configured.+bot network ​to make repeated subscription requests to multiple lists, 
 +with the intent of SLUUG sending thousands of subscription confirmation 
 +e-mails ​to one address, that would eventually change to the next target. 
 +Looking in the logs, this had been happening for years, with SLUUG sending 
 +over 80k spam confirmations. 
 +This also taxed the SLUUG mail server when hundreds of e-mails were 
 +rejected ​and sitting in the mail queue for retry.
  
 +In response, a local mod was made to ''​subscribe.py''​
 +<code python>
 +sluug_sub_mod1_value = cgidata.getvalue('​sluug_sub_mod1',​ ''​)
 +    if not sluug_sub_mod1_value:​
 +        syslog('​mischief',​ '​Subscribe w/o local mod form field as: %s: %s', email, remote)
 +        results.append(_('​Subscription failed due to internal error!'​))
 +    elif sluug_sub_mod1_value != '​sluug_sub_mod1-20180517':​
 +        syslog('​mischief',​ '​Subscribe w/ wrong local mod form field value %s as: %s: %s', sluug_sub_mod1_value,​ email, remote)
 +        results.append(_('​Subscription failed due to internal error!'​))
 +</​code>​
  
-GENERAL MAIL TODO:+Also modified the generic ''​listinfo.html''​ template and the customized 
 +version for the announce list (no other customized versions needed changes) 
 +to add: 
 +<code html> 
 +<input type="​hidden"​ name="​sluug_sub_mod1"​ value="​sluug_sub_mod1-20180517">​ 
 +</​code>​
  
-  * We need to get virtual users set up in some way. Craig and I discussed +The permanent fix to stop all spam subscriptions is to upgrade ​to a current 
-this, and two options are postfix maps and mysql database. We didn't +release of mailman that includes ​''​SUBSCRIBE_FORM_SECRET''​ and probably other 
-make any decisions. Two particular challenges will be copying existing  +new features ​to combat the bots.
-passwords for POP3/IMAP access and mail filtering (procmail/​maildrop/​whatever). Once we get users set up, we'll need to migrate their mail spools.+
  
-  * We need to get spamassassin working. We particularly need to take a +=== GENERAL MAIL TODO: ===
-look at the performance impact of scanning list mail. This should +
-probably be done incoming list mail only.+
  
-  * We need to get some form of webmail up and running. I'm partial to +  ​* We need to get virtual users set up in some way. Craig and I discussed this, and two options are postfix maps and a mysql database. We didn't make any decisions. Two particular challenges will be copying existing passwords for POP3/IMAP access and mail filtering (procmail/​maildrop/​whatever). Once we get users set up, we'll need to migrate their mail spools. 
-Horde becauses it has a powerful interface and a ton of cool modules. I +  * We need to get spamassassin working. We particularly need to take a look at the performance impact of scanning list mail. This should probably be done incoming list mail only. 
-haven'​t used the password module, but it might be particularly useful +  ​* We need to get some form of webmail up and running. I'm partial to Horde becauses it has a powerful interface and a ton of cool modules. I haven'​t used the password module, but it might be particularly useful for us. See http://​www.horde.org/​accounts/​screenshots/​accounts.png. If we use horde, we'll be using mysql, so that might be the way to go for virtual users.
-for us. See http://​www.horde.org/​accounts/​screenshots/​accounts.png. If +
-we use horde, we'll be using mysql, so that might be the way to go for +
-virtual users.+
  
 That should do it for now - enjoy the rest of the weekend. That should do it for now - enjoy the rest of the weekend.
Line 499: Line 574:
  
 ---- ----
- 
 ====== Notes ====== ====== Notes ======
  
Line 509: Line 583:
   * vette - results of admin actions to deferred posts   * vette - results of admin actions to deferred posts
   * bounce - tracks bounces, so failing members can automatically be purged   * bounce - tracks bounces, so failing members can automatically be purged
 +  * mischief - Detected attempts to use mailman that failed, including subscriptions by bots that were blocked.
 +  * error - Problems.
 +  * smtp - All mail activities?
 +  * smtp-failure - All mail problems?
  
 TODO: We should move these to /var/log and put them under log rotation. TODO: We should move these to /var/log and put them under log rotation.
 +See ''​contrib/​*redhat_fhs.patch''​ for a source modification to change
 +log and data file locations.
  
 Important information about problems might also be in the Apache server logs.  Currently in ''/​var/​log/​apache2/''​. Important information about problems might also be in the Apache server logs.  Currently in ''/​var/​log/​apache2/''​.
- 
 ===== Problems seen ===== ===== Problems seen =====
  
build/lists.1527492344.txt.gz · Last modified: 2018/05/28 02:25 by SLUUG Administration