The Whole Mailman FAQ

Last changed on Wed Sep 15 17:56:04 2004 CEST

(Entries marked with ** were changed within the last 24 hours; entries marked
with * were changed within the last 7 days.)

-------------------------------------------------------------------------------

1. Introduction: What is GNU Mailman?

  * 1.1. What is GNU Mailman?
  * 1.2. How do I spell Mailman?
  * 1.3. Is GNU Mailman free?
  * 1.4. Failure to exec script. WANTED gid 0, GOT gid 12 (group mismatch
    error)
  * 1.5. How do I turn off passwords completely?
  * 1.6. How do I run Mailman on Win2k (Win98, etc)
  * 1.7. I've set up mailman, created a list, and added myself to the list, but
    I don't get any messages!
  * 1.8. How do I turn off HTML messages/attachments?
  * 1.9. What does "message has implicit destination" mean?
  * 1.10. There is no message threading in my web-archives
  * 1.11. How do I make the archives searchable
  * 1.12. How do I automatically delete held messages?
  * 1.13. What is an MTA? What is an MUA? What is an MDA or LDA?
  * 1.14. Can I run Mailman under thttpd?
  * 1.15. What is the largest list Mailman can run? *
  * 1.16. Moved -- see FAQ entry 4.39
  * 1.17. Do you offer Mailman hosting?
  * 1.18. How do I search the archives of the mailman-users mailing list?
  * 1.19. How do I use Mailman with Exim?
  * 1.20. How do I use Mailman with Redhat Linux?
  * 1.21. How do I use Mailman with MacOS X?
  * 1.22. I have a basic question -- What kind of information do I need to
    provide when posting a question to this mailing list?
  * 1.23. I have a specific/detailed question -- What kind of information do I
    need to provide when posting a question to this mailing list?
  * 1.24. What is the minimum recommended configuration for running Mailman? *

-------------------------------------------------------------------------------

2. Help for mailing list members

  * 2.1. How can I post from 2 or more addresses to a "members-only" list?
  * 2.2. So what is this VERP stuff *
  * 2.3. From field displayed by MS Outlook
  * 2.4. How do I send email to people on my list?

-------------------------------------------------------------------------------

3. List administrator tasks

  * 3.1. How do I change a member's password?
  * 3.2. How can I track subscriber's real names?
  * 3.3. How can I remove a post from the list archive / remove an entire
    archive?
  * 3.4. How do I move a list to a different server/Mailman installation.
  * 3.5. What is an Umbrella list - and why doesn't it do what I want?
  * 3.6. What can I do about users with broken autoresponders?
  * 3.7. Setting up Web access using MM list passwords - for Apache
  * 3.8. I forgot my list password! How do I get it back?
  * 3.9. How do I edit a held message before approving it for the list?
  * 3.10. How do I enforce a text/plain posts only policy?
  * 3.11. How do I create a newsletter/announcement/one-way list?
  * 3.12. How can I remove duplicate Re: markers from the Subject: field of
    messages?
  * 3.13. How do I remove a user name or email address with an illegal
    character in it?
  * 3.14. Troubleshooting: No mail going out to lists members
  * 3.15. How do I enable personalization (messages tailored to the recipient)?
  * 3.16. How do I assign a single password to all subscribers in a list?
  * 3.17. Moved -- see FAQ 4.40
  * 3.18. How do I hide the fact that I'm using a mailing list management
    system?
  * 3.19. Deleted
  * 3.20. How to I get the list administrator to look at all email with
    attachments, html etc.
  * 3.21. How do I unsubscribe a member without them receiving an unsubscribe
    notice?
  * 3.22. Deleted
  * 3.23. Deleted -- see FAQ 3.3
  * 3.24. Tracking down elusive subscriber bounces
  * 3.25. How do I remove subscribers from a list and put them on a different
    list?
  * 3.26. Is there an easy way to discard all messages waiting to be reviewed?
  * 3.27. Who receives mail to -owner and -admin?
  * 3.28. What are all the "special" email addresses used for a list
  * 3.29. Deleted
  * 3.30. Deleted
  * 3.31. What if I don't want to save any messages for the moderator?
  * 3.32. What's the regex syntax for header filter rules (in privacy options/
    spam filters)?
  * 3.33. How do I accept all addresses from a particular domain?
  * 3.34. How do I spoof-proof my one-way (announcements or newsletter) list?
  * 3.35.
  * 3.36. Processing old mbox archives with procmail/formail
  * 3.37. I want to do extensive customization of outgoing messages -- can
    Mailman do this?
  * 3.38. Why am I receiving moderation requests that read ...mailing list has
    -1 request(s) waiting... ?
  * 3.39. How do I protect my mailing lists against viruses?
  * 3.40. I want to completely customize the look and feel of Mailman -- how do
    I do that?
  * 3.41. What variables can I use in the headers and footers added to messages
    by Mailman?
  * 3.42. Help!!! All messages from my list are being rejected by AOL as spam,
    or being silently thrown away!
  * 3.43. DELETED
  * 3.44. How can I Mass Subscribe a list with real names *

-------------------------------------------------------------------------------

4. Site administrator tasks

  * 4.1. How to get rid of the List-* headers?
  * 4.2. Which MTA should I use with Mailman?
  * 4.3. Mailman's internal archiver (Pipermail) doesn't understand MIME! What
    can I do?
  * 4.4. How can I use an external archiver with Mailman like MHonArc?
  * 4.5. Why don't the footers show up on some list messages? (Mailman 2.0.x)
  * 4.6. How can I backup my Mailman lists/membership/etc on a regular basis?
  * 4.7. How can I rotate my Mailman logs?
  * 4.8. How can I add Namazu as a search engine for my Mailman list archives?
  * 4.9. Summary of the mailman/bin/ commands
  * 4.10. How to use Mailman on a linux/grsec kernel
  * 4.11. What about performance?
  * 4.12. What about VERP?
  * 4.13. How do I prevent MIME attachments/HTML/Viruses being sent to lists
  * 4.14. How do I upgrade from Mailman 2.0.x to Mailman 2.0.y
  * 4.15. How do I filter incoming mail before it hits mailman (e.g., using
    procmail)
  * 4.16. How do I prevent duplicate copies of email coming to the list
  * 4.17. Why are lists missing from the listinfo page?
  * 4.18. How do I backup my lists and their configurations/membership rosters?
  * 4.19. I get "Could not acquire qrunner lock" in ~mailman/logs/qrunner...
  * 4.20. Who should deal with transient DNS errors?
  * 4.21. Why make changes in mm_cfg.py NOT Defaults.py ?
  * 4.22. Messages that are translated from HTML to plain text have a body like
    ""/root/8e8Ta4: Permission denied"
  * 4.23. How do I use SpamAssassin with Mailman?
  * 4.24. How do I create unadvertised lists?
  * 4.25. What is the purpose of the site-wide mailing list?
  * 4.26. How do I remove a list that has a space in the list name?
  * 4.27. Securing Mailman's web GUI by using Secure HTTP/SSL
  * 4.28. How do I fix HTML->text conversions which just give a "Permission
    denied" message?
  * 4.29. Where can I change a list or the default URL used for the web
    interface?
  * 4.30. How do I configure the admin webpage to show more members per page?
  * 4.31. How do I remove a list via the web
  * 4.32. How Do I Upgrade version 2.0.x to version 2.1.x?
  * 4.33. How do I put a subscribe form for my list on a web page?
  * 4.34. I don't want the "download full raw archive" link which provides
    non-antispammed email addresses
  * 4.35. What do I do with a shunt (qfiles/shunt) directory full of files?
  * 4.36. Creating a new list from Web CGI result in "Error: Unknown virtual
    host"
  * 4.37. How do I stop the archiver from "scrubbing" HTML messages sent to the
    list?
  * 4.38. How to change some configuration for ALL lists?
  * 4.39. HELP! Mailman is munging HTML & MIME-formatted messages before they
    are sent out? (Mailman 2.1.x)
  * 4.40. HowTo Patch Mailman Source Code - a basic overview
  * 4.41. How do I deal with qrunner processes that suck up all available CPU?
  * 4.42. I installed/upgraded Mailman, but now I get the error "NameError:
    global name 'True' is not defined". What can I do?
  * 4.43. How can I add auto-numbering to the subject line of mailing list
    messages?
  * 4.44. I've installed/upgraded Mailman, but now I get the error "KeyError:
    'getgrnam(): name not found'" What can I do?
  * 4.45. The admin web interface is not saving my changes -- what is wrong?
  * 4.46. file extension in pipermail archive is obj or bin, not xls or doc.
    why?
  * 4.47. Virtual domain hosting with Mailman?
  * 4.48. How can I change the HTML templates used by my mailing lists?
  * 4.49. How do I enable automatic alias generation with sendmail?
  * 4.50. I just did a portupgrade on FreeBSD, but now I get the error
    "ImportError: No module named os" -- what can I do?
  * 4.51. How do I limit the rate at which Mailman sends mail?
  * 4.52. How do I change the subject line format for subscription
    confirmations?
  * 4.53. Why has my change to mm_cfg.py been ignored?
  * 4.54. How do I get qrunner to run under daemontools?

-------------------------------------------------------------------------------

5. Downloading and installing Mailman

  * 5.1. How do I import an archive into a new mailing list?
  * 5.2. Will Mailman work with Windows 2000?
  * 5.3. localhost.localdomain is used inside admin email URLs
  * 5.4. Configuring cron to run with the correct privileges, troubleshoot cron
    error.
  * 5.5. "Site list is missing: mailman" error after upgrade
  * 5.6. Cannot Find/Receive Bounce Messages
  * 5.7. Where are my icons?
  * 5.8. What version of Python is required for Mailman 2.1.5?
  * 5.9. Adding a second version of Python to a server
  * 5.10. Deleted
  * 5.11. DELETED
  * 5.12. What MTA features are required for use with Mailman 2.1.5?

-------------------------------------------------------------------------------

6. Integration issues (with mail or web servers)

  * 6.1. Failure to exec script. WANTED gid 31, GOT gid -1. (Reconfigure to
    take -1?)
  * 6.2. MTA Performance Tuning Tips for EXIM
  * 6.3. MTA Performance Tuning Tips for Sendmail
  * 6.4. MTA Performance Tuning Tips for Postfix
  * 6.5. MTA Performance Tuning Tips for Qmail
  * 6.6. Mailman Performance Tuning for Mail Delivery
  * 6.7. Initial Mailman v2.0 with TMDA and MIME filtering HOW-TO
  * 6.8. Improving performance by local DNS caching
  * 6.9. I get a "RuntimeError: command failed: /usr/sbin/postalias" when
    creating lists and Postfix is my MTA
  * 6.10. Mail bouncing when tried to send mail to
    "mailman-owner@some.domain.com"
  * 6.11. Mailman and CPanel
  * 6.12. Mailman + postfix + amavisd-new HOWTO *
  * 6.13. Mailman and Allegroserve

-------------------------------------------------------------------------------

1. Introduction: What is GNU Mailman?

-------------------------------------------------------------------------------

1.1. What is GNU Mailman?

GNU Mailman is mailing list management software. It allows you to create and
manage electronic mail mailing lists. It provides a web front-end for easy
administration, both for list owners and list members. It supports digests (RFC
934 and RFC 1153), archiving, spam protection, bounce detection, Usenet
gateways, and many more features. Version 2.1 supports multilingual mailing
lists.

Edit this entry / Log info / Last changed on Thu Nov 8 18:48:34 2001 by Barry
Warsaw

-------------------------------------------------------------------------------

1.2. How do I spell Mailman?

Mailman is spelled with a capital initial "M" and a lowercase second "m". It is
incorrect to spell it "MailMan". To be completely pedantic, it should be
spelled "GNU Mailman".

Edit this entry / Log info / Last changed on Thu Nov 8 18:50:30 2001 by Barry
Warsaw

-------------------------------------------------------------------------------

1.3. Is GNU Mailman free?

Yes. It is free in two senses: it costs nothing to download, install, or use.
And it is Free Software, meaning you can change it and redistribute the
changes, as long as you adhere to its license.

Mailman is part of the GNU Project, and is covered by the GNU General Public
License. For details see:

The Free Software Foundation: http://www.fsf.org

The GNU Project: http://www.gnu.org/gnu/the-gnu-project.html

The GNU General Public License: http://www.gnu.org/licenses/gpl.html

The Mailman pages at gnu.org: http://www.gnu.org/directory/mailman.html

Edit this entry / Log info / Last changed on Thu Nov 8 20:13:02 2001 by Barry
Warsaw

-------------------------------------------------------------------------------

1.4. Failure to exec script. WANTED gid 0, GOT gid 12 (group mismatch error)

NOTE: This is the most common and frequest question asked on the Mailman
support lists.

See also the Mailman FAQ entry at <http://www.python.org/cgi-bin/faqw-mm.py?req
=show&file=faq06.001.htp>.

Have you ever gotten the following error message?

The original message was received at Tue, 6 Nov 2001 16:27:56 -0500 from IDENT:
smmsp@localhost [127.0.0.1]

   ----- The following addresses had permanent fatal errors -----

"|/home/mailman/mail/mailman_wrapper post test"

    (reason: 2)
    (expanded from: <test@ns.abs-comptech.com>)

   ----- Transcript of session follows -----

Message delivered to mailing list <test@ns.abs-comptech.com> Failure to exec
script. WANTED gid 0, GOT gid 12. (Reconfigure to take 12?) 554 5.3.0 unknown
mailer error 2

Normally you will need to reconfigure the software. You can do this by issuing
the following commands in the Source Code directory.

  # make clean

  # echo "NOTE: The following values for the GID's are example
          values.  The ones needed my your system will likely be
          different.  Check your MTA, web server configuration, and
          logged error messages for details and the correct values.

  # configure --with-cgi-gid=233 --with-mail-gid=12
  # make install

Sometimes this still does not solve the problem. In these cases, you need to
check the following for the source of the problem:

For those people using Sendmail as their MTA under Mailman :

1. Does the smrsh directory contain a LINK to the actual Binary. Placing a
Binary in the directory will have sendmail use the binary file, not the file
which you are compiling.

2. If you renamed the file (due to a confilict with Majordomo), are you using a
Link to the MailMan binary wrapper file?

    [root@ns mail]# ls -l
    total 32
    lrwxrwxrwx    1 root     root           26 Nov  7 09:37 mailman_wrapper -> /home/mailman/mail/wrapper*
    -rwxr-sr-x    1 root     mailman     31000 Nov  7 09:39 wrapper*
    [root@ns mail]#

Note that in Mailman 2.1.x the wrapper program is called 'mailman' while in
2.0.x it was called 'wrapper'.

eg - on a redhat system

 [root@server smrsh]# ls -al
 lrwxrwxrwx    1 root     root           31 Jul 15 15:07 mailman -> /usr/local/mailman/mail/mailman

For those unfamiliar with unix - the command to make a link is

 ln -s <filename> in the directory you want to make a link from.

on the redhat system above, this will look like this.

 ln -s /usr/local/mailman/mail/mailman

If you installed from RPMs or some other form of binary package, you either
need to get different packages which are configured correctly, or you need to
install the correctly configured code from source.

Go to your source for your binary packages to see if they can help you with
this problem.

Edit this entry / Log info / Last changed on Wed Jul 28 00:12:48 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.5. How do I turn off passwords completely?

This is an upcoming feature and may be available in a current alpha or beta
release.

Try http://satyap.csoft.net/satyap/download.html#mailmanw in the meantime.

Edit this entry / Log info / Last changed on Thu Feb 19 21:57:24 2004 by Robert
Hirn

-------------------------------------------------------------------------------

1.6. How do I run Mailman on Win2k (Win98, etc)

Mailman is designed for Unix systems. It may run on Windows systems after a lot
of tweaking, but everyone recommends against it.

If you must, you will need Python and a mail transfer agent at the least.

Edit this entry / Log info / Last changed on Thu Nov 8 21:23:59 2001 by Satya

-------------------------------------------------------------------------------

1.7. I've set up mailman, created a list, and added myself to the list, but I
don't get any messages!

Start out by checking the Mailman and MTA logs. (MTA means Message Transfer
Agent e.g., Sendmail.) They are your best guide and will, every time, contain
the necessary clues as to both the exact definition of the problem and its
solution.

The most frequent reasons for this are:

a) You didn't setup the cronjobs needed by Mailman

-- Check the crontab for the mailman user (or whatever user your Mailman
installation runs as) and make sure the Mailman entries are there. If necessary
restart cron to pick up the changes.

b) You haven't properly setup the aliases for your list.

-- The newlist command prints out the necessary aliases for the list. You will
need to copy/add them to the alias file you are using for your list setup
(often ~mailman/data/aliases).

c) You haven't run `newaliases` to "compile" the new aliases into a hash table
or DBM. (Sendmail and Postfix)

-- Run `newaliases` as root. For postfix you'll also need to do a `postfix
reload` for Postfix to pick up the changes.

d) You haven't configured your MTA to find the alias file you are using for
your lists.

-- See the documentation for your MTA.

e) You haven't run the Mailman qrunner daemon (~mailman/scripts/mailman or
~mailman/bin/mailmanctl). (You have to do this regardless of your MTA.)

-- This is explained in the installation documentation. Remember to configure
your system to run the daemon when it boots!

f) The message you sent is being held for moderation by the list.

-- Check into the administrative interface for your list and then check the
moderation queue.

g) Make sure that you have set these variables:

DEFAULT_EMAIL_HOST = 'your.host.name'

DEFAULT_URL_HOST = 'your.url.addess'

as well as added:

add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)

in mm_cfg.py (default /usr/local/mailman/Mailman/mm_cfg.py). If these have not
been set Mailman may be sending pending requests from %list%-owner@127.0.0.1
which could be bounced.

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.014.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.007.htp>.

Edit this entry / Log info / Last changed on Tue Jun 15 12:11:17 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.8. How do I turn off HTML messages/attachments?

See also the Mailman FAQ entries at <http://www.python.org/cgi-bin/faqw-mm.py?
req=show&file=faq04.013.htp> and <http://www.python.org/cgi-bin/faqw-mm.py?req=
show&file=faq04.039.htp>.

In Mailman 2.1, on the admin page, go to the Content Filtering section. Change
"Should Mailman filter ..." to Yes. If you leave the rest at default values,
including Yes for "Should Mailman convert text/html parts to plain text?", your
list will only distribute plain text messages with no attachments.

Mailman 2.0.* has no MIME supports. You will need to use an external MIME
stripping tool for Mailman 2.0.*.

  demime: http://scifi.squawk.com/demime.html
  mimefilter: http://ftp.br.debian.org/debian/pool/main/m/mimefilter/
  stripmime: http://www.phred.org/~alex/stripmime.html
  stripmime: http://www.clarity.net/~adam/stripmime/

Notes: There are two different tools called "stripmime".

MimeFilter is written (and packaged for Debian Linux by Davide G. M. Salvetti,
who also packages and maintains the Debian Mailman package.

Typically such tools are used by either inserting them into the aliases used
for the Mailman lists, or by having the Mailman aliases deliver to procmail
which then runs the MIME stripper via a procmail recipe and then delivers the
message to Mailman.

There is also a patch for mailman 2.0.8 at http://sourceforge.net/tracker/
index.php?func=detail&aid=413752&group_id=103&atid=300103 from Geoffrey
Dairiki.

Mailman 2.1.* adds some support for manipulating MIME srtuctures.

Edit this entry / Log info / Last changed on Sun Jul 11 16:14:05 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.9. What does "message has implicit destination" mean?

It means that the address for the list was not found in the To: or CC: headers.

Typically this is due to one of a couple reasons:

1) Your list has a different domain name (FQDN) than you think it does, or than
was used as the target address of the message. See the second to last entry on
the main list configuration page to ensure that you have this set properly.

2) The message was BCC'ed (blind carbon copied) to the list, and was actually
not addressed to the list directly (not that you can see).

If you're running an umbrella list, you may want this message to be accepted by
the sub-list. To do that, in the sub-list's admin screen, go to Privacy Options
/ Recipient Filters / Alias Names, and enter the name of the umbrella list as a
valid alias.

The reason this happens is that mail being distributed by a mail list retains
the name of the mail list in the To: header and the name of the original sender
in the From: header. So, when the mail gets to the sub-list, it is still marked
as being To the umbrella list. By entering the name of the umbrella list as a
valid alias, you tell your sub-list that it's OK to accept mail which was
originally sent to the umbrella list.

Alternatively, if you want to accept all BCCs to the list and get rid of this
message entirely, go to Privacy Options / Recipient Filters and set the
require_explicit_destination option to be "No". This should be the first option
on the page, and by default, Mailman will set this option to be "Yes".

Edit this entry / Log info / Last changed on Thu Aug 12 02:33:37 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.10. There is no message threading in my web-archives

Threaded mail readers like Pipermail don't assemble threads based on the
'Subject:' of the discussion - They use two special message headers designed
for this purpose: 'In-Reply-To:' and 'References:'.

If your archives are flat, then this may be because your Mail User Agent (MUA)
isn't creating these headers when you reply to a message (This isn't a problem
with modern MUAs), or an over-enthusiastic mailserver somewhere in the chain
between your MUA and mailman is stripping them out Lotus Notes, CC:Mail, and
(some) Microsoft Exchange servers are notorious for doing this).

Edit this entry / Log info / Last changed on Tue Nov 27 18:41:22 2001 by J C
Lawrence

-------------------------------------------------------------------------------

1.11. How do I make the archives searchable

The archives (unless made explicitly private) are indexable and searchable by
standard web search tools (for example you could encourage google to index your
archive pages and then do searches with google restricting results to your
domain).

  eg Search google for, "chever technique site:your.domain.dom"

Other commonly used and recommended external search tools enclude:

  MnoGoSearch: http://www.mnogosearch.org/
  HT:Dig:  http://www.htdig.org/

As an alternative, there are some patches for pipermail (the Mailman internal
archiver) to support direct indexing and searching of the archive pages. This
also adds a search box to the list archive index pages.

The patches to handle this are on the Mailman sourceforge patches/bugs pages,
with ids:-

  [ #728836 ] corrects defects in some HTML templates (only for MM 2.1.2)
  [ #661138 ] corrects defects in some HTML templates (only for MM 2.1 and 2.1.1)
  [ #668685 ] corrects error in sys.exit handling by script/driver (only for MM 2.1 and 2.1.1)
  [ #444879 ] Archive indexer control to improve indexing
  [ #444884 ] Integration of Mailman & htdig for archive search

The sourceforge URLs for the patches are given below but if you have any
difficulty retrieving them from sourceforge then they are also posted on the
following site:

  http://www.openinfo.co.uk/mailman/index.html

  http://sourceforge.net/tracker/index.php?func=detail&aid=728836&group_id=103&atid=100103
  http://sourceforge.net/tracker/index.php?func=detail&aid=661138&group_id=103&atid=300103
  http://sourceforge.net/tracker/index.php?func=detail&aid=444879&group_id=103&atid=300103
  http://sourceforge.net/tracker/index.php?func=detail&aid=668685&group_id=103&atid=300103
  http://sourceforge.net/tracker/index.php?func=detail&aid=444884&group_id=103&atid=300103

Instructions for applying these patches (you need the copy of the Mailman
source tree you installed from) are given on the referenced pages.

Edit this entry / Log info / Last changed on Fri Oct 10 18:39:33 2003 by 
Richard Barrett

-------------------------------------------------------------------------------

1.12. How do I automatically delete held messages?

Mailman does not delete unwanted messages, instead it holds the message and
sends a notice to the administrators of the list that a message is waiting. One
of the admins must login via the web interface and deny or discard any unwanted
messages.

There is an alternative solution added at the bottom (by tkikuchi 08/13/2004).

There is a work-around that can be implemented on a per list basis.

On a list where you want all held messages deleted (these include attempts to
subscribe, when subscribes must be approved by an admin), you can do the
following:

1) Delete all current requests for the list using the web Admin. This will make
the file request.db empty for that list.

2) Change directory to your lists configuration directory

      cd /home/mailman/lists/<list-name>/

3) Copy the file request.db to orig.request.db

      cp request.db orig.request.db

4) Login as the mailman user

      su mailman

5) Create a bash script to delete the held messages for the list and then
replace the database file request.db with the blank one we copied in step 3:

  #!/bin/bash
  # This script removes held messages that have accumulated for a list
  # Clear out the heldmessages in ~mailman/data  then
  # Replace request.db in ~mailman/lists/<listname>/ with the original
  #
  # Example using a list named "hr"
  # rm ~/mailman/data/heldmsg-hr-* 2> /dev/null
  # cp ~mailman/lists/hr/orig.request.db ~mailman/lists/hr/request.db

  rm ~mailman/data/heldmsg-<list-name>-* 2> /dev/null
  cp ~mailman/lists/<list-name>/orig.request.db ~mailman/lists/<list-name>/request.db

  # repeat for all lists where you want held messages auto-deleted

6) Make the script executable

      chmod a+x  /home/mailman/bin/rm_held_msgs

7) Edit crontab to have the script run hourly

      crontab -e

  # Run auto-delete script to remove unwanted messages from my lists
  59 * * * *      /home/mailman/bin/rm_held_msgs

Done.

===

The Details:

Held messages are stored in ~mailman/data. You can easily read, edit, or delete
them. The messages are stored in files with the name:

      heldmsg- <list-name> - <number> .txt.

As an example held messages for hr would have names like this:

      heldmsg-hr-97.txt
      heldmsg-hr-98.txt
      heldmsg-hr-99.txt
      heldmsg-hr-100.txt

These files contain individual emails that were sent to your lists. Mailman
stores these files here (as a sort of queue) and sends out either an immediate
message or a daily message alerting the admin of the waiting messages. Mailman
will also load some information from these held messages into a database for
your list. The name of this database is request.db.

The amount of information Mailman loads into request.db for every message is
determined by various settings in either ~mailman/Mailman/mm_cfg.py or
Defaults.py

When you browse via the web-admin and look at the waiting requests, you are
accessing the request.db database and not the actual files in ~mailman/data/..
For our purposes, we just want to blank out request.db.

We cannot just delete it. But it is safe to replace request.db with an empty
version.

I have my lists set to NOT alert me so that I only get the daily alert from
Mailman (in the web-admin under General Options):

   "Should administrator get immediate notice of new requests,
   as well as daily notices about collected ones"            = no

The Mailman alert script then runs at exactly 5:00pm every day, so I could set
the script to kick off once a day at 4:59pm and I would never get bugged about
held messages (for lists in the script). Instead I like to run the script
hourly at 59 minutes past the hour. The effect is the same, I just don't like
letting spam hang out on my server longer than an hour.

The first active line in the script uses a redirect to get rid of warnings -
just in case there are currently no held messages to be deleted. A redirect
with a "2" in front of it specifically catches console warning messages:

      .... 2> /dev/null   # sends warnings into the bit bucket

--------------------------------------------------------------

ALTERNATIVE SOLUTION:

Apply a patch from sourceforge mailman patch area #790494. http://
sourceforge.net/tracker/index.php?func=detail&aid=790494&group_id=103&atid=
300103

You can specify how many days to hold the pending message and automatically
discard when expires.

Edit this entry / Log info / Last changed on Fri Aug 13 09:44:54 2004 by Tokio
Kikuchi

-------------------------------------------------------------------------------

1.13. What is an MTA? What is an MUA? What is an MDA or LDA?

We really need a separate section for more general "Huh?" questions.

An MTA, or Message Transfer Agent (a.k.a., Mail Transfer Agent) is the software
and other systems that are responsible for sending and receiving mail between
systems. That is the ONLY things MTAs do: they send and receive messages
between systems. MTAs use the SMTP (Simple Mail Transfer Protocol) to send and
receive messages.

Another (not very precise) term for this is "mail server". MTAs commonly used
with Mailman include:

  Exim: http://www.exim.org/
  Postfix:  http://www.postfix.org/
  QMail:  http://cr.yp.to/qmail.html
  Sendmail:  http://www.sendmail.com/

An MUA or Mail User Agent is the program that an end user uses to read and
process mail. Typical examples include Pegasus, exmh, mutt, Eudora, TheBat,
pine, elm. Sometimes imprecisely (and confusingly) also called a "mail client".

An LDA is a Local Delivery Agent. An MDA is a Mail Delivery Agent. The two are
essentially synonymous. (Actually there are subtle differences between the two,
but they only concern those involved in the deep arcana of mail systems.) LDAs/
MDAs are the software responsible for receiving a message from an MTA and
arranging for it to be received by the local system (eg delivered to a
mailbox). procmail is commonly used as an LDA on Unix systems. Some IMAP
servers (eg Courier and Cyrus) have their own custom LDAs to deliver mail into
their storage systems.

An MTA may be configured to use different LDAs for different addresses. This is
quite common.

Edit this entry / Log info / Last changed on Thu Jul 31 13:16:46 2003 by Brad
Knowles

-------------------------------------------------------------------------------

1.14. Can I run Mailman under thttpd?

Yes.

  --<cut>--
  Subject: [Mailman-Users] thttpd
  From: Justin Sheehy <justin@iago.org>
  Date: Sat, 01 Dec 2001 13:20:42 -0500 (10:20 PST)
  To: barry@digicool.com (Barry A. Warsaw)
  Cc: mailman-users@python.org

  Several months ago I sent mail to mailman-users when I was trying to
  figure out how to get Mailman to run under thttpd.

  I figured it out fairly quickly, but forgot to post the answer back
  here.  I'm doing so now, for the benefit of others who might want to
  run Mailman on thttpd.

  I copied the cgi-bin directory from Mailman's install directory into a
  "mailman" subdirectory of the thttpd data directory.  This could be
  done with symlinks, but then you'd have to turn off some of thttpd's
  useful security checks.

  Then, I added this entry to the thttpd configuration file:

  cgipat=/mailman/**

  The double-asterisk is important.  Without it, thttpd will not process
  CGI requests of the sort that Mailman expects.

  That's all.  Mailman has been running flawlessly for me under thttpd
  for over four months now.

  -Justin
  --<cut>--

Edit this entry / Log info / Last changed on Sat Dec 1 20:18:31 2001 by J C
Lawrence

-------------------------------------------------------------------------------

1.15. What is the largest list Mailman can run? *

There are no built-in limits to the size of a Mailman list. The largest list
reported to date had around 147,000 members. (If you have run a list larger
than that with Mailman, please email <gward at python dot net> and I'll update
this FAQ entry.)

However, there's more to worry about than the size of your largest list; if
you're concerned with system load and bandwidth consumption, then you have to
consider the total population of all your lists, the activity level of each
one, and the total aggregate activity. For example, 10 lists with 10,000
members each is pretty much equivalent to a single 100,000 member list --
assuming the level of traffic is the same. And a 10,000-member list that gets a
single announcement per week is much less load than 10 active 1,000-member
discussion lists.

Some case studies:

As of December 2001, ActiveState runs around 50 external mailing lists with a
total subscriber population of about 93,500. Their largest "announce" list has
just under 50,000 subscribers; they have a handful of discussion lists with
around 2,000 subscribers each.

A system administrator with The Guardian (the British newspaper) described
running an announce-only list with 147,000 subscribers (only used a few times a
year). The same organization also has a daily distribution list with 60,000
members, and for a time had an unmoderated discussion list with 20,000 members.
(No technical problem, the lawyers put an end to it.)

As of July 2004, FreeBSD.org has over 100 lists, two announce-only that have
about 10,000 to 15,000 subscribers, and two discussion lists with around 5000
subscribers (which can be pretty busy with dozens to almost a hundred messages
per day), and several more busy discussion lists with around 3000 subscribers.

As of September 2004, lists.apple.com has around 115 lists, gets 600-650
messages posted per day, resulting in around 14 million messages being
delivered per month.

SourceForge may be the largest Mailman-hosted mailing list site currently in
operation, but we are still working on obtaining details of their operation.

Conclusion:

If you're going to be running lists with more than a few hundred members, you
should take a look at the entries on tuning your MTA and Mailman to work best
together. They're in Section 6 of this FAQ.

If you're going to be running lists with more than a few thousand members, then
you need to have a thorough understanding of how your MTA and Mailman work, and
a deep and intimate familiarity with Internet e-mail in general. And of course,
you will have to carefully consider how best to tune your MTA and Mailman to
work best together. If you're missing any of these prerequisites, you're in for
a difficult time.

Edit this entry / Log info / Last changed on Sat Sep 11 02:36:08 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.16. Moved -- see FAQ entry 4.39

Moved. See FAQ entry 4.39 at <http://www.python.org/cgi-bin/faqw-mm.py?req=show
&file=faq04.039.htp>.

Edit this entry / Log info / Last changed on Tue May 11 10:24:15 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.17. Do you offer Mailman hosting?

The Mailman developers do not offer hosting, however several companies do
provide Mailman and mailing list hosting services. We generally recommend that
hosting facilities "advertise" on the general Python-Friendly Web Hosting page:

http://www.python.org/cgi-bin/moinmoin/PythonHosting

This page is a community driven Wiki, so you are free to add your site, but
please make it clear that you are providing Mailman hosting, even if you host
other Python-related services.

Edit this entry / Log info / Last changed on Fri Nov 21 15:43:39 2003 by Barry
Warsaw

-------------------------------------------------------------------------------

1.18. How do I search the archives of the mailman-users mailing list?

The mailman-users mailing list is archived at <http://mail.python.org/pipermail
/mailman-users>. However, this site does not provide its own search function
for the archive.

The folks at gmane.org have done a mailing list/USENET news gateway for the
mailman-users list (see <http://news.gmane.org/gmane.mail.mailman.user>), and
they include a search interface.

There is also a searchable interface at mail-archive.com (see <http://
www.mail-archive.com/mailman-users@python.org/>).

Alternatively, you can use Google and include the tag "inurl:mail.python.org/
pipermail/mailman-users/" along with your other search criteria.

Edit this entry / Log info / Last changed on Tue Jun 8 18:00:39 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.19. How do I use Mailman with Exim?

This information is included in the Mailman "tarball", in the file README.EXIM.
However, additional/alternative information is maintained by Nigel Metherington
(and other contributors to Exim) at exim.org.

For Mailman 2.1.x and Exim 4.x, please see <http://www.exim.org/howto/
mailman21.html>.

If you have an old Mailman 2.0.x/Exim 3.x installation, please see <http://
www.exim.org/howto/mailman20.html>.

Edit this entry / Log info / Last changed on Thu Jun 10 17:35:12 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.20. How do I use Mailman with Redhat Linux?

If you are using Redhat Linux, note that many of the older RPMs they provide
are known to be broken.

If you search the archives of the mailman-users mailing list, you should be
able to easily find messages such as <http://mail.python.org/pipermail/
mailman-users/2003-June/029911.html> and <http://mail.python.org/pipermail/
mailman-users/2003-April/028215.html>, which discuss bugs specific to the RPMs
provided by Redhat.

If you search the archive of Redhat bugs, you should be able to easily find
such pages as <http://bugzilla.redhat.com/bugzilla/long_list.cgi?buglist=88083>
and <http://bugzilla.redhat.com/bugzilla/long_list.cgi?buglist=89695>, both of
which reference <http://rhn.redhat.com/errata/RHBA-2003-179.html>.

You can either get the latest RPMs from Redhat (which may be based on more
recent versions of Mailman), or you can install Mailman from source.

Either way, these problems are NOT bugs in Mailman. These are errors introduced
during the process of building the binary packages.

Edit this entry / Log info / Last changed on Tue Aug 31 23:24:00 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.21. How do I use Mailman with MacOS X?

Mailman includes a file named README.MACOSX in the source code "tarball" which
you can download. Assuming the SourceForge CVSweb interface is working
correctly, you can find a copy of the latest version of this file at <http://
cvs.sourceforge.net/viewcvs.py/mailman/mailman/README.MACOSX?view=markup>.

In short, this file references the message at <http://mail.python.org/pipermail
/mailman-users/2002-October/022944.html>, which includes a lot of links at the
bottom to various other articles and messages which may be relevant.

However, note that this work was done under MacOS X 10.2 "Jaguar", with Mailman
2.0.x and does not appear to be applicable to MacOS X 10.3 "Panther", or
Mailman 2.1.x or later. In particular, Apple changed the MTA they use between
10.2 and 10.3 (they switched from using sendmail to using postfix), and they
also changed the Python version that they ship.

Dan Phillips notes that there are newer articles on the afp548 site which have
been updated for use with Mailman 2.1.x. See <http://www.afp548.com/Articles/
Jaguar/mailman21-new.html> and <http://www.afp548.com/Articles/Jaguar/
mailman21-upgrade.html>. They still reference OS X 10.2, but are not MTA
specific, and AFAICT those, along with Mailman's README.POSTFIX should be
adequate to get it up and running on 10.3 client.

Unofficially, Larry Stone has posted his excellent description of how he
installed Mailman 2.1.4 on his MacOS X 10.3 (client, not Server) installation
at <http://mail.python.org/pipermail/mailman-users/2004-July/037909.html>. They
should be applicable for version 2.1.5 as well.

On MacOS X 10.3 Server, Apple provides their own version of Mailman that is
pre-installed out of the box. This includes a GUI interface to manage the mail
server and Mailman, and there is a chapter in the Mail Server Administration
book as well. Unfortunately, both seem to be very limited when it comes to
managing Mailman. The mailman executables, configure files, etc. are all found
in /usr/share/mailman with the local configuration file at:

 /usr/share/mailman/Mailman/mm_cfg.py

Our thanks to Robert Snyder for finding and providing this information to us.
Unfortunately, this has been yet another case of the community having to step
up to fill an information gap left by the vendor.

In terms of upgrading the version of Mailman provided with MacOS X 10.3 Server,
officially you need to talk to Apple about that. They provided the original
binary package version, and unless you want to rip out everything they've got
and install your own copy from the official Mailman source, there is no other
known upgrade path available.

If you are having problems with permissions problems under MacOS X 10.3 Server,
see the Apple KnowledgeBase article found by Steve Burling, mentioned at <http:
//mail.python.org/pipermail/mailman-users/2004-July/037899.html>. You may also
be interested in the article at <http://docs.info.apple.com/article.html?artnum
=107906>.

Edit this entry / Log info / Last changed on Tue Jul 27 22:57:45 2004 by Brad
Knowles

-------------------------------------------------------------------------------

1.22. I have a basic question -- What kind of information do I need to provide
when posting a question to this mailing list?

The first thing you should do is indicate that you've done your homework. Look
at the Mailman documentation linked from <http://www.list.org/docs.html>.
Search the FAQ via the FAQ Wizard at <http://www.python.org/cgi-bin/faqw-mm.py
>. Search the archives of the mailing list (see <http://www.python.org/cgi-bin/
faqw-mm.py?req=show&file=faq01.018.htp>).

Once you've looked through all the relevant pieces of documentation, FAQ
entries, archive messages, etc... and you still haven't found your answer,
please give us additional information as well as the question itself.

Specifically, we would like to know:

1. What methods did you use to look through the documentation and search the
FAQ, mailing list archives, etc...?

2. If there were things that initially sounded relevant but ended up not being
useful to you, which ones were they?

If you did miss something that is relevant, then having this information will
help us go back and improve the documentation/FAQ/etc... so that the next
person who does the same search will hopefully hit the correct answer.

In addition, we would appreciate it if you could provide URLs and precise
descriptions of the information you found but which was not helpful to you.

See also FAQ entry <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq03.014.htp>.

With this information, we are much more likely to be able to provide you
assistance with your question.

If you have a specific/detailed question, please procede to FAQ entry 1.23 at <
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.023.htp>.

*** Please turn off any spam-blocking software that requires anyone to jump
through hoops to email you. ***

Edit this entry / Log info / Last changed on Fri Aug 20 22:56:33 2004 by Jeff
Barger

-------------------------------------------------------------------------------

1.23. I have a specific/detailed question -- What kind of information do I need
to provide when posting a question to this mailing list?

In addition to the steps listed in the FAQ entry 1.22 at <http://www.python.org
/cgi-bin/faqw-mm.py?req=show&file=faq01.022.htp>, we would like some additional
information. Specifically:

1. Your precise hardware configuration. This means brand (e.g., HP, Dell,
Apple, etc...), CPU type (e.g., Intel x86, Athlon, UltraSPARC-IIi, PowerPC G4,
etc...), number of CPUs, CPU speed, amount of physical RAM, amount of virtual
memory, amount of disk space on the various filesystems, what kind of
filesystems are installed where (e.g., softupdates is enabled on this
filesystem but not that one, ext2fs in async mode is enabled on this filesystem
but IBM JFS is installed on that one, etc...), what kind and speed of disk
drives/drive arrays you have and which filesystems are where, your disk
controller types, and so on.

2. Your precise software configuration. This means operating system and version
(e.g., FreeBSD 4.10.2-REL, Red Hat Linux 9, Digital Unix 4.0a, etc...), web
server software and version (e.g., Apache 2.0.48), mail server software and
version (e.g., sendmail 8.12.1, postfix-2.1.3, etc...), Mailman version (e.g.,
2.1.5), etc.... Please also tell us if any of this software is as installed
with the OS version or if it has been upgraded, and if the programs in question
were installed as binary packages or if they were installed from source. If the
programs were installed as binary packages, please tell us where you got those
packages from (e.g., are they from the vendor, or did you get them a third
party), etc....

3. Tell us where the software is installed, and where the configuration files
are. For the MTA, tell us what anti-spam procedures you are following (e.g.,
amavisd-new+SpamAssassin+Razor+DCC, crm114, DSPAM, SpamBayes, etc...), what
anti-virus processing you are doing (e.g., amavisd-new+ClamAV, Sohpos, NAV,
etc...), any black listing/message filtering you are doing within the MTA
itself, etc....

4. For the mailman configuration, you may need to provide the contents of the
file "mm_cfg.py" below the line that says:

 ##################################################
 # Put YOUR site-specific settings below this line.

However, you don't need to include blank lines or lines that are just comments
-- all we need to see are the parts that were added which actually define
things.

5. Please tell us about your mailing lists -- how many users you have, how many
lists you have, how active the users are, whether the list is open/closed/
moderated, whether or not you have enabled features like +confirm and/or
+approve, digest mode, any MIME stripping that is configured, any anti-spam/
anti-virus filtering that is done, etc....

6. Be prepared to give us detailed information regarding what is in your mail
server logs, what is in your mail queue, etc.... Also be prepared to give
detailed information regarding the configuration of your logging system. If you
dump all your mail on an outgoing mail server via your provider, you're likely
to need their help in determining what they currently have in their queues
which you sent. To do this, you'll need to have a large sample of queue-ids and
message-ids for them to look for, and you'll need to get that information from
your own logs.

See also FAQ entry <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq03.014.htp>.

With this information, we are much more likely to be able to provide you
assistance with your question.

*** Please turn off any spam-blocking software that requires anyone to jump
through hoops to email you. ***

Edit this entry / Log info / Last changed on Fri Aug 20 22:55:29 2004 by Jeff
Barger

-------------------------------------------------------------------------------

1.24. What is the minimum recommended configuration for running Mailman? *

Question:

What Operating Systems are supported by Mailman? Are there any OSes that are
supported better than others? Can it run under a virtual machine configuration
(e.g., VMWare)? What about hardware requirements -- memory, hard disk, network,
etc...?

Answer:

In theory, Mailman should work on any hardware/operating system that fully
supports the current version of Python (e.g., 2.3), and includes compatible web
and mail servers. This does include virtual machine configurations. In
practice, there are a wide variety of hardware/OS combinations that may (or may
not) work well, but which have not been reported back to the maintainers of
Mailman. It would be impossible to list all the possible permutations and
combinations on either side.

More importantly, the requirements are going to depend on a wide variety of
factors such as how many recipients you will have, how they are distributed on
the receiving systems, how many messages they will send per day, how large
those messages are, how much processing must occur for each message (e.g.,
content and spam filtering, etc...), whether or not you are making use of
personalization and/or VERP, what kind of network and I/O latency and capacity
you have, what kind of network and I/O latency bottlenecks there may be between
you and your recipient users, and so on.

For a very small list, a Compaq Armda 4131T laptop with 48MB of RAM, 1GB hard
drive, 133MHz Pentium-1 processor, add-in PCMCIA 10/100-Base T/TX network
interface card, etc... might be completely suitable. For a very large list, you
might have to distribute the load across a wide variety of systems, each
designed to handle one part of the overall load -- and each one a very large/
high-capacity machine in its own right.

However, the choice of OS and hardware is usually more an individual choice of
the person(s) who will be directly responsible for going and physically laying
hands on the machine if something needs to be done at the console.

If you have a small mailing list and want to run your own system as opposed to
using a mailing list hosting service (see <http://www.python.org/cgi-bin/
faqw-mm.py?req=show&file=faq01.017.htp>), you may want to take a look at
providers that would give you a virtual machine account (provided with tools
such as VMWare), and this is likely to be much less expensive than a getting a
real machine at a co-location facility (e.g., $20/month as opposed to $99-250/
month).

As one data point, the python.org Mailman mailing list server runs on Debian
Linux (Mailman is written in Python, and an integral component of the Python
project). As another data point, the extremely large Mailman mailing lists at
freebsd.org are running on FreeBSD (obviously). In addition, Apple has recently
upgraded lists.apple.com to run on an XServe Dual-G4 running MacOS X Server
that is around 95% idle, whereas it was previously served by a Solaris box that
was frequently very overloaded.

Edit this entry / Log info / Last changed on Sun Sep 12 18:18:07 2004 by Brad
Knowles

-------------------------------------------------------------------------------

2. Help for mailing list members

-------------------------------------------------------------------------------

2.1. How can I post from 2 or more addresses to a "members-only" list?

Many lists are restricted to "members-only", meaning if you post from an
address that isn't a member of the list, the message is held for moderator's
approval. Members-only lists are increasingly popular because they reduce the
amount of spam seen by the list.

The problem then is for people who want to post to the list from two or more
addresses, but only want one of those addresses to receive list postings. E.g.
say I sometimes post to the list from barry@dom1.ain and sometimes as 
warsaw@dom2.ain. I'd like both postings to be treated as coming from a list
member, but I don't want duplicates of list messages; I only want 
barry@dom1.ain to get the messages.

The solution for MM2.0 and MM2.1 is to actually subscribe both addresses to the
list, but to disable delivery from all but one of the addresses. So in the
above example, I'd subscribe barry@dom1.ain and warsaw@dom2.ain, but I would
disable delivery to warsaw@dom2.ain.

You can disable delivery to an address by going to the personal options page
for that address and setting "Mail delivery" to "disabled" (for MM2.1 -- the
wording is slightly different in MM2.0).

This is admittedly a hack, and it will be fixed in future versions.

Edit this entry / Log info / Last changed on Wed Jan 2 23:39:39 2002 by Barry
Warsaw

-------------------------------------------------------------------------------

2.2. So what is this VERP stuff *

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.012.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq05.012.htp>.

VERP = "Variable Envelope Return Paths"

The purpose of VERP'ing is primarily to improve bounce detection; to help
identify to which subscriber's e-mail address a piece of returned mail was
sent.

Any e-mail has a return path which remains the same until it is either
delivered or bounced. This is distinct from the From: header and the Sender:
header (although MS Outlook seems to have a penchant for, at times, conflating
the Sender: and From: headers when it displays information about incoming
e-mail).

In Mailman 2.0.x outgoing mail from lists normally has a return path of
<listname>-admin@mailman.domain.com

With Mailman 2.1 outgoing mail from lists normally has a return path of
<listname>-bounces@mailman.domain.com

When a piece of e-mail cannot be delivered, it is e-mailed back to the e-mail
address in the return path of the rejected e-mail.

The change in aliases between MM 2.0.x and MM 2.1 was to separate incoming
bounced e-mail from other administrative inputs received by Mailman.

So, with stock MM 2.1 bounces are sent back to the e-mail address <listname>-
bounces@mailman.domain.com

E-mails delivered to this address on the Mailman server are analysed with the
objective of identifying to whom the outgoing mail was addressed and taking
steps to disable distribution to them if e-mail to them exceeds bounce limits
set by the list administrator.

Problems can arise with bounce detection if an MTA (Mail Transfer Agent)
handling a piece of e-mail sent out by Mailman rewrites the destination address
of that e-mail from that which Mailman originally used. This might happen if,
for instance, someone has moved from one service provider to another and
arranged to have their e-mail forwarded.

Without VERP'ing, an e-mail with a rewritten destination, which then bounces,
is sent back to the Mailman return path OK but it has lost its original
destination. If Mailman only sees the rewritten destination in the returned
mail and it cannot correlate that with any known subscriber of the list, it
cannot handle the bounce appropriately or automatically.

With VERP'ing the return path of each outgoing mail is specific to the
destination e-mail address and encodes that destination in the e-mail's return
path. So, for instance, VERP'ed e-mail to me from a list would have a return
path of:

  <listname>-bounces+r.barrett=openinfo.demon.co.uk@mailman.domain.com.

Note how my e-mail address of r.barrett@openinfo.demon.co.uk is encoded in that
return path.

With a VERP'ed return path the original destination is embedded in the address
to which the bounced e-mail is returned. Thus regardless of re-writing of the
destination of mail going out, if it is returned, Mailman can reliably identify
the subscriber whose mail is being bounced.

So, why not do it with every outgoing e-mail? Well there is a price for
VERP'ing everything outgoing. Instead of being able to pass a single message
for say 500 subscribers to an MTA in the one go, Mailman has to pass a single
message for a single addressee to an MTA, 500 times. This impacts on
performance and the load applied to both the Mailman software and the SMTP
server it passes its mail to and potentially other, downstream, MTAs.

If you turn on VERP'ing for everything and have lengthy subscriber lists, do
not be surprised if your ISP shouts at you.

If you decide to use personalization for a list, then you have committed to
sending out to your subscribers one-at-a-time anyway, so you might as well VERP
such outgoing mail as doing so will not add to the cost of doing the
personalization: see the VERP_PERSONALIZED_DELIVERIES configuration variable in
$prefix/Mailman/Defaults.py (but make your changes in mm_cfg.py)

Likewise, if you send out monthly password reminders to subscribers, Mailman
has to do message personalization and will send out an individual message to
each subscriber anyway, so VERP'ing these message doesn't carry any extra cost:
see the VERP_PASSWORD_REMINDERS configuration variable in $prefix/Mailman/
Defaults.py (but make your changes in mm_cfg.py)

Mailman can also do VERP'ing on a sampling basis, so that over time it should
capture all those hard to find bouncing subscriber addresses: see the
VERP_DELIVERY_INTERVAL configuration variable in $prefix/Mailman/Defaults.py
(but make your changes in mm_cfg.py)

Finally, you could alter the way return paths are changed for VERP'ing by
altering (very carefully) the VERP_FORMAT and VERP_REGEXP variables in $prefix/
Mailman/Defaults.py (but making your changes in mm_cfg.py). But you would need
a very good reason and lots of confidence if you are going this route.

Oh, and do not forget to check that your incoming MTA can handle VERP'ed return
addresses OK.

Note that Mailman 2.1.5 has a new bounce processor. Now when an address's
bounce score reaches the threshold, that address will be sent a probe message.
Only if the probe bounces will the address be disabled. The score is reset to
zero when the probe is sent. Mailman 2.1.5 also supports the variables
VERP_PROBE_FORMAT and VERP_PROBE_REGEXP in Defaults.py.

A/the seminal document on VERP'ing is at http://cr.yp.to/proto/verp.txt

Edit this entry / Log info / Last changed on Sun Sep 12 18:48:10 2004 by Brad
Knowles

-------------------------------------------------------------------------------

2.3. From field displayed by MS Outlook

Some users of the MS Outlook MUA find that mail they receive that has been sent
out by a Mailman list server is displayed on the Outlook GUI with a 'From'
field that is structured like this:

  From listname-admin@mailman.server.com On behalf of fred@poster.domain.com

or this,

  From listname-bounces@mailman.server.com On behalf of fred@poster.domain.com

or even this:

  From listname-bounces+jim=reader.domain.com@mailman.server.com On behalf of fred@poster.domain.com

This is because, for reasons unknown to this author, some versions of MS
Outlook appear to conflate the values of the Sender (or is it the Return-Path)
and From mail headers and render this as the From field on their GUI. In
contrast many other MUAs including version of MS Outlook Express just render
the From field using the value of the From mail header.

This author knows of no way to stop this behaviour by Outlook; if you do then
add it to this FAQ page.

Of the three examples given above, the first, using the -admin list alias is
what comes from a MM 2.0.x installation.

The second, using the -bounces list alias comes from a MM 2.1 installation.

The third is from a MM 2.1 installation that is sending out VERP'ed e-mail (for
more information about VERP see http://www.python.org/cgi-bin/faqw-mm.py?req=
show&file=faq02.002.htp).

It appears that not all version of MS Outlook have this behaviour. If you know
of versions that do or do not, then add the information to this FAQ page.

MS MUA/Version with the behaviour described:

 MS Outlook 2002

MS MUA/Version without the behaviour described:

  MS Outlook Express 6

Note: The change from -admin suffix to -bounces suffix (introduced by MM 2.1)
was to separate the incoming e-mail to Mailman between what was bounced e-mail
being returned and what was other administrative stuff. This was part of the
changes in 2.1, including VERP'ing, aimed at improving bounce handling.

A partial 'solution' to this problem you can try, is a hack at the file $prefix
/Mailman/Handlers/SMTPDirect.py in the MM 2.1.2 released code. If you change
line 342 in the function bulkdeliver() from:

    msg['Sender'] = envsender

to:

    msg['Sender'] = mlist.GetListEmail()

then it will (hopefully) shorten:

  test-bounces@listdomain.tld

to:

  test@listdomain.tld

in what the subscriber's Outlook MUA displays in its 'From' field'.

The risk with this hack is that any response sent to the address in the Sender:
header gets sent to the list not to the list's bounce alias. But an MTA
generating a bounce message should not be using the address in the Sender:
header of the message but the address of the sender from the envelope of the
SMTP transaction.

But no warranty with the hack. Take a copy of SMTPDirect.py before you start.
Be careful with the editing. Remember source code indentation is syntactically
significant with Python. Try some tests, including adding duff mail addresses
to your test list to check what happens when things bounce.

Edit this entry / Log info / Last changed on Tue Jul 29 17:58:34 2003 by 
Richard Barrett

-------------------------------------------------------------------------------

2.4. How do I send email to people on my list?

How do I send email to people on my list?

Messages sent to the list alias are distributed to the list's subscribers, for
example a message sent to mailman-users@python.org gets distributed to the
subscribers to the mailman-users list.

Edit this entry / Log info / Last changed on Fri Jul 25 21:47:00 2003 by 
Richard Barrett

-------------------------------------------------------------------------------

3. List administrator tasks

-------------------------------------------------------------------------------

3.1. How do I change a member's password?

Use the member's personal page to change his/her password. The "Membership
Management" page contains a link to that page. If you don't know the member's
password, use the list's admin password. (It's also possible to use the site
password, of course.)

Don't forget that members can have their password sent to them anytime they
like, by clicking the appropriate button on their personal page. There is no
need for you to change their password if they forget their password.

Edit this entry / Log info / Last changed on Thu Nov 22 14:28:32 2001 by Avalon

-------------------------------------------------------------------------------

3.2. How can I track subscriber's real names?

Mailman 2.0.*: This is not directly possible. Mailman 2.0.* only tracks the
subscription address, password, and subscription options/configuration.

Mailman 2.1.*: Mailman will attempt to track the GECOS field (ie "real name")
of the subscription account.

Note: The GECOS field is an unreliable data entry. It is not safe to expect
that the value of the GECOS field with either remain constant for a given
subscriber, or that it can be used in any reasonable sense as an authentication
or identification mechanism. It is trivially edited, changed, tweaked, and
otherwise rewritten by almost every MUA in existance.

Edit this entry / Log info / Last changed on Tue Nov 27 18:09:40 2001 by J C
Lawrence

-------------------------------------------------------------------------------

3.3. How can I remove a post from the list archive / remove an entire archive?

You need to edit the raw archive "mbox" directly, and then regenerate the
archive. This requires login access on the Mailman host; you'll either need to
be in the "mailman" group, or be the "mailman" user. (You could of course do
this as root, but that's not recommended -- don't do anything as root that you
don't absolutely need to do as root.)

Here's a detailed example. Let's say you want to remove a post from the archive
for the "test" list. First, you need to find out where the archive lives:

  cd ~mailman
  ls -ld archive/private/test*

If that finds nothing try:

  ls -ld archive/public/test*

If that too finds nothing, then something is wrong. Ask for help on the
mailman-users lists.

Let's assume you found the "test" archive in archive/private:

  cd archive/private

Now you need to edit the raw archive:

  xemacs test.mbox/test.mbox

(You can use whatever text editor you like, but be sure that it 1) doesn't try
to wrap lines or change anything implicitly, and 2) can handle large files, if
this is a big list archive.)

Find the message you want to remove, and remove it. Each message starts with a
line that looks like:

  From joe@example.com Wed Sep 26 16:39:08 2001

where, obviously, the sender address and delivery date will vary. You need to
delete from the "From " line that starts your target message to the next "From
" line, ie. the start of the next message. You MUST have a blank line between
the end of the previous message and the start of the next message. For example,
after deleting message N, you should have something like this:

   [...last line of text from message N-1...]
   [blank line]
   From jane@example.net Wed Sep 26 17:53:02 2001
   [...headers from message N+1...]

Now you need to rebuild the archive visible from the web. First, move the
existing archive directory (test, not test.mbox) out of the way:

  mv test test.save

and then go rebuild the archive:

  cd ~mailman
  ./bin/arch test

NOTE: you must move the directory out of the way, instead of simply copying it.
If you don't do this, the arch command will add additional copies of previous
messages to the existing archives and you will end up with duplicates.

Or you can use the --wipe option with ~mailman/bin/arch to initialize rather
than add to the existing archive. See ~mailman/bin/arch --help

To make sure it worked, visit the list archive with your web browser. Once
you're sure it's fine, you can delete the "test.save" directory.

CAVEAT: If you delete entire messages from the archive two side effects occur:

1) Threading may be broken - if C is In-Reply-To: B which is In-Reply-To: A,
and B is deleted, C will no longer be threaded with A.

2) Messages will be renumbered - this may be important if there are saved links
to archive messages. Since the message number is part of the URI, the saved
link will no longer work or will retrieve the wrong message.

To avoid these problems, instead of deleting the entire message, leave the
headers intact and replace the body with "Message deleted" or some other
meaningful text.

----------------------------------------

NOTE ON HOW TO DELETE AN ENTIRE ARCHIVE:

1. Browse to the list-administration panel, turn off archiving (option archive)
and switch archive from public to private if it's public (option
archive_private).

2. Do the things as described above, when editing the raw archive (mbox), just
delete all entries.

If you like to reenable archiving later, just turn on archiving in the
list-administration panel (and set it to public if you like to).

----------------------------------------

I've had good luck using a command-line email program (such as mutt) to delete
messages from the archive mbox. I've found this is especially useful for
archives that contain many attachments and/or encoded text sections.

Edit this entry / Log info / Last changed on Fri Aug 20 23:00:59 2004 by Jeff
Barger

-------------------------------------------------------------------------------

3.4. How do I move a list to a different server/Mailman installation.

Robin Rowe's quick tip: There's already a mass subscribe feature in the
interface. All one needs is the member list from the old list. To get that,
send a blank email to yourlistname-request@yourlistserver.com with a subject of
'who yourpassword'. The list admin password is only necessary if you have the
members list locked to admin access only. Otherwise, you could just use 'who'.
The automated email response will contain the list of names from the list. You
can cut-n-paste that text into the {Membership Management}{Mass Subscription}
field in the interface of the new list.

Barry Warsaw has described the process he went through to upgrade from 2.0.10
to 2.1b2, which is similar to the process of moving a list from one server to
another, at the following link:

http://www.mail-archive.com/mailman-developers@python.org/msg03127.html

Proceed at your own risk, it may or may not be relevant.

The process of removing subscribers from one list and moving them to another is
described in the FAQ answer at:

http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.025.htp

If you can only move part of the list due to not having full read permissions
to the old list, you can regenerate the archive from the .mbox of the old
archive using the techniques described in this FAQ answer:

http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.003.htp

Finally, you also need to update the MTA aliases. Run <mailman>/bin/genaliases
to generate the list (or actually update if you've configured Mailman for you
MTA).

Kevin Gagel's steps for moving Mailman from different OS and upgrading to a
newer version.

Going from Redhat 7.3, Mailman 2.1.4 to SuSE 8.0, Mailman 2.0.4 while upgrading
2.0.4 to 2.1.5. This can be done in three basic steps.

1.)Copy the lists from the old server to the new server.

2.)Upgrade Mailman from 2.0.4 to 2.1.5

3.)Set the permissions and correct the url.

1 To copy the lists, cd into the mailman directory that holds "data",
"archives", and "lists". Use tar to compress them into three files like this:

tar -cpvW -f /root/mmdata ./data

tar -cpvW -f /root/mmarchives ./archives

tar -cpvW -f /root/mmlists ./lists

Then copy these three files to the new server under the mailman directory where
the above directories exist. Once you've done that, cd into that directory and
begin the expansion with the following:

tar -xvf mmdata

tar -xvf mmarchives

tar -xvf mmlists

2 Set the permissions from this directory with:

chown -R nobody:mailman .

chmod -R a+cx,g+ws .

3 Now your ready to upgrade from 2.0.4 to 2.1.5, follow all instrunctions for
your upgrade in the UPGRADE document. You'll find that in the source package
you've downloaded. For the configure process I had to use these paramaters:

--prefix=/usr/lib/mailman --with-var-prefix=/var/lib/mailman --with-groupid=
nogroup --with-python=/usr/bin/python

I had use these because SuSE splits the package and installed it to different
locations. You can go with the default Mailman install if you'd like. Just be
sure to locate the three tar files and expand them to the correct mailman
directory.

4 Now your ready to fix the installation. By now you should have mailman
running and working but your old lists and archives don't yet show up. This
step fixes that. Cd into the mailman/bin directory. Run the following command
for each list you have transfered:

withlist -l -r fix_url listname

All should be well and working now. One caveat that might occur is you'll start
getting notices about a missing stringIO modules. If this occurs to you then
delete the following directories from the mailman directory and redo the above
steps:

archives data lists locks logs qfiles spam

Edit this entry / Log info / Last changed on Wed Aug 11 22:41:23 2004 by Kevin
W. Gagel

-------------------------------------------------------------------------------

3.5. What is an Umbrella list - and why doesn't it do what I want?

An Umbrella List is a list of other lists.

For example, you have the following lists running in Mailman:

  ThreeBlindMice: Minnie@farm.org, Mighty@cartoon.net, Mickey@disney.com
  ThreeBears: Smokey@usda.gov, Little@pbs.org, Momma@mgoose.net
  ThreeMenNaTub: Rub@tub.org, Dub@tub.org, Little@pbs.org

You can create an Umbrella List called, "Threesomes" and instead of people, it
will contain the above list names. When you send email to Threesomes, the
message goes to each of the individual lists...

  Threesomes: ThreeBlindMice@mylists.com, ThreeBears@mylists.com, ThreeMenNaTub@mylists.com

When mailman sends out a monthly password reminder, it sends an email to every
person in its lists. If it did that for the Umbrella List "Threesomes", then it
would treat "ThreeBlindMice" as an ordinary user. Mailman would send an email
to "ThreeBlindMice" that contained the password for subscribing or
unsubscribing it to the list "ThreeSomes". Everyone on the "ThreeBlindMice"
list would receive that message with the password. Now that is fine in the case
of Minnie and Mighty, but Mickey has been known to do a few silly things in his
time...

You can keep Mickey from getting the password by telling Mailman to send this
type of information to "ThreeBlindMice-owner" instead of to "ThreeBlindMice".

By making "Threesomes" into an Umbrella List, you can tell Mailman to add
"-owner" to any notices, confirmations and passwords that it sends out to list
members.

I would like to see Umbrella Lists do more than just that. IMO, they should
only send one message out to each actual user, regardless of how many lists
that user is on. Currently, if a message is sent to an Umbrella list and a user
is on multiple sub-lists under that Umbrella list, then they will receive
multiple copies of the message - one for each of the sub-lists that they are
on. In the example above, Little@pbs.org would receive two messages everytime
someone sent an email to "Threesomes".

===

The work-around for this is to setup a script that builds a Mega-list out of
other lists on your server.

  #!/bin/bash
  # Create and sync the Pasus_All@mydomain.com mailling list
  # The list includes everyone on a Pasus project mailling list.
  # All the Pasus mailing lists start with the word: pasus
  #
  # create an empty file
  echo " " >/tmp/pasus_list
  #
  # dump out all lists names that contain the word pasus, but
  #   be sure to not include the "_all" list, since that is the list
  #   that we are recreating with this script
  LISTS="`~mailman/bin/list_lists |grep -i pasus | \
    grep -vi _all| awk '{ print $1 }'`"
  #
  for i in "$LISTS"
  do
     # dump all the user emails out to a file for all the pasus lists
     ~mailman/bin/list_members $i >>/tmp/pasus_list
  done
  #
  # this sorts the list and removes duplicate entries
  cat /tmp/pasus_list |sort -u >/tmp/pasus_all_list
  ### the above step is really unnecessary as "sync_members" already 
  ### does this automatically - still I like a clean orderly list...
  #
  # feed the list of all pasus users into the "_all" list
  # Note: welcome messages are turned off for this list.
  ~mailman/bin/sync_members -f /tmp/pasus_all_list pasus_all >/dev/null
  #
  # Take only pictures, leave only foot prints...
  rm /tmp/pasus_list
  rm /tmp/pasus_all_list

  # Have cron run this script hourly and the Pasus_all list will be
  # updated automatically every hour.

===

Thanks to Mark Merlins, Jon Carnes, and Shane Beasley for the above work
around.

Note this workaround does not take into account per-member options, especially
digest and plain delivery options, and the nomail option. A comprehensive
script that does it all can be found at http://www.loowit.net/~james/
gen-combined-list

Edit this entry / Log info / Last changed on Thu Oct 9 22:38:27 2003 by James T
Perkins

-------------------------------------------------------------------------------

3.6. What can I do about users with broken autoresponders?

A common problem on mailing lists is users who have setup an "autoresponder",
or software that automatically responds to all mail for them, that doesn't
understand mailing lists. Autoresponders are most commonly used for "I'm on
vacation"-style messages, but they might also be seen bouncing mail that
appears to be spam or virus-infected. There are varying degrees of broken-ness,
but there do not appear to be any limits on the stupidity of broken
auto-responders.

The least harmful kind of stupid auto-responder is one that replies to the list
administrator -- eg. if you are running foo-list@example.com, you keep getting
"I'm on vacation" messages from one of your list members at 
foo-list-admin@example.com. The problems here might be 1) the user did not
configure her autoresponder to exclude mailing lists, and 2) the autoresponder
doesn't have a rate-limiting feature (eg. "Only send an automatic response to
each sender once per day"). In this case, the autoresponder software is
probably working correctly (in particular, it's sending responses to the
envelope sender only), and the user has probably just configured it
incorrectly. Since it's just you, the list administrator being bothered, how to
deal with this is up to you. Disabling the user's subscription is an option (if
they don't get any more list mail, they won't send you any more pointless "I'm
on vacation" messages), but it's probably just as easy to ignore the messages
(and send the user a polite note asking her to fix her autoresponder before she
takes another vacation).

A more annoying kind of brain-damage is when every poster gets an inexplicable
"I'm on vacation" message from someone he's never heard of (and who just
happens to be on the same list). For example, let's say joe@some.domain sends
mail to your list, foo-list@example.com. Mailman processes the message and
sends a copy to everyone on the list, including bob@other.domain. If 
bob@other.domain has a stupid auto-responder that sends responses to some
address in the header of the message (as opposed to the envelope), then 
joe@some.domain will get bob@other.domain's auto-response. joe will presumably
be confused by this, since he doesn't know who the heck bob is. However, you --
the list admin -- won't find out about, unless you also post to the list -- or
if joe figures out what's going on and tells you about it. In this case, the
autoresponder software is broken, and sending a polite note to 
postmaster@other.domain, asking that the broken autoresponder be fixed or
disabled, is legitimate. You should cc bob@other.domain so he knows what kind
of trouble he's causing. If the problem persists, disabling bob's subscription
is reasonable; you should send bob another note to let him know you have done
so.

The worst kind of autoresponder brain-damage is when "I'm on vacation" messages
get sent to the list itself. This should be inconceivable with Mailman (since
the list address is usually only in the "To" or "Cc" headers), but remember:
there is no limit to the stupidity of broken autoresponders. If this happens,
the best option is to immediately disable the subscription of the user causing
the trouble: you want to stop clogging your list with completely useless noise
as quickly as possible. Again, you should send mail to the postmaster of the
offending domain and the user responsible for the autoresponder, letting them
know what kind of havoc their amazing broken autoresponder is causing.

The key things to keep in mind here are:

* it's your list; you have the right to disable the subscription for anyone
who's causing problems

* be polite: don't curse or swear at distant postmasters; they probably don't
even realize what's happening

* be accurate: don't accuse the wrong user of having a broken autoresponder,
and don't mis-characterize the problem when explaining it to them or their
postmaster. (If you don't understand the difference between envelope sender and
header addresses, you should think twice before complaining.)

Edit this entry / Log info / Last changed on Mon Dec 3 20:01:15 2001 by Greg
Ward

-------------------------------------------------------------------------------

3.7. Setting up Web access using MM list passwords - for Apache

The Mailman database stores its user passwords in plain text and by default
sends them out to the users once a month, so these passwords are by no means
secure. They do however make a convenient database for allowing access to
Mailman related web-sites.

Apache uses .htaccess files to limit access to certain sites on a web server.
For details, please read:

  http://httpd.apache.org/docs-2.0/howto/auth.html

Basically you place a .htaccess file into a directory on your webserver and in
that file point to a password database. The server makes folks login before
allowing them access to that part of the site.

The script below takes as input the name of one of your Mailman lists and dumps
out each users email address and password. It then plugs those values into a
password database /etc/httpd/.htpasswd.<list-name>. You can use a .htaccess
file on your website to point to that password database and authenticate
against it.

You should be able to read the above link in less than an hour and using the
script below as a base, get everything working in another hour.

  #!/bin/bash
  #  Script: mm_htaccess
  #  Updated on Nov 22, 2002 by Linda L. Julien
  #  Creates a .htpasswd file based on a Mailman list
  #    user = the email address of the user
  #    password = the Mailman password for the user
  #       blank passwords are assigned the value "password"
  #
  #  Script assumes Mailman was installed to the ~mailman/..
  #
  #  Input to the script is the name of the mailing list
  #  Output is a .htpasswd file suitable for web access
  #
  LIST="`~mailman/bin/list_lists |awk '{print $1}' | \
          grep -iw ^$1$ 2>/dev/null |tr [A-Z] [a-z] `"
  if [ "xxx$LIST" = "xxx" ]; then
     echo "mm_htaccess: missing valid listname from Mailman"
     echo "    mm_htaccess  <list-name>"
     echo "  "
     echo "This program creates a .htpasswd.<list-name> file in /etc/httpd"
     echo "It uses the users and passwords stored in the Mailman list."
     echo "  "
     echo "You must choose a valid Mailman list on this server:"
     ~mailman/bin/list_lists
     echo " "
     echo "mm_htaccess command aborted!"
     exit 0
  fi
  touch /var/tmp/.htpasswd.$LIST
  CONFIG=~mailman/lists/$LIST/config.db
  #
  # Isolate the password secton of the config file
  PWLN=`~mailman/bin/dumpdb $CONFIG | \
        grep -n "'passwords': {" |cut -f 1 -d: `
  ~mailman/bin/dumpdb $CONFIG |sed -n "$PWLN, \$ p" >/var/tmp/.htlist.$LIST
  #
  # Retrieve the password info for each member of the list
  for i in `~mailman/bin/list_members $LIST `
  do
    PASS=`grep -i $i /var/tmp/.htlist.$LIST|  head -1 | \
          cut -f2- -d@ | cut -f3 "-d'"`
    if [ "xxx$PASS" = "xxx" ]; then PASS="password" ; fi
    htpasswd -b /var/tmp/.htpasswd.$LIST $i $PASS
    # echo $i " : " $PASS
  done
  #
  mv /var/tmp/.htpasswd.$LIST /etc/httpd/.htpasswd.$LIST
  rm /var/tmp/.htlist.$LIST

===

Note that the above script uses the "htpasswd" program which comes with Apache,
and is normally installed in /usr/bin/..

Thanks to Jon Carnes for the above script.

===

Ok, building on Linda and Jon's script here, I have come up with the following
method of creating the htpasswd databases automagically in the directory /etc/
httpd:

1) Here is mm_make_htaccess.pl:

        #!/usr/bin/perl
        ###############
        ##
        ## mm_make_htaccess.pl
        ##
        ## Create /etc/httpd/Makefile for Mailman use.
        ## version 1.12
        ## by ges, 10/17/2002
        ##
        ## This script is designed to take the directory names
        ## from ~mailman/lists and put them into a Makefile
        ## which, when run, will create htpasswd files for
        ## all your mailman lists, for use in Apache.
        ##
        ###############
        ##
        ## Version 1.1 10/15/02: Tests to see if $list_dir/$list
        ## is a directory or not, before adding the entry to
        ## the Makefile!
        ##
        ###############
        ##
        ## Version 1.11 10/16/02: Update: Made printing to the
        ## Makefile a little more elegant (thanks, Frank!),
        ## and now making use of $variables in the Makefile.
        ##
        ###############
        ##
        ## Version 1.12 10/17/02: Update: Added more comments
        ## to the code.
        ##
        ###############
        ##
        ## First we define where Mailman lists reside.
        ##
        ## Then we define what directory the Makefile
        ## lives in, as well as where the htpasswd files
        ## will live.
        ##
        ## Then we will set a variable to change for
        ## admins using Mailman 2.1 or 2.0
        ##
        ## Change $configext to be db for 2.0
        ## or pck for 2.1
        ##
        $list_dir="/etc/mailman/lists";
        $file_dir="/etc/httpd";
        $configext="pck";
        ##
        ##
        ## First let's open the list directory!
        ## Once that's done, we'll read the filenames into @names
        ## for later use!
        ##
        opendir(LISTDIR,$list_dir) || die("Cannot Open $list_dir!");
        @lists = readdir(LISTDIR);
        closedir(LISTDIR);
        ##
        ## Now we open the makefile
        ##
        open(FILE,">$file_dir/Makefile") || die("Cannot Open File");
                # This changes print's default to FILE
                my($oldHandle) = select(FILE);
                # Now we start off the Makefile with two variables
                # and some nice spacing.
                print "\n\nMAILMAN=$list_dir";
                print "\nHTACCESSDIR=$file_dir";
                # We ensure that "all" means make the list htpasswd files.
                print "\n\nall: mailmanstuff\n\n";
                # H'ok--now we have the work...
                print "mailmanstuff: ";
                        # This steps through each entry in @lists, which we read
                        # earlier, using readdir
                        foreach $list (@lists)
                        {
                                # Ok--we really don't want to match
                                # current directory & parent...
                                next if ($list eq ".");
                                next if ($list eq "..");

                                # And here we test to make sure that
                                # the entry is actually a directory,
                                # not a file, then it adds the
                                # Makefile line to call each list entry.
                                if (-d "$list_dir/$list") {
                                        print "\$(HTACCESSDIR)/htpasswd.$list ";
                                        next;
                                }
                        }
                # Ok--let's null out $list for re-use.
                $list="";
                # Making it pretty...
                print "\n\n";
                        # And here again we step through...
                        foreach $list (@lists)
                        {
                                next if ($list eq ".");
                                next if ($list eq "..");

                                # This time we set up the individual entries
                                # for each list.
                                if (-d $list_dir . "/" . $list) {
                                        print "\$(HTACCESSDIR)/htpasswd.$list: ";
                                        print "\$(MAILMAN)/$list/config.$configext\n";
                                        print "\t/home/adm/bin/mm_htaccess $list\n\n";
                                        next;
                                }
                        }
        # Now, let's close the Makefile...
        close(FILE)

2) Create a shell script-wrapper for ~mailman/bin/newlist (I call it
newmaillist):

        #!/bin/sh
        ~mailman/bin/newlist
        /path-to/mm_make_htaccess.pl

2) Create a shell script to run Make in /etc/httpd (I call it
htpasswd_mailman.sh):

        #!/bin/sh
        cd /etc/httpd
        make

3) Put an entry in root's crontab (this example runs the script every 10
minutes):

        0,10,20,30,40,50 * * * *  /path-to/htpasswd_mailman.sh

Since this uses make, it only runs on the list databases that have actually
been touched since the last make.

I hope people find this useful. :)

I know that I'm doing the "typical newbie" way of walking a filesystem in Perl,
I figured this was easier to port, since it didn't require any modules to be
installed in Perl.

If anyone has an idea of getting the perl script to run when lists are created
via the GUI interface, that would make this a very handy solution indeed!

Thanks to Jon Carnes and Linda Julien for the original mm_htaccess script, and
the inspiration for this!

Edit this entry / Log info / Last changed on Sun Jan 12 04:01:58 2003 by Glenn
Sieb

-------------------------------------------------------------------------------

3.8. I forgot my list password! How do I get it back?

The site password can be used to log into anywhere a member password or list
password would go. So you can reset the password to something you do know: use
the site password to log into the list and change the password (you don't need
to know the old list password to do this). If you haven't got a site password,
use $(MAILMAN_HOME)/bin/mmsitepass to set one.

(If you are not also the site administrator, have the site administrator change
the list password for you. Once the site administrator has set it, you have a
new list password, and you can change it again to something that only you
know.)

Edit this entry / Log info / Last changed on Thu Oct 10 23:48:48 2002 by Dan
Mick

-------------------------------------------------------------------------------

3.9. How do I edit a held message before approving it for the list?

Here's how to do this:

- Use the "forward this message" feature in the admindb page to forward the
message to yourself.

- At the same time, discard the original held message. You may want to do this
later, after you're sure the message was properly forwarded to you.

- Edit the message in your mail reader. You should include a notice in the
message explaining that the list moderator has edited the message. Please use
proper netiquette!

- Resend the message to the list using a Resent-To: header containing the list
posting address. Also include an Approved: header containing the list's admin
password.

Viola! The edited message should be forwarded to the list membership.

==========

Alternatively, you can edit the actual held message found in ~mailman/data/...
The message will be in a file called heldmsg-<listname>-<number>.txt

You can use "grep" to find the message, and your favorite editor to change it.

Once you have modified the message, feel free to approve it.

The web-interface shows you a portion of the message that has been copied into
a database called request.db. When you edit the message via the web-interface
you are actually editing the portion captured in request.db and not the
original message.

Approval of the message sends the actual held message from ~mailman/data/..

==========

Note for Mailman 2.1: the second suggestion about doesn't work so well because
by default, held messages are stored on disks in a more efficient binary
representation called a "pickle". Pickles aren't normally editable. If you
still want to use the second suggestion, you might want to set
HOLD_MESSAGES_AS_PICKLES in ListAdmin.py to 0, which tells Mailman to store
them using the older, less efficient, plaintext representation instead of
pickles.

I still don't like the second suggestion much because it is limited to only
those who have access to the system disk and command line. Also, in MM2.1 the
admindb interface doesn't fool you into thinking you can edit the message, as
the text areas are now read-only.

Edit this entry / Log info / Last changed on Thu Apr 18 23:31:06 2002 by Barry
Warsaw

-------------------------------------------------------------------------------

3.10. How do I enforce a text/plain posts only policy?

In Privacy Options -> Spam-specific posting filters

Hold posts with header value matching a specified regexp.

    Content-Type: .*octet
    Content-Type: .*oda
    Content-Type: .*audio
    Content-Type: .*image
    Content-Type: .*alternative
    Content-Type: .*digest
    Content-Type: .*mixed
    Content-Type: .*related
    Content-Type: .*rich
    Content-Type: .*html
    Content-Type: .*video

Sample Rejection Message:

  Content-Type: (from message header)
  X-Mailer: (from message header)

  Please configure your email client [1] to send text/plain messages to
  this list. Instead of attachments, you should include any diagnostic
  information as in-line text in the main message body.

  [1] http://www.expita.com/nomime.html

  Details:
  Only messages with a Content-type: of "text/plain" and
  "multipart/signed" are automatically posted to the list. All other
  content-types are held for administrative action.

  If you have any questions about this policy, please send them to
  <yourlist-admin@yourdomain>.

An alternate solution is to use a mime stripping script. FAQ 1.8 explains how
to do this.

Edit this entry / Log info / Last changed on Tue Nov 12 19:31:21 2002 by Mike
Noyes

-------------------------------------------------------------------------------

3.11. How do I create a newsletter/announcement/one-way list?

From 2 posts by Barry Warsaw to the mailman-users list on February 10/11 2003,
with edits by JC Dill:

How to setup an announcement/newsletter/one-way email list with Mailman:

Features: Members join by filling in a form on your website (and replying to
the confirmation email). They will be sent a welcome message that does not
mention how to post to the list. They will receive your newsletters, with a
footer that gives simple instructions for unsubscribing. Only authorized
persons can post to the list (send the newsletters). (Note: For this FAQ we are
discussing how to setup an announcement list for a band.)

How to create a customized welcome message that avoids mentioning how to post
to the list:

  In Mailman 2.1.x you can customize the welcome message.  Create
  a directory lists/<yourlist>/en (assuming English :) and copy
  templates/subscribeack.txt to this directory.  Then edit this file 
  for your specific wording.  Mailman will use this specialized 
  template for the English welcome messages.

How to minimize password issues and unsubscription problems commonly faced on
this type of list:

  Turn on personalization under the NonDigest section[1] (and disable
  digests under the Digest section) so people get their options page
  URL included in the footer of every message.  While not totally
  avoiding the use of a password, most unsubs won't need them.  The
  easiest instructions (include in the footer) are to send a message to
  <yourlist>-leave@dom.ain, and then simply reply to the confirmation
  message, so they never need to know their password.  Disable monthly
  password reminders.  You may wish to occasionally manually run the
  reminders to cull dead addresses.

[1] In 2.1 personalization is not enabled by default -- adding:

  OWNERS_CAN_ENABLE_PERSONALIZATION = 1

to ${prefix}/Mailman/mm_cfg.py will reveal this functionality.

How to restrict the list so only authorized persons can post:

  Turn on the moderation flag for all your existing users.  Go to the
  membership management page, and use the Additional Member Tasks to
  turn on the mod flag for all users.

  Then go to Privacy Options -> Sender filters and set the
  default_member_moderation flag to Yes so that new users are
  automatically set as moderated.

  Set the member_moderation_action to Reject and add a nice rejection
  notice text to the following text box.  Say something like "this is
  an announcement list, to reach the band, please email band@dom.ain"

  Set the generic_nonmember_action to Reject or Discard.

How to set an announcement list to reply to a contact address:

  Go to the General options and scroll down to Reply-To munging.
  Turn on first_strip_reply_to and set reply_goes_to_list to Explicit
  Address.  Set reply_to_address to band@dom.ain.  This way anyone
  following up to an announcement will send the message to your band
  contact address.  If your newsletters are likely to be widely
  forwarded on, you may want to use a special contact address for
  this purpose, one you can change from time to time when the address
  starts receiving a lot of spam.

  Scroll down on the General options and set include_list_post_header
  to No, but leave include_rfc2369_headers to Yes.

How to post to the announcement list:

  For the list subscribers who are authorized to post announcements to
  the list, turn off their moderation flag so their postings go straight
  through.  (Note: this leaves open the possibility of someone forging
  email as coming "from" one of the authorized posters, and spamming
  your list.  This also opens a door for virus software that takes a
  random address from the infected computer and uses it as the "from",
  and then sends to another random address on the same computer, to use
  the authorized sender address and the list address, and end up sending
  the virus to your list.)  A more secure alternative is for your
  approved posters to add an Approved header to their postings as a
  header, or as the first line of the post). 

=============================================================

If you want to create a list to which only the admin or a restricted number of
people can post (such as a newsletter), set member_posting_only to "no" and add
the addresses of all allowed posters to posters.

This is done on the Privacy Options page. The relevant items are "Restrict
posting to list members" and "Addresses of members accepted...".

There is a link to more detailed information for the items which you might want
to read as well.

One problem you will encounter is that the headers of each list message still
mention a post address, and some users will think that public posting is
allowed. This can be turned off in Mailman v.2.1, but for the 2.0 series, it
seems you have to live with it.

In 2.1b, "Restrict posting to list members" is no longer available,
unfortunately. It appears the best solution for 2.1 is to make all members
require moderation, give them an auto-action if they try to post (discard or
bounce, e.g.), give all non-subscribers an auto-action, and then either
specifically allow certain non-subscribed addresses or un-mod certain
subscribed addresses to allow them to post. (If there is an easier way, please
correct this.)

=================================================

How to do it in Mailman 2.1: you want a list where only the list moderator can
post. If list subscribers (or nonsubscribers) try to post, you want their
messages to be rejected. The list moderator should be able to send email to the
list without having to tend to administrative requests -- you just want the
email to go. The quickest way to fix the list configuration settings to get
this scenario is to dump the configuration list via "config_list", edit the
file, and read it back in.

Here is what you need to do for list "foobar", assuming that "joeblow@big.com"
is the moderator:

   config_list -o foobar foobar

Then edit the foobar file and set the following options:

   moderator = ['joeblow@big.com']
   default_member_moderation = 1
   member_moderation_action = 1   (reject member posts)
   member_moderation_notice = 'Sorry, subscribers cannot post to this list.'
   generic_nonmember_action = 2   (reject nonmember posts)

Now reload the foobar settings file:

   config_list -i foobar foobar

One final step is needed so that the moderator can post to the list without
doing "tend administrative requests". Go to the admin webpage for the list and
UNCHECK the "mod" button for the moderator ONLY. All subscribers to the list
should have the "mod" button turned ON, except for the moderator. This assumes
that the moderator is also subscribed to the list.

Edit this entry / Log info / Last changed on Thu Jul 24 20:46:25 2003 by Ajay

-------------------------------------------------------------------------------

3.12. How can I remove duplicate Re: markers from the Subject: field of
messages?

I've managed to do this with a simple sed script.

(1) Find a suitable place for your script, and then make a file

    like 'mylist.sed' containing:

    ----- start of mylist.sed -----
    s/^Subject:\(.*\)\[SUBJECT_PREFIX\] /Subject:\1/
    s/^Subject: *\(Re: \)\+/Subject: Re: /
    ------ end of mylist.sed ------

    (Change the SUBJECT_PREFIX to the subject prefix for your list)

(2) Make a file like 'mylist.sh' containing:

    ----- start of mylist.sh -----
    cat | sed -f /PATH/TO/mylist.sed | /PATH/TO/mailman/mail/wrapper post listname
    ------ end of mylist.sh ------

    (Again, change the /PATH/TO's to a suitable value for your system)
    (Note that qmail users need to insert the 'preline' command between the pipe and the 'wrapper' command)

(3) Change the 'mylist.sh' script to mode 755 (executable)

(4) Change /etc/aliases or ~alias/.qmail-listname to be:

    | /PATH/TO/mylist.sh

    (Change this as you require)

(5) Test by posting to the list with a subject line containing Re:'s and a
[subject_prefixes] and more Re:'s

Edit this entry / Log info / Last changed on Tue Aug 13 05:04:56 2002 by Phil
Stoneman

-------------------------------------------------------------------------------

3.13. How do I remove a user name or email address with an illegal character in
it?

This sort of thing might result in a subscriber address being corrupted, and
errors like this:

Traceback (most recent call last):

  File "/usr/local/mailman/scripts/driver", line 87, in run_main
    main()
  File "/usr/local/mailman/Mailman/Cgi/admin.py", line 198, in main
    show_results(mlist, doc, category, subcat, cgidata)
  File "/usr/local/mailman/Mailman/Cgi/admin.py", line 501, in show_results
    form.AddItem(membership_options(mlist, subcat, cgidata, doc, form))
  File "/usr/local/mailman/Mailman/Cgi/admin.py", line 873, in membership_options
    all = [_m.encode() for _m in mlist.getMembers()]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 22: ordinal not in range(128)

To fix this, try using bin/withlist

 % python -i bin/withlist -l mylist
 >>> del m.members['na,me@example.com']
 >>> m.Save()
 >>> ^D

Instead of the del m.members line, you might want to use a higher level method,
such as removeMember(), e.g.:

 % python -i bin/withlist -l mylist
 >>> m.removeMember('na,me@example.com')
 >>> m.Save()
 >>> ^D

The latter is probably preferrable since it cleans out any other information in
the database associated with the bogus address, but if it raises exceptions you
should use the former (and submit a bug report!)

Email addresses may only contain ASCII characters. Older versions of Mailman
allowed non-ASCII characters as email addresses sometimes. This has since been
fixed, but some older lists may have bogus addresses in them. Here's a command
line recipe that might help clear all of these out:

 % bin/find_members "[^\w\-+@.%]" listname > file
 % bin/remove_members -f file listname

You may have to adjust that regular expression.

Edit this entry / Log info / Last changed on Wed Jun 2 01:18:54 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.14. Troubleshooting: No mail going out to lists members

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq01.007.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.007.htp>.

Here are some common things to check when no mail is going out from your lists.

======

I'm going to assume Sendmail as the MTA (its still the most commonly found -
though postfix is gaining ground):

0) Check_perms.

   In all cases you should start by checking the
   permissions on the files that were setup:

     ~mailman/bin/check_perms

1) Cron.

   Make sure that the cron daemon is running

     ps -aux |grep cron |grep -v grep

   This will print out the process information about the cron
   daemon. If it returns a blank line, then cron is NOT running.
   NOTE: Mailman 2.0.13 and lower does not run an independent 
   daemon. The qrunner process is run via cron.

     Later versions of Mailman, versions 2.1 and above, also
     run an independent daemon called mailmanctl. You need to
     make sure that the mailmanctl daemon is also running
       ps -aux| grep mailmanctl |grep -v grep

     If this returns a blank line then the Mailman daemon is not
     running! 

2) Aliases.

   To create a mailman list you ran "newlist" and it
   printed out four lines that you needed to copy to the 
   /etc/aliases file (or wherever your MTA goes to find its
   aliases). Check that the aliases are in /etc/aliases:

     grep wrapper /etc/aliases

   Note for version 2.1: "wrapper" has been replaced with
   "mailman",
     grep mailman /etc/aliases

   Even if the aliases are there, you may still need to reset
   the aliases hash table so that it includes this new alias
   information:

     newaliases 

    Here is a typical alias listing for a group called "sys":

     ## Mailman verion 2.0.x system mailing list
     sys:           "|/home/mailman/mail/wrapper post sys"
     sys-admin:     "|/home/mailman/mail/wrapper mailowner sys"
     sys-request:   "|/home/mailman/mail/wrapper mailcmd sys"
     sys-owner:      sys-admin

    If you are using Sendmail with virtual domains, you will 
    have to also add entries into the into the 'virtusertable' 
    to make the mapping for the virtual domain to the local 
    aliases previously setup, similiar to the following:

     ## mailman virtual mappings
     sys@virtdomain.com                    sys
     sys-admin@virtdomain.com              sys-admin
     sys-request@virtdomain.com            sys-request
     sys-owner@virtdomain.com              sys-admin

     then run 'make' or 'makemap' to rebuild the 
     virtusertable.db file and restart sendmail.  This solution
     is limited in that you can not have multiple virtual
     domain with the same local address/list name.

       NOTE: in mailman version 2.1 and above you may have a
       secondary aliases file that is used specifically
       for Mailman and the lists that it creates:
         ~mailman/data/aliases

        You will need to make sure that this alias file is
        recognized by your MTA (postfix, sendmail, etc), and
        that it is up-to-date with your latest aliases.

        If you do not have integration turned on, or you do 
        not have this alias file, then all your aliases *must*
        go in the standard /etc/aliases file as specified
        above.

3) Smrsh.

   Check to see if your MTA uses smrsh.  Red Hat as well
   as a few other distributions automatically setup Sendmail to
   use smrsh.  Smrsh stops Sendmail from running a script or
   other program that is included in an alias. Mailman uses a
   program called "wrapper" to run all of its aliases (see the
   alias examples above):

     grep "smrsh" sendmal.cf

   If this comes up blank then Sendmail does not use smrsh; 
   if not, then your server is probably running smrsh and you
   need to make sure that smrsh is setup to allow Mailman's
   wrapper program to run. Locate the smrsh directory and do
   an ls -l of that directory.  On Red Hat:

     ls -l /etc/smrsh

   and the output should be similar to:

     wrapper -> /home/mailman/mail/wrapper

   Note: some systems setup the smrsh directory in:
      /usr/adm/sm.bin/

   Note for version 2.1: the wrapper program is now simply
   called "mailman",
      mailman -> /var/mailman/mail/mailman

4) Interface.

   Some distributions in a noble "attempt" to limit
   the number of open relays on the internet, default Sendmail
   so that it listens to a limited number of interfaces. The
   default interface that Mailman list's use is "localhost" 
   (127.0.0.1) - this is configurable in Mailman's mm_cfg.py 
   file. To check Sendmail's configuration file:

     grep "Port" sendmail.cf

   This will list out the DeamonPortOption and indicate the
   interfaces it listens on (0.0.0.0 would mean all interfaces).

   You can also check out which interfaces your MTA is listening
   on by using:

     netstat -na |grep ":25 "

   In a similar vein, some folks have reported that
   "localhost" is not defined in their /etc/hosts file
   (it should be set to 127.0.0.1)

   SMTPHOST
   Mailman would accept messages but they wouldn't go anywhere.
   logs/smtp would just show "All recipients refused: (61, 
   'Connection refused')". My mailserver is configured to listen
   only on its external IP address, no localhost or anything like
   that. I had to add to ~mailman/Mailman/mm_cfg.py:
      SMTPHOST = '<ip of my mailserver>'
   After that Mailman started working perfectly.

   The settings in ~mailman/Mailman/mm_cfg.py (or Defaults.py)
   that are used to set these "Delivery Defaults":
     DELIVERY_MODULE = 'SMTPDirect'
     MTA = 'Manual'
     SMTPHOST = 'localhost'
     SMTPPORT = 0               # default from smtplib
     SENDMAIL_CMD = '/usr/lib/sendmail'

5) qrunner.

   If you are running Mailman 2.0.x then qrunner is
   run every minute via a cron job (that is why cron *must* be
   running for Mailman to work). Try running the program by
   hand. The exact syntax can be found in Mailman's cron jobs:

     su mailman
     crontab -l

   Here is an example of running qrunner by hand:

     su mailman
     /usr/bin/python -S /home/mailman/cron/qrunner

   If this generates any errors then send those to the list
   for diagnosis - or look at the last few lines of errors and
   search the list for key words from the error messages.

   If you are running Mailman 2.1.x then the qrunners are daemons that
   are started by $prefix/bin/mailmanctl, which itself may be being 
   run via a 'mailman' startup script. This is described in the 
   INSTALL document for the version of MM you are running.

6) Locks.

   A errant lock file can stop a list from processing as
   Mailman waits for the lock to be removed. Since your list is
   not sending, we shall assume that no lock files should be on
   the list and that it is safe to delete any we find.

     ls -l ~mailman/locks

   The output will be something like:

     qrunner.lock.moya.trilug.org.22845

   This indicates that process # 22845 created the lock. To look
   at this process and see what it is (if it still exists):

     ps aux |grep 22845 |grep -v grep

7) Logs.

   If you don't have any of the common problems above,
   then you should look for errors in your log files.

   First look for errors in your MTA log files. On Red Hat that
   would be in /var/log/maillog.

   Look in the log starting at the time you sent a test message.
   You should see your initial message come in and be passed
   on to your Mailman list, afterwards you may see warnings
   or errors caused by Mailman trying to send out mail to the
   members of the list.

   Next look in Mailman's logs. The files are in ~mailman/logs/.
   There are several logs to look in for problems:
     error
     smtp-failure
     smtp
     vette
     config
     post

   Note: if you look in the qrunner log you will see several
   warnings about "Could not acquire qrunner lock", these are
   actually normal and are NOT a problem.

   Every line in the log files is dated so you should be able to
   isolate the place in the log files to start looking, based on
   when your problem started. 

     - What do the logs and the times of the entries in your MTA 
       show.  Is it passing the mail into the mailman alias
       properly? Check the time.
     - Now check the Mailman post log and when it shows the
       message accepted for posting.  The time should be very
       close to the MTA's time.
     - Now back to the MTA, does it show the message going out
       to the various folks on the list? 

8) Qfiles.

   You may have a malformed email (or one that is simply
   too big) clogging up the flow of mail to your lists. Mail
   that is queued up by Mailman is stored in the directory:
     ~mailman/qfiles

   Move any files out of the directory and into a temporary 
   directory, then send a new test message to your list. If that
   works, then you can move some of the old queued up files back
   and let those process.  If it stops working again then you
   have a bad message in that batch - delete them or copy them to
   a different temporary directory.

9) SMTPHOST

   Mailman would accept messages but they wouldn't go anywhere.
   logs/smtp would just show "All recipients refused: (61, 
   'Connection refused')". My mailserver is configured to listen
   only on its external IP address, no localhost or anything like
   that. I had to add to ~mailman/Mailman/mm_cfg.py:
      SMTPHOST = '<ip of my mailserver>'
   After that Mailman started working perfectly.

===

If none of the above steps helps, then try seeking out help on the 
mailman-users@python.org mailing list. When submitting your problem to the list
please include the following information:

  - version of mailman you are running,
  - how it was installed (via rpm, apt-get, or source),
  - version of OS your server is running,
  - the MTA (sendmail, postfix, exim, qmail, etc) running on your server.

As a courtesy to the good folks on the list please use a Subject that describes
your problem (and not simply the word "Help).

======

Please feel free to criticize and expand on this. I'm hoping that it proves
useful as a starting point for folks having mail-flow problems.

Edit this entry / Log info / Last changed on Tue Jun 15 12:08:57 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.15. How do I enable personalization (messages tailored to the recipient)?

Personalization means customizing certain parts of the message for each
individual recipient. The most obvious use is to set the To: header to each
individual's address, but there are many other possible uses, such as
customized footers. Personalization was introduced in Mailman 2.1.

Personalization may considerably increase your bandwidth usage; see the
discussion on VERP for more details.

To allow list owners to personalize messages, set the
OWNERS_CAN_ENABLE_PERSONALIZATION variable to true. (Read about it in $prefix/
Mailman/Defaults.py, but change it in $prefix/Mailman/mm_cfg.py.) Note that
"true" in Python is the value 1 (one).

Now the owner of each list can turn on personalization of that list. To turn on
personalization, login as list owner and look at the second option under
"Non-Digest Options". Follow the "details" link for a helpful discussion of how
you can personalize the messages.

As long as you are sending personalized messages, you might want to set
VERP_PERSONALIZED_DELIVERIES to true as well.

After you have changed VERP_PERSONALIZED_DELIVERIES, you need to restart
qrunner for the change to actually take effect.

Edit this entry / Log info / Last changed on Thu Jun 10 22:59:48 2004 by 
malgosia askanas

-------------------------------------------------------------------------------

3.16. How do I assign a single password to all subscribers in a list?

This scripting trick works for 2.x versions of Mailman, and

relies on bin/withlist to work.

1) Become the "mailman" user, eg "su - mailman".

2) Put the following small file "changeuserpw.py" in the bin subdirectory,
making

sure that ~/bin is in user mailman's default path, so the file

can be found and used:

======================================

from Mailman.Errors import NotAMemberError

def changeuserpw(mlist, addr, newpasswd):

    try:

        mlist.setMemberPassword(addr, newpasswd)

        mlist.Save()

    except NotAMemberError:

        print 'No address matched:', addr

========================================

3) Create the following small shell script, named "change.userpw"

and make it executable (chmod 700 change.userpw):

========================================

#!/usr/bin/ksh

if [ $# -ne 2 ]; then

        print "Usage is: $0 list password"

        exit 1

else

        list=$1

        passwd=$2

fi

#---get the list members

list_members $list > /tmp/$list.members

#---change the user passwords

for member in `cat /tmp/$list.members`

do

        print "==== $member"

        withlist -l -r changeuserpw $list $member $passwd

done

rm /tmp/$list.members

=======================================

4) Run the shell script to change all subscriber passwords

in a specified list, eg:

change.userpw mylist foobar

will change the password for all subscribers in list "mylist"

to foobar.

5) Check your work and check the status of the database files

for the list. You should do a "check_db mylist" to see if you

get any complaints (you should not). You should also do a dumpdb

on the list's database file and take a look at it. You will find

that all subscriber passwords in the database are what you

specified for the shell script (if things worked). For example:

cd /web/data/mailman/lists/mylist

ls -l config.*

dumpdb config.pck > /tmp/mylist.dump

vi /tmp/mylist.dump

And then look for something like:

'passwords': { 'joeblow@somewhere.com': 'foobar',

and so on through all of your subscribers for that list. If all

of the passwords are "foobar" then things worked.

Edit this entry / Log info / Last changed on Mon Jan 6 20:25:24 2003 by Jeff
Earickson

-------------------------------------------------------------------------------

3.17. Moved -- see FAQ 4.40

Moved. See FAQ entry 4.40 at <http://www.python.org/cgi-bin/faqw-mm.py?req=show
&file=faq04.040.htp>

Edit this entry / Log info / Last changed on Tue May 11 10:29:30 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.18. How do I hide the fact that I'm using a mailing list management system?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.037.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.040.htp>.

Question: I want to change the configuration of the mailing list so that all
messages are addressed directly to the recipient, and the mailing list name
does not appear anywhere in the headers. I also want all e-mail to appear to
come from Customer.Service@mycompany.example.com, since most users will
probably reply to the sender address if they have any questions. How do I set
this up?

Answer: The features you're asking for are more appropriate to a Customer
Relationship Management (CRM) system, and not a mailing list management system
(such as Mailman). Any attempt to abuse a mailing list management system to
function more like a CRM system is likely to be both painful and unsuccessful.

Many ISPs use a simple "POP Bulletin" system to achieve these goals, with the
messages appearing to come from Customer Service and all replies being sent
back to them. No mailing list management system involved at all -- indeed no
MTAs involved, and no extra copies of this message stored anywhere on the
server, and no "deliveries" of this message to individual private mailboxes.
You can achieve the same sorts of things with a shared IMAP folder system, and
subscribing all users to the appropriate shared folder(s).

If you're not an ISP and you can't use "POP Bulletin" or IMAP shared folder
solutions, then you'd be left with traditional CRM systems which do actually
send out real e-mail messages.

Try going to www.google.com and search for "Customer Relationship Management
system" to see what solutions are available that are likely to be more
appropriate to your needs.

On the other hand, if you'd like to add these kinds of features to Mailman, we
encourage you to join the mailman-developers mailing list and work with us to
add this kind of functionality. To quote Richard Barrett on a related subject:

 MM is free, Open Software; so go for it. I am sure a well implemented enhancement
 will find other grateful users.

Edit this entry / Log info / Last changed on Wed Aug 11 13:10:09 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.19. Deleted

Edit this entry / Log info / Last changed on Tue May 11 10:32:20 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.20. How to I get the list administrator to look at all email with
attachments, html etc.

On Sat, 04 May 2002 02:33:39 -0400 Tim Miller wrote:

> > My only hassle with it so far is we would all prefer plain text messages.
Can't use stripmime (see below) because that would affect other users of the
Mailman facility who want to retain html.

> You can BLOCK HTML and attachments by going to your Privacy page and finding
the line that says: "Hold posts with header value matching a specified regexp."
Put the 3 lines below in there.

Content-Type: .*multipart

Content-Type: .*mixed

Content-Type: .*rich

> Again, this won't strip out the unwanted stuff but holds offending messages
for approval.

> Create a text message in your word processor or mail client that you can
paste into the explanation box when you reject the message telling your poster
that his message was rejected because he did not use plain text, and save it.
You'll need this over and over. ;-)

(see also FAQ 4.13 at <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq04.013.htp>, entitled "How do I prevent MIME attachments/HTML/Viruses being
sent to lists")

Edit this entry / Log info / Last changed on Wed Sep 8 13:00:48 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.21. How do I unsubscribe a member without them receiving an unsubscribe
notice?

In MM2.1 you'll find a switch to suppress the good bye message on the "General
Options" page

Edit this entry / Log info / Last changed on Fri Sep 26 17:49:31 2003 by Arthur
Teschler

-------------------------------------------------------------------------------

3.22. Deleted

Edit this entry / Log info / Last changed on Fri Jun 25 21:09:36 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.23. Deleted -- see FAQ 3.3

See <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.004.htp>

Edit this entry / Log info / Last changed on Tue Jun 15 03:18:25 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.24. Tracking down elusive subscriber bounces

Some bounce messages returned to Mailman lists as a result of the lists mailing
out to its subscribers do not correlate with the known subscribers to the list.
These bounce messages refer to mail addresses which are not subscribers.

One possible cause of these bounce notices is the rewriting of mail aliases
subscribed to the list, to which mail has been sent by the list, by some MTA
handling that outgoing mail.

For example, when someone leaves a job, arrangements may be made for their
email alias to be rewritten to some other mail alias. Or arrangements may be
made for copies of someone's email to be directed to a work colleague.

Problems arise if the mail address following rewriting is then undeliverable.
The bounce message that is generated by the MTA that fails the delivery refers
to the rewritten mail address not the original mail address to which Mailman
sent the mail. As a consquence, unless the return path to Mailman has been
VERP'ed, Mailman cannot correlate the bounce message with any of the list
subscribers and can do no more than log the problem in its bounce log.

See http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq02.002.htp for
information about VERP if you are not familiar with it.

The source and cause of these bounces can be a real pain to track down.

One approach you can use, with MM 2.1.x, to track down elusive bounces is to:

1. check that your site admin has configured Mailman so that you can enable
personalization for a list and that personalized mail is VERP'ed. If they have
not then the technique described below will not work.

2. temporarily enable personalization and VERP for a list with bounce problem.

3. set a maximum bounce score < 1.0 for the list.

4. ensure the list is configured to notify the list owner when bounces cause
unsubscribe.

5. send a test message to the list.

6. see what the VERP'ed return addresses are on the bounce notifications of the
users unsubscribed because of bounces.

If the address in the VERP'ed return path is not the same as the email address
referred to by the MTA generating the bounce then it says that the problem
discussed above has been the likely cause of your elusive bounces problem.

btw: Do not forget to disable personalization and VERP for the list unless you
can affored the extra resource use needed to support continued use of these
features.

Without VERP there is no simple answer to tracking this sort of problem down
with MM 2.0.x or 2.1.x

Edit this entry / Log info / Last changed on Tue Aug 5 00:20:58 2003 by Richard
Barrett

-------------------------------------------------------------------------------

3.25. How do I remove subscribers from a list and put them on a different list?

There are situations where you want to move subscribers off of one list and
replace them with other subscribers. Or there are situations where you add a
number of people to your new list and you realize you added the wrong list.

It might be easiest to remove everyone and start over. Unfortunately, there is
no button that allows you to do that from mailman. But with the help of your
system administrator (and about 15 seconds of their time), you can clear out
your subscribers list.

For simplicity I have divided this FAQ entry into three sections:

1) How to get a copy of the list of subscribers on the old list.

2) How to add those subscribers to a new list en masse.

3) How to remove all subscribers from a list.

I hope this is useful!

-r.c.

1) How to obtain the list of current members.

To do that, send a message to listname-request@listhost.org, and in the body of
the message specify:

who

Note that the list must be set to allow list members to view the list. If you
have set it to have only the administrator view the list, Mailman is overly
cautious and will not mail out the subscriber list, even if this command is
coming from an administrator.

2) To add the subscribers to a different list, just copy and paste the names of
subscribers (there is no limit, and adding 1000 names at a time seems fine)
into the appropriate box in the membership management page of mailman. It might
be a good idea to suppress your standard "welcome" message and just send out a
regular message to the list to let everyone know of the change.

3) To remove the subscribers from your list, you will need to have access to
the machine on which mailman runs. On that machine, you have to run the
sync_members command. Now, if you don't actually need to use the old list right
away you can just disable automatic subscription to the list. Go to the
"privacy options" page and set the second option, what steps are required to
subscribe, to "confirm+approval" or just "approval" so that as list owner, you
can intercept these attempts and redirect people. And another setting you might
want to modify is the "unsubscribe" message. There does not seem to be a way to
turn unsubscribe messages off completely, but at least you can change the
message to let people know the unsubscribe is really just a migration.

Anyway, getting back to the work of actually removing the subscribers... here
is what your admin should do to clear out the old list:

./sync_members [options] -f file listname

So that you can actually specify a replacement for your list. (Using this
command, you could actually maintain your mailing list with a piece of desktop
software, and have a programmer create scripts to automatically upload the
people who are going to get your e-newsletter.)

Getting back to the point of this faq, what your admin has to do to wipe the
list clean is just leave out the file name:

admin@yourhost: cd /var/lib/mailman

admin@yourhost: bin/sync_members -f - <nameof list to change>

^D (i.e. you just type Control-D on your keyboard)

So when you leave out the file name, mailman waits for you to type the file,
and pressing Control-D means you have not specified any file, so the list will
be "sync'd" with a blank file. Then you will see a series of unsubscribe
messages slowly scrolling:

Removed: padnet@xxxx.net ( padnet@xxxx.net)

Removed: cataldo@xxxtell.com ( cataldo@xxxtell.com)

Removed: scott@xxm.org ( scott@xxm.org)

Until the list is cleared out and has zero subscribers.

Edit this entry / Log info / Last changed on Tue Aug 26 03:51:20 2003 by rich
cowan

-------------------------------------------------------------------------------

3.26. Is there an easy way to discard all messages waiting to be reviewed?

If you have a list where non members can post but their messages are moderated,
you'll have to deal with a lot of spam. After sometime, just a small percentage
of them will be good posts. It's boring to click to discard each messages. A
better solution is to approve the good ones, and use a javascript link to mark
to discard all.

First, open your bookmark manager and add the following link as an URL:

 javascript:var el=document.forms[0].elements;for (var i=0; i<el.length;i++) if (el[i].type=='radio' && el[i].value==3) void(el[i].checked=true); 

Now visit the moderation page, and visit the bookmark. All the messages will be
marked to discard, just press the button.

I've tested it just with mozilla, but it should work in all browser versions
that support javascript links.

For Mailman 2.1.5, we find the following update which is mentioned in the
mailman-2.1.5/NEWS file:

    - The admindb page has a checkbox that allows you to discard all held
      messages that are marked Defer.  On heavy lists with lots of spam holds,
      this makes clearing them much faster.

Edit this entry / Log info / Last changed on Thu Jun 17 23:48:02 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.27. Who receives mail to -owner and -admin?

The administrative options for Mailman mailing lists include fields for "owner"
and "moderator" email addresses. It would make sense if mail to -owner would go
to the "owners" but since subscription requests go to that address, I suspect
it includes moderators. So, does mail to -owner go to "owners", "moderators",
or both? What about -admin?

Along the same lines, if I put an address in the "owner" field, is there any
point in putting it in the "moderator" field, or is that completely redundant?

Edit this entry / Log info / Last changed on Thu Sep 4 08:15:06 2003 by Michael
Askew

-------------------------------------------------------------------------------

3.28. What are all the "special" email addresses used for a list

-request Users send commands to this address to subscribe, get help, or perform
other functions

-admin AnswerMe

-owner AnswerMe

Are there any others?

Edit this entry / Log info / Last changed on Wed Sep 8 17:45:41 2004 by George
Dinwiddie

-------------------------------------------------------------------------------

3.29. Deleted

Edit this entry / Log info / Last changed on Thu Dec 4 13:48:01 2003 by Joan
Mize

-------------------------------------------------------------------------------

3.30. Deleted

Edit this entry / Log info / Last changed on Thu May 20 14:24:01 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.31. What if I don't want to save any messages for the moderator?

For Mailman 2.1.5, go to the "Privacy options..." page for the list in
question, then down to the "Sender filters..." sub-page.

At the top of this page, there is an option for what you want to happen by
default to messages that come from users that have their moderation bit set
(member_moderation_action). You most likely want this to be set to "Hold".

At the bottom of this page (the second from last option), there is a setting
for the action to take for postings from non-members for which no explicit
action is defined (generic_nonmember_action). Set this to "Discard".

The last option on this page is "Should messages from non-members, which are
automatically discarded, be forwarded to the list moderator?"
(forward_auto_discards). You want to set this to be "No", so that you aren't
bothered with all this spam or whatever.

Edit this entry / Log info / Last changed on Thu Jun 17 23:54:01 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.32. What's the regex syntax for header filter rules (in privacy options/spam
filters)?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.033.htp>

Use this regexp to match messages with the word "Phentermine" (a well-known
spam word) in their subject:

Subject: .*Phentermine

_Don't_ try to match the whole line by enclosing your pattern between beginning
of line anchor ('^') and end of line anchor ('$'). E.g. this won't match any
message (contrary to what you'd expect):

^Subject: .*Phentermine.*$

Edit this entry / Log info / Last changed on Tue Jun 29 17:47:14 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.33. How do I accept all addresses from a particular domain?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.032.htp>

Q: If I want postings from all non-members at a particular domain to be
accepted, how do I go about doing that without adding them each individually?

A: It's in the instructions in the list admin interface. Go to the Privacy
Options > Sender Filters. For "List of non-member addresses whose postings
should be automatically accepted." it says:

Add member addresses one per line; start the line with a ^ character to
designate a regular expression match.

For Python RegEx see: http://www.python.org/doc/current/lib/module-re.html

Well, here's an example:

  ^[^@]+@example\.com$

This tells Mailman to accept Emails received by any sender from example.com -
no matter if the one's subscribed or not.

Edit this entry / Log info / Last changed on Tue Jun 29 17:47:52 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.34. How do I spoof-proof my one-way (announcements or newsletter) list?

If you've had problems with virus-generated messages with spoofed senders
getting through to a one-way list (we have), you can completely spoof-proof
your list by requiring Web-based approval of every message even if it is sent
by the list's moderator (the author, in case of a one-way newsletter).

Under Privacy Options, Recipient Filters, set max_num_recipients to 1. This
will cause every message posted by the moderator to require approval via the
Web (Reason: too many recipients)

If there's a hole in this logic somewhere, please let us know.

Edit this entry / Log info / Last changed on Tue Mar 16 13:27:09 2004 by 
Prairienet Mailman Site Admin

-------------------------------------------------------------------------------

3.35.

A message sent to a List by a non-subscriber and was held for administrator
review along with many other "spam" email messages. I selected Accept for this
particular message since it was something that should legitimately be posted to
the list, so that's what I chose. As in the past, after choosing Accept, and
closing the session by clicking the Submit All Data button, I expected the
message to be sent to the list, but that did not happen. This has happened a
couple of times to this one Specific List. Do you have any explainations/
suggestions?

Edit this entry / Log info / Last changed on Wed Apr 14 14:29:10 2004 by Linda
Sheldon

-------------------------------------------------------------------------------

3.36. Processing old mbox archives with procmail/formail

My mailman-2.1.4 breaks my old mbox archives when processing them into html
(pipermail). As a result, some messages get merged together. Often those HTML
versions have no Subject line etc. It's a mess.

It helps a bit to cleanup the old archives. Get procmail from http://
www.procmail.org and install it. You will get procmail(1) and formail(1)
binaries.

Create /usr/local/mailman/etc/clean-mbox-file.rc as follows (5 lines):

  :0fh
  #
  # formail -b -s procmail -m /usr/local/mailman/etc/clean-mbox-file.rc < \
  # /usr/local/mailman/archives/private/$listname.mbox
  #
  # You will receive cleaned up old archive file in
  # $HOME/new-mbox-archive.mbox. The original file is left untouched.

  :0fh
  |formail -Y -q- -I"Posted-Date:" -I"Received-Date:" -I"Old-From:" \ 
  -I "Old-To:" -I "Old-Subject:" -I "Old-Date:" -I "Old-Reply-To:" \
  -I"X-Mailer:" -I"Received:" -I"Status:" -I"X-Status:" \
  -I"X-MIME-Autoconverted:" -I"Organization:" -I"Sender:" \
  -I"Reply-To:" -I"List-Id:" -I"List-Unsubscribe:" -I"List-Archive:" \
  -I"List-Post:" -I"List-Help:" -I"List-Subscribe:" \
  -I"X-List-Received-Date:" -I"Errors-To:" -I"X-Errors-To:" \
  -I"X-Listname:" -I"Listname:" -I"Status:" -I"Content-Length:" \
  -I"Lines:" -I"X-Loop:" -I"X-Mailing-List:" -I"List-Info:" \
  -I"X-Info:" -I"Precedence:"

  # http://www.natur.cuni.cz/~mmokrejs/procmail/mobiles
  # to be sure that quoted-printable in headers got converted to 8 bit
  # De-mangle RFC 2047 header mangling
  :0Hhfw
  * =\?[^?]+\?[qb]\?[^?]+\?=
  |/usr/local/bin/dmmh

  #
  # Fix dates like below
  #From cita@xxxxx.cz  Thu Jan 20 03:27:35 2000
  #From: Ctirad Hruby <cita@xxxxx.cz>
  #Date: Thu, 13 May 1999 08:40:57 +0200

  # convert the date extracted from the From_ line to Date: format
  # From line: Fri Jan 14 11:22:23 2000
  # Date: line: Fri, 14 Jan 2000 11:21:31 +0100
  :0
  * ^From +[a-zA-Z0-9_@.-]+[       ]+\/[^  ].*
  {
    FR1=$MATCH
    FR11=`echo $FR1 | perl -e 'while (<>) { $l=$_; $l =~ m/^(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+/; print "$1, $3 $2 $5 $4\n";}'`
  }

  :0
  * ^Date: +\/[^ ].*
  { FR2=$MATCH }

  :0
  * ^Date: +\/[a-zA-Z\,]+
  { OVERLAP=$MATCH }

  #
  # TODO
  # figure out both dates differ significantly, if yes then recreate
  # Date:
  #
  # I'm lazy now, so I'll blindly overwrite the Date: field with the
  # date extracted from the From_ line. This is particularly bad as an   
  # email sent just before midnight might end up dated now on next day
  # (as it could have been delivered on mailserver just after midnight
  # ... and if *that* very next day was in new month we shift the
  # message mistakenly ...).
  #
  :0f
  | formail -I "Date: $FR11"

  # the next line will store all emails into a file in current directory
  :0:
  mylistname

  # the next line will store all emails into a file in $HOME/
  #:0:
  #$HOME/new-mbox-archive.mbox

Then run command (one line):

  $ formail -b -s procmail -m /usr/local/mailman/etc/clean-mbox-file.rc <
  /usr/local/mailman/archives/private/$listname.mbox

You will receive cleaned up old archive file in your home directory called
mylistname or in $HOME/new-mbox-archive.mbox. The original file is left
untouched. Append/prepend the archive file to

  $prefix/archives/private/$listname.mbox/$listname.mbox.

Would be nice if someone would describe how to define procmail as an external
archiver. It could be used to mask email addresses in raw archives, execute
pipermail/MhonArc/glimpse, just any program after each email archived.

Edit this entry / Log info / Last changed on Tue Apr 20 11:19:53 2004 by Martin
Mokrejs

-------------------------------------------------------------------------------

3.37. I want to do extensive customization of outgoing messages -- can Mailman
do this?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.018.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.040.htp>.

Mailman is a mailing list management system. The general function is to take
messages that are submitted, pass them through some filtering rules, then send
out the message to all subscribers -- assuming it passes inspection, and
perhaps with certain parts of the message removed for the benefit of mailing
list members. Mailman has some limited functionality to allow the list-owner
the option of customizing a few bits of information as those messages pass
through the system, but that's about it.

If you're interested in what Mailman can do in terms of customizing outgoing
mailing list messages, see <http://www.python.org/cgi-bin/faqw-mm.py?req=show&
file=faq02.002.htp>, <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq04.012.htp>, <http://www.python.org/cgi-bin/faqw-mm.py?query=template&
querytype=simple&casefold=yes&req=search>, and <http://www.python.org/cgi-bin/
faqw-mm.py?query=filter&querytype=simple&casefold=yes&req=search>.

However, Mailman does not have any hooks to any back-end database systems, nor
does it have any features that would allow you to customize large parts of an
outgoing message. If you want to do that, you need a proper mail-merge
management system, which is closely related to a Customer Relations Management
system (see the FAQ entry mentioned above).

Try going to www.google.com and search for "mail-merge management system" to
see what solutions are available that are likely to be more appropriate to your
needs.

On the other hand, if you'd like to add these kinds of features to Mailman, we
encourage you to join the mailman-developers mailing list and work with us to
add this kind of functionality. To quote Richard Barrett:

 MM is free, Open Software; so go for it. I am sure a well implemented enhancement
 will find other grateful users.

Edit this entry / Log info / Last changed on Wed Aug 11 13:10:43 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.38. Why am I receiving moderation requests that read ...mailing list has -1
request(s) waiting... ?

A newly created mailman 2.1.5 list will email bogus moderator requests until a
genuine one has been handled. The email posted says:

 "Subject: -1 LISTNAME moderator request(s) waiting

 The LISTADDRESS mailing list has -1 request(s) waiting 
 for your consideration at:
        WEBADDRESS
 Please attend to this at your earliest convenience.  This notice of 
 pending requests, if any, will be sent out daily."

The cause is that request.pck is either missing or it doesn't contain the
version info.

The workaround is simply to create a real moderation request (which
automatically creates a request.pck) or to copy a valid request.pck file over
from another list.

There's also a patch available at sourceforge, which should fix this. http://
sourceforge.net/tracker/index.php?func=detail&aid=970383&group_id=103&atid=
300103

You can also fix it by hand:

 $ bin/withlist -l mylistwithoutrequests
 >>> print  m.NumRequestsPending()
 -1
 >>> m.Save()
 >>> m.Unlock()
 >>> {ctrl-D}

See: http://mail.python.org/pipermail/mailman-users/2004-June/037330.html

Edit this entry / Log info / Last changed on Thu Aug 26 11:06:18 2004 by Enrico
Ardizzoni

-------------------------------------------------------------------------------

3.39. How do I protect my mailing lists against viruses?

From the thread on the mailman-users mailing list at <http://mail.python.org/
pipermail/mailman-users/2004-July/038196.html>:

From <http://mail.python.org/pipermail/mailman-users/2004-July/038201.html>,
one option is to make the entire list moderated, so that only posts which are
approved by the moderators are accepted by mailman. However, this creates a
heavy load on the moderators.

From <http://mail.python.org/pipermail/mailman-users/2004-July/038202.html>,
another option is to set the maximum allowed message size something really low,
such as 15KB. Most viruses are unlikely to be this small, so messages which
pass this check are unlikely to be infected, and the moderators can always go
in and manually approve the larger messages.

You can also choose to strip all types of attachments, save a few specific
types which are known to be safe. Look under the "Content Filtering" section
for the web administration page for your mailing list. However, this only works
for attachments that are in MIME format, and would not catch attachments that
are "uuencoded", for example.

One last option is to implement anti-virus scanning software on your mail
server, so that these things are caught and dealt with before they ever get the
chance of getting to Mailman.

The information at <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq06.012.htp> is oriented towards using amavisd and SpamAssassin in an
anti-spam configuration, but you can also use amavisd to tie into anti-virus
scanning solutions, such as ClamAV. Of course, you can do similar things with
other MTAs (e.g., sendmail, exim, etc...) or other scanning solutions.

Edit this entry / Log info / Last changed on Thu Jul 22 03:05:31 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.40. I want to completely customize the look and feel of Mailman -- how do I
do that?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.018.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq03.037.htp>.

Mailman gives the administrator a limited degree of control over the "look and
feel" of the user interface, by allowing you to edit the HTML templates of
certain web pages. However, some other parts are generated internally, and
short of changing the Python source code, you have no control over what those
pages will look like.

There is no site-wide header or footer that can be modified, nor is there any
way to remove the trailer at the bottom which includes a link to the admin
interface. There are also no facilities to use CSS, or to incorporate Mailman
output within a frames-based page layout for your site.

If you'd like to add these kinds of features to Mailman, we encourage you to
join the mailman-developers mailing list and work with us to add this kind of
functionality. To quote Richard Barrett on a related subject:

 MM is free, Open Software; so go for it. I am sure a well implemented enhancement
 will find other grateful users.

See also the threads on the mailman-users mailing list at <http://
mail.python.org/pipermail/mailman-users/2004-August/038605.html> and <http://
mail.python.org/pipermail/mailman-users/2004-August/038612.html>.

Edit this entry / Log info / Last changed on Wed Aug 11 13:14:54 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.41. What variables can I use in the headers and footers added to messages by
Mailman?

real_name

    This is the value of the real_name configuration variable
    in the General options category.

list_name

    This is the canonical name of the mailing list.  In other words
    it's the posting address of the list.

        Note: For backward compatibility, the variable _internal_name is
    equivalent.

host_name

    This is the domain name part of the email address for this list.

web_page_url

    This is the base url for contacting the list via the web.  It can
    be appended with listinfo%(list_name)s to yield the
    general list information page for the mailing list.

description

    The brief description of the mailing list.

info

    This is the full description of the mailing list.

cgiext

    This is the extension added to CGI scripts.  It might be the empty
    string, .cgi, or something else depending on how your site
    is configured.

---

Each of these is used in the form %(fieldname) where fieldname is one of the
names given above.

For example, the simple footer that gives the mailing list name, the posting
address, and the listinfo address is as follows:

   _______________________________________________
   %(real_name)s mailing list
   %(real_name)s@%(host_name)s
   %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

Edit this entry / Log info / Last changed on Tue Aug 31 21:37:54 2004 by Terri
Oda

-------------------------------------------------------------------------------

3.42. Help!!! All messages from my list are being rejected by AOL as spam, or
being silently thrown away!

AOL has a tendency to declare that sites are sending them spam and to start
silently throwing all messages away that come from that site, sometimes on the
basis of a single spam complaint from a single user.

Sometimes, that user is a legitimate member of a mailing list from that site,
and mistakenly clicked on the wrong message and clicked the "report as spam"
button. Sometimes that user doesn't remember subscribing to it, so they report
the message as spam. Or, sometimes the user doesn't remember how to
unsubscribe, so they report the message as spam.

There's not really anything you can do about this. You are subject to the whims
of the clueless AOL users and the occasional fat-finger mistake.

The best you can do is apply to be a member of the AOL whitelist system. See <
http://postmaster.aol.com/tools/whitelist_guides.html>.

Please note that AOL has a "Feed Back Loop" form that you can fill out (see <
http://postmaster.aol.com/tools/fbl.html>), allowing you to see all spam
reports that are generated for given IP addresses. If you run a large network,
it would probably be a very good idea to register your IP addresses and get
reports sent back to you regarding your mail servers.

Edit this entry / Log info / Last changed on Tue Sep 7 16:03:33 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.43. DELETED

Edit this entry / Log info / Last changed on Sun Sep 5 16:52:44 2004 by Brad
Knowles

-------------------------------------------------------------------------------

3.44. How can I Mass Subscribe a list with real names *

The list for a mass subscribe can contain lines in any of the following
formats:

  jdoe@example.com
  <jdoe@example.com>
  John Doe <jdoe@example.com>
  "John Doe" <jdoe@example.com>
  jdoe@example.com (John Doe)

The last three all associate the real name "John Doe" with the subscription.
The formats can be intermixed in one mass subscribe.

Edit this entry / Log info / Last changed on Wed Sep 15 17:56:04 2004 by Mark
Sapiro

-------------------------------------------------------------------------------

4. Site administrator tasks

-------------------------------------------------------------------------------

4.1. How to get rid of the List-* headers?

These headers are recommended and defined by RFC 2369 (<http://www.faqs.org/
rfcs/rfc2369.html>) and should not be removed. Supporting MUAs will read the
RFC 2369 headers and add controls for users to easily subscribe, unsubscribe,
ask for help from the list owner, check/see the description of the list, etc.

For example the MUA I use (J C Lawrence <claw@kanga.nu>), exmh (<http://
www.beedub.com/exmh/>), will add a new menu containing List help, subscribe,
unsubscribe, archive, and post commands when a message containing RFC 2369
headers is viewed.

Note: Some MUAs, most notably Eudora, will display the headers when the message
is open, but take the appropriate action when the user clicks on the links.
Some users may consider this to make their display cluttered and less readable,
however in the case of Eudora this was an explicit design decision.

By default Eudora shows extra headers, including list headers in open messages,
but not in the preview pane. Unless you WANT to see the headers, there is
really no reason to open messages. If you really wants to read from open
messages rather than with preview, there is a way to enable this behavior.

To suppress display of the headers under Eudora on Windows you need to add the
header names to the TabooHeaders or "Boring Headers" option. Try a Google
search for "TabooHeaders" for more information on how to do this, or see the
Eudora FAQ item <http://www.eudora.com/techsupport/kb/2241hq.html> and <http://
www.eudora.com/techsupport/kb/2186hq.html>.

For Eudora 6.0 on MacOS X, you can make use of the "Esoteric Settings" plug-in
which is shipped with Eudora, but is turned off by default. To do this, see the
instructions at <http://www.eudora.com/techsupport/kb/1779hq.html>. Once
restarted, go to the "Special" menu, and select "Settings...". Scroll down to
the "Boring Headers" icon, and click on it. In the field provided, add the
appropriate "List-" headers that you do not want to see any longer.

For older versions of Eudora on the Macintosh, you can also use SuperSleek. It
is no longer developed/supported, but it still works and it can be downloaded
from <http://dradul.tripod.com/binaries/super-sleek-10.hqx>.

While not recommended, Mailman 2.1 supports suppressing all the RFC 2369
headers list-wide. See the admin interface under the General Options category,
specifically the include_rfc2369_headers option. MM2.1 also supports removing
just the List-Post header for announce-only or one-way lists.

Edit this entry / Log info / Last changed on Sat Jul 10 17:47:11 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.2. Which MTA should I use with Mailman?

The following MTAs are commonly used with Mailman:

  Courier: http://www.courier-mta.org/
  Exim:      http://www.exim.org/
  Postfix:   http://www.postfix.org/
  Qmail:     http://cr.yp.to/qmail.html
  Sendmail:  http://www.sendmail.com/

This doesn't mean that other MTAs can't be used or that they won't work.
Mailman is MTA agnostic and will generally support anything that understands
standard SMTP and can deliver a message to a local program.

As for which MTA to use? That is really a local site decision and should be
governed by local expertise, and performance and security requirements.
However, some commentary may help:

Courier: courier is a new player when compared to other MTA's. Courier is an
MTA that contains the standard SMTP server, as well as IMAP, POP3, webmail, and
other features. This turns out to be quite an advantage as far as integrating
these services is concerned. All of the tools use the same databases for
authentication, read and write the same mailboxes, etc. The documentation is
relatively good, and configuration can be done with little difficulty. If you
don't want to edit configuration files, Courier also comes with a tool called
webadmin to help with this. The performance and security of Courier is not as
well tested as with other MTA's because of its young age. Also, if you are
using mailboxes in the standard mbox format rather than maildirs, it is not
recommended that you use Courier because some of its features don't operate
properly with mbox.

Exim: Of all the commonly used MTAs Exim is probably the easiest for
non-experts to setup and configure. The configuration file is easily read and
understood by non-experts, the documentation is unusually clearly written and
thorough, and the author and user community actively support and help users.
Exim is also among the most configurable and easily extended of all MTAs. Exim
makes it easy to do very clever and strange things with mail systems. Nigel
Metheringham has written a particularly good HOWTO (http://www.exim.org/howto/
mailman.html) on configuring Exim for Mailman. Exim is highly recommended for
smaller sites, people new to mail systems, and sites that need strange/unusual
mail configurations. Note: Exim is the default MTA for Debian/Linux.

Postfix: Postfix aims to be an extremely secure and fast MTA. It is built on
the principle of many small, simple, easily reviewed (for security concerns)
programs that don't trust each other but operate together to process mail.
Postfix is an unusually fast MTA, has an excellent security history, an active
and helpful developer, and with a very highly commented configuration file and
lots of reasonable built-in defaults, and is one of the easiest MTAs on the
planet to setup and configure (experts are split on whether exim or postfix is
easier in this regard). Postfix is highly recommended for sites that have
security concerns, or high performance requirements (though Exim's performance
curves are quite similar). Postfix is one of the only other MTAs available that
actively tries to be a drop-in replacement for sendmail, working with the same
command names and command-line options, etc.... If you have been using sendmail
and are looking for a replacement, postfix is likely to be the easiest and
least painful way to make that change. Postfix also has many default features
that make it extremely well suited to handling mailing lists with
out-of-the-box configurations, such as hashed mail queue directory structures,
parallel delivery, bounded exponential backoff on errors (so that you don't get
a "thundering herd" of servers that all try to bury a particular remote site
right after it comes back up), etc.... Glenn Graham has an excellent article on
postfix entitled "Postfix: A Secure and Easy-to-Use MTA" (http://www.onlamp.com
/lpt/a/4099/). Note: Postfix is the default MTA for Mandrake/Linux.

Qmail: Qmail is the dark horse. Fast, stable, good security history, similar
architecture to Postfix (in some ways Postfix was inspired by Qmail), funky
licensing (it is NOT Open Source), highly abrasive author, very odd/
non-standard configuration. Qmail can also be, umm, entertaining to integrate
with other external mail systems (such as Mailman). It does not follow commonly
used standards and patterns.

Sendmail: The old man on the block and the default MTA for most commercial
Unixes. Probably the closest thing you'll ever see to a totally unreadable
gibberish config file and the documentation is not a whole lot better.
Historically, Sendmail has had a poor security history and quite horrible
performance. Recent versions of Sendmail have significantly improved both
problems, bringing Sendmail moderately close (if not quite matching) to other
more modern MTAs like Exim and Postfix. Due to its unreadable configuration
file, default performance and security history Sendmail is not generally
recommended unless you happen to already have a local Sendmail expert.

However, sendmail is the default in many flavors and distributions of Unix-like
operating systems, and may be the easiest to implement simply because it's
already there. Configuration has been made quite easy when done properly
through m4 instead of sendmail.cf, and many graphical tools are available for
intuitive configuration.

Moreover, with appropriate work, sendmail can be configured to be the fastest
and highest-volume general-purpose MTA on the planet. This takes knowing what
you're doing, making suitable configuration changes, and daily
care-and-feeding, but if you want to run one of the largest mailing list
systems in the world, sendmail is likely to be your only feasible choice.

Edit this entry / Log info / Last changed on Mon Jul 26 05:52:22 2004 by Joseph
c. Lininger

-------------------------------------------------------------------------------

4.3. Mailman's internal archiver (Pipermail) doesn't understand MIME! What can
I do?

Mailman 2.0.* uses a version of Pipermail to archive messages that doesn't
understand MIME. If upgrading to Mailman 2.1.* is not an option, there's not
much you can do about this other than using an external archiver.

Suggested external archivers:

  MHonArc:  http://www.mhonarc.org/
  HyperMail: http://www.hypermail.org/

MHonArc is probably the most commonly used, and is certainly the most flexible
and configurable of the current Open Source mail archivers.

Example MIME messages archived under MHonArc:

  http://www.kanga.nu/archives/MUD-Dev-L/1998Q2/msg00854.php
  http://www.kanga.nu/archives/MUD-Dev-L/1998Q2/msg00893.php
  http://www.kanga.nu/archives/MUD-Dev-L/1998Q2/msg01256.php
  http://www.kanga.nu/archives/MUD-Dev-L/2000Q3/msg00661.php

One of the other significant advantages of MHonArc is that it properly
understands national charactersets and will generate the HTML the properly
renders and displays messages written in national charactersets. This can be
particularly valuable/important for non-english lists and authors.

Edit this entry / Log info / Last changed on Mon Aug 4 12:18:20 2003 by 
Jonathan Ah Kit

-------------------------------------------------------------------------------

4.4. How can I use an external archiver with Mailman like MHonArc?

There are two basic ways to use an external archiver with Mailman:

1) You can configure Mailman to use the external archiver instead of the
internal pipermail.

2) You can subscribe an address to each list you want archived which archives
all mail sent to it.

Configuring Mailman to use an external archiver instead of the internal
Pipermail is slightly finicky. A HOWTO for this was posted to the
mailman-developers list and should be copied in here once someone finds it.

The MHonArc mailing list also had a copy posted:

  http://www.mhonarc.org/archive/html/mhonarc-users/2001-10/msg00021.html

One (possible) arrangement for using an archiving address is:

-- First setup an account which is subscribed to the various lists you want to
archive.

-- Next setup the scripts so that mail received on that address is passed to
MHonArc (don't bother with MHonArc RCs at this point) such that the messages
are archived into the directories etc as you wish. I built/automated most of
this section with various procmail recipes in a way that's not portable or
particularly useful to anyone else (I stash all messages locally in MH folders
and then run MHonArc against those folders etc). You'll need to build something
that fits what you want to do.

-- Finally, build RCs that archive the messages in the manner you want and plug
those RCs into your MHonArc invocation in the scripts you just wrote.

Or more specifically:

1) Subscribe archiver@yourdomain.com to the list. 2) Configure your MTA dn 
archiver@yourdomain.dom to use procmail as the LDA 3) Write a procmail recipe
which filters out the list posts (and ignores SPAM, password reminders, etc)
and then invokes your archiver (eg MHonarc) on the message.

J C Lawrence (claw@kanga.nu) uses a slight variation on this arrangement. His
notes:

As archiving is CPU and disk I/O expensive I don't do it at the time the
message is received, but via a cron job. This reduces system contention and
allows work to be scheduled when the system is less busy.

Some documentation and references on my system:

My entire setup except for the procmail recipes can be found here:

  ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives/

In particular the following file does the automatic index generation:

  ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives/PHP/listroot.php

As seen (for instance) here:

  http://www.kanga.nu/archives/MUD-Dev-L/

My procmail recipes save to MH folders, so the make* scripts under here:

  ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives/scripts/

do the work of building the MHonArc archives off those MH folders, and:

  ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives/scripts/moveQuarters

is run via cron and does the symbolic link swizzling for the various archiving
periods (trivial to change for months, week, or other periods)..

As for what it all looks like when running, see the various pages under here:

  http://www.kanga.nu/archives/

With the cannonical example of:

  http://www.kanga.nu/archives/MUD-Dev-L/

The above setup is PHP4 specific and needs the template supports from PHPLIB.
The main reason for using the PHP aspects are that you get the ability to reply
to archives messages on the web.

An example of a message, posted via the web-reply feature and then archived by
the above code:

  http://www.kanga.nu/archives/MUD-Dev-L/2001Q3/msg00745.php

If you need a setup which isn't reliant on PHP (and you don't need/want the
ability to reply to list messages on the web), you can find my older RCs here:

  ftp://ftp.kanga.nu/pub/Kanga.Nu/WebArchives.old/

No PHP required and generate a very similar look'n'feel to my current setup.

Edit this entry / Log info / Last changed on Tue Jul 22 07:52:14 2003 by James
Cameron

-------------------------------------------------------------------------------

4.5. Why don't the footers show up on some list messages? (Mailman 2.0.x)

For Mailman 2.1.x, see also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&
file=faq04.039.htp>.

This is a known problem and limitation with Mailman 2.0.*. Mailman 2.1.* fixes
this area.

Under Mailman 2.0.* footers are added by appending a text block to the end of
each message. This works well for plain text messages. It does not work for
MIME messages as the footer does not lie within the MIME structure of the
message.

Edit this entry / Log info / Last changed on Wed Jun 16 01:08:36 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.6. How can I backup my Mailman lists/membership/etc on a regular basis?

You should of course have regular, reliable, and tested site backup procedures
for all your data, not just your Mailman lists. However, there are advantages
to making and/or distributing backups of your list configurations in other and
possibly more flexible and frequent manners than your site backup procedures.

J C Lawrence <claw@kanga.nu> uses the following approach:

Note: You'll need to have a reasonably recent version of nmh installed as the
scripts rely on the MH tools to build a MIME message:

  http://www.nongnu.org/nmh/

The scripts:

  ~/bin/mimemail:
    --<cut>--
    #!/bin/bash
    #set -x

    subject=$1
    file=$2
    addr=$3

    echo "To: ${addr}
    From: nobody
    Subject: ${subject}

    #application/octet-stream [${subject}] ${file}

    " | /usr/bin/mh/mhbuild - > /tmp/mimemail.tmp.${$}
    /usr/lib/mh/post -verbose -watch /tmp/mimemail.tmp.${$}
    rm /tmp/mimemail.tmp.${$}
    --<cut>--

  ~/bin/mailman.backup
    --<cut>--
    #!/bin/bash
    #set -x

    datestr=`date +%Y%m%d`
    host=`hostname -f`
    file=mailman.lists.${datestr}.tar.gz
    filepath=~/backups/${file}
    sendto="root"

    cd /var/lib/mailman
    tar zcf ${filepath} lists
    cd ~/backups
    mimemail ${file} ${filepath} ${sendto}
    #rm ${file}
    --<cut>-

Yeah, there are some hard coded paths in there. Sue me. I never said they were
pretty, merely that they worked. Brief explanation:

  mimemail takes three arguments: the Subject: of the message its
  going to send, the file it is going to send, and the address it is
  going to send it to.  mimemail depends on the nmh tools to build
  the MIME message.

  mailman.backup takes no arguments.  It creates a compressed
  tarball of ~mailman/lists in the file
  ~/backups/mailman.lists.YYMMDD.tar.gz

    eg: mailman.lists.20011128.tar.gz

  And then uses mimemail to send that file to root@localhost
  (edit/change if you want) before deleting it.

To use drop them in a cronjob something like:

  0 6 * * 0 /home/archiver/bin/mailman.backup

A sample execution should look something like:

  $ ./mailman.backup
   -- Posting for All Recipients --
    -- Local Recipients --
    root: address ok
   -- Recipient Copies Posted --
  Message Processed

The result will be that every time the cronjob runs it will send you (in this
case root@host.dom) a message looking something like:

  Subject: mailman.lists.20011128.tar.gz
  From: nobody@kanga.nu
  Date: Wed, 28 Nov 2001 21:12:55 -0800
  To: root@kanga.nu

  <MIME attachment application/octet-stream> 

Where the MIME attachment is the tarball constructed by ~/bin/mailman.backup.
Edit the value of sendto in mailman.backup if you want them going somewhere
else.

Edit this entry / Log info / Last changed on Mon Dec 1 00:16:23 2003 by Peter
Jaros

-------------------------------------------------------------------------------

4.7. How can I rotate my Mailman logs?

Eric Toran's logrotate tool is recommended:

  http://rpmfind.net/linux/redhat/code/logrotate/

LogRotate is installed by default with most Linux distributions. Debian/Linux
users are particularly well off in that the Debian Mailman package
automatically installs a LogRotate configuration wlong with Mailman.

You may encounter problems using logrotate with MM 2.1.x because the qrunners
run as daemons rather than cron jobs. They thus tend to hold open file
descriptors to log files for their (indefinite) lifetime. You may need to wrap
invocation of mailmanctl stopping and then starting the qrunners around the
invocation of logrotate in order to get things to work right. Well thats what I
found worked but if you have a better idea then update this FAQ page.

Example configuration stanza for Mailman for LogRotate (based on but not
identical to the Debian defaults):

  /var/log/mailman/bounce {
        weekly
        missingok
        create 0664 list list
        rotate 4
        compress
        delaycompress
  }

  /var/log/mailman/digest {
        monthly
        missingok
        create 0664 list list
        rotate 4
        compress
        delaycompress
  }

  /var/log/mailman/error {
        weekly
        missingok
        create 0664 list list
        rotate 4
        compress
        delaycompress
  }

  /var/log/mailman/post {
        monthly
        missingok
        create 0664 list list
        rotate 12
        compress
        delaycompress
  }

  /var/log/mailman/smtp-failure {
        daily
        missingok
        create 0664 list list
        rotate 7
        compress
        delaycompress
  }

  /var/log/mailman/smtp {
        daily
        missingok
        create 0664 list list
        rotate 7
        compress
        delaycompress
  }

  /var/log/mailman/locks {
        daily
        missingok
        create 0664 list list
        rotate 7
        compress
        delaycompress
  }

  /var/log/mailman/fromusenet {
        daily
        missingok
        create 0664 list list
        rotate 7
        compress
        delaycompress
  }

  /var/log/mailman/qrunner {
        daily
        missingok
        create 0664 list list
        rotate 7
        compress
        delaycompress
  }

  /var/log/mailman/subscribe {
        monthly
        missingok
        create 0664 list list
        rotate 12
        compress
        delaycompress
  }

  /var/log/mailman/vette {
        weekly
        missingok
        create 0664 list list
        rotate 4
        compress
        delaycompress
  }

Ok, Richard that's the long way. I use a shorter way :-)

File /etc/logrotate.d/mailman

/home/mailman/logs/* {

        weekly
        missingok
        rotate 52
        nomail
        compress
        delaycompress
        notifempty
        create 664 mailman mailman
        olddir /home/mailman/logs/old
        sharedscripts
        postrotate
           if [ -f /home/mailman/data/master-qrunner.pid ]; then \
                /etc/init.d/mailman restart > /dev/null; fi
        endscript

}

For the non-Linux folks, the included "newsyslog" in *BSD will do a similar
job.

Example /etc/newsyslog.conf entry:

/usr/local/mailman/logs/bounce mailman:mailman 640 5 1000 * Z /usr/local/
mailman/data/master-qrunner.pid

...and so forth for each log file.

Edit this entry / Log info / Last changed on Thu Jul 1 00:23:39 2004 by Charles
Sprickman

-------------------------------------------------------------------------------

4.8. How can I add Namazu as a search engine for my Mailman list archives?

See also the message in the mailman-users archive at <http://mail.python.org/
pipermail/mailman-users/2004-June/037580.html>, from Tom Morrison, dated Fri,
25 Jun 2004 09:45:23 -0500.

Subject: Re: [Mailman-Users] Mailman for maillist, namazu for search

  From: Philip S Tellis
  Date: Fri, 30 Nov 2001 14:13:40 +0530 (IST)
  To: <mailman-users@python.org>

Sometime on Nov 29, Jon Carnes assembled some asciibets to say:

  > Looks cool.  How did you do it, and what are the current problems
  > with the way it's setup/working?

How I did it:

Namazu

There's this product called namazu (http://www.namazu.org/). It's a search
engine that uses a full text index. Can index local paths, not remote paths.

Put the namazu cgi in the mailman/cgi-bin/ directory, and called it search.
Also had to hack namazu a bit to get it to read the index from the PATH_INFO
from the url (/linuxers/, /life/, etc.).

Also changed the namazu templates to look like mailman's listinfo pages.

The last thing was to index the pages.

So far, no changes were required to mailman, but the products aren't fully
integrated yet.

The only thing that isn't done is automatic reindexing everytime a new message
is added. I thought it may make sense to do the reindexing at the same time
that a new archive is created (ie, weekly or monthly), but couldn't find the
place to put that.

The next best idea I had was to put it the reindexing as a cron job. Problem
here is that there is a chance that the reindexer would run while a new message
was only partially written to disk.

Mailman

I also had to add either a link to the search page on the listinfo page, and a
search box or link on the archive pages. While this isn't up on the urls yet,
it is done by hacking Mailman/Archiver/HyperArch.py.

The real problem is getting the link to show up only on pages that have
actually been indexed.

How this can be fully integrated

A config variable stating whether the list needs to be indexed for searching,
and based on that, these links can be written to the archive pages. The
reindexer can be run through mailman or through cron, depending on what others
think is a better solution.

Philip

-- Darth Vader:

        The force is with you young Skywalker, but you are
        not a Jedi yet.

Edit this entry / Log info / Last changed on Fri Jun 25 17:01:17 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.9. Summary of the mailman/bin/ commands

This is a summary of the mailman commands available via the console in ~mailman
/bin/..

These commands are essential for building mailing lists and for backing up list
information (users, user options, list configuration). The commands also
greatly extend the functionality of Mailman.

===

add_members: add regular or digested users to a list.

arch: rebuild a mailing list's archives.

check_db: check a mailing list database for corruption.

check_perms: check the permissions on the Mailman installation.

clone_member: add a list member with identical settings as an existing list
member (including password).

config_list: examine or change list configuration from the command line.

digest_arch: convert majordomo archives into mailbox format. Old program, use
with extreme care!

dumpdb: dump the contents of a Mailman .db file.

find_member: find all lists that a specified user is on.

list_lists: list all the Mailman mailing lists.

list_members: list all the members of a mailing list.

mmsitepass: set the site password, good for admin-ing any list.

move_list: reset where the database looks for its files, if you change the name
of the dirs because you change the name of the list.

newlist: create a new mailing list.

remove_members: remove specified members from a list.

rmlist: remove an old mailing list - does not remove the archives unless you
specify -a.

sync_members: synchronizes mailing list membership with a flat text file.

update: upgrade from previous version of Mailman to current version.

version: print out the version of Mailman you are using.

withlist: advanced interactions with mailing list objects.

===

Other notable files:

paths.py - module used by many Mailman scripts to tell it where its files are
stored.

mailman/Mailman/Defaults.py - The default values for configuring Mailing lists.
Also controls how Mailman interacts with your MTA (sendmail, postfix,
qmail...), and how it interacts with your web server. Note: NEVER change this
file - copy any section you want to change into the file below and make the
changes there.

mailman/Mailman/mm_cfg.py - Your sites customized settings. Used to set the
defaults for any new mailing lists. It is highly recommended that you read
through Defaults.py before making any changes, but that you make any desired
changes in this file.

Edit this entry / Log info / Last changed on Tue Dec 4 02:28:57 2001 by Jon
Carnes

-------------------------------------------------------------------------------

4.10. How to use Mailman on a linux/grsec kernel

Subject: [Mailman-Users] Running mailman on a linux/grsecurity kernel

  From: Marc MERLIN
  Date: Tue, 4 Dec 2001 12:10:43 -0800
  To: mailman-users@python.org

  ...<deletia>...

----------------------------------------------------------------------------

Steps you need to take to run mailman on a linux grsecurity protected kernel

1) Install and run securelinux_fix.py -f

2) Grsecurity will not run binaries if the directory they're in is writable by
a user other than root:

   panoramix:/var/local/mailman/mail# chown root.mailman .
   panoramix:/var/local/mailman/mail# chmod 755 .
   panoramix:/var/local/mailman/mail# chattr +i .

3) Apply the same fix to the CGIs

   panoramix:/var/local/mailman/cgi-bin# chown root.mailman .
   panoramix:/var/local/mailman/cgi-bin# chmod 755 .
   panoramix:/var/local/mailman/cgi-bin# chattr +i .

Marc

----------------------------------------------------------------------------

Edit this entry / Log info / Last changed on Tue Dec 4 23:02:32 2001 by J C
Lawrence

-------------------------------------------------------------------------------

4.11. What about performance?

There are many aspects to tuning a Mailman system, but they essentially resolve
to:

  1) Mailman

  2) MTA

  3) Supporting infrastructure.

Taking them in order:

Tuning Mailman is normally not needed. The only real control which is of
significance is MAX_SMTP_RCTPS in ~mailman/Mailman/mm_cfg.py (see ~mailman/
Mailman/Defaults.py for details). The optimal value will vary depending on your
lists, the distribution of their memberships across domains, and the
responsiveness of the various domain's MXes. Typically the sweet spot is in the
range of 2 - 5. See Chuq von Rospach's analysis of VERP expense under "What
about VERP?" for further details.

MTA tuning is incredibly subject to local conditions, MX demographics of your
lists, list load and traffic patterns etc. There are however a some critical
common aspects:

-- Configure your MTA to not do DNS verifies on receipt from localhost. Most
MTAs default to doing verifies by default. Leaving that turned on will slow
delivery rates from mailman to your MTA significantly, especially for larger
lists.

  EXIM

  In Exim, the value to edit is receiver_verify_hosts. See 
  README.EXIM in the Mailman distribution, or h
  http://www.exim.org/howto/mailman.html for details.  

  POSTFIX

  QMAIL

  SENDMAIL

    From: http://mail.python.org/pipermail/mailman-developers/2001-August/009329.html

    You can do this without modifying your sendmail files at all. 
    Instead, in your startup script, add:

        /usr/sbin/sendmail -bd -ODeliveryMode=defer \
               -ODaemonPortOptions=Name=MSA,Port=NNNN,M=E,Addr=127.0.0.1

    Where NNNN is some port number not otherwise used (you can test 
    if something's in use by doing "telnet localhost NNNN" -- if it's
    refused, there's no daemon listening)

    This sets up a sendmail process listening to the alternate port, 
    in DEFER mode, but set to talk only to the localhost interface, 
    so it's not accessible by anyoneother than your local machine: 
    no open relay problems.

    To make mailman access that port, add this to your mm_cfg.py:

        # define alternate SMTP port
        SMTPPORT = 1313

-- Watch your mail spools and observe the percentage and number of slow MXes
you have for your lists. Based on that you may want to tune your retry times/
rates and total time a message can live in your spool (minimum 4 days). If you
have large populations of slow MXes, there can be significant advantage to
putting in a domain routing rule in your MTA to send all that traffic to
another mail server running as a smarthost which is specifically configured/
tuned for slow MXes,

MTAs are inherently IO bound, especially disk IO bound. There are a number of
thins you can do at the system level to help alleviate this:

-- Run a local cacheing nameserver. MTAs do very large numbers of DNS lookups.
Taking network latency out of the picture can speed performance significantly.
BIND has been proven to be one of the fastest caching nameservers available
(see http://www.shub-internet.org/brad/papers/dnscomparison/), and although
BIND 8 is faster than BIND 9, the latter is more secure and recommended (see 
http://www.isc.org/products/BIND/). A commercial alternative that is even
faster is Nominum Foundation Caching Nameserver (see http://www.nominum.com/
product.php?id=CNS). Another alternative is pdnsd (http://www.phys.uu.nl/
~rombouts/pdnsd.html) due to its very elegant behaviour in handling slow/bad
nameservers. You may want to avoid djbdns/dnscache unless you are already
familiar with the djb family of programs and you know what you're getting
yourself into (http://cr.yp.to/djbdns.html).

-- If you are using syslog(8) logging with your MTA, make sure that your
syslog.conf is not configured to sync() after every write to your mail logs.
You can do this by prefixing the entry in syslog.conf with a '-'. See the man
page for syslog.conf(5) for details.

-- Put your syslog logs and your MTA spool on physically different spindles (ie
different drives). It makes little sense to have two very IO active processes
competing for who gets to read or write from the same device at any given time.

-- After carefully checking your MTA documentation, consider mounting your MTA
spool device with the "noatime" option. This can significantly reduce IO levels
to Inodes, which will help the general IO load of your mail spool device.

-- If designing a new system for this purpose, optomise for synchronous disk
transactions - ie your disks should have low latency, the ability to handle
several queued operations (ie SCSI), and be arranged in a configuration that
reduces latency (ie RAID0 or RAID1+0, not RAID5).

-- If you are running extremely high mail loads (think tens of millions of
deliveries per day) you may want to consider using a solid state disk for your
mail spool instead of a normal magnetic drive. Much faster. Much lower latency.
Very expensive. A theoretical alternative on recent linux kernels is to use a
journalling file system (such as ext3) in data journalling mode with the
journal on a separate fast device - either a very fast disk or solid state disk
(still expensive, but cheaper than everything on solid state).

Edit this entry / Log info / Last changed on Thu Sep 25 12:59:05 2003 by Brad
Knowles

-------------------------------------------------------------------------------

4.12. What about VERP?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq02.002.htp>
and <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq05.012.htp>.

Here's one example of how this works in practice, summarizing various messages
in the thread at <http://mail.python.org/pipermail/mailman-developers/2004-July
/017015.html>:

Enabling VERP can be a serious performance hit for large lists. One example
list with over 150,000 recipients saw a factor of 25-75 reduction in
throughput, when VERP was enabled:

   May 27 16:43:46 2004 (440) <20040527135407.74A5F368143 at alan.rezo.net> smtp
        for 151942 recips, completed in 1231.438 seconds
   Jun 11 19:05:45 2004 (440) <20040611163245.ED9CB3680AE at alan.rezo.net> smtp
        for 152333 recips, completed in 649.634 seconds
   Jun 30 15:39:26 2004 (435) <20040630132741.F1A0836811C at alan.rezo.net> smtp
        for 152717 recips, completed in 428.891 seconds
   Jul 13 02:05:22 2004 (435) <20040712150834.DCA153680BD at alan.rezo.net> smtp
        for 152991 recips, completed in 31782.241 seconds

Note:

   428.891/152717 = 0.0028084 seconds per recipient (average)
   1231.438/151942 = 0.0081046 seconds per recipient (average)
   31782.241/152991 = 0.2077392 seconds per recipient (average)

This may not sound like much, but keep in mind that 31782.241 seconds is
529.704 minutes, or 8.828 hours, and during that time the mailing-list system
was pretty much stuck (the other mail, that did not need to pass through
Mailman, was OK).

If you can modify your Mailman configuration to let the MTA do the VERPing, you
won't get the personalization benefits, but you will get the improved bounce
management benefits. Both Exim and postfix are capable of doing this. For
postfix, see <http://www.postfix.org/VERP_README.html#smtp>, although this will
currently require that you modify your Mailman source code to use the "XVERP"
option.

Of course, in this example the machine may not have been tuned for maximum
performance, so your results might be better, if you pay more attention to the
tuning of your systems. Of course, your results could also be worse.

So much for one example of the practice.

Now, here's the theory, from a message by Chuq von Rospach at <http://
mail.python.org/pipermail/mailman-developers/2001-June/008928.html>:

  1000 subscribers -- no digest subscribers to simplify this. Assume 
  just individual messages.

  The message size is 10K, including header.

  The bandwidth needed to generate a connection to send a message is 1K 
  (which is pretty close)

  The bandwidth needed to add an address to an existing message is about 
  1/10 of a K (also pretty close).

  The practical limit to the number of messages you can piggyback is 
  100, since this is specified in RFC2821 as the smallest number a site 
  is REQUIRED to take. In practice, due to non-conformant sites, you 
  have to be careful setting it beyond 50 these days, because sites set 
  this number down because they think it slows down the spammers (I'm 
  yet to be convinced it makes a damn bit a difference, especially since 
  MTAs like postifx recognize the 452 and auto-adjust now. This is 
  another place where sendmail seems behind the technology curve, FWIW)

  How much bandwidth is used depends on these factors:

  what your piggyback value is (in mailman, it's SMTP_MAX_RCPTS)

  how many domains have > 1 subscriber.

  Here's how plaidworks breaks down:

    3101 subscribers across 1287 domains. that's an average of 2.3 
    subscribers per domain, but the numbers skew wildly, so averages 
    are meaningless.

  Here's how my site breaks down:

    # of subscribers                    # of domains/# of users
    ---------------------                       -----------------
    1                                           263/263
    2                                           142/284
    3                                           40/120
    4                                           19/76
    5                                           16/80
    6                                           10/60
    7                                           7/49
    8                                           3/24
    9                                           6/54
    10                                          2/20
    11                                          2/22
    12                                          2/24
    13                                          1/
    14                                          1/
    16                                          1/
    17                                          1/ (worldnet.att.net)
    22                                          1/(juno.com)
    29                                          1 (mindspring.com)
    30                                          1 (pacbell.net)
    35                                          1 (plaidworks.com)
    43                                          1 (sympatico.ca)
    53                                          1 (earthlink.net)
    150                                         1 (home.com)
    173                                         1 (yahoo.com)
    228                                         1 (hotmail.com)
    441                                         1 (aol.com)

    if you're scoring at home, 37% of subscribers come from that last 
    4 domains: 5% for home and yahoo, 7% for hotmail, and 14% for aol. 
    those are your 500 pound gorillas (AOL is 800 pounds), and piss 
    them off at your own risk.

  At the other end, 8% of your users are the only subscriber from 
  a domain. 16% are 1 or 2 per domain. 26% are on sites with 5 or 
  fewer subscribers.

  Time for some numbers.

  Back to the 1000 member list for simplicity. The subscriber list 
  breaks down to:

    85  -       1/85
    45  -       2/90
    12  -       3/36
    6   -       4/24
    [...]
    48  -       1
    55  -       1
    73  -       1
    142 -       1

  That's 553, or 55% of the subscribers, wedged tightly on both ends 
  of the curve. We can extrapolate what they'll do to bandwidth from 
  the end cases if we need to.

    Extreme case: SMTP_MAX_RCPTS = 1.

    1000 subscribers * (10K message size + 1K overhead) = 11,000K bytes 

bandwidth.

    Extreme case: SMTP_MAX_RCPTS = 100

  These get sent  down the line this way:

    85 * 11K
    45 * (1 * 11K + 1 * .1K)
    12 * (1 * 11K + 2 * .1K
    6 * (1 * 11K + 3 * .1K)
    [...]
    1 * 11K + 47 * .1K
    1 * 11K + 54 * .1K
    1 * 11K + 72 * .1K
    2 * 11K + 140 * .1K

  Do you see how I got these numbers? In the case of the 12 domains 
  with three subscribers, you have to make an 11K connection for the  
  first message, and piggy back on the other two addresses at .01K each. 
  You don't really see huge savings until the big domains, and you'll 
  see AOL goes over the 100 address limit so gets split into two 
  different messages.

  For this 55%, the SMTP=1 is 6050K. For 100, it's 1711K bytes. That's 
  28% of the first number, so we're cutting 72% of the bandwidth by 
  chunking at 100. The tradeoff is performance, though -- it takes a 
  lot longer to deliver those AOL addresses, because if you split it 
  into two batches, you can't parallelize the delivery. Package up 100 
  AOL addresses in one batch, none of them get delivered until all 100 
  addresses are sent to AOL and accepted. It's much faster to send them  
  as ten batches of ten in parallel -- but that's the trade off here. 
  Cut network bandwidth but slow delivery to the larger domains.

  Okay, let's look at a case in the middle. SMTP_MAX = 5. The ones with 
  less than 5 don't change, but the big domains do

    85 * 11K
    45 * (1 * 11K + 1 * .1K)
    12 * (1 * 11K + 2 * .1K
    6 * (1 * 11K + 3 * .1K)
    [...]
    1 * (10 * 11K + 38 * .1k)
    1 * (11 * 11K + 44 * .1K)
    1 * (15 * 11K + 58 * .1K)
    1* (29 * 11K + 113 * .1K)

  that works out to (trust me) about 2378K, or about a 60% reduction.

  Let's try SMTP_MAX = 2.

    85 * 11K
    45 * (1 * 11K + 1 * .1K)
    12 * (2 * 11K + 1 * .1K
    6 * (2 * 11K + 2 * .1K)
    [...]
    1 * (10 * 11K + 38 * .1k)
    1 * (11 * 11K + 44 * .1K)
    1 * (15 * 11K + 58 * .1K)
    1* (29 * 11K + 113 * .1K)

  that works out to 2575K, or about a 57% cut.

  By a rough look at those domains in the middle, I'd say these numbers 
  are good +-10%.

  What's this mean? Here's the executive summary:

  The network penalty between SMTP_MAX = 1 (effectively VERP) and any 
  kind of batching (SMTP > 1) is roughly 50%. To get VERP or customized
  footers or customized anything, you double your network bandwidth.

  There is very little advantage to setting SMTP_MAX > 5, UNLESS your 
  subscriber base is heavily stratified onto very few sites. If you 
  have really large groups of subscribers on AOL or Hotmail, it can help 
  cut network bandwidth, but at best, it seems to be about a 10% 
  improvement. 

  If you plot the numbers I did on a curve, you can see just how 
  little advantage you get by increasing the number. You get almost 
  all of the advantage by going to 2, and the line past 5 is very 
  flat....

  Interesting -- I honestly didn't expect to see THIS big a difference 
  -- I was expecting more like 25-30% increase in bandwidth for a 
  VERP-type delivery.

  My thoughts on what this means to future directions:

    Customized messages (VERPing, or encoded unsub URLs, or all of 
    that...) should definitely be an option in Mailman 2.1.

  I would set Mailman's 2.1 default to have this turned ON, giving us 
  the customized unsub links and etc, but to document this for users so 
  they know to turn it off on slow networks.

  If users turn it off, I recommend that SMTP_MAX be set by default to 
  5, and that we document that it makes little sense to change it 
  unless a site is horribly network limited, because even setting to 
  the max only gains them another 10% (and if they're THAT network 
  limited, they're seriously asking for trouble anyway), and only if 
  their subscriber base fits a profile that lends itself to the 
  compression. Setting it large also leaves them open to spamblocking 
  by systems that don't necessarily follow the standards or act right, 
  too.

  We should ALSO note here that some MTAs (postfix, for instance) 
  might override SMTP_MAX anyway -- you could set it to 100, but 
  postfix might be configured smaller, so they have to be aware of 
  those potential interactions. you then get into the issues of 
  tuning all this, with few delivery threads with lots of addresses 
  vs many threads in parallel.. and all that fun -- I guess I'm 
  trying to say that you can't tune mailman in isolation from the 
  MTA (and down that road lies a huge rathole of attempting to 
  document this stuff...)

  But from these numbers, any 2.0.x version of mailman should set 
  SMTP_MAX to between 2 and 5, unless they're horribly network 
  limited. it makes no sense to be larger than 5, and it makes no 
  sense to be 1 unless you've done some kind of VERPing patch.

  for 2.1, we want to implement these customizations and default them 
  on, but with a 50% network hit, we definitely want to make it clear 
  what's going on and make it possible for them to turn it off and 
  return to a generic URL and non-customized e-mail.

  Barry's mileage may vary on his preferences for default, of course, 
  and it's his show. I think the advantages of the customized URL/email 
  capability is a huge one and most sites will benefit from it -- but 
  the network hit might kill some sites, so we have to give them an easy 
  ability to turn the feature off.

  What do y'all think? I've included mailman-developers on this reply, 
  since while this started on mm-users, it really ought to be discussed 
  on the developers list...

Edit this entry / Log info / Last changed on Tue Jul 13 17:43:47 2004 by Fil

-------------------------------------------------------------------------------

4.13. How do I prevent MIME attachments/HTML/Viruses being sent to lists

See also the Mailman FAQ entries at <http://www.python.org/cgi-bin/faqw-mm.py?
req=show&file=faq01.008.htp> and <http://www.python.org/cgi-bin/faqw-mm.py?req=
show&file=faq04.039.htp>.

[From posting by Alf Christophersen, to mailman-developers on 7 Dec 01]

Most of these viruses/attachments come in as MIME encoded, so you can always
use stripmime[1], or demime[2] to get that stuff off of the message prior to
submitting it to the list. You can go one step further, and look into
MIMEDefang[3], which does a plethora of things for you. There's also Quarantine
[4], which will strip out any type of attachments to your message. All of these
solutions are external to Mailman (in fact MIMEDefang hooks into sendmail - if
you're running it), so you can always end up with "clean" messages going
through your list.

    [1] - http://www.phred.org/~alex/stripmime.html
    [2] - http://scifi.squawk.com/demime.html
    [3] - http://www.roaringpenguin.com/mimedefang/
    [4] - http://www.johncon.com/john/QuarantineAttachments/

An alternative is a MIME handler patch for mailman itself - see

 http://sourceforge.net/tracker/index.php?func=detail&aid=413752&group_id=103&atid=300103

Edit this entry / Log info / Last changed on Sun Jul 11 16:14:35 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.14. How do I upgrade from Mailman 2.0.x to Mailman 2.0.y

Unfortunately you do require a little downtime for this, however it should be
measured in single digit minutes, possibly less:

1) Retrieve the sources for the new Mailman. eg

    $ snarf http://prdownloads.sourceforge.net/mailman/mailman-2.0.10.tgz

2) Follow the instructions for building Mailman exactly all the way up the
point of installing it, but DO NOT install it. You can find the ./configure
line you used last time by looking in the config.status file in old source
tree.

2a) You may want to make a backup of your current installation, e.g. by doing a
"tar cvf - /home/mailman | gzip -c -9 > mymmbackup.tgz".

3)Stop the following daemons: MTA (exim/postfix/qmail/whatever), web server
(apache/thttpd/etc), and cron. eg

    # /etc/init.d/cron stop
    # /etc/init.d/apache stop
    # /etc/init.d/sendmail stop

4) Install your newly configured and built Mailman:

    # make install

5) Restart the services you previously stopped.

    # /etc/init.d/cron start
    # /etc/init.d/apache start
    # /etc/init.d/sendmail start

This gives a short outage, but shutting down those services ensures that
nothing is fiddling with your installation at the time when it is being
upgraded.

BAW: I'll note that if you're feeling reeaaallly gutsy and you're just
upgrading within a minor rev series (e.g. MM2.0.8->2.0.9->2.0.10) you might be
able to do a live install without shutting down your servers. Just install the
new version on top of the old one. I definitely wouldn't recommend this for
busy sites though.

Edit this entry / Log info / Last changed on Tue Aug 26 00:59:46 2003 by rich
cowan

-------------------------------------------------------------------------------

4.15. How do I filter incoming mail before it hits mailman (e.g., using
procmail)

I spam filter incoming mail on my public lists using procmail. It doesn't
require changing either procmail or Mailman. What I did was put my procmail
filter in /etc/procmailrcs/mailman (your procmail may want it somewhere else)
and change the ownership so that it belongs to the group that I configured
mailman's "--with-mail-gid=", in my case nobody.nobody.

I replaced the mail list alias, from

    lugor-announce:          "|/usr/home/mailman/mail/wrapper post lugor-announce

to

    lugor-announce:          "|/usr/bin/procmail -m MAILMAN=lugor-announce /etc/procmailrcs/mailman

The procmail filter I use stores spam in a folder for review in mailman's Mail
directory, but they belong to "nobody", not "mailman", which can be a minor
problem at times. The last lines of the procmail filter pass the mail (if it
wasn't filed as spam) to mailman with the commands:

    :0
    |/usr/home/mailman/mail/wrapper post ${MAILMAN}

Because I used to use an older version of Mailman that mistook the "Sender:"
header for the envelope sender (somebody misread RFC 822!), I also used that
procmail filter to strip out the Sender header, using the lines:

    :0 f
    | formail -R "Sender:" "X-Former-Sender:"

before the lines above.

Edit this entry / Log info / Last changed on Wed Jun 2 00:49:43 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.16. How do I prevent duplicate copies of email coming to the list

Occassionally I get the same email message coming to a list. Sometimes this is
because people with broken MUAs (like Outlook) do a "group reply" and it puts
the list address in twice, and sometimes it's a mail server hiccup. In order to
prevent that, I run the mailing list through procmail as shown in the previous
FAQ entry. Near the top of my procmail recipe file, I put the following code:

 :0 Whc: msgid.lock
 | formail -D 8192 /tmp/msgid.cache

 :0 a:
 /dev/null

The first bit keeps a message id cache of the last 8192 message ids recieved on
the list, and the second bit deletes any messages whose message id matches one
in the cache.

If you run more than one email list through the same procmail recipe file, be
sure and use a different msgid.cache file for each list, possibly by calling it
/tmp/msgid.cache.${MAILMAN} If you don't, people won't be able to send a
message to two of your mailing lists at a time. That may be desirable to you.

Edit this entry / Log info / Last changed on Wed Jan 9 18:57:48 2002 by Paul
Tomblin

-------------------------------------------------------------------------------

4.17. Why are lists missing from the listinfo page?

Assuming MM 2.0.X, the factors affecting whether a list appears on the listinfo
(as opposed to the admin page) are:

1. the value of the list's 'advertised' option (the first one on the page) of
the Privacy Options Section of the list's admin web GUI.

2. the value of VIRTUAL_HOST_OVERVIEW in $prefix/Mailman/Defaults.py or
mm_cfg.py. If this is set to true (usually 1) then you may be experiencing the
effect of Mailman Virtual Host feature. In that case the address in the URL
used to access listinfo is compared to the address in the list's 'web_page_url'
option (the last one) of the General Options Section of the list's admin web
GUI. If they are not the same, the list is not added to the listinfo page
returned.

Edit this entry / Log info / Last changed on Mon Sep 29 06:32:31 2003 by rhonda

-------------------------------------------------------------------------------

4.18. How do I backup my lists and their configurations/membership rosters?

You can use the following scripts in a cronjob:

  #!/bin/bash
  #set -x
  # Sends a file to an email address as a MIME attachment.
  # Syntax: mimemail <subject> <file> <address>
  # Note: Requires MH or nmh to be installed (uses MHN)

  subject=$1
  file=$2
  addr=$3

  echo "To: ${addr}
  From: nobody
  Subject: ${subject}

  #application/octet-stream [${subject}] ${file}

  " | /usr/bin/mh/mhbuild - > /tmp/mimemail.tmp.${$}
  /usr/lib/mh/post -verbose -watch /tmp/mimemail.tmp.${$}
  rm /tmp/mimemail.tmp.${$}

----------

  #!/bin/bash
  #Put in a cronjob to archive your lists and email them to you
  #set -x

  datestr=`date +%Y%m%d`
  host=`hostname -f`
  file=mailman.lists.${datestr}.tar.gz
  filepath=~/backups/${file}
  sendto="root"

  cd /var/lib/mailman
  tar zcf ${filepath} lists
  cd ~/backups
  ~/bin/mimemail ${file} ${filepath} ${sendto}

The result is a tarball emailed to you by the cronjob which contains the entire
lists directory hierarchy of yuor Mailman installation.

Edit this entry / Log info / Last changed on Tue Apr 9 17:30:42 2002 by J C
Lawrence

-------------------------------------------------------------------------------

4.19. I get "Could not acquire qrunner lock" in ~mailman/logs/qrunner...

qrunner is a little program that processes the queued list mail, i.e. sends it
to your MTA. It is invoked by cron, usually once a minute. Before it starts up,
it checks for a lock file in ~mailman/locks, because only one qrunner may run
at a time (otherwise, messages might be sent out more than once or there could
even be deadlocks).

If there is none, it assumes that no other qrunner is running, creates a lock
for itself and begins to work through the files in the queue (~mailman/qfiles).
If there is a lock, it aborts and prints said error message.

There are two possible reasons for this to happen:

(1) The queue is large, or sending the mail takes long, so the old qrunner has
not yet finished when cron starts a new one. This is normal behaviour.

(2) A qrunner has crashed, and was not able to remove the lock it had acquired,
or qrunner is choking on some message forever. This is a problem, because no
list mail will be delivered until you clean up.

Marc Merlin suggested on mailman-users to check if a qrunner is active, for
example by doing "ps aux|grep qrunner". If no, you can safely delete the
qrunner lock file in ~mailman/locks/ by hand. If yes, he recommends to do a
"strace -f -p <process-id of qrunner>" to see what it's doing and whether it is
stuck with a certain message. If so, remove the offending message from the
queue, kill qrunner and remove the lock.

This should rarely if ever happen with Mailman releases > 2.0.9. If you run
into this problem frequently, you should probably ask for assistance on the
mailman-users list.

Edit this entry / Log info / Last changed on Fri May 3 13:55:31 2002 by Jrn
Nettingsmeier

-------------------------------------------------------------------------------

4.20. Who should deal with transient DNS errors?

Sometimes, Mailman tries to send mail to domains which exist but do not have a
MX (mail exchanger) or an A (address) record. This may happen for example when
a spam comes from a newly reserved domain, who has not been setup to receive
mail.

Most MTA are configured to reject mail for such a domain with a temporary
failure exit code (such as 450), because the absence of those records may be
caused by a transient network outage. If your local MTA has been configured
this way, it will reject mail from Mailman with this temporary failure exit
code, and Mailman will try to resent the mail every minute. Considering that
the DNS lookup may easily take up to 30 seconds in case of a network problem,
this may slow down Mailman mail delivery by a huge factor.

One solution is to let your MTA deal with this situation instead of Mailman.
Configure your MTA so that it always accepts mail coming from Mailman. In
Postfix for example, this is done by using a "client_access" restriction
(allowing mail from localhost if Mailman is running on the same machine as
Postfix) before the "reject_unknown_recipient_domain" restriction.

Edit this entry / Log info / Last changed on Sat Feb 1 00:09:39 2003 by Samuel
Tardieu

-------------------------------------------------------------------------------

4.21. Why make changes in mm_cfg.py NOT Defaults.py ?

This just an expansion of what Mailman's INSTALL document says and is
reiterated in Defaults.py and mm_cfg.py.

The reason for not editing Defaults.py is that it is replaced when you install
a new version of MM over an existing version: the normal case when updating an
operational installation. Hence, you will lose the configuration variable
changes you have made to a running system if you made them in Defaults.py

In contrast, mm_cfg.py is _NOT_ replaced when a new version of MM is installed
over an existing version and hence your explicit local configuration changes
are retained over the update.

Remember that the Mailman source modules only import mm_cfg (never Defaults)
which, alone, imports Defaults _before_ doing its own assignments; this means
that the config variable assignments in mm_cfg override any assignment of the
same variable in Defaults. If it isn't mentioned in mm_cfg, then the assignment
in Defaults holds sway.

Admins should follow the same logic, in which case there should be little room
for confusion.

Edit this entry / Log info / Last changed on Tue Feb 11 12:14:46 2003 by 
Richard Barrett

-------------------------------------------------------------------------------

4.22. Messages that are translated from HTML to plain text have a body like ""/
root/8e8Ta4: Permission denied"

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.028.htp
>.

Chances are you are running Mailman on a Red Hat version of Linux ...

Turns out that Red Hat decided to change the temporary directory that lynx uses
to be the users home directory. Since qrunner is suid to mailman, the processes
environment information it runs with is not actually mailman's (home directory,
etc) ... so if you start qrunner when signed on as root, the processes home
directory is still root's home directory (in my case '/root').... so, you have
a process that has a home directory that it has no authority to write to.

To fix the problem, we have to change the temporary directory that lynx uses.

Change the HTML_TO_PLAIN_TEXT_COMMAND to the following ...

HTML_TO_PLAIN_TEXT_COMMAND = 'LYNX_TEMP_SPACE=/tmp /usr/bin/lynx -force_html
-dump %(filename)s'

This causes the temporary directory that lynx will use to be /tmp (which, btw,
is the default for a freshly built lynx).

(thanks to John Buttery for the modified lynx command)

Edit this entry / Log info / Last changed on Fri Jul 16 02:11:02 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.23. How do I use SpamAssassin with Mailman?

There are 3 ways.

1. Integrate it into the MTA, such as described in FAQ entry 6.12 at <http://
www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq06.012.htp>.

2. Using patches by Jon Parise located at http://sourceforge.net/tracker/
index.php?func=detail&aid=640518&group_id=103&atid=300103 as described at http:
//www.jamesh.id.au/articles/mailman-spamassassin/

3. Using procmail to execute spamassassin before handing the message to
mailman. (see rest of FAQ entry).

From a post to the mailman-users list by Dave Stern <dave@umiacs.umd.edu> on 26
Feb 2003. Editted to use default paths.

-[ Begin Message ]-

Seeing how others have asked about this, I've gotten an older version of
mailman (2.0.12) and spamassassin (2.1) working together. Let's say the list in
question is called mylist

- add to you aliases file

  mylist:     "/usr/bin/procmail -m /etc/procmailrc"

- /etc/procmailrc include things like

-[ Start /etc/procmailrc ]-

 :0fw
 | /usr/bin/spamassassin

 :0:
 * ^X-Spam-Status: Yes
 /tmp/Likelyspam

 :0:
 |/etc/smrsh/wrapper post mylist

-[ End /etc/procmailrc ]-

-[ End Message ]-

The second method will work with any version of mailman giving the wrapper call
is correct. For mailman 2.0 /etc/smrsh/wrapper

For mailman 2.1 /etc/smrsh/mailman

Edit this entry / Log info / Last changed on Thu Aug 19 15:01:22 2004 by 
Alexander GQ Gerasiov

-------------------------------------------------------------------------------

4.24. How do I create unadvertised lists?

How do I create lists that don't appear in the list info page?

From the list admin page, select "Privacy Options". There is a choice for
"Advertise this list when people ask what lists are on this machine?".

Edit this entry / Log info / Last changed on Sat Oct 4 00:40:29 2003 by Eric
Smith

-------------------------------------------------------------------------------

4.25. What is the purpose of the site-wide mailing list?

The site-wide mailing list is for errors and warnings generated by Mailman. It
is also where password reminders appear to come from.

You have to create this list yourself after you install Mailman. But Mailman
assumes this list exists. It also assumes its name is "mailman". You can use a
different name but then you need to modify MAILMAN_SITE_LIST in mm_cfg.py.

Since you create this list just like any other, it is by default public. You
should probably make it private. You may also want to turn archiving off. Or
make the archives private.

Normal users should not subscribe to this list. Your site administrators should
subscribe to the list.

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq05.005.htp
>.

Edit this entry / Log info / Last changed on Thu Jun 24 12:02:14 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.26. How do I remove a list that has a space in the list name?

Version 2.1.x of Mailman no longer allows for the creation of list names with
spaces in them... here is an archive message on it: ======

Basically you can delete the directories associated with the list and that's
that. The following directories are created when you create a list:

  ~mailman/lists/<list name>
  ~mailman/archives/private/<list name>
  ~mailman/archives/private/<list name>.mbox
  ~mailman/archives/public/<list name>
  ~mailman/archives/public/<list name>.mbox

If you delete those directories, then you have deleted the list! One of the
nice things about Mailman is its simplicity.

(From a post to mailman-users by jonc AT nc.rr.com)

Edit this entry / Log info / Last changed on Fri Mar 14 21:29:15 2003 by JC
Dill

-------------------------------------------------------------------------------

4.27. Securing Mailman's web GUI by using Secure HTTP/SSL

For extra security when using Mailman's web GUI you may want to make access to
it use Secure HTTP with URLs using the https scheme.

To do this you will need to:

1. Make the appropriate changes to your Apache installation (adding a module to
provide SSL support, for example) and httpd.conf so that your server will only
make the Mailman web interface URIs available via Secure HTTP. For more
information about configuring Apache visit http://httpd.apache.org.

Something like:

    RedirectPermanent /mailman/ https://<server_name>/mailman/

should do the trick.

2. Assuming MM 2.1.1, assign the value of the DEFAULT_URL_PATTERN Mailman
configuration variable in mm_cfg.py to use the https scheme, e.g.

    DEFAULT_URL_PATTERN = 'https://%s/mailman/'

This will ensure that the web_page_url attribute of new lists use the https
scheme

3. Use $prefix/bin/fix_url.py to get the change to DEFAULT_URL_PATTERN
propagated to the web_page_url attributes of existing lists.

Edit this entry / Log info / Last changed on Fri Jun 27 16:43:25 2003 by Jon
Earle

-------------------------------------------------------------------------------

4.28. How do I fix HTML->text conversions which just give a "Permission denied"
message?

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.022.htp
>.

An HTML message comes through the list with only the body:

   /root/9Si1aE: Permission denied

and the error log reads something like:

   HTML->text/plain error: 256

According to David Gibbs's post to mailman-users <http://mail.python.org/
pipermail/mailman-users/2003-January/025369.html>, this is caused by a Redhat
configuration issue with lynx where the temp directory is set to the user's
home directory, but since mailman runs suid mailman, it doesn't work.

To solve this, change the HTML_TO_PLAIN_TEXT_COMMAND to the following. (Make
sure that this is all on one line.)

HTML_TO_PLAIN_TEXT_COMMAND = 'LYNX_TEMP_SPACE=/tmp /usr/bin/lynx -force_html
-dump %(filename)s'

Edit this entry / Log info / Last changed on Fri Jul 16 02:15:36 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.29. Where can I change a list or the default URL used for the web interface?

For MM 2.1.2:

 Changing hostnames
 ------------------

These changes should be made $prefix/Mailman/mm_cfg.py. The applicable MM
config variables are described in $prefix/Mailman/Defaults.py. Typically
something like what follows, at the end of your mm_cfg.py, will do the trick:

 DEFAULT_EMAIL_HOST = 'your.mailhostname.tld'
 DEFAULT_URL_HOST = 'your.webhostname.tld'
 VIRTUAL_HOSTS.clear()
 add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)

and some more add_virtualhost(URL_FQDN, EMAIL_FQDN) if you are using virtual
hosts.

 Virtual host issues
 -------------------

If you call the function add_virtualhost (which is defined in Defaults.py if
you want to look at the code) with no EMAIL_FQDN then the function computes the
email hosts from the url host by removing one leading element of the name of
the url host. So for instance, calling the function like this:

 add_virtualhost('listserver.mydomain.tld')

is equivalent to:

 add_virtualhost('listserver.mydomain.tld', 'mydomain.tld')

VIRTUAL_HOSTS is a Python dictionary (equivalent to a Perl Hash) with URL_FQDNs
as the keys and EMAIL_FQDNs as the values. Do ensure the different
add_virtualhost() calls have unique values for their first parameter or you may
get a surprise.

 Non-standard web server ports
 -----------------------------

If your Apache server uses non-standard ports you may also want to assign,
again in mm_cfg.py, different values for the DEFAULT_URL_PATTERN and
PUBLIC_ARCHIVE_URL MM config variables. Insert the servers non-standard port
number into the pattern.

 Using Secure HTTP
 -----------------

If you want to use Secure HTTP instead of regular HTTP you may also want to
assign, again in mm_cfg.py, different values for the DEFAULT_URL_PATTERN and
PUBLIC_ARCHIVE_URL MM config variables.

Depending on how you want your site to operate you may change one or other or
both of these patterns to use the 'https' scheme rether than the default 'http'
scheme.

Check in Defaults.py for the description and current values of the variables.
In this case you will also have to adjust your Apache setup to support SSL
which is outside the scope of this FAQ but see http://www.python.org/cgi-bin/
faqw-mm.py?req=show&file=faq04.027.htp

 Do not forget to restart mailmanctl
 -----------------------------------

Then run 'mailmanctl restart'. (or the changes will not take hold).

 Existing versus new lists
 -------------------------

Changing these things in mm_cfg.py will now effect new lists.

If you want to update existing lists the $prefix/bin/withlist script $prefix/
bin/fix_url.py is your friend for resetting things. Just run fix_url.py to get
its usage.

 Changing existing archives
 --------------------------

If you are using the MM pipermail archiver with HTML archives, you might also
need to run $prefix/bin/arch if there is archived email with attachments that
have been extracted with links to the attachments left in the email. These
links seem to use the web_page_url of the list concerned at the time the email
was added to the archive. Running arch rebuilds these links using a list's
current web_page_url.

Edit this entry / Log info / Last changed on Thu Jul 24 17:43:13 2003 by 
Richard Barrett

-------------------------------------------------------------------------------

4.30. How do I configure the admin webpage to show more members per page?

Change the setting in DEFAULT_ADMIN_MEMBER_CHUNKSIZE in mm_cfg.py The
description is in Defaults.py.

Edit this entry / Log info / Last changed on Sat Aug 2 16:47:47 2003 by JC Dill

-------------------------------------------------------------------------------

4.31. How do I remove a list via the web

To delete a list via the web interface you must set the variable
OWNERS_CAN_DELETE_THEIR_OWN_LISTS to yes in the mm_cfg.py file. It is no by
default. See Defaults.py for more info on what the variable does.

If you tried to remove a list via the web interface and were told "You're being
a sneaky list owner!" you have the variable set to No.

See the python source file in your mailman source tree under ./Mailman/Cgi/
rmlist.py to better understand this whole process.

Edit this entry / Log info / Last changed on Sun Dec 7 03:06:28 2003 by Tom
Oakes

-------------------------------------------------------------------------------

4.32. How Do I Upgrade version 2.0.x to version 2.1.x?

The documentation for the upgrade process is located in a file called
UPGRADING, which is in the root folder of the set of files that you download
when you upgrade.

Note that mailman is not 100% backward compatible, and you can lose some of
your settings when you upgrade. It is also possible that the archive links for
your mailing lists will no longer work. So this is written in an attempt to lay
out some of the issues that need to be better documented and addressed, in the
hopes that others will help fill in the gaps.

Upgrade Paths:

a) if you only have 1 or 2 lists set up, you could make a copy of the archive
for the lists (the large MBOX file containing all the posts that were ever made
to the list) and also a copy of the user database (where is it located? do you
use dumpdb to back it up?) and then manually reconstruct your list once you
install Mailman 2. This requires uninstalling mailman.

b) If you wish, you can set up mailman 2.1 simultaneously with 2.0 and then
upgrade your lists individually from 2.0, as described in the UPGRADING file.

c) The best option is to upgrade everything at once. However there relatively
few scripts or tools available at this time to ease the process.

Some tools to know about: Mailman includes a number of scripts in the mailman/
bin folder... however the documentation for these scripts is a bit lacking...
if you are looking at this FAQ: http://www.python.org/cgi-bin/faqw-mm.py you
will find the scripts documented under section 4.9. Some of the commands worth
knowing are listed below, but the FAQ does not specify if these commands are in
all versions of 2.0.x or if some are only available in version 2.1. Here are a
few for starters...

fix_url: only in version 2.1; I take it this is something you do after
upgrading? Fix_url will update the web_page_url variable in the config file
(which is not available from the Web interface). If you not changing the name
of your list server then you will not have to use this variable, but if you are
moving from one computer to another then you will have to update this variable
or the public lists will not show up on the listinfo page and whenever you
submit a change in the configuration you will jump back to the original
computer. The documentation is a little unclear but if you run it without any
arguments then it will get the values from mm_cfg.py and stick them into the
config file of the named list.

i.e. bin/withlist -l -r fix_url your_list_name

resets the web_page_url to the value of DEFAULT_URL_HOST in mm_cfg.py

This is mostly designed for virtual hosts but it is also needed in changing
from one host to another.

dumpdb: the documentation for this says that it "dumps the contents of a
mailman.db file" but what does this mean? Does "dump" mean erase, or does it
mean "back up" And does it dump the contents of the database for the all lists
or do you call it with a list name as a parameter?

sync_members: this will take as input the name of a list and a flat text file
and will change the list database to reflect the contents of the flat text
file. So I take it you can remove all members of a list this way, so that if
you wanted to rename listA to ListB and then reuse ListB for a slightly
different purpose, you could do that without having to do an "rmlist" and then
a "newlist".

update: update from previous version to current version

Some issues to consider:

When you upgrade, it is very possible that the links to your archive will no
longer work: the link from your listinfo page will point to /var/www/pipermail
(which does not exist) instead /var/lib/mailman/archives.

Also, a different version of Pipermail is used by Mailman 2.1 from the one used
by version 2.0. It is not documented whether this is the reason for some of the
archive problems.

It is also possible that messages sent to your list after an upgrade will not
be received properly. So it is important to test your lists after upgrading.

The mailman UPGRADING file includes several other issues... an excerpt follows.

"Mailman will NOT upgrade the template files for existing lists. Chuq Von
Rospach gives some useful advice in this message to the users mailing list:

    http://mail.python.org/pipermail/mailman-users/2000-September/006826.html
    [Actually, the upgrade to MM2.1a2 /will/ shuffle template files,   deleting any that it detects are unchanged from the original defaults (calculated via md5 checksums)."

If you are also moving from one machine to another at the same time you are
upgrading look at: http://acd.ucar.edu/~fredrick/linux/mailman/upgrading.html

"In Mailman 2.1, the qrunner subsystem has been completely rewritten. You no
longer start qrunner from cron! Instead, there is a bin/mailmanctl script which
is used to start, stop, and restart mail delivery. This script is appropriate
to use as a Unix init script. Be sure to update your crontab with the new cron/
crontab.in file."

Note that you do not need to regenerate your aliases files; this is dated
information left over from earlier versions of mailman... so I guess you can
ignore the comments along these lines.

Edit this entry / Log info / Last changed on Sat May 8 05:49:39 2004 by Michael
Mansour

-------------------------------------------------------------------------------

4.33. How do I put a subscribe form for my list on a web page?

Cut and paste the code below into the html of your web page, changing <SERVER>
and <LISTNAME> to the correct values.

The only required parts of this form are the email text box and the submit
button - edit or remove the others as you like.

 --start code --

 <hr ALIGN="center" WIDTH="730"><br>
 <H2><B>Join the XYZ list</b></H2>
 <FORM Method=POST ACTION="<SERVER>/mailman/subscribe/<LISTNAME>"><br>
 Your E-mail address: <INPUT type="Text" name="email" size="30" value=""><br>
 Your Name (optional): <INPUT type="Text" name="fullname" size="30" value=""><br><br>
 You may enter a privacy password below. This provides only mild security, but should<br>
 prevent others from messing with your subscription. <b>Do not use a valuable password</b> as it<br>
 will occasionally be emailed back to you in cleartext.<br><br>
 If you choose not to enter a password, one will be automatically generated for you, and it will<br>
 be sent to you once you've confirmed your subscription. You can always request a mail-back<br>
 of your password when you edit your personal options.<br><br>
 Password choice: <INPUT type="Password" name="pw" size="15"><BR>
 Confirm Password: <INPUT type="Password" name="pw-conf" size="15"><BR><BR>
 Would you like to receive list mail batched in a daily digest? (You may choose NoMail after you join.)<BR><br>
 <input type=radio name="digest" value="0" CHECKED> No <input type=radio name="digest" value="1"> Yes<br><br>
 <INPUT type="Submit" name="email-button" value="Subscribe">
 </FORM>
 <p><p>
 <hr ALIGN="center" WIDTH="730">

 -- end code --

Edit this entry / Log info / Last changed on Sun Nov 16 21:37:14 2003 by Paul H
Byerly

-------------------------------------------------------------------------------

4.34. I don't want the "download full raw archive" link which provides
non-antispammed email addresses

SPAM PROBLEM:

 HYPOTHESIS
 SOLUTION 2.1.4

 HACK SOLUTION 2.0.11 (AND PROBABLY 2.1.1)
 (1) remove link to *.mbox in   
 /var/lib/mailman/Mailman/Archiver/HyperArch.py
 (2) remove compiled versions HyperArch.py[oc]
 (3) test on an archive (e.g. <mylist>)
 (4) make the *.mbox/*.mbox files inaccessible (since google links remain)
 (5) think about newlist s (maybe nothing needs doing)

SPAM PROBLEM:

i think this is a known spam problem - by default, it looks like the archive
files linked under "download the full raw archive" like the following etc are
NOT antispammed:

http://lists.mydomain.org/mailman/public/mylist.mbox/mylist.mbox

It seems to be present in 2.0.11 and 2.1.1

HYPOTHESIS

IMHO the reason why this is probably not easy to solve is that this is where
mail is automatically saved when it's received. If this is filtered by " at "
-> "@" then it means that overall there are typically 4 copies of the entire
mailbox (e.g. html version, monthly archives, true mailbox with @ hidden from
external access, and " at " version for web access).

i couldn't find if this has been discussed, but it looks like there's a simple
solution in 2.1.4.

SOLUTION 2.1.4

It looks like the solution in mailman-2.1.4 is to offer different templates:

 mailman-2.1.4/templates/en/archtoc.html
 mailman-2.1.4/templates/en/archtocentry.html
 mailman-2.1.4/templates/en/archtocnombox.html   ->  this one has no mbox

e.g. http://mail.python.org/pipermail/mailman-announce/ does not link to *.mbox
/*.mbox

However, the raw mbox does exist in http://mail.python.org/pipermail/
mailman-announce.mbox/ and it is publicly accessible. (Please don't post the
exact URL - no need to google a hidden file.)

HACK SOLUTION 2.0.11 (AND PROBABLY 2.1.1)

In 2.0.11, the line pointing to the .mbox is in

/var/lib/mailman/Mailman/Archiver/HyperArch.py (for Debian anyway ;)

      You can get <a href="%(listinfo)s">more information about this list</a>
      or you can <a href="%(fullarch)s">download the full raw archive</a>
      (%(size)s).
     </p>

solution:

(1) remove the link to the full .mbox in /var/lib/mailman/Mailman/Archiver/
HyperArch.py

To do this,

replace

      You can get <a href="%(listinfo)s">more information about this list</a>
      or you can <a href="%(fullarch)s">download the full raw archive</a>
      (%(size)s).
      </p>

by

      You can get <a href="%(listinfo)s">more information about this list.</a>
      </p>

(2) remove compiled versions (in my case the .pyc gets automatically
recompiled)

 rm /var/lib/mailman/Mailman/Archiver/HyperArch.py[co]

(3) test this

 cd /var/lib/mailman/archives/public/
 /usr/lib/mailman/bin/arch mylist

Then check out:

 http://lists.mydomain.org/pipermail/mylist/

Hopefully there will be no link to the .mbox However, direct access by a spam
harvester/spider knowing the URL will still be possible.

(4) So, make the .mbox file inaccessible - since google links will still hang
around for some time

 chmod go-rw /var/lib/mailman/archives/private/*.mbox/*.mbox

(5) probably nothing needed when running newlist for new lists.

Making new lists will, by default, write *.mbox/*.mbox which are web
accessible, but nobody reasonable and careful is going to link to them.

Known threads on this issue:

 http://mail.python.org/pipermail/mailman-developers/2004-February/016569.html
 http://zope.org/Members/bwarsaw/MailmanDesignNotes/MailmanProblems (wiki)

Edit this entry / Log info / Last changed on Thu Feb 19 01:41:04 2004 by boud

-------------------------------------------------------------------------------

4.35. What do I do with a shunt (qfiles/shunt) directory full of files?

I was using 2.1.2 (RedHat's 2.1.2-2), and got the problem where the digester
(?) would bomb out with "ValueError: Empty module name". By the time I got
around to dealing with it, my shunt directory had about 226 megabytes of stuff
in it. I upgraded to 2.1.4 (RedHat's 2.1.4-2 from rawhide) and that problem
seems to be solved, but now I get:

 /var/mailman/pythonlib/japanese/c/iso_2022_jp.py:4: RuntimeWarning: Python C API version mismatch for module
 _japanese_codecs: This Python has API version 1011, module _japanese_codecs has version 1012.
 import codecs, japanese.c._japanese_codecs

(on a RedHat 9 system with all patches) and meanwhile, the stuff is still all
stuck in the shunt directory.

I'm not having much luck finding docs that even explain the shunt directory.

So:

1. What is the shunt directory exactly?

2. (more importantly) How can I persuade Mailman to deliver the stuff that's in
the shunt directory?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

ANSWER: I had a similar problem and figured out the answer today.

1. The shunt directory is a place where messages go when there is an error with
some functions that process them. In my case, it was because of an archiving
error in version 2.1.3 that was fixed with an upgrade to 2.1.4

2. To fix it, there is a program called unshunt in the ~/mailman/bin directory.

You simply run it from the command line; you don't need any special parameters:

% [path_to]/mailman/bin/unshunt

Edit this entry / Log info / Last changed on Thu Aug 5 21:07:27 2004 by Jeff
Barger

-------------------------------------------------------------------------------

4.36. Creating a new list from Web CGI result in "Error: Unknown virtual host"

even if "newlist" from command line was successful.

Answer: Virtual domains appearing in URLs that are used to access the Mailman
CGI have to be registered with Mailman. This is because Mailman can't know the
corresponding mail virtual domain from looking at the URL alone.

Common example: URL to create list is

 http://www.do.main/mailman/create but

list mail for this list will go to

 listname@do.main.

In this case, you would have to register this domain by adding

 add_virtualhost( 'www.do.main', 'do.main' )

to mm_cfg.py.

Alternatively you can specify all such mappings in one go by adding

 VIRTUAL_HOSTS = { 'www.do.main.one': 'do.main.one',
                   'www.do.main.two': 'do.main.two',
                   ... }

to mm_cfg.py.

In the above example, it would even have been enough to just say

 add_virtualhost( 'www.do.main' )

and so on, because the add_virtualhost() helper function strips off the first
part of the given domain name and uses the remainder as the mail domain name,
if only one argument has been given.

See Defaults.py for more explanations and examples.

Edit this entry / Log info / Last changed on Sat Mar 6 04:40:50 2004 by Frank
Luithle

-------------------------------------------------------------------------------

4.37. How do I stop the archiver from "scrubbing" HTML messages sent to the
list?

Question:

I want html messages to appear in the public archive. In other words I want a
person to click on the link for a message and when it opens up it is in the
original html format it was sent.

What do I need to do so that it is not 'scrubbed'?

Answers

1) Try setting ARCHIVE_HTML_SANITIZER = 3 in mm_cfg.py.

2) Try another archiving agent. One which can do this is mhonarc:

See http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.004.htp (4.4.
How can I use an external archiver with Mailman like MHonArc?) for more info.

Edit this entry / Log info / Last changed on Sun Apr 4 19:48:28 2004 by Terri
Oda

-------------------------------------------------------------------------------

4.38. How to change some configuration for ALL lists?

Q: I am the site administrator and I'd like to change the configuration of ALL
existing lists. How to do that?

A: Use /usr/lib/mailman/bin/config_list

Here's an example:

 1. Login your shell
 2. Create a file (/path/to/configfile) and add your configuration *) 
 3. Change the configuration using this shell script:
     for i in $(/usr/lib/mailman/bin/list_lists -b) ; do /usr/lib/mailman/bin/config_list -i /path/to/configfile/ $i ; done

To check for errors before doing the changes, you can simulate your changes
using the -c flag. see:

 config_list --help

-----------------------------------------------------------------------------------

*) Annotations:

- Only use lower case directives!

- Use 0 or 1 for boolean variables

- You can add as many directives as you like

- To get an example of a list's configuration, type

 /usr/lib/mailman/bin/config_list -o - [list_name]

Edit this entry / Log info / Last changed on Sun Apr 18 11:09:13 2004 by 
Steffen Mller

-------------------------------------------------------------------------------

4.39. HELP! Mailman is munging HTML & MIME-formatted messages before they are
sent out? (Mailman 2.1.x)

a.k.a., "Why are footers being shown as attachments to HTML-formatted messages?
"

For Mailman 2.0.x, see also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&
file=faq04.005.htp>.

The problem is that there is no general-purpose method of determining where a
footer should be properly attached to a message that is HTML or MIME formatted.
To figure that out for HTML-formatted messages, you'd have to implement a
complete HTML imaging engine, so that you could see what was the apparent
"bottom" of the page (as it appears to the user), and then work backwards from
there to figure out where to perform your surgery.

Doing so would also lock you into a solution that only works for those MUAs
which implement the same (or compatible) HTML engines, and would almost
certainly make the situation worse for everyone else.

So, what Mailman does instead is to take the entire HTML message as sent, put
that into a MIME bodypart, add another MIME bodypart for the footer, and then
wrap all that up and send it out as a message that is formatted a different
kind of MIME message. Some MUAs will display the result in a fashion reasonably
close to the format in which the message was submitted, and some won't. Those
that have problems may result in the footer being displayed as an attachment,
or worse.

Note that this problem is not unique to HTML-formatted messages. Messages that
have MIME multipart/mixed or multipart/alternative bodyparts have the same
problem -- Mailman can't figure out where it could safely append the footers to
the existing bodyparts, so it has to wrap the whole message up in another MIME
structure and add a bodypart at the bottom for the footers. Even if the message
is formatted as text/plain, it may have a different charset or
content-transfer-encoding than Mailman would use for the footers. In that case,
the incoming text/plain message would still get wrapped up as one bodypart with
the footers created as a second bodypart, and the whole thing sent out as a
multipart/mixed.

To solve this problem, your options are:

 1. Configure Mailman to remove the footer
 2. Configure Mailman to strip all HTML and MIME formatting and send out all
      messages as text-only (text/plain)
 3. Get everyone to change the MUA they use to one that is more HTML/MIME-aware
 4. Live with the problem

Note that this issue is not specific to a particular version of Mailman. This
is a general problem with messages that are formatted in HTML or MIME, which is
one reason why Mailman supports methods of stripping and/or converting HTML and
MIME to more readable formats.

Keep in mind that the default Mailman content filtering includes more than just
text/plain. If you change the configuration to strip everything but text/plain,
you will break cryptographic signatures from S/MIME and PGP/MIME MUAs, and will
probably cause additional problems for your users. Choosing option #2 from the
above list is quite a bit more difficult to configure than you may think it is.
Even if you can get the system to do this, the result would most likely be
quite a bit more drastic than you think.

There are many examples of this issue being discussed in the archives, with the
threads at <http://mail.python.org/pipermail/mailman-users/2004-April/
036374.html> and <http://mail.python.org/pipermail/mailman-users/2004-July/
037999.html> being two examples.

Edit this entry / Log info / Last changed on Sun Jul 11 16:10:32 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.40. HowTo Patch Mailman Source Code - a basic overview

This entry is a basic overview of how to patch Mailman's source code - this
applies really to any Unix source code. Patching source code is important for
Mailman users: fixes and new features often come out as patches.

Patches are changes in existing code (and only the changes!). Applying a patch
can affect one or more files of Mailman's source code. You only need to apply a
patch once.

Most patches only work with a specific release or version of Mailman. Make sure
the patch you are trying to apply will work for the version of Mailman that you
are using!!! Each release of Mailman gives more features and greater stability.
Much of the source code changes with each release/upgrade, so different patches
may be needed for each release.

In general patches are only written for the current version of Mailman. If you
installed Mailman from rpm or apt (or from anything other than source, then
patching will NOT work for you).

Patching:

 - Download the source code and expand it. It will create a directory with all sorts of files and subdirectories (this will be called the "build" directory)

 - Download the patch and gunzip it (if necessary)

 - Make the "build" directory your current directory. Apply the patch with the MM build directory as current working directory using the command: 
   patch -p1 < path-to-patch-file

The patch runs through the source files and makes modifications. Now you build
Mailman like normal (read the file INSTALL in the build directory):

  ./configure --any switches you need
  make install

If you keep the build directory hanging around then you can apply patches at
any point, and then simply re-install.

One of the really nice features of Mailman is that you can re-install it at any
time and it leaves your data/config alone, but refreshes the binaries - so
applying a patch is a snap! Be sure to turn off mailmanctl before doing a
rebuild or you might loose some mail that is being processed inside a queue.

If you have any more questions, don't be shy in asking on the 
mailman-users@python.org mailing list.

Jon Carnes - (this one is for you Paul!)

Edit this entry / Log info / Last changed on Tue May 11 10:26:51 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.41. How do I deal with qrunner processes that suck up all available CPU?

This may be a problem with the integrated pipermail archive system. See the
thread in the archive of the mailman-users mailing list, starting with <http://
mail.python.org/pipermail/mailman-users/2004-May/036822.html>.

In short, you may need to change the archive period to be more frequent (i.e.,
from one month to one week), so as to give the qrunner process less work to do
each time a new message comes in.

Note that Mailman version 2.1.4 incorporated a change to help deal with this
issue, and 2.1.5 incorporated further changes to help deal with other
situations that can also cause "runaway" qrunner processes.

You could also be having a problem with excessive Python "pickle" contention.
Try splitting your list into smaller sub-lists, each of which is subscribed to
a parent "umbrella" list.

See <http://mail.python.org/pipermail/mailman-users/2004-May/037058.html> for
more information on this topic.

Edit this entry / Log info / Last changed on Mon May 31 00:41:21 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.42. I installed/upgraded Mailman, but now I get the error "NameError: global
name 'True' is not defined". What can I do?

See the thread including the message by Barry Warsaw at <http://mail.python.org
/pipermail/mailman-developers/2004-May/016889.html>.

In short, recent versions of Mailman 2.1.x need recent versions of Python to
support them, specifically Python 2.3. If you're using RedHat or Debian-Stable
versions of Linux, or MacOS X 10.2 "Jaguar", you may need to install Python 2.3
from source.

For further details, see <http://mail.python.org/pipermail/mailman-developers/
2004-May/016903.html>, <http://mail.python.org/pipermail/mailman-developers/
2004-May/016892.html>, and <http://mail.python.org/pipermail/mailman-developers
/2004-May/016893.html>.

Edit this entry / Log info / Last changed on Thu May 20 14:47:43 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.43. How can I add auto-numbering to the subject line of mailing list
messages?

There is a patch to enable such feature (add sequencial number in subject
prefix).

http://sourceforge.net/tracker/?func=detail&aid=601117&group_id=103&atid=300103

Edit this entry / Log info / Last changed on Tue May 25 13:58:09 2004 by Tokio
Kikuchi

-------------------------------------------------------------------------------

4.44. I've installed/upgraded Mailman, but now I get the error "KeyError:
'getgrnam(): name not found'" What can I do?

This question has been moved to section 1. See <http://www.python.org/cgi-bin/
faqw-mm.py?req=show&file=faq01.020.htp>.

Edit this entry / Log info / Last changed on Fri Jun 11 03:10:57 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.45. The admin web interface is not saving my changes -- what is wrong?

Have you changed the URL for the list, without completely following the
instructions at <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq04.029.htp>?

Have you changed the list to secure it via HTTPS/SSL/TLS, without completely
following the instructions at <http://www.python.org/cgi-bin/faqw-mm.py?req=
show&file=faq04.027.htp>?

Otherwise, you may be accessing the site via a redirect, which is tossing away
the POST changes. See also the entire thread including <http://mail.python.org/
pipermail/mailman-users/2004-April/035964.html> and another entire thread
including <http://mail.python.org/pipermail/mailman-users/2003-February/
026095.html>.

Edit this entry / Log info / Last changed on Wed Jun 2 14:34:55 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.46. file extension in pipermail archive is obj or bin, not xls or doc. why?

From the mailing list:

 > whenever I access it through the archives it gets relinked to certain
 > location where the extension is either .obj or .bin ..problem is whenever a
 > user is accessing it.. he is asked on what application to open it with.. can
 > this prevented... that is still retains its attachments original extension
 > files. ex. .xls .doc?

Mailman archiver is designed to attain safety against malicious files like
viruses. Typical virus mail has an attachment like this:

 > ------=_NextPart_000_001B_01C0CA80.6B015D10
 > Content-Type: audio/x-wav;
 >         name="message.scr"
 > Content-Transfer-Encoding: base64
 > Content-ID: <031401Mfdab4$3f3dL780$73387018@57W81fa70Re>
 >
 > TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Windows will execute this code if the extension is kept as .scr which is not an
audio/x-wav file extension.

Some mailers are not compliant with the IANA registered mime extension and put
application/octet-stream content-type for excel or msword. In this case,
mailman is very cautious not to trust the original extention and change it to
.bin. If these files are sent by a compliant mailer like netscape, the
content-type will be application/msword or application/vnd.ms-excel and the
extension will become .doc or .xls.

So, if you really don't care the safety, you must hack the code at your own
risk.

Oh, there is an option to use an external archiver... http://www.python.org/
cgi-bin/faqw-mm.py?req=show&file=faq04.003.htp

Edit this entry / Log info / Last changed on Tue Jun 8 13:16:31 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.47. Virtual domain hosting with Mailman?

From a recent thread on the mailman-users mailing list (see <http://
mail.python.org/pipermail/mailman-users/2004-June/037415.html>), Richard
Barrett said:

 With unmodified, standard Mailman source code, there is a single 
 namespace for list names shared by all virtual host supported by a 
 given Mailman installation. Multiple installations on the same machine 
 can be used to avoid the list naming restrictions this creates.

 The modified version of Mailman, shipped by Cpanel as part of their 
 commercial hosting product for ISPs, adopts a different, 
 list-name-munging solution to the problem but Cpanel have not made the
 modified source code generally available in the public domain.

I would like to add that running the CPanel software, or hosting your list at
CPanel, will require that you get all your Mailman support from them. See <
http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq06.011.htp> for
details.

Another option would be to look at patches to try to supply this kind of
functionality with older versions of Mailman. Looking through the set of
Mailman patches at SourceForge, we run across <http://sourceforge.net/tracker/
index.php?func=detail&aid=943827&group_id=103&atid=300103>, which was developed
for use under Debian stable with their binary package version 2.1.1-5.1 of
Mailman. However, this patch may require significant effort to get it to work
with more modern versions of Mailman (e.g., 2.1.5).

Searching the archives of the mailman-users mailing list will likely turn up
other discussions of this subject, especially the thread at <http://
mail.python.org/pipermail/mailman-users/2004-June/037439.html>, which is
another excellent discussion of the subject by Richard Barrett.

Edit this entry / Log info / Last changed on Mon Jun 14 18:46:32 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.48. How can I change the HTML templates used by my mailing lists?

Richard Barrett has answered this question in two excellent messages on the
mailman-users mailing list. See <http://mail.python.org/pipermail/mailman-users
/2004-June/037493.html> and <http://mail.python.org/pipermail/mailman-users/
2004-June/037646.html>.

Summarizing from these messages:

If you change templates or add new per-list templates, for instance, you have
to run mailmanctl -restart to get the qrunner to restart so that it will
rebuild its template cache and in the process pick up new or changed templates.

Note that /bin/arch always starts with its own empty template cache each time
it runs.

Also:

You should put your site and list specific changes as described in these
comments form $prefix/Mailman/Utils.py:

     # Make some text from a template file.  The order of searches depends on
     # whether mlist and lang are provided.  Once the templatefile is found,
     # string substitution is performed by interpolation in `dict'.  If `raw'
     # is false, the resulting text is wrapped/filled by calling wrap().
     #
     # When looking for a template in a specific language, there are 4 places
     # that are searched, in this order:
     #
     # 1. the list-specific language directory
     #    lists/<listname>/<language>
     #
     # 2. the domain-specific language directory
     #    templates/<list.host_name>/<language>
     #
     # 3. the site-wide language directory
     #    templates/site/<language>
     #
     # 4. the global default language directory
     #    templates/<language>
     #
     # The first match found stops the search.  In this way, you can specialize
     # templates at the desired level, or, if you use only the default
     # templates, you don't need to change anything.  You should never modify
     # files in the templates/<language> subdirectory, since Mailman will
     # overwrite these when you upgrade.  That's what the templates/site
     # language directories are for.
     #
     # A further complication is that the language to search for is determined
     # by both the `lang' and `mlist' arguments.  The search order there is
     # that if lang is given, then the 4 locations above are searched,
     # substituting lang for <language>.  If no match is found, and mlist is
     # given, then the 4 locations are searched using the list's preferred
     # language.  After that, the server default language is used for
     # <language>.  If that still doesn't yield a template, then the standard
     # distribution's English language template is used as an ultimate
     # fallback.  If that's missing you've got big problems. ;)

Edit this entry / Log info / Last changed on Sat Jun 26 22:43:17 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.49. How do I enable automatic alias generation with sendmail?

Question: When you use Mailman with the postfix MTA, there are some nice
additional tools to automate the generation of the aliases that are required,
when you create a new mailing list. Can you do the same with Sendmail?

Answer: Yes. Ed Greenberg posted a message to the mailman-users mailing list on
this subject. See <http://mail.python.org/pipermail/mailman-users/2004-June/
037514.html> for details.

Hopefully we'll be able to convince him to upload this patch to SourceForge and
get this incorporated into a future version of Mailman.

Edit this entry / Log info / Last changed on Sun Jun 27 23:26:43 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.50. I just did a portupgrade on FreeBSD, but now I get the error
"ImportError: No module named os" -- what can I do?

Question: I just did a portupgrade on FreeBSD, so that I could get the latest
version of all the port definitions. However, after I did so, the mailman web
interface broke and I got the following errors in my mail log:

 Command output: Could not find platform independent libraries <prefix>
 Could not find platform dependent libraries <exec_prefix>
 Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
 Traceback (most recent call last):
 File "/usr/local/mailman/scripts/post", line 31, in ?     import paths
 File "/usr/local/mailman/scripts/paths.py", line 27, in ?     import os
 ImportError: No module named os

Answer: This problem appears to happen occasionally when you do a portupgrade
on a system where one port (such as mailman) depends on another port (such as
python), and may be exacerbated when the system is upgraded while they are
running.

If this happens to you, the first thing you need to do is to shut off the mail
system and to turn off mailman, so as to minimize the damage while you try to
fix the system.

Searching the archives of the FreeBSD mailing lists, the typical solution
appears to be to go to the directory of the depending port (e.g., /usr/ports/
lang/python), and do the following (as root):

 make deinstall; make clean; make install

Then, stop and start mailman. This has worked for us.

Edit this entry / Log info / Last changed on Tue Jun 29 13:26:21 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.51. How do I limit the rate at which Mailman sends mail?

From the thread on the mailman-users mailing list at <http://mail.python.org/
pipermail/mailman-users/2004-July/038195.html>:

Some people choose to install Mailman at home on a consumer-grade Internet
connection, or with a hosting company, and the provider will only allow them to
send so many messages in a given period of time.

Unfortunately, Mailman has no way to control the rate at which messages are
sent. Depending on how you have configured various settings, Mailman may dump
100 or 500 recipients or more into what it considers to be a single outgoing
message. The MTA which receives this long list of subscribers might then break
that up into smaller chunks of five or ten recipients for some medium-size
sites, fifty or a hundred or more recipients for large sites, and lots of
smaller sites with just one or two recipients.

Your ISP would look at this flow of outgoing messages, and could decide to
count this in a variety of ways. They could count the total number of
recipients, regardless of the number of messages. They could count the number
of connection attempts, regardless of number of recipients. They could count
the number of bytes flowing across the connection in a given period of time.
They could do almost anything.

If you want to do rate limiting, you need to do that within the MTA, but this
is an issue that you should bring up on mailing lists, newsgroups, etc... that
are specific to your MTA -- there's nothing that Mailman can do to help you. If
your MTA does not provide any features to allow you to control this, you should
consider upgrading your account, switching ISPs to one that does not rate limit
you, or moving your list to a hosting provider (see <http://www.python.org/
cgi-bin/faqw-mm.py?req=show&file=faq01.017.htp>).

Edit this entry / Log info / Last changed on Thu Jul 22 03:13:39 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.52. How do I change the subject line format for subscription confirmations?

From the thread on the mailman-users mailing list at <http://mail.python.org/
pipermail/mailman-users/2004-July/038235.html>:

Question: Is there any way to modify the Subject of the invite message?
"confirm aad5624d316c46234928426cb4a6c57c691d0e7c" is a DEAD WRINGER for spam,
and is likely to be deleted by many of our users.

Answer: In $MAILMAN_HOME/Mailman/Defaults.py, we find the following section of
comments & code:

 # For nicer confirmation emails, use a VERP-like format which encodes the
 # confirmation cookie in the reply address.  This lets us put a more user
 # friendly Subject: on the message, but requires cooperation from the MTA.
 # Format is like VERP_FORMAT above, but with the following substitutions:
 #
 # %(confirm)s -- the list-confirm mailbox will be set here
 # %(cookie)s  -- the confirmation cookie will be set here
 VERP_CONFIRM_FORMAT = '%(addr)s+%(cookie)s'

 # This is analogous to VERP_REGEXP, but for splitting apart the
 # VERP_CONFIRM_FORMAT.
 VERP_CONFIRM_REGEXP = r'^(?P<addr>[^+]+?)\+(?P<cookie>[^@]+)@.*$'

 # Set this to Yes to enable VERP-like (more user friendly) confirmations
 VERP_CONFIRMATIONS = No

So, all you need to do is put "VERP_CONFIRMATIONS = Yes" into your mm_cfg.py in
$MAILMAN_HOME/Mailman, then stop and restart mailman.

Note that this feature is not enabled by default within mailman, due to the
fact that it requires VERP support from the MTA, which may not be available.
Before enabling this feature, ensure that your MTA is configured to support
VERP.

There are a number of other FAQ entries which discuss VERP, and you can read
them by clicking on this link: <http://www.python.org/cgi-bin/faqw-mm.py?query=
verp&querytype=simple&casefold=yes&req=search>.

Also note that this only works for the invitation messages. This feature does
not work for other types of confirmation messages. See <http://mail.python.org/
pipermail/mailman-developers/2004-February/016513.html> and <http://
mail.python.org/pipermail/mailman-developers/2004-February/016558.html> for
more on this and a patch.

Thanks to Mark Sapiro!

Edit this entry / Log info / Last changed on Fri Jul 23 23:08:28 2004 by Brad
Knowles

-------------------------------------------------------------------------------

4.53. Why has my change to mm_cfg.py been ignored?

Changes made to mm_cfg.py will not automatically affect existing mailing lists.
It is not sufficient to simply stop, then restart, mailmanctl.

As an example, if you had changed the DEFAULT_URL_PATTERN record in etc/mailman
/mm_cfg.py from :

  DEFAULT_URL_PATTERN = 'http://%s/cgi-bin/mailman'

... to:

  DEFAULT_URL_PATTERN = 'https://%s/cgi-bin/mailman'

... then you will also need to run:

  ${PREFIX}/bin/fix_url crochet-discuss

... where 'crochet-discuss' is the name of the mailing list.

After you have done this, and after you have restarted mailmanctl, the links in
the generated admin screen (for example, https://www.foo.org/mailman/admin/
crochet-discuss/general) will correctly begin with 'https://', instead of
'http://', that is, the generated html will now look similar to:

  ...
  <li><a href="https://www.foo.org/mailman/admin/crochet-discuss/general"> <strong>[General Options]</strong></a>
  <li><a href="https://www.foo.org/mailman/admin/crochet-discuss/passwords"> Passwords</a>
  <li><a href="https://www.foo.org/mailman/admin/crochet-discuss/language"> Language&nbsp;options</a>
  ...

Edit this entry / Log info / Last changed on Sun Jul 25 03:56:05 2004 by James
Sinnamon

-------------------------------------------------------------------------------

4.54. How do I get qrunner to run under daemontools?

it is possible to run qrunner under daemontools, but you have to do a little
setup. Because of the fact that mailmanctl forks, you can't use it to start and
stop the qrunner processes under daemontools. The solution is to run /usr/local
/mailman/bin/qrunner directly.

To start, create a directory called /usr/local/mailman/supervise:

 mkdir /usr/local/mailman/supervise

We will need two scripts in this directory. The first should be called /usr/
local/mailman/supervise/run, and should contain the following text:

 #!/bin/sh
 exec setuidgid mailman ./qrunner

Now, create /usr/local/mailman/supervise/qrunner with the following text:

 #!/bin/sh
 # execute a pass through the queues once per minute
 while `/bin/true`; do
    # perhaps code to check for and remove locks should go here???
    /usr/local/mailman/bin/qrunner --runner=All --once
    sleep 60
 done

We should fix up the permissions on this directory now. This is not strictly
necessary, but it will stop check_perms from complaining:

 cd /usr/local/mailman
 chmod 2755 supervise
 chmod 755 supervise/*
 chown mailman:mailman supervise -R

Finally, just link /usr/local/mailman/supervise into your /service directory:

 ln -s /usr/local/mailman/supervise/ /service

The qrunner process should start within 5 seconds. Use svstat to verify things
are working properly.

Edit this entry / Log info / Last changed on Sun Jul 25 20:27:21 2004 by Joseph
c. Lininger

-------------------------------------------------------------------------------

5. Downloading and installing Mailman

-------------------------------------------------------------------------------

5.1. How do I import an archive into a new mailing list?

If your archive is in mbox format copy it to

   archives/private/<list>.mbox/<list>.mbox  (where <list> is your list name)

then run

    bin/arch <list> archives/private/<list>.mbox/<list>.mbox

If it's not in mbox format then you need to figure out how to convert it from
whatever form it's in to mbox.

Note, that public archive files are just a softlink to

  archives/private/<list>.mbox/<list>.mbox

so you don't have to do the same job twice.

Edit this entry / Log info / Last changed on Tue Apr 20 11:21:58 2004 by Martin
Mokrejs

-------------------------------------------------------------------------------

5.2. Will Mailman work with Windows 2000?

Yes. Here are the steps required to make it work using exim as an MTA and IIS
as a web server.

Pre-requisites

Warning: This list of cygwin required components may be incomplete. Send
changes to transom@extremelogic.com.

Cygwin (www.cygwin.com)

    * Admin:
          o Cron
          o Cygrunsrv
    * Base
    * Devel
          o Autoconf
          o Automake
          o Make
          o GCC
    * Interpreters
          o Python

You must set the system environment variable CYGWIN=ntsec for mailman to work

Exim (ftp://ftp.uni-erlangen.de/pub/pc/gnuwin32/cygwin/porters/Humblet_Pierre_A
/V1.1/)

   1. Open the cygwin bash prompt
   2.  cd /
   3. tar xvzf /path/to/exim-4.04-cygwin-1.3-bin.tar.gz
   4. You should read /usr/local/exim/README.CYGWIN. It has detailed info on exim.
   5. add exim user to windows 2000
   6. from the Windows 2000 command prompt: net user exim /add
   7. add group mm
   8. from the Windows 2000 command prompt net localgroup mm /add
   9. add exim to the mm group
  10. from the Windows 2000 command prompt net localgroup mm exim /add
  11. open cygwin prompt and fix passwd and group files
  12.  mkpasswd -l | grep exim >> /etc/passwd
  13. mkgroup -l | grep mm >> /etc/group
  14. change exim to be in mm group in /etc/passwd
  15. update /etc/aliases (include in doc)
  16. get /usr/local/exim/configure from CVS or production (include in doc)
  17. change hostname in configure to reflect the server's FQDN
  18. install the exim service cygrunsrv -I exim -e CYGWIN=ntsec -p /usr/local/bin/exim -a "-bdf -q15m"
  19. Change ownership on the /usr/local/exim directory
  20. chown -R exim.mm /usr/local/exim
  21. start the exim service cygrunsrv -S exim
  22. check the exim.log in /var/log/ for errors

Mailman (www.list.org)

Note: GNU Mailman running under cygwin on Windows 2000 cannot be used to create
private mailing lists. All mailing lists will be public.

   1. extract the mailman archive to a directory
   2. create a user named mailman
   3. from the Windows 2000 command prompt: net user mailman /add
   4. add mailman to the mm group
   5. from the Windows 2000 command prompt net localgroup mm mailman /add
   6. Set a password for the mailman user
   7. net user mailman newpassword
   8. create the cygwin user with mkpasswd
   9.  mkpasswd -l | grep mailman >> /etc/passwd
  10. make mm mailman's group in /etc/group
  11. Run the configure script for mailman
  12. cd /path/to/extracted/mailman
  13. Make sure mailman's home directory exists and is set GID
  14. chown -R mailman.mm /home/mailman/
  15. chmod g+s /home/mailman/
  16. ./configure --with-mail-gid=mm --with-cgi-gid=mm --with-groupname=mm --with-cgi-
      ext=.exe
  17. make
  18. make install
  19. copy wrapper.exe.exe from the installation (under src) directory to /home/mailman/mail/wrapper.exe
  20. login as mailman and run ~/bin/checkperms -f until it stops bitching
  21. change to the /home/mailman/mail/ directory and mv each .exe.exe file to .exe (remove the extraneous .exe)
  22. create the virtual directory mailman and point to c:\cygwin\home\mailmain\cgi-bin
  23. The vdir should have script and execute permissions
  24. Open properties on the mailman vdir, directory security, anonymous access, edit, click on edit under anonymous access - change the anonymous access user to mailman. Do not check "allow IIS to manage password"
  25. Add read permissions to "other" from the private archive directory
  26. chmod o+r /home/mailman/archives/private
  27. create the virtual directory pipermail and point to c:\cygwin\home\mailman\archives\private
  28. set the default doc to index.html
  29. copy cygwin1.dll into a dir on the path like c:\winnt\system32
  30. Install the cron service
  31. cygrunsrv -I cron -p /usr/sbin/cron -a -D
  32. logon as mailman and add the crontab
  33. cd /home/mailman/cron/
  34. crontab crontab.in

Common Problems

When you compile mailman, you must tell it an appropriate group id to run the
scripts under. The checkperms -f script will correct these permissions.
However, programs running under cygwin cannot bet setuid or setgid without
being patched specifically for cygwin because Windows does not incorporate
setuid or setgid functionality as UNIX does. For this reason several parts of
mailman must be tweaked manually to run as the mailman user.

Administration

To add new lists:

   1. run /home/mailman/bin/newlist
   2. add the output of the script to /etc/aliases

Test your mailman installation:

   1. Add a new list
   2. Open http://yourhost/mailman/listinfo.exe and subscribe yourself to the list (the list admin does not get list mail by default)
   3. send a test message to the list

Edit this entry / Log info / Last changed on Tue Aug 13 17:15:40 2002 by Todd
Ransom

-------------------------------------------------------------------------------

5.3. localhost.localdomain is used inside admin email URLs

When you have created a new list and send the list admin an email, that email
has localhost.localdomain for the URL (ie. http://localhost.localdomain/mailman
/)

How do I change this to reflect the correct hostname/path?

An obscure solution:

edit the mm_cfg.py that is under <path to mailman>/Mailman

Change the value for DEFAULT_HOST_NAME to your host.

Edit this entry / Log info / Last changed on Wed Jan 22 21:19:42 2003 by mECSR

-------------------------------------------------------------------------------

5.4. Configuring cron to run with the correct privileges, troubleshoot cron
error.

Note that this entry is only valid for Mailman version 2.0.x. Version 2.1 (and
later) uses a different mechanism that is not based on cron.

When you install mailman either from source or from rpm you need to configure
the cron jobs to run under the context of the user root. In your installation
directory it will create this file: (This is my installation directory, yours
might be /usr/local/mailman.)

/var/mailman/cron/crontab.in

You need to import this file as root with the following command crontab -u root
/var/mailman/cron/crontab.in (note that this will overwrite your existing
crontab, the correct way to do this is crontab -u root -l>>/var/mailman/cron/
crontab.in; and then crontab -u root /var/mailman/cron/crontab.in) Then reload
your crond daemon. service crond reload

It took me 4 months to track down this particular problem. This will spare you
the effort of being so incredible stupid about figuring this out. To reiterate
run the mailman deamons under user root, if you look at the scripts that run
you will see they have the suid bit set to run as mailman when called by root.
You will get these sorts of messages if you run cron jobs with the incorrect
error message:

Traceback (most recent call last):

  File "/var/mailman/cron/senddigests", line 94, in ?
    main()
  File "/var/mailman/cron/senddigests", line 82, in main
    mlist = MailList.MailList(listname, lock=0)
  File "/var/mailman/Mailman/MailList.py", line 101, in __init__
    self.Load()
  File "/var/mailman/Mailman/MailList.py", line 573, in Load
    dict, e = self.__load(file)
  File "/var/mailman/Mailman/MailList.py", line 539, in __load
    fp = open(dbfile)

IOError: [Errno 13] Permission denied: '/var/mailman/lists/mailman/config.pck'

Note that this entry is only valid for Mailman version 2.0.x. Version 2.1 (and
later) uses a different mechanism that is not based on cron.

Edit this entry / Log info / Last changed on Tue May 11 10:51:11 2004 by Brad
Knowles

-------------------------------------------------------------------------------

5.5. "Site list is missing: mailman" error after upgrade

After upgrading from 2.1b3 to 2.1.2, Mailman refused to restart. I used the
previously valid startup command:

   /usr/bin/python2 /home/mailman/bin/mailmanctl -s -q start

but received the error

   Site list is missing: mailman

This version of mailman requires a "site wide" list. Instructions for this are
included in section 4 (Final system set-up) of the INSTALL file.

See also <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.025.htp
>.

Edit this entry / Log info / Last changed on Thu Jun 24 12:02:38 2004 by Brad
Knowles

-------------------------------------------------------------------------------

5.6. Cannot Find/Receive Bounce Messages

I am trying to review the bounce messages - as with the new SPAM blockers if
you are not on the person's "approved" list you email may not go through. I
cannot seem to get the bounce message that Mailman cannot process - with or
without Mailman bounce processing on. I have tried setting to send the messages
to the list owner, and have set the posting by non-members to be held and
forwarded. I get one or two messages sent to the list admin, but when I turn
off processing and purposely send to a non-existent address - I receive nothing
back in either the master account for the domain, the list admin account or the
posters account.

What am I doing wrong?

Thanks in advance

Edit this entry / Log info / Last changed on Sat Feb 28 12:41:46 2004 by Lloyd
Tennison

-------------------------------------------------------------------------------

5.7. Where are my icons?

After you install Mailman on your server (as we did), you may not see any icons
at the bottom of the web pages Mailman generates (like the user interface web
page). Here's what we had to do in our situation (according to the tech support
people at VERIO Web Hosting):

Apache comes with the following directive by default.

Alias /icons/ "/usr/local/apache/icons/"

Mailman is looking for these images in the "/icons" directory so I copied them
to this location.

/usr/local/mailman/icons# cp * /usr/local/apache/icons/

Here's a tip. If you want to change one of the icons (say, to your company
logo), just place your company logo file in the icons folder (as per above),
and then change the name to the name of one of the icons Mailman uses. In our
situation, the "Powered by FreeBSD" logo was missing. The actual file name for
this logo was "powerlogo.gif." We added our logo to the icons folder and
changed the name to "powerlogo.gif" and that's it. Now at the bottom of our
Mailman web pages are the Mailman, Python, and company logos.

Edit this entry / Log info / Last changed on Tue Mar 9 19:19:54 2004 by 
Frederick Leonhardt

-------------------------------------------------------------------------------

5.8. What version of Python is required for Mailman 2.1.5?

This is the other side of the same issue discussed in Mailman FAQ entry 4.42 at
<http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq04.042.htp>.

See the thread including the message by Barry Warsaw at <http://mail.python.org
/pipermail/mailman-developers/2004-May/016889.html>.

In short, Mailman 2.1.5 (and later) need recent versions of Python to support
them, specifically Python 2.3. If you're using RedHat or Debian-Stable versions
of Linux, or MacOS X 10.2 "Jaguar", you may need to install Python 2.3 from
source.

For further details, see <http://mail.python.org/pipermail/mailman-developers/
2004-May/016903.html>, <http://mail.python.org/pipermail/mailman-developers/
2004-May/016892.html>, and <http://mail.python.org/pipermail/mailman-developers
/2004-May/016893.html>.

For more on the requirements of Mailman 2.1.5, see <http://www.python.org/
cgi-bin/faqw-mm.py?req=show&file=faq05.012.htp>.

Edit this entry / Log info / Last changed on Thu Jul 29 14:22:31 2004 by Brad
Knowles

-------------------------------------------------------------------------------

5.9. Adding a second version of Python to a server

You need to have Python 2.2 or better for Mailman - but what if you can't
upgrade your python because it will break other dependencies? Install an
additional version of Python, and point to it in your Mailman configure
statement. Python 2.3.3 is recommended as of the writing of this FAQ, check for
current recommendation and alter as needed.

 # wget http://www.python.org/ftp/python/2.3.3/Python-2.3.3.tgz
 # tar xvzf Python-2.3.3.tgz
 # cd Python-2.3.3
 # ./configure 
 # make 
 # make altinstall

The altinstall gives you a new version of Python without messing with the old
one(s).

Python2.3 is now at /usr/local/bin/python2.3 Your configure statement will then
include:

 # ./configure --with-mail-gid=mail --with-python=/usr/local/bin/python2.3

Edit this entry / Log info / Last changed on Mon Jun 7 19:27:32 2004 by Paul H
Byerly

-------------------------------------------------------------------------------

5.10. Deleted

Edit this entry / Log info / Last changed on Sun Jul 4 15:14:28 2004 by Brad
Knowles

-------------------------------------------------------------------------------

5.11. DELETED

Edit this entry / Log info / Last changed on Wed Jul 28 00:09:33 2004 by Brad
Knowles

-------------------------------------------------------------------------------

5.12. What MTA features are required for use with Mailman 2.1.5?

Your MTA needs to be configured to support VERP addressing. If you don't know
what VERP is, see <http://www.python.org/cgi-bin/faqw-mm.py?req=show&file=
faq02.002.htp>.

From the Mailman-2.1.5/NEWS file:

    - The bounce processor has been redesigned so that now when an address's
      bounce score reaches the threshold, that address will be sent a probe
      message.  Only if the probe bounces will the address be disabled.  The
      score is reset to zero when the probe is sent.  Also, bounce events are
      now kept in an event file instead of in memory.  This should help
      contain the bloat of the BounceRunner.

      New supporting variables in Defaults.py: VERP_PROBE_FORMAT,
      VERP_PROBE_REGEXP

      REGISTER_BOUNCES_EVERY is promoted to a Defaults.py variable.

These probe messages are sent out in VERP format, so if your MTA doesn't
support VERP, you're likely to have serious problems with users being
unsubscribed, etc....

Note that if you are running Mailman under CPanel, you should not upgrade to
version 2.1.5, unless you got that upgrade from CPanel themselves. All Mailman
support issues under CPanel should be directed to them. See <http://
www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq06.011.htp>.

If your server is at a hosting provider, you need to ask your provider if the
MTA is currently configured to support VERP. Do not upgrade Mailman to version
2.1.5 until you confirm that this support is in place.

There are a number of other FAQ entries which discuss VERP, and you can read
them by clicking on this link: <http://www.python.org/cgi-bin/faqw-mm.py?query=
verp&querytype=simple&casefold=yes&req=search>.

For more information on the requirements of Mailman 2.1.5, see also <http://
www.python.org/cgi-bin/faqw-mm.py?req=show&file=faq05.008.htp>.

Edit this entry / Log info / Last changed on Thu Jul 29 14:23:27 2004 by Brad
Knowles

-------------------------------------------------------------------------------

6. Integration issues (with mail or web servers)

-------------------------------------------------------------------------------

6.1. Failure to exec script. WANTED gid 31, GOT gid -1. (Reconfigure to take
-1?)

See also the Mailman FAQ entry at <http://www.python.org/cgi-bin/faqw-mm.py?req
=show&file=faq01.004.htp>.

The above is an example. If you get this message while accessing one of
mailman's CGI pages, it means you must run configure again and go through the
install process again, only this time use --with-cgi-gid=-1 (according to the
example in the question).

If you got this in a mail context (not CGI), you might try --with-mail-gid=-1.

You should also try running bin/check_perms -f , which may fix this problem.

If you installed from RPMs or some other form of binary package, you either
need to get different packages which are configured correctly, or you need to
install the correctly configured code from source.

Go to your source for your binary packages to see if they can help you with
this problem.

Edit this entry / Log info / Last changed on Sun Jul 4 15:18:50 2004 by Brad
Knowles

-------------------------------------------------------------------------------

6.2. MTA Performance Tuning Tips for EXIM

[Basically a placeholder for now. This information is taken from my general
Exim/Mailman howto at http://www.exim.org/howto/mailman.html which is also in
the Mailman distribution as README.EXIM]

List Handling Configuration

[These comments are for Mailman version 1.x and 2.0. It is likely that 2.1 may
have different requirements] Exim can be run in a similar way to sendmail -
with mailman run from aliases, or there is a recipe for automagic list support
by adding exim directors and transports to the exim config file. This is
described in the files refered to above.

Performance

[Also make sure you look at the generic MTA performance FAQ entries]

This is a set of configuration directives I used on the list boxes I admin.
Some of these are necessary, others are cosmetic, a few are probably
superfluous - they work for me!

 # definition of injecting IP addresses
 LOCAL_NETS=127.0.0.1/32
 #
 # Extra logging data - not necessary but makes the logs more
 # useful, but bigger
 # lookup all hostnames - puts hostnames into log as well as ips
 host_lookup = 0.0.0.0/0
 # tweak logging
 log_all_parents
 log_file_path = /var/log/exim/%s.log
 log_received_recipients
 log_refused_recipients
 log_received_sender
 log_smtp_confirmation
 #
 #
 # relay control - from our local network only
 host_accept_relay = LOCAL_NETS
 #
 #
 # Verify receipient addresses on everything except local injects
 # DO NOT verify addresses from mailman - this would slow down
 # the acceptance of messages dramatically
 receiver_verify_hosts = !127.0.0.1/8:0.0.0.0/0
 sender_verify
 #
 # performance tweaks - 1st is good for linux, maybe less so for others
 split_spool_directory
 remote_max_parallel = 15

Edit this entry / Log info / Last changed on Mon Mar 10 11:59:46 2003 by Nigel
Metheringham

-------------------------------------------------------------------------------

6.3. MTA Performance Tuning Tips for Sendmail

If you are interested in this topic, beg, borrow, or steal a copy of Nick
Christensen's book "Sendmail Performance Tuning" (see http://www.jetcafe.org/
~npc/book/sendmail/). This is the definitive work.

If you are looking for online resources with regards to this subject, see
Nick's slides for his talk entitled "Performance Tuning Sendmail Systems" (see 
http://www.jetcafe.org/~npc/doc/performance_tuning.pdf). Then there are the
slides for the talk by Brad Knowles entitled "Sendmail Performance Tuning for
Large Systems" (see http://www.shub-internet.org/brad/papers/sendmail-tuning/).
You may also be interested in Brad's paper "MTA Performance Comparison:
sendmail, postfix, & exim on *BSD" (see http://www.shub-internet.org/brad/
papers/mtacomparison/).

For papers that are specific to the issue of tuning sendmail for use with
mailing lists, read Rob Kolstad's paper "Tuning Sendmail for Large Mailing
Lists" (see http://www.usenix.org/publications/library/proceedings/lisa97/
21.kolstad.html), and Strata Chalup's paper "Drinking from the Fire(walls)
Hose: Another Approach to Very Large Mailing Lists" (see http://www.usenix.org/
publications/library/proceedings/lisa98/chalup.html).

================

Other comments (taken from the mailing lists):

  http://mail.python.org/pipermail/mailman-users/2001-May/011257.html
  http://mail.python.org/pipermail/mailman-developers/2001-August/009329.html

If you are using sendmail, consider moving to postfix, where you can configure
the system safely for deferring DNS lookups. That significantly speeds up
mailman's ability to queue messages. Sendmail has the same option (DeliveryMode
=defer) but you can't use it, because it disabled anti-spam checks and turns
you into an open relay. Grr. If you're running anything less than sendmail 8.11
-- upgrade, and spend some time configuring subdirectories to reduce I/O load
and contention in the directory structures (QueueDirectory=/var/spool/mqueue/q*
where * = at least 5 directories, with appropriate df/qf/xf subdirs).

If you aren't running DNS on that local machine, set up a caching-only name
server. You'll see a significant increase in yor performance by removing the
network interactions, even if you're dealing with a DNS server on your local
LAN.

-------

Okay, I've had a chance to test this, and tweak it a bit. Sorry for the delay
in responding -- I moved ISPs this last week, and the dust is just settling.

I did this a little differently, since I know that tweaking sendmail.cf files
gives many people hives, and so people aren't likely to do it. It's also
unneccesary.

You can do this without modifying your sendmail files at all. Instead, in your
startup script, add:

        /usr/sbin/sendmail -bd -ODeliveryMode=defer \
                -ODaemonPortOptions=Name=MSA,Port=NNNN,M=E,Addr=127.0.0.1

Where NNNN is some port number not otherwise used (you can test if something's
in use by doing "telnet localhost NNNN" -- if it's refused, there's no daemon
listening)

This sets up a sendmail process listening to the alternate port, in DEFER mode,
but set to talk only to the localhost interface, so it's not accessible by
anyoneother than your local machine: no open relay problems.

To make mailman access that port, add this to your mm_cfg.py:

 # define alternate SMTP port
 SMTPPORT = 1313

I've been running this fine for about a week, and I'm quite comfortable with
it. It works with sendmail 8.10 and later and doesn't require a sendmail god to
implement. Ready for the FAQ, I'd say.

 ---

Both of these answers culled from messages by Chuq Von Rospach 
chuqui@plaidworks.com

Edit this entry / Log info / Last changed on Thu Jul 31 13:41:10 2003 by Brad
Knowles

-------------------------------------------------------------------------------

6.4. MTA Performance Tuning Tips for Postfix

See also the Postfix Performance Tuning page at <http://www.postfix.org/
TUNING_README.html>.

[Initial content taken from a group of messages on list with first in thread at
http://mail.python.org/pipermail/mailman-users/2001-December/016271.html - this
needs further editing. Nearly all content was from messages by Dan Wilder ]

As lists get larger, Postfix delivery with out-of-the box configuration really
slows down. The bottlenecks I've found are queue length and number of SMTP
processes, both of which default to values too small for large lists. I began
noticing pretty severe rate limiting at about 10,000 deliveries on a list.

To get the number of SMTP processes up, change "default_process_limit" in
main.cf.

  default_process_limit = 150

gave results I could live with. Note that this depends on your hardware,
especially on the speed of your disks.

Also, you can increase the max. number of recipients per message that Postfix
is willing to accept: smtpd_recipient_limit = 1000 to something larger (10.000
maybe)

There's an active message queue which also became a sticky wicket for us. Two
limits which affect this, and the values I arrived at by experiment are:

  qmgr_message_active_limit = 40000
  qmgr_message_recipient_limit = 40000

Default on both of these is 20.000. The comments in the config file say:

  # The qmgr_message_active_limit parameter limits the number of
  # messages in the active queue.

  # The qmgr_message_recipient_limit parameter limits the number of
  # in-memory recipients. This parameter also limits the size of the
  # short-term, in-memory destination status cache.

Rather use nqmgr than qmgr.

With these changes in place, a K6-350 with 128M RAM delivers messages averaging
perhaps 3K over a medium-speed DSL, without entirely saturating the DSL, at
load average below 2.0, without impinging on swap. It'll reach 20,000
recipients in a couple of hours.

Another common recommendation with Postfix is to set

  disable_dns_lookups = yes

a measure others have reported favorably on, but which I have not yet tried.

After changing parameters in main.cf, run "postfix reload" then look at the
process table to see if the postfix processes are running, and how many smtp
processes are working at it.

  mailq

gives a report of what's in process and is a great help in tuning.
Unfortunately, running mailq competes for the same resource (the disk). Thus
running mailq often will considerably slow down your delivery! A crude index of
what's out there is

  mailq | grep '^[A-Z0-9]' | wc -l

and variations.

[Following some other comments on mailing list, this from Ralf Hildebrandt]

I began to see signs of thrashing some place between a couple of thousand
recipients, and 10,000. There was what appeared to be a phase transition of
sorts, above which waits began to predominate and I believe delivery rate may
have even gone down.

I'm not getting anything like 1400 deliveries/minute (I don't think). I'll try
measuring delivery rate during this week's large announce-lists posting, then
next week set disable_dns_lookups and see what change there may be.

The DSL does get to be pretty much of a nuisance to run vi on over an ssh
connection, when the list mail is going out.

The local copy of BIND is something I'd neglected to mention. We use split DNS,
with internal copies of BIND claiming to be definitive for our domain,
forwarding queries with respect to other domains outside. The mailing list
server does auxilliary duty as second internal MX server, so there's a copy of
BIND running on it, as well as Postfix/Mailman. It looks to itself first for
all name resolution.

Edit this entry / Log info / Last changed on Tue Jul 13 01:08:05 2004 by Brad
Knowles

-------------------------------------------------------------------------------

6.5. MTA Performance Tuning Tips for Qmail

[Placeholder - can someone provide content]

Edit this entry / Log info / Last changed on Fri Dec 7 11:28:02 2001 by Nigel
Metheringham

-------------------------------------------------------------------------------

6.6. Mailman Performance Tuning for Mail Delivery

These are generic performance tuning tips - for tips for specific MTAs see the
other FAQs in this section. This stuff is likely to be Mailman 2.0 specific

General points:-

  - Disk throughput is the primary driver for any mail system,
    so don't cheat on your disk config
  - DNS lookups will be needed in bulk by your MTA - put a 
    local DNS cache on the machine itself or a machine very 
    close on a fast network.  See DNS performance FAQ entry
  - choose an MTA that you can handle and fits your needs 
    (postfix and exim are the most often recommended ones on the list)

[From http://mail.python.org/pipermail/mailman-users/2001-May/011257.html ]

A few things to check...

Make sure your batch size is small:

 SMTP_MAX_RCPTS = 10

Set your qrunner proc to live longer, and extend the lock life:

 QRUNNER_LOCK_LIFETIME = hours(10)
 QRUNNER_PROCESS_LIFETIME = minutes(15)
 QRUNNER_MAX_MESSAGES = 300

Set these to 20 hours, 2 hours and 50000

Here's why: qrunner doesn't process the queue FIFO. Instead, it opens up the
directory and processes the entries sequentially. This implies that if you get
a lot of stuff in the queue, qrunner will quit after 15 minutes and start over.

As qrunner processes stuff, it deletes those files. So as new stuff comes in,
they get stored as close to the start of the directory inode as possible, so
when qrunner quits and restarts, if it hasn't processed everything in the
directory, it starts over with newer stuff, leaving older stuff deep into the
inode -- and it'll never GET to that stuff deep in the inode until the system
quiets down and it's given a change to catch up. That means something could
literally stay in the queue forever if the system never slows down enough to
allow qrunner to clean up.

By extending qrunner's lifetime, you're allowing it to go much deeper into the
inode. It's STILL not FIFO (and this is why replies get seen before messages
being replied to, and why digests have messages scrambles; Barry and I have
talked at length about this and the queueing should be FIFO in 2.1....), but
you're a lot less likely to have a given message buried in the queue for hours.
It doesn't fix the problem -- but significantly (from my system'ss operations)
reduces it, and limits the impact when it does happen.

If you find you still have stuff waiting around -- after you make this change,
if you're still running 2 hours behind, extend it to 3 hours. But be aware that
if for some reason qrunner decides to wedge, you're locking up your system for
longer periods of time. Keeping an eye on things is good.

Edit this entry / Log info / Last changed on Mon Mar 10 12:02:59 2003 by Nigel
Metheringham

-------------------------------------------------------------------------------

6.7. Initial Mailman v2.0 with TMDA and MIME filtering HOW-TO

I built the system with Mailman 2.0, TMDA 0.58, Exim v3, mimefilter,
mimedecode, and procmail. You don't need mimedecode or mimefilter, but if you
want to do mime filtering (even with say strip mime or demime instead) adapting
the below is should be simple and obvious. Several aspects and details are
going to be specific to the default configurations and installations of the
relevant packages under Debian Linux. I will assume that you'll be Linux/Unix
savvy enough to both detect those points and work around them as needed.

Background:

  -- I decided to integrate the system via procmail as I also wanted to
  (easily) add MIME filtering and other forms of message rewriting to my
  list setup.

  -- A primary goal is that the system operates without requiring either
  SysAdm or list owner attention.

  -- As discussed on-list One of my wishes was that lists maintain
  whitelists which are private to them and thus not shared across the
  entire set of lists serviced by that host.  I will not detail how I
  went about that except in passing as current TMDA development
  initiatives will render that much simpler in future.

  -- Elegance was not a goal.  Workability and were.  There are several
  rough edges left.  Some are noted below.

How-To: -------

Getting the system working essentially has three steps:

  1) Getting Mailman working transparently under Exim without aliases.

  2) Getting TMDA configured, installed, and tested.

  3) Getting TMDA integrated with Exim and Mailman, using procmail as a
  glue layer.

You'll be wise to check and test each step as you go along. Finding the correct
source of faults with the whole system in place can be a confusing and
difficult exercise. Its easier to test and retest as you go along so as to
limit the set of possible items that need examination for error.

Step one is fairly simple: Just configure Exim as per the README.Exim or
Nigel's HOW-TO on exim.org. Make sure its all working and that Mailman
functions correctly in all regards as regards the MTA. You will have to update
and extend these configurations later to suit TMDA, but start out making sure
you have the base right. If Mailman doesn't work on your system its hardly
worth trying to get TMDA also not working.

Install TMDA on your system (Debian uses a $PREFIX of /usr). Make sure you get
at least version 0.58. For Debian this means installing from /testing. If you
use a later version (such as current CVS) read the changelog carefully as
you'll have to make some adaption in your setup to the new features in TMDA.
Those changes are left as an exercise for the reader.

Under ~list/ create the following directory hierarchy, with 0700 permissions
and owned by list.list (the Debian Mailman user):

    ~list/.tmda
    ~list/.tmda/cache
    ~list/.tmda/lists
    ~list/.tmda/logs
    ~list/.tmda/filters
    ~list/.tmda/pending
    ~list/.tmda/templates

Within that tree create the following empty files, each of zero length, owned
by list.list and with 0600 permissions:

    ~list/.tmda/lists/released
    ~list/.tmda/lists/whitelist.auto
    ~list/.tmda/lists/blacklist
    ~list/.tmda/lists/whitelist
    ~list/.tmda/filters/outgoing

The above files need to be created as otherwise TMDA will error when it fails
to find them. If you have particular scaling or performance issues with regard
to these address lists you might want to use the dbm or cdb forms instead (see
the TMDA documentation for details). If so, make sure you configure ~list/.tmda
/config to match.

Next copy the default TMDA message templates to:

    ~list/.tmda/templates/bounce.txt
    ~list/.tmda/templates/confirm_request.txt
    ~list/.tmda/templates/confirm_accept.txt

And rewrite them as suitable for your mailing lists.

Now create your TMDA crypt key in:

    ~list/.tmda/crypt_key

and your TMDA configuration in:

    ~list/.tmda/config

My ~list/.tmda/config reads:

    DATADIR = "/var/lib/mailman/.tmda/"
    MAIL_TRANSFER_AGENT = "exim"
    DELIVERY = "|/usr/bin/procmail -p -f $SENDER"
    RECIPIENT_DELIMITER = "+"
    CRYPT_KEY_FILE = "/var/lib/mailman/.tmda/crypt_key"
    BARE_APPEND = "/var/lib/mailman/.tmda/lists/whitelist"
    CONFIRM_APPEND = "/var/lib/mailman/.tmda/lists/whitelist.auto"
    CONFIRM_MAX_MESSAGE_SIZE = None
    TEMPLATE_DIR = "/var/lib/mailman/.tmda/templates/"
    ACTION_INCOMING = "confirm"
    ACTION_OUTGOING = "bare"
    FINGERPRINT = ["message-id", "from", "date"]
    FULLNAME = "My Extremely silly name for my filters that I need to change"
    HOSTNAME = "the.FQDN.of.my.system"
    LOGFILE_DEBUG = "/var/lib/mailman/.tmda/logs/tmda_debug.log"
    LOGFILE_INCOMING = "/var/lib/mailman/.tmda/logs/tmda_incoming.log"
    PENDING_BLACKLIST_APPEND = "/var/lib/mailman/.tmds/lists/blacklist"
    PENDING_BLACKLIST_APPEND = "/var/lib/mailman/.tmds/lists/deleted"
    PENDING_RELEASE_APPEND = "/var/lib/mailman/.tmda/lists/released"
    PENDING_WHITELIST_APPEND = "/var/lib/mailman/.tmda/lists/whitelist"
    TIMEOUT = "2w"

You'll specifically need to tailor the values of FULLNAME and HOSTNAME from the
above cases.

Note:

  I use a RECIPIENT_DELIMITER of "+" as I use "-" in several list names
  and don't want the two appearing to run together.  Make sure you read
  the caveats on using "+" in the TMDA FAQ before chosing the
  RECIPIENT_DELIMITER you want to use.

In my case ~list/.tmda/logs is actually a symlink to /var/log/TMDA which is
owned by list.list and of 0700 permissions and a matching logrotate stanza:

    /var/log/TMDA/procmail.log {
      monthly
      compress
      missingok
    }
    /var/log/TMDA/tmda-debug.log {
      monthly
      compress
      missingok
    }
    /var/log/TMDA/tmda-incoming.log {
      monthly
      compress
      missingok
    }

Next, and almost finally is to create the incoming filter with ownership
list.list and permissions 0600:

    ~list/.tmda/filters/incoming

The contents should be similar to:

    #
    # Note: First match wins, so sequence of rules is significant
    #

    #
    # Bounce blacklisters
    #

    from-file ~list/.tmda/lists/blacklist bounce

    #
    # Explicitly listed whitelist addresses get thru (explicitly because 
    # this is a hand maintained file)
    #

    from-file ~list/.tmda/lists/whitelist ok    

    #
    # Normal list members get thru. Repeat this line as needed for each
    # of your lists, replacing "listname" with each listname.
    #

    from-mailman -attr=members ~list/lists/listname ok

    #
    # Digest members get thru.  Repeat this line as needed for each of
    # your lists, replacing "listname" with each listname
    #

    from-mailman -attr=digest_members ~list/lists/listname ok

    #
    # Confirmed whitelist addresses get thru (the whitelist.auto file is
    # built by posters confirming themselves to TMDA)
    #

    from-file ~list/.tmda/lists/whitelist.auto ok

    #
    # Mail from the list control gets thru. Repeat these lines as needed
    # for each of your lists, replacing "listname" with each listname.
    #

    from listname-admin@myhost.dom ok
    from listname-owner@myhost.dom ok

Tailor to suit as noted. Soon the complexity (and hand effort of adding the
list-specific lines) will no longer be necessary as future versions of TMDA
will have a more flexible filter schema that supports variable replacement.

At this point you should be able to test your TMDA configuration (not including
the Mailman specific bits), manually injecting messages into tmda-inject while
having the environment variables SENDER and RECIPIENT appropriately set and
seeing that it doesn't error (check the bounces and ~list/.tmda/logs/
tmda-debug.log for specifics on errors).

Check your TMDA setup carefully and pay particular attention to file
permissions and ownership errors. Make sure you do all your testing as the
Mailman user ("list" for Debian).

You're now ready for the last step: integrating TMDA with Exim via procmail.
Place the procmail rc in:

  ~list/.procmailrc

Again, the ownership is list.list and permissions 0600. It should read
something like:

    VERBOSE=off
    SHELL=/bin/sh
    PATH=/usr/bin:/usr/local/bin:/usr/lib:/usr/local/lib
    LOGFILE=/var/log/TMDA/procmail.log
    MAILDIR=/var/lib/mailman/locks

    #
    # Commands don't get filtered
    #

    :0 w
    * ACTION ?? mailcmd
    |/var/lib/mailman/mail/wrapper $ACTION $TARGET

    #
    # Pass bounces thru directly.
    #

    :0 w
    * ACTION ?? mailowner
    * ^Return-path: <>
    | /var/lib/mailman/mail/wrapper $ACTION $TARGET

    #
    # Mail that has already been filtered by TMDA.  This is an added
    # header that is used to check for messages which have already been
    # processed by this TMDA/Mailman procmail filter.
    #

    :0 w
    * $^X-Mail-Filter-Recipient: $RECIPIENT
    {

      #
      # Send the message to Mailman for processing.  This is a separate
      # recipe to make the extensions for MIME filter etc below clearer
      # and simpler.  
      #

      :0 w
      * ACTION ?? post
      | /var/lib/mailman/mail/wrapper $ACTION $TARGET

      :0 w
      * ACTION ?? mailowner
      | /var/lib/mailman/mail/wrapper $ACTION $TARGET

    }

    #
    # Add the flag header so we can detect messages coming out of TMDA,
    # even as distinct from mail from other TMDA users.
    #

    :0 fw
    | /usr/bin/formail -A "X-Silly-Me-Mail-Filter-Recipient: $RECIPIENT"

    #
    # Everything else (-admin, -owner, and posts which haven't been
    # passed by TMDA) get filtered thru tmda-filter.  If you want
    # list-specific inbound filters instead globally shared lists,
    # insert a small script here which writes a message customised
    # inbound filter in (say) /tmp/tmda-inbound.$$, call tmda-filter
    # with a matching -I argument, and then delete the temporary inbound
    # filter afterwards.
    #

    :0 w
    | /usr/bin/tmda-filter -c /var/lib/mailman/.tmda/config

    #
    # Take the exit code from TMDA.  This allows procmail to return in
    # error to the MTA.
    # 

    EXITCODE=$?

Notes:

  1) Please use a different loop detection header than
  X-Silly-Me-Mail-Filter-Recipient.  Make something up that fits your needs.  

  2) The added loop detection will not be necessary with TMDA versions
  after 0.58 as they add their own custom header which you can use for
  host and recipient specific loop detection.

As I do want to do MIME and other message pre-processing, and I want the
ability to send messages which bypass the MIME filters, my ~list/.procmailrc is
a little more complex. Adapt as appropriate to your installation:

    VERBOSE=off
    SHELL=/bin/sh
    PATH=/usr/bin:/usr/local/bin:/usr/lib:/usr/local/lib
    LOGFILE=/var/log/TMDA/procmail.log
    MAILDIR=/var/lib/mailman/locks

    #
    # Commands don't get filtered
    #

    :0 w
    * ACTION ?? mailcmd
    |/var/lib/mailman/mail/wrapper $ACTION $TARGET

    #
    # Pass bounces thru directly.
    #

    :0 w
    * ACTION ?? mailowner
    * ^Return-path: <>
    | /var/lib/mailman/mail/wrapper $ACTION $TARGET

    #
    # Mail that has already been filtered by TMDA.  This is an added
    # header that is used to check for messages which have already been
    # processed by this TMDA/Mailman procmail filter.
    #

    :0 w
    * $^X-XXX-Filter-Recipient: $RECIPIENT
    {

      #
      # Posts sent to a list
      #

      :0 w
      * ACTION ?? post
      {

        # 
        # Got an X-MimeStripNo header?  Don't MIME strip
        #

        :0 w
        * ^X-MimeStripNo
        {

          #  
          # Remove the X-MimeStripNo header
          #

          :0 fw
          | formail -I X-MimeStrip

        }

        :0 w
        * ! ^X-MimeStripNo
        {

          #
          # Mimefilter -- As mimefilter requires that the local directory
          # be writable and that various environment variables are defined, 
          # we use a wrapper script.
          #

          :0 fw
          | ~archiver/bin/MimeFilter 

          #
          #  Get rid of quoted-printable
          #

          :0 fw
          | /usr/bin/mimedecode
        }

        #   
        # If there's nothing left of the message (less than 20 bytes), don't
        # bother sending it forward to Mailman.
        #

        :0 wb
        * < 20
        | /dev/null

        #
        # We have a post!
        #

        :0 w
        | /var/lib/mailman/mail/wrapper $ACTION $TARGET

      }

      #
      # Messages to -admin and -ower
      #

      :0 w
      * ACTION ?? mailowner
      | /var/lib/mailman/mail/wrapper $ACTION $TARGET

    }

    #
    # Add the flag header so we can detect posts coming out of TMDA,
    # even as distinct from mail from other TMDA users.
    #

    :0 fw
    | /usr/bin/formail -A "X-XXX-Mail-Filter-Recipient: $RECIPIENT"

    #
    # Everything else (-admin, -owner, and posts) get filtered thru TMDA
    #

    :0 w
    | /usr/bin/tmda-filter -c /var/lib/mailman/.tmda/config

    #
    # Take the exit code from TMDA.  This allows procmail to return in
    # error to the MTA.
    # 

    EXITCODE=$?

Yeah, I could make that procmailrc cleaner and simpler. It works. Its a 1.0. I
don't argue with working 1.0 versions.

The contents of ~archiver/bin/MimeFilter are:

    #!/bin/bash
    #set -x

    #
    # Syntax:
    #
    #   MimeStrip
    #
    # Author: J C Lawrence <claw@kanga.nu>
    #

    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    # Config (Edit this bit to suit your setup!)
    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

    # Where is mimefilter?

    mimefilter="/usr/bin/mimefilter"

    # Note: Make sure that the value of tmpdir is writeable by the UID or GID 
    # that this script will execute under.

    tmpdir="/tmp/mimestrip.${$}"

    # Do you want to save an mbox of the messages before filtering 
    # in $savefile_pre?   (0/1)

    feature_savepre=1

    # Do you want to save an mbox of the messages after filtering 
    # in $savefile_post?  (0/1)

    feature_savepost=1

    # File to save pre-filtered messages in

    savefile_pre="/tmp/mimestrip.prefilter"

    # File to same post filtered messages in

    savefile_post="/tmp/mimestrip.postfilter"

    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    # MimeFilter setup (Don't edit from here down)
    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

    # The name of the mailing list this message is intended for.  Used
    # as the return address of the warning issued to the orginal author
    # if the message is not already clean.

    export list=${USER} 

    # The address of the mailing list this message is intended for.
    # Used in the X-Loop field of the warning issued to the original
    # author if the message is not already clean.

    export listaddr=${USER}@${DOMAIN}

    # The administrative (owner) address of the mailing list this
    # message is inteded for.  Used in the return address of the warning
    # issued to the original author if the message is not already clean.

    export listreq=${USER}-admin@${DOMAIN} 

    # The email address of the maintainer of the mailing list this
    # message is inteded for.  If it is defined, it is used to send the
    # maintainer original carbon copies of messages that have been
    # modified by this filter -- if filter_mime_cc_maintainer is
    # affermative, of course.

    export maintainer=${USER}-owner@${DOMAIN} 

    # A boolean flag: if affermative (i.e., if it matches the /y/i Perl
    # regular expression), the mimefilter script will send carbon copies
    # of every cleaned (modified) message to the maintainer of the
    # mailing list the message is intended for.

    export filter_mime_cc_maintainer=${copy_maintainer}

    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
    # Start working
    # -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

    # Build a safe working directory for mimefilter.  We need to do this as 
    # postfix by default will execute this script in /var/spool/postfix and 
    # mimefilter will try and create temp directories and files under there,
    # which will fail on permissions.

    # Note: Make sure that the value of tmpdir is writeable by the UID or GID 
    # that this script will execute under.

    /bin/rm -rf ${tmpdir} 2> /dev/null
    /bin/mkdir ${tmpdir}
    cd ${tmpdir}

    /bin/cat | ${mimefilter}

    # Save copies of the before and after messages off to a temp file for 
    # analysis/debugging

    if [ ${feature_savepre} -eq 1 ]
    then
      /bin/cat ${holdfile} >> ${savefile_pre}
      echo >> ${savefile_pre}
    fi

    if [ ${feature_savepost} -eq 1 ]
    then
      /bin/cat ${workfile} >> ${savefile_post}
      echo >> ${savefile_post}
    fi

    # Clean up

    /bin/rm -rf ${tmpdir}
    exit 

Most of all that hassle is to get the environment variables mimefilter needs
properly set and get the current working directory set to something I've write
permissions to (eg /tmp). I could have done all that inside procmail (and
probably should have). I didn't for purely historical reasons.

mimedecode is a little tool which flattens quoted-printable into US ASCII. I
find it rather useful. Take care here if you have posters posting with other
character sets.

As always: caveat emptor.

At this point you should be able to test your procmail and TMDA configuration
my injecting messages into procmail as below:

    # List posts
    cat message_file | procmail -mpt ACTION='post' TARGET='listname' ~list/.procmailrc

    # -request commands
    cat message_file | procmail -mpt ACTION='mailcmd' TARGET='listname' ~list/.procmailrc

    # -admin and -owner messages
    cat message_file | procmail -mpt ACTION='mailowner' TARGET='listname' ~list/.procmailrc

Test it all out. Make sure TMDA is properly holding messages. Release,
whitelist, blacklist etc them via:

  tmda-pending -c ~list/.tmda/config

and watch the contents of ~list/.tmda/lists to ensure that the right things are
happening.

Work the whole system until you're certain that everything in the procmailrc
and your TMDA configuration has been exercised and shown correct.

Don't forget to go back and delete all your test addresses from the files under
~list/.tmda/lists. Remember to NOT delete the files and to do all your testing
as the list user (file permissions are critical).

You're now ready for the last step: Integrating with your MTA. Before we get to
the message details some discussion on why I used Exim for this:

  I started out wanting and trying to use Postfix.  The core problem I
  hit was making the filter process hung off the alias execute as the
  right user.  I needed it to execute as the list user so it would be
  able to read the Mailman list configurations to verify subscribers
  from there.  If you want to entirely separate posting rights from list
  subscription this wouldn't be a problem (just remove the mailman bits
  from ~list/filters/incoming).

  Getting the filter process to execute as 'list' under postfix proved a
  bitch.  Its not impossible, but it requires either messing with
  content filters (which have a high performance overhead for ALL
  messages) or doing a vtable setup with a custom transport, which
  requires that you run your lists from a sub-domained virtual host (eg
  lists.kanga.nu).  I am unwilling to make either compromise.  YMMV.

  Exim conversely makes defining the specific effective UID and GID for
  an alias filter process very easy.  I also like, know, and use Exim on
  other systems, so moving my list server back to Exim was no great
  pain.

  I don't like and refuse to use either Sendmail, and so can't comment
  on how TMDA may be integrated with it.  I'm also not a QMail user and
  so can't comment for similar reasons.  For those familiar with those
  respective MTAs you should be able to base off the below recipes (with
  the above caveats) successfully.

The full set of Mailman-specific sections needed in /etc/exim/exim.conf are as
below. Compare them carefully with the Nigel's HOW-TO for basic Exim/Mailman
integration. The changes aren't large but you should understand each of them.
The documentation at exim.org is excellent.

These configurations are mildly Exim v3 specific. Exim v4 is noticeably but not
fundamentally different. Use the documentation at exim.org to guide you thru
the differences.

  In the main configuration section:

    # Add the Mailman user (list) to the value of trusted users so
    # procmail can use -p to pass environment variables through
    # (required by TMDA)

    trusted_users = mail:list

    # $HOME dir for Mailman

    MAILMAN_HOME=/var/lib/mailman

    # Wrapper script for Mailman

    MAILMAN_WRAP=MAILMAN_HOME/mail/wrapper

    # UID and GID for Mailman

    MAILMAN_UID=list
    MAILMAN_GID=mail

  In the transports section:

    list_transport:
      driver = pipe
      command = /usr/bin/procmail -mpt ACTION='post' TARGET='${lc:$local_part}' MAILMAN_HOME/.procmailrc
      # command = MAILMAN_WRAP post ${lc:$local_part}
      current_directory = MAILMAN_HOME
      home_directory = MAILMAN_HOME
      user = MAILMAN_UID
      group = MAILMAN_GID
      return_path_add
      delivery_date_add
      envelope_to_add
      environment = EXTENSION=${substr_1:$local_part_suffix}:\
                    RECIPIENT=$local_part$local_part_suffix@$domain 

    list_request_transport:
      driver = pipe
      command = /usr/bin/procmail -mpt ACTION='mailcmd' TARGET='${lc:$local_part}' MAILMAN_HOME/.procmailrc
      # command = MAILMAN_WRAP mailcmd ${lc:$local_part}
      current_directory = MAILMAN_HOME
      home_directory = MAILMAN_HOME
      user = MAILMAN_UID
      group = MAILMAN_GID
      return_path_add
      delivery_date_add
      envelope_to_add
      environment = EXTENSION=${substr_1:$local_part_suffix}:\
                    RECIPIENT=$local_part$local_part_suffix@$domain 

    list_admin_transport:
      driver = pipe
      command = /usr/bin/procmail -mpt ACTION='mailowner' TARGET='${lc:$local_part}' MAILMAN_HOME/.procmailrc
      # command = MAILMAN_WRAP mailowner ${lc:$local_part}
      current_directory = MAILMAN_HOME
      home_directory = MAILMAN_HOME
      user = MAILMAN_UID
      group = MAILMAN_GID
      return_path_add
      delivery_date_add
      envelope_to_add
      environment = EXTENSION=${substr_1:$local_part_suffix}:\
                    RECIPIENT=$local_part$local_part_suffix@$domain 

  In the directors section:

    list_owner_director:
      driver = smartuser
      require_files = MAILMAN_HOME/lists/${lc:$local_part}/config.db
      suffix = "-owner"
      new_address = "${lc:$local_part}-admin@${domain}"

    owner_list_director:
      driver = smartuser
      require_files = MAILMAN_HOME/lists/${lc:$local_part}/config.db
      prefix = "owner-"
      new_address = "${lc:$local_part}-admin@${domain}"

    list_admin_director:
      driver = smartuser
      suffix = -admin
      require_files = MAILMAN_HOME/lists/${lc:$local_part}/config.db
      transport = list_admin_transport

    list_request_director:
      driver = smartuser
      suffix = -request
      require_files = MAILMAN_HOME/lists/${lc:$local_part}/config.db
      transport = list_request_transport

    list_director:
      driver = smartuser
      require_files = MAILMAN_HOME/lists/${lc:$local_part}/config.db
      transport = list_transport
      suffix = +*
      suffix_optional   

Pay particular attention to the value of "suffix" in list-director and make
sure it agrees with RECIPIENT_DELIMITER in ~list/.tmda/config.

You should now be up and running with TMDA integrated as the front face of your
Mailman lists. If you want to monitor your TMDA pending queue for your Mailman
installation you can periodically run

  tmda-pending -c ~list/.tmda/config

Or just drop the following cronjob in the Mailman user's crontab:

  * * * * * /usr/bin/tmda-pending -CTb | mail you@domain.tld

to be emailed regular status updates on new messages in TMDA pending.

Edit this entry / Log info / Last changed on Tue Oct 15 06:38:29 2002 by J C
Lawrence

-------------------------------------------------------------------------------

6.8. Improving performance by local DNS caching

The MTA used by Mailman will in general require to do a whole pile of DNS
lookups to route the addresses handed to it by Mailman (there are exceptions,
like if you just push all the addresses to an upstream smart host or over UUCP
- in which case you have moved the problem to the next hop).

To improve the performance of the MTA - especially the latency on message
sending - it is worth ensuring that there is a local fast DNS cache. By local
we mean on the same machine where the MTA is running. Another alternative,
usually not as good, would be on a local 100Mbit network (or faster), so within
a millisecond of ping time. Since the MTA for any resonably loaded lists will
tend to do a whole bunch of very similar lookups in any period, adding a local
cache for the DNS lookups can be a huge performance boost - to the point where
its worth putting the cache on the local machine even though it is stealing
some cycles from the MTA itself. You could even do a two-level system of
caching, where the DNS server on the local machine forwards unknown queries to
a central caching DNS server on the network.

The selection and configuration of such a DNS cache is outside what can be
explained here. There are two primary options that you can use:

1. Dan Bernstein's DNS cache - part of his djbdns tools. This is a small single
function program which performs as a DNS cache for this purpose and is written
to be secure. The downside is that some people have problems with the licensing
of DJB's code, and it works in its own little universe of programs with strange
methodology for those coming at it from outside. See <http://cr.yp.to/
djbdns.html> for details.

2. The standard BIND DNS server, configured as a DNS cache. This is typically
not configured to be secure "out-of-the-box", has been in existence much longer
and therefore has historically had more security issues, and may use larger
amounts of RAM on your system. Your Unix distribution should provide
appropriate packages for this -- but make sure any configuration is for caching
only restricted to local hosts (see the Team Cymru "Secure BIND Template" for
one example of how to configure your BIND server securely, at <http://
www.cymru.com/Documents/secure-bind-template.html>).

Note that BIND is much better documented than djbdns, given that there are many
books written on this subject (especially "DNS and BIND" by Albitz and Liu from
O'Reilly & Assoc. at <http://www.oreilly.com/catalog/dns4/index.html>, and "DNS
and BIND Cookbook", by Cricket Liu from O'Reilly & Assoc. at <http://
www.oreilly.com/catalog/dnsbindckbk/index.html>), as well as the main BIND page
from the Internet Systems Corporation at <http://www.isc.org/sw/bind/>. There
is also commercial support available for BIND, both from the ISC (see <http://
www.isc.org/ops/support/>) and other vendors. Finally, there are many
commercial packages from other vendors which are based on BIND, see <http://
www.isc.org/sw/bind/vendorware.php>.

--------

Moreover, real-world experience with regards to performance of these two
options can differ. Read the slides for the talk by Brad Knowles entitled
"Domain Name Server Comparison: BIND 8 vs. BIND 9 vs. djbdns vs. ???" (see 
http://www.shub-internet.org/brad/papers/dnscomparison/) and note that, on the
test systems in question, dnscache never came close to the level of performance
provided by BIND 8 and BIND 9.

For example, for the root zone, dnscache did 98 queries per second vs. almost
500 queries per second for BIND 8 and over 300 queries per second for BIND 9,
and for the .tv zone dnscache could only do 16 queries per second vs. 90 for
BIND 8 and 55 for BIND 9. See page 90 of the version of these slides presented
at the RIPE 44 meeting in Amsterdam for more information.

In addition, djbdns (which includes dnscache) depends on a number of external
packages which must also be installed in order for the software to function. It
appears to have been developed in response to security issues in much older
versions of BIND. However, while BIND has adapted and evolved (to make most of
these concerns a non-issue), djbdns has not.

Moreover, there are many aspects to the DNS protocol that djbdns does not
support, at least not by default. If you want or need any of these features,
then you have to figure out how to turn them on, or find a patch from a
third-party (not accepted by the author, otherwise it would have been in the
package) which adds the feature in question.

Edit this entry / Log info / Last changed on Sun Jul 4 15:38:17 2004 by Brad
Knowles

-------------------------------------------------------------------------------

6.9. I get a "RuntimeError: command failed: /usr/sbin/postalias" when creating
lists and Postfix is my MTA

If you use Postfix as your MTA and get an error like this when creating a new
list:

    Traceback (most recent call last):
      File "/usr/local/mailman/scripts/driver", line 87, in run_main
        main()
      File "/usr/local/mailman/Mailman/Cgi/create.py", line 55, in main
        process_request(doc, cgidata)
      File "/usr/local/mailman/Mailman/Cgi/create.py", line 217, in process_request
        sys.modules[modname].create(mlist, cgi=1)
      File "/usr/local/mailman/Mailman/MTA/Postfix.py", line 226, in create
        _update_maps()
      File "/usr/local/mailman/Mailman/MTA/Postfix.py", line 47, in _update_maps
        raise RuntimeError, msg % (acmd, status, errstr)
    RuntimeError: command failed: /usr/sbin/postalias /usr/local/mailman/data/aliases (status: 1, Operation not permitted)

The problem is most likely that your permissions are wrong on data/aliases and/
or data/aliases.db. As README.POSTFIX states:

    Make sure that the owner of the data/aliases and data/aliases.db
    file is `mailman' and that the group owner for those files is
    `mailman'.  E.g.:

    % su
    % chown mailman:mailman data/aliases*

You should also ensure that data/aliases and data/aliases.db are group
writable. E.g.:

    % chmod 0664 data/aliases*

BAW: Note that I think it's also possible to get this error if you have Postfix
installed in a non-default location, e.g. /usr/local/bin/postfix. In that case,
be sure the variables POSTFIX_ALIAS_CMD and POSTFIX_MAP_CMD point to the right
paths for the respective executables. If not, make the appropriate changes in
your mm_cfg.py file.

Edit this entry / Log info / Last changed on Tue Jul 8 06:55:10 2003 by Barry
Warsaw

-------------------------------------------------------------------------------

6.10. Mail bouncing when tried to send mail to "mailman-owner@some.domain.com"

Hi,

I downloaded and installed Mailman, When a list is created i got a mail from "
mailman-owner@localhost.com" , but when i tried to mail to the id 
mailman-owner@localhost.com, the mail bounces back.

The head of the bounced mail:

Date: Thu, 17 Jul 2003 14:37:36 +0530 From: Mail Delivery System <
Mailer-Daemon@localhost.com> To: root@localhost.com Subject: Mail delivery
failed: returning message to sender

This message was created automatically by mail delivery software (Exim).

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

  mailman-owner@localhost.com
    unknown local-part "mailman-owner" in domain "localhost.com"

------ This is a copy of the message, including all the headers. ------

Can any tell me whats the problem and how to enable the mailid "mailman-owner".

Thanks in advance

Edit this entry / Log info / Last changed on Thu Jul 17 11:10:45 2003 by Ankur

-------------------------------------------------------------------------------

6.11. Mailman and CPanel

A lot of folks use Mailman through CPanel, and they come to the Mailman mailing
lists for help when they have trouble. There's usually little we can do to help
with these specific problems, and you should contact CPanel for assistance. If
they contact us, we'll work with them as necessary to fix any bugs in Mailman
that affect its use with CPanel.

Here's some information provided by CPanel for the Mailman FAQ:

    The primary place [for support] would be our forums
    http://forums.cpanel.net/ That's the most public place 
    we have for airing problems with cpanel and usually we try 
    to make sure there's resolutions to those problems. If it's a more
    specific problem they can put a support ticket in with us at 
    http://support.cpanel.net/  (The support site also has links to 
    our forums, faq page, documentation, and access to submit bugs and 
    feature requests)

Edit this entry / Log info / Last changed on Tue Nov 4 05:41:27 2003 by Barry
Warsaw

-------------------------------------------------------------------------------

6.12. Mailman + postfix + amavisd-new HOWTO *

2004-04-08 - This is a first draft. Comments are welcome. This file is released
under the GNU Free Documentation License (FDL, see below).

2004-06-28 - See also the amavisd-new page at <http://www.ijs.si/software/
amavisd/>, and the various "HowTo" documents and other information available,
further down the page at <http://www.ijs.si/software/amavisd/#doc>.

2004-07-14 - Note that this example uses the postfix "after-queue content
filter" technique. See <http://www.postfix.org/FILTER_README.html>. This means
that you have to accept the spam, process is through amavisd-new &
SpamAssassin, and if the message is to be rejected, then the MTA (or amavisd)
has to generate a bounce back to the envelope sender address. An alternative
method is to use the postfix "before-queue content filter" technique (see <
http://www.postfix.org/SMTPD_PROXY_README.html>), a.k.a., "smtpd proxy". The
after-queue method is more scalable and more robust in the face of high loads,
but has the problem that you're left trying to bounce garbage once you've
accepted it. The before-queue method is less scalable and easier to get into
situations where you effectively cause a DoS attack on yourself when loaded,
but rejects the spam outright so that the sending machine has to try to deal
with any bounce. This is generally considered to be better behaviour, so that
your server is more secure against being abused as a "joe-job" amplifier.

2004-09-13 - Alt.: reinject mail via port 10025

-----

INTRODUCTION: Installing the antispam/antivirus amavisd-new on a mailing-list
server poses a serious performance issue: when the server sends out thousands
of emails to the mailing-list subscribers, some of these subscribers return
bounce messages, which can number in the hundreds and might clog the antivirus
daemon if you're not careful.

Here's how we do it on http://listes.rezo.net/

1) Before all, make sure you run postfix v2.x, otherwise the FILTER feature
will not be here. Configure postfix so that it accepts scanned messages from
amavisd-new on localhost:10025

Add to /etc/postfix/master.cf the following lines:

    localhost:10025 inet n  -       n       -       -       smtpd
        -o content_filter=
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o strict_rfc821_envelopes=yes
        -o smtpd_error_sleep_time=0  
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000

2) Configure amavisd-new the usual way, so that it accepts incoming mail on
localhost:10024 (or any other port you choose) and sends it back into the mail
queue via localhost:10025; this is very standard, but I guess the settings is
as follows, in /etc/amavis/amavis.log:

    $inet_socket_port = 10024;
    @inet_acl = qw( 127.0.0.1 );
    $max_servers  =  2; # two servers max at the same time

3) Define a smtp-amavis service on postfix, so that it can be invoked later:

Add to /etc/postfix/master.cf:

    smtp-amavis unix -      -       n       -       2       lmtp
        -o smtp_data_done_timeout=1200

Note here that the maximum number of processes running in parallel (2) is the
same as in the amavisd-new configuration. You can increase both a bit if you
experience delays in delivery because of amavis, but that's out of the scope of
this HOWTO. 2 is fine for us, with a daily average of 10 emails to check per
minute (and a powerful computer).

4) Test your filter by sending messages locally through SMTP:10024

5) Configure postfix to send all emails through the filter EXCEPT those
messages that are only addressed to a list-bounces address :

Create the address regexp in /etc/postfix/amavis_check (do 'man regexp_table'
to get more information):

    !/-bounces@(my\.domain\.tld|other\.domain\.net)$/i
        FILTER smtp-amavis:[127.0.0.1]:10024

Modify /etc/postfix/main.cf to have the check_recipient_access use this regexp
table:

    smtpd_recipient_restrictions = permit_mynetworks
        check_client_access hash:$config_directory/access
        reject_unauth_destination
        check_recipient_access regexp:$config_directory/amavis_check
        # other UCE checks here

An alternative could be to place this line into mm_cfg.py:

    SMTPPORT = 10025

This way Mailman will use the same port as amavisd-new when returning scanned
mail to Postfix.

6) You're done. Check your log files and enjoy an almost spam- and virus-free
server.

7) Now you can focus on the viruses and politics that kill people in the real
world, and read "Global Aids: Myths and Facts" by Alec Irwin and Joyce Millen,
published by South End Press.

REFERENCES:

  AMaViS:          http://www.amavis.org/
  amavisd-new:     http://www.ijs.si/software/amavisd/
  Mailman:         http://www.list.org/
  postfix:         http://www.postfix.org/

        Copyright (c) 2004 PHILIPPE RIVIERE.
        Permission is granted to copy, distribute and/or modify this document
        under the terms of the GNU Free Documentation License, Version 1.2
        or any later version published by the Free Software Foundation;
        with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
        Texts.

        Updated references to amavisd-new home page by Brad Knowles.
        Added notes at the top for after-queue versus before-queue content
        filtering methods, and reasons for choosing one over the other.

Edit this entry / Log info / Last changed on Mon Sep 13 15:55:05 2004 by Pascal
Oberndoerfer

-------------------------------------------------------------------------------

6.13. Mailman and Allegroserve

Final system setup requires you to do something along the lines of

    ScriptAlias   /mailman/     $prefix/cgi-bin/

and

    Alias         /pipermail/   $varprefix/archives/public/

The above is very Apache! Here follows the Allegroserve counterpart. Make sure
you specify :setuid and :setgid arguments to net.aserve:start, otherwise the
server will attempt to run cgi scripts as root, which is a Bad Thing.

  ;;;;;;;;;;;

  (in-package "NET.ASERVE")

  ;; what is the url prefix for mailman queries?

  (defparameter *mailman-prefix* "/mailman/")

  ;; where is mailman installed? 

  (defparameter *mailman-root* "/usr/local/mailman/")
  (defun mailman-location (where)
    (merge-pathnames where *mailman-root*))

  ;; We want to convert "/mailman/admin/mylist" into
  ;; "/usr/local/mailman/cgi-bin/admin" and "/mylist"

  (defun mailman-cgi-names (uri-path)
    (let* ((uri-relative (uri-path (enough-uri uri-path 
                                               *mailman-prefix*)))
           (where (position #\/ uri-relative))
           (program-name (subseq uri-relative 0 where))
           (program (merge-pathnames program-name
                                     (mailman-location "cgi-bin/"))))
      (values (namestring program)
              (and where
                   (subseq uri-relative where)))))

  (defun cgi-error-output (req ent istream)
    (declare (ignore req ent))
    (let ((ostream (error-stream)))
      (loop
       (let ((char (read-char-no-hang istream nil :eof)))
         (cond ((null char) (return nil))
               ((eq :eof char) (return t))
               (t (write-char char ostream)))))))

  (defun mailman-invoke-cgi (req ent) 
    (let ((uri-path (request-decoded-uri-path req)))
      (multiple-value-bind (program path-info)
          (mailman-cgi-name uri-path)
        (unless (probe-file program)
          (error "Mailman program ~s not found." uri-path))
        (run-cgi-program req ent program
                         :path-info path-info
                         :error-output #'cgi-error-output))))

  (publish-prefix #| :host *my-vhost-names* |#
                  :prefix *mailman-prefix*
                  :function #'mailman-invoke-cgi)

  (publish-directory #| :host *my-vhost-names* |#
                     :prefix "/pipermail/"
                     :destination (namestring (mailman-location "archives/public/")))

Edit this entry / Log info / Last changed on Tue Jul 6 18:03:51 2004 by Nick
Levine

-------------------------------------------------------------------------------
The GNU Mailman mailing list / Mailman FAQ Wizard 1.0.3 / Feedback to Mailman
FAQ Owner

Python Powered

Note: to make changes to a FAQ entry, or to add a new entry, use the password 
Mailman when prompted. The password is case-sensitive. This FAQ is community
maintained under the honor system.
