====== Setup of the mailing lists ======

===== postfix =====

==== main.cf ====

<code>
# for administration of mlmmj
recipient_delimiter = +

# to talk to rspamd for dmarc munging

smtpd_milters=inet:127.0.0.1:11332
non_smtpd_milters=inet:127.0.0.1:11332

# to map the ml with the proper transport
virtual_alias_maps = hash:/usr/local/mlmmj/virtual
transport_maps = hash:/usr/local/mlmmj/transport
propagate_unmatched_extensions = virtual

</code>

==== master.cf ====

<code>
mlmmj    unix  -       n       n       -       -       pipe
    flags=ORhu user=mailman argv=/usr/local/bin/mlmmj-receive -F -L /usr/local/mlmmj/lists/$nexthop
</code>

===== rspamd =====
==== /usr/local/etc/rspamd/local.d/dmarc.conf ====

<code>
munging {
        list_map =  "/usr/local/mlmmj/rspamd-munging";
        mitigate_strict_only = true;
}
</code>

===== mlmmj =====

==== Layout ====

| /usr/local/mlmmj/bin | Home made scripts |
| /usr/local/mlmmj/lists | Directory containing all the mailing lists |
| /usr/local/mlmmj/www | static html assets |
| /usr/local/mlmmj/mboxes | list of mboxes corresponding to archive backups, if archives should be rebuilt they will be rebuilt from there |
| /usr/local/mlmmj/archived/mboxes | mboxes for archived mailing lists |
| /usr/local/mlmmj/archived/archives | html generated mailing archive for archived mailing lists |
| /usr/local/mlmmj/tmp | directory where the subscriptions will be handled |
| /usr/local/mlmmj/virtual* | virtual mapping of mailing lists email and internal email for postfix (run postmap on it once modified) should contain 1 entry per valid email for a given mailing list |
| /usr/local/mlmmj/transport* | mapping between internal email and transport for postfix) run porstmap on it once modified  should contain 1 entry per mailing list |
| /usr/local/mlmmj/webarchive | Directory where the html archives will be generated |
| /usr/local/mlmmj/rspamd-munging | list of mailing list to activate munging on, one per line |

==== maintenance (bounce etc) ====

the mlmmj-maintd daemon is run every 5 minutes to handle all the maintenance via crontab

==== crontabs ====

<code>
$ cat /usr/local/etc/cron.d/mlmmj
SHELL=/bin/sh
PATH=/bin:/usr/bin:/usr/local/bin
*/5     *       *       *       *       mailman /usr/local/mlmmj/bin/webarchives.sh
*/2     *       *       *       *       mailman /usr/local/mlmmj/bin/mlsubscriptions.sh
*/5     *       *       *       *       mailman /usr/local/bin/mlmmj-maintd -F -d /usr/local/mlmmj/lists
</code>

===== archives =====
Archives are run via hypermail
if a mailing lists have at its root directory a file named noarchive, then the incremental upgrade will stop running
The configuration is done in hmrc

==== /usr/local/etc/hmrc ====

<code>
folder_by_date = "%Y-%B"
usegdbm = "1"
indextable = "0"
reverse = "1"
reverse_folders = 1
describe_folder = "%B %Y"
require_msgids = 0
discard_dup_msgids = 0
gmtime = 1
monthly_index = 1
message_pattern = %.6d
file_by_thread = true
href_detection = true
thrdlevels = 2
monthly_index = 1
yearly_index = 1
thread_file_depth = 1
append=1

ihtmlheaderfile=        /usr/local/etc/hypermail/index-header.hyp
mhtmlheaderfile=        /usr/local/etc/hypermail/msg-header.hyp
ihtmlfooterfile=        /usr/local/etc/hypermail/index-footer.hyp
mhtmlfooterfile=        /usr/local/etc/hypermail/msg-footer.hyp
</code>

==== Regenerating from scratch the archives ====

:!: Do this as user mailman :!:

<code>
$ hypermail -c /usr/local/etc/hmrc -o append=0 -a "/index" -m /usr/local/mlmmj/mboxes/freebsd-test.mbox -l "freebsd-test: Test posting area" -u -x -d /usr/local/mlmmj/webarchive/freebsd-test
</code>

==== Incremental updates ====

The incremental updates are performed by crontab via the webarchives.sh script

It will take all untreated files from /usr/local/mlmmj/lists/**ml**/archive and pass them through mexport (from the mblaze package) to generate a mbox formatted output which will be piped to hypermail

The untreated files are the files which filename is a number higer then the last number recorded in the lastindex file

As a result the mbox of the mailing will be prepended with the new mails and the html archives updated

The script runs every 5 minutes

===== mlmmj-webview =====

Small CGI to handle subscription and links to archive
It should run as user %%**%%*nobody%%**%%*

It takes /usr/local/mlmmj/list as an entry file to determine what it should show to the user

The subscription will be written in files in /var/spool/mlmmj-webview/subscribe
The unsubscription will be written in files in /var/spool/mlmmj-webview/unscubsribe

a crontab will every 2 minutes look at new (un)subscriptions here and dedup them and proceed
The script being /usr/local/mlmmj/bin/mlsubscriptions.sh

===== Tools =====

| list_members [mailinglist] | list the members of a given mailing list |
| list_lists | list the public mailing lists |
| mkml.sh -n name -d description | create a new mailing list |
| webarchives.sh | incrementally update the web archives and mbox |
| mlsubscriptions.sh | deal with pending (un)subscriptions |
| msgconv.sh | lookup all html files in webarchive and try to convert to unicode what can be converted to it |