public inbox for [email protected]
help / color / mirror / Atom feed[PATCH] pglister: Add example script to import from mailman
6+ messages / 2 participants
[nested] [flat]
* [PATCH] pglister: Add example script to import from mailman
@ 2021-11-03 19:15 Célestin Matte <[email protected]>
2021-11-10 11:50 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Célestin Matte @ 2021-11-03 19:15 UTC (permalink / raw)
To: [email protected]
--
Célestin Matte
Attachments:
[text/x-patch] 0001-Add-example-script-to-import-from-mailman-tools-impo.patch (3.1K, 2-0001-Add-example-script-to-import-from-mailman-tools-impo.patch)
download | inline diff:
From 40ffdea780c631c30ceff14ca18496ff4aeecb23 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?C=C3=A9lestin=20Matte?= <[email protected]>
Date: Wed, 3 Nov 2021 20:13:03 +0100
Subject: [PATCH] Add example script to import from mailman
tools/import_from_mailman.sh
---
tools/import_from_mailman.sh | 54 ++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)
create mode 100755 tools/import_from_mailman.sh
diff --git a/tools/import_from_mailman.sh b/tools/import_from_mailman.sh
new file mode 100755
index 0000000..c8a46ec
--- /dev/null
+++ b/tools/import_from_mailman.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+
+# Script to import subscribers and email archives from mailman to pglister.
+# Lists must already be created in pglister.
+# Here is where to find some options in mailman administrative interface for each list:
+# - Subscription policy:
+# Privacy options -> "What steps are required for subscription?"
+# - Members view membership:
+# Privacy options -> "Who can view subscription list?"
+# - Moderation level:
+# Privacy options -> Sender filter -> "By default, should new list member postings be moderated?"
+
+ORIGIN="" # server hosting mailman
+TARGET="" # server hosting pglister
+PUBLIC_LISTS="" # space-separate list of public lists to import
+PRIVATE_LISTS="" # space-separate list of private lists to import
+DOMAIN="" # fqdn used for lists
+DJANGO_USER="list"
+PYTHON_VENV_PGLISTER="/srv/pglister/local/web/pglister/bin/python"
+PYTHON_VENV_PGARCHIVES="/srv/pgarchives/local/django/bin/python"
+PYTHON_VENV_PGARCHIVES_PRIVATE="/srv/pgarchives-private/local/django/bin/python"
+PGARCHIVES_PATH="/srv/pgarchives/local/"
+PGARCHIVES_PRIVATE_PATH="/srv/pgarchives-private/local/"
+PGLISTER_PATH="/srv/pglister/local/"
+
+function import_list() {
+ listname_full="$1"
+ listname=$(echo "$listname_full" | tr '[:upper:]' '[:lower:]')
+ echo "Loading subscribers from $listname_full"
+ ssh $ORIGIN /var/lib/mailman/bin/list_members -f -n -o "$listname"_nomail.txt "$listname_full"
+ ssh $ORIGIN /var/lib/mailman/bin/list_members -f -o "$listname"_mail.txt "$listname_full"
+ scp $ORIGIN:"$listname"_nomail.txt $TARGET:lists/
+ scp $ORIGIN:"$listname"_mail.txt $TARGET:lists/
+ ssh $TARGET sudo -u $DJANGO_USER $PYTHON_VENV_PGLISTER $PGLISTER_PATH/bin/load_subscribers.py -l "$listname"@$DOMAIN -a -f lists/"$listname"_nomail.txt --nomail
+ ssh $TARGET sudo -u $DJANGO_USER $PYTHON_VENV_PGLISTER $PGLISTER_PATH/bin/load_subscribers.py -l "$listname"@$DOMAIN -a -f lists/"$listname"_mail.txt
+ echo "Loading archives from $listname_full"
+ scp $ORIGIN:/var/lib/mailman/archives/private/"$listname".mbox/"$listname".mbox $TARGET:lists/
+ if [ "$2" == "public" ]
+ then
+ ssh $TARGET sudo -u $DJANGO_USER $PYTHON_VENV_PGARCHIVES $PGARCHIVES_PATH/loader/load_message.py -m lists/"$listname".mbox -l "$listname"
+ else
+ ssh $TARGET sudo -u $DJANGO_USER $PYTHON_VENV_PGARCHIVES_PRIVATE $PGARCHIVES_PRIVATE_PATH/loader/load_message.py -m lists/"$listname".mbox -l "$listname"
+ fi
+}
+
+for i in $PUBLIC_LISTS
+do
+ import_list "$i" public
+done
+
+for i in $PRIVATE_LISTS
+do
+ import_list "$i" private
+done
--
2.33.1
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [PATCH] pglister: Add example script to import from mailman
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
@ 2021-11-10 11:50 ` Magnus Hagander <[email protected]>
2021-11-11 10:49 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Magnus Hagander @ 2021-11-10 11:50 UTC (permalink / raw)
To: Célestin Matte <[email protected]>; +Cc: [email protected]
On Wed, Nov 3, 2021 at 8:15 PM Célestin Matte <[email protected]>
wrote:
>
> --
> Célestin Matte
This script makes a connection between the pglister and pgarchives code
which I'm not sure we want to "codify" quite so clearly. I think it might
for that reason be better suited for a wiki page or such?
It also makes a lot of assumptions about paths etc -- which would be OK for
a sample on a wiki page, but if we *do* want to makt it into a "proper part
of the codebase", it needs to be more generic.
--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/;
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/;
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [PATCH] pglister: Add example script to import from mailman
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-10 11:50 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
@ 2021-11-11 10:49 ` Célestin Matte <[email protected]>
2021-11-17 17:56 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Célestin Matte @ 2021-11-11 10:49 UTC (permalink / raw)
To: Magnus Hagander <[email protected]>; +Cc: [email protected]
> It also makes a lot of assumptions about paths etc -- which would be OK for a sample on a wiki page, but if we *do* want to makt it into a "proper part of the codebase", it needs to be more generic.
>
I moved all variables to the beginning of the script to make it as generic as possible — variables are given as examples.
If you prefer it to be on a wiki page instead of the main repository, fine.
Can I get editor access to the wiki?
Cheers,
--
Célestin Matte
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [PATCH] pglister: Add example script to import from mailman
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-10 11:50 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
2021-11-11 10:49 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
@ 2021-11-17 17:56 ` Magnus Hagander <[email protected]>
2022-01-19 10:35 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Magnus Hagander @ 2021-11-17 17:56 UTC (permalink / raw)
To: Célestin Matte <[email protected]>; +Cc: [email protected]
On Thu, Nov 11, 2021 at 11:49 AM Célestin Matte <[email protected]>
wrote:
> > It also makes a lot of assumptions about paths etc -- which would be OK
> for a sample on a wiki page, but if we *do* want to makt it into a "proper
> part of the codebase", it needs to be more generic.
> >
>
> I moved all variables to the beginning of the script to make it as generic
> as possible — variables are given as examples.
> If you prefer it to be on a wiki page instead of the main repository, fine.
> Can I get editor access to the wiki?
>
I'd be happy to do that -- but I can't figure out how :) It' already set to
"everyone with access", but I can't figure out how to grant you edit
permissions on it without granting you a high-level maintainer access to
the whole project.
Do you happen to know how to do that? Or does somebody else here with a bit
more gitlab experience perhaps know?
--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/;
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/;
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [PATCH] pglister: Add example script to import from mailman
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-10 11:50 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
2021-11-11 10:49 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-17 17:56 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
@ 2022-01-19 10:35 ` Célestin Matte <[email protected]>
2022-02-06 13:06 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
0 siblings, 1 reply; 6+ messages in thread
From: Célestin Matte @ 2022-01-19 10:35 UTC (permalink / raw)
To: Magnus Hagander <[email protected]>; +Cc: [email protected]
> I moved all variables to the beginning of the script to make it as generic as possible — variables are given as examples.
> If you prefer it to be on a wiki page instead of the main repository, fine.
> Can I get editor access to the wiki?
>
>
> I'd be happy to do that -- but I can't figure out how :) It' already set to "everyone with access", but I can't figure out how to grant you edit permissions on it without granting you a high-level maintainer access to the whole project.
>
> Do you happen to know how to do that? Or does somebody else here with a bit more gitlab experience perhaps know?
Back to this patch.
Wouldn't this be easier to put the script on postgresql's wiki? The gitlab wiki seems empty for now. (With a redirect from gitlab's wiki, maybe?)
--
Célestin Matte
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: [PATCH] pglister: Add example script to import from mailman
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-10 11:50 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
2021-11-11 10:49 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-17 17:56 ` Re: [PATCH] pglister: Add example script to import from mailman Magnus Hagander <[email protected]>
2022-01-19 10:35 ` Re: [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
@ 2022-02-06 13:06 ` Magnus Hagander <[email protected]>
0 siblings, 0 replies; 6+ messages in thread
From: Magnus Hagander @ 2022-02-06 13:06 UTC (permalink / raw)
To: Célestin Matte <[email protected]>; +Cc: [email protected]
On Wed, Jan 19, 2022 at 11:35 AM Célestin Matte
<[email protected]> wrote:
>
> > I moved all variables to the beginning of the script to make it as generic as possible — variables are given as examples.
> > If you prefer it to be on a wiki page instead of the main repository, fine.
> > Can I get editor access to the wiki?
> >
> >
> > I'd be happy to do that -- but I can't figure out how :) It' already set to "everyone with access", but I can't figure out how to grant you edit permissions on it without granting you a high-level maintainer access to the whole project.
> >
> > Do you happen to know how to do that? Or does somebody else here with a bit more gitlab experience perhaps know?
>
> Back to this patch.
> Wouldn't this be easier to put the script on postgresql's wiki? The gitlab wiki seems empty for now. (With a redirect from gitlab's wiki, maybe?)
Yeah, that would be a perfectly fine idea as well. And we certainly
know how the permissions work there :)
--
Magnus Hagander
Me: https://www.hagander.net/
Work: https://www.redpill-linpro.com/
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2022-02-06 13:06 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-11-03 19:15 [PATCH] pglister: Add example script to import from mailman Célestin Matte <[email protected]>
2021-11-10 11:50 ` Magnus Hagander <[email protected]>
2021-11-11 10:49 ` Célestin Matte <[email protected]>
2021-11-17 17:56 ` Magnus Hagander <[email protected]>
2022-01-19 10:35 ` Célestin Matte <[email protected]>
2022-02-06 13:06 ` Magnus Hagander <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox