Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lYWbb-0006Io-5p for pgadmin-hackers@arkaria.postgresql.org; Mon, 19 Apr 2021 16:18:35 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1lYWba-0004vP-2D for pgadmin-hackers@arkaria.postgresql.org; Mon, 19 Apr 2021 16:18:34 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lYMkq-0004JW-8L for pgadmin-hackers@lists.postgresql.org; Mon, 19 Apr 2021 05:47:28 +0000 Received: from mail-oi1-x22e.google.com ([2607:f8b0:4864:20::22e]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lYMkn-0008AI-T2 for pgadmin-hackers@postgresql.org; Mon, 19 Apr 2021 05:47:27 +0000 Received: by mail-oi1-x22e.google.com with SMTP id u80so806494oia.0 for ; Sun, 18 Apr 2021 22:47:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=supabase.io; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=A3Xgp/8JjpeWjvPYnrBntgoMmIeyXBWX/MTJ9JzDiE0=; b=Z+5GgvImVJznwv0kqQy+eN3KEU2WLHAdJfgVnMwFvBvxK1+bsX4MZwlnY7oUltPPxl /j3LAqa/UAkyPBwoL8INoB6mcyPPVd7SJ7v8cHpHOg1RxiP9JyRJtY1y4sCmBX3F3Ka9 NA7phnocab9y1JNL2z7Seuxg2jf/R1Ho45uTp65Dd59x7uoZjfQGCQzvrwLtCX5HNb1K dax6xa8ZTWZPV25qwIfq4ywuRIy4mobubUn5YyzT7AIpPCNXW51Uwn8J/2m44tV8tbvO 7ZZ4G6VYz0dJgyoTJUWvKQ49t1gZnV9hChyKot4j1AAw8pwbvUPKRWN0GHcVAPtx7DG8 xGhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=A3Xgp/8JjpeWjvPYnrBntgoMmIeyXBWX/MTJ9JzDiE0=; b=s+9OuHk4bcpc3Gi9lhbPiA4X/JwNQek+qG969JQoj6L0pyjheqxVQ2YJo5QVvDt711 b1onUyIeCsHulg0Q0RQDKaghUituKfKzNCCXrLdochnyfXsSjzQrV1c0zpOYXWlFL9Kc J5/2gXvi+tDCiDtQ/D+fXylZRmKEIKys1AnMQKFpucuFDm0hDNImVA19eulGL1pQJMyu 5/VC34/Q3rvFoTll6m7ZdY3AHuEEHjNGBxRoISLsZ+KLPLXEmldB2ce7+lxzP587fW6/ B25oEJrmtv0DGt2GEMIbIYTIMl3ZOVfXWbV2+jEjF6tJDcSpu7iG2RO5MRGGI5xdXNn9 /bVg== X-Gm-Message-State: AOAM530YtxAcUrObkHU8z00NHd/kboEmPv55v/0zSLl/8eC0egfsNaUt pZ7h66ddT6HGADL/ozRfSAZwR3p5mn3AxET3Eb3WjgLcRuyZ1w== X-Google-Smtp-Source: ABdhPJzcgSneu5EXMPstonLPJIuwTwjtkYFIwiEi9jADWYM+WCm72cBDAMP7f4LucFARXbOLNOcJM4JamFvK28rRPXU= X-Received: by 2002:aca:5d43:: with SMTP id r64mr14514782oib.19.1618811244799; Sun, 18 Apr 2021 22:47:24 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Steve Chavez Date: Mon, 19 Apr 2021 00:47:13 -0500 Message-ID: Subject: Re: CLI for Schema Diff To: Dave Page Cc: pgadmin-hackers Content-Type: multipart/alternative; boundary="000000000000347ce405c04cddf6" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000347ce405c04cddf6 Content-Type: text/plain; charset="UTF-8" Hi Dave, Thanks for the thoughtful reply. I got sidetracked but now I'm able to continue work on the CLI. > 1) Agree a standard for command line syntax. > 3) Make the code modular and extensible; perhaps using a Python module per CLI module. Cool, I agree with the standard. From my understanding, the "core" CLI module would be in charge of registering the servers, and thus it'll be needed by all the other CLI modules right? Regarding the grant wizard, I'm not familiar with the module so I can't be of much help in implementing a CLI mode for it. However, I can definitely help with both core and schema diff. Would you agree if I scope my contribution to only those 2 modules? > 2) Figure out how to avoid the hacks in your existing code to do things like: Absolutely. I've been meaning to speed up the CLI start-up time, my implementation currently loads many unneeded modules. I'll also avoid the hack to hide stdout. > 4) Allow the use of saved connections as well as arbitrary connection strings. Agree. I think saved connections will come with the "core" CLI module implementation(the register-server command). On Thu, 1 Apr 2021 at 03:44, Dave Page wrote: > Hi > > On Thu, Apr 1, 2021 at 12:38 AM Steve Chavez wrote: > >> > but is this something you'd be interested in working on to become a >> more fully featured and production quality CLI? >> >> Yes, absolutely! >> > > Cool! > > >> What would be the next step? I'm all for doing a CLI that covers pgAdmin >> quality standards. >> > > I think there are a few things we'd need to do: > > 1) Agree a standard for command line syntax. CLIs can easily become very > messy - I've spent time designing such standards in the past > (unfortunately, I can't just share that), but some thoughts might be: > > - have a consistent > syntax. For example; > > pgacli --output-format=json schema-diff generate-diff --source=... > --target=... > pgacli core register-server --host=... --maintenance-db=... ... > pgacli grant-wizard grant-acl --select --type=tables > --objects=schema.tb_* > > (I've assumed a module name of 'core' for things directly related to > the heart of pgAdmin) > > - Actions are in the form "verb-noun" > - Every common option has both a short and a long form > - Uncommon options have a long form only > - Support various help options, showing appropriate info; > > pgacli -h > pgacli --help > pgacli core -h > pgacli schema-diff --help > > 2) Figure out how to avoid the hacks in your existing code to do things > like: > > - Hide initialisation output > - Setup the dummy SQLite database > > 3) Make the code modular and extensible; perhaps using a Python module per > CLI module. > > 4) Allow the use of saved connections as well as arbitrary connection > strings. > > I think 2 is the hardest issue. It may be that we add a flag to the main > application code that tells it it's running under the CLI, so that it can > avoid doing things it doesn't need to in those cases. One example is the > module loading; we might want to skip that entirely under the CLI, and have > the CLI module that's being called load just the modules it needs. > > >> >> On Wed, 31 Mar 2021 at 03:08, Dave Page wrote: >> >>> Hi >>> >>> On Tue, Mar 30, 2021 at 3:36 PM Steve Chavez wrote: >>> >>>> Hey all, >>>> >>>> In case anyone is interested, I've managed to enable a CLI mode for the >>>> Schema Diff on this repo: >>>> >>>> https://github.com/steve-chavez/pgadmin4/blob/cli/web/cli.py >>>> >>>> It basically works by using a Flask test client that interacts with the >>>> Schema Diff endpoints. >>>> It's a single isolated file, I haven't patched any of the existing >>>> modules. >>>> >>>> For a quickstart, there's also a docker image that can be used like: >>>> >>>> docker run supabase/pgadmin-schema-diff \ >>>> 'postgres://postgres@host/diff_source' \ >>>> 'postgres://postgres@host/diff_target' \ >>>> > diff.sql >>>> ## the stderr output shows the same messages as the Schema Diff GUI: >>>> >>>> Starting schema diff... >>>> Comparision started......0% >>>> Comparing FTS Dictionaries ...35% >>>> Comparing Functions ...50% >>>> Comparing Trigger Functions ...60% >>>> Comparing Sequences ...70% >>>> Comparing Tables ...80% >>>> Comparing Views ...90% >>>> Done. >>>> >>>> >>> That's an interesting approach! Obviously the code is just a proof of >>> concept at the moment (redirecting stdout is masking errors for example), >>> but is this something you'd be interested in working on to become a more >>> fully featured and production quality CLI? >>> >>> -- >>> Dave Page >>> Blog: http://pgsnake.blogspot.com >>> Twitter: @pgsnake >>> >>> EDB: http://www.enterprisedb.com >>> >>> > > -- > Dave Page > Blog: http://pgsnake.blogspot.com > Twitter: @pgsnake > > EDB: http://www.enterprisedb.com > > --000000000000347ce405c04cddf6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Dave,

Thanks for the thoughtful reply. I got sid= etracked but now I'm able to continue work on the CLI.

> 1) A= gree a standard for command line syntax.
> 3) Make the code modular a= nd extensible; perhaps using a Python module per CLI module.

Cool, I= agree with the standard. From my understanding, the "core" CLI m= odule would be in charge of registering the servers, and thus it'll be = needed by all the other CLI modules right?

Regarding the grant wizar= d, I'm not familiar with the module so I can't be of much help in i= mplementing a CLI mode for it.

However, I can definitely help with b= oth core and schema diff. Would you agree if I scope my contribution to onl= y those 2 modules?

> 2) Figure out how to avoid the hacks in your= existing code to do things like:

Absolutely. I've been meaning = to speed up the CLI start-up time, my implementation currently loads many u= nneeded modules. I'll also avoid the hack to hide stdout.

> 4= ) Allow the use of saved connections as well as arbitrary connection string= s.

Agree. I think saved connections will come with the "core&qu= ot; CLI module implementation(the register-server command).

On Thu, 1 Ap= r 2021 at 03:44, Dave Page <dpage@p= gadmin.org> wrote:
Hi

On Thu, Apr 1, 2021 at 12:38 AM Steve Ch= avez <steve@supab= ase.io> wrote:
> but is this something you'd be interested i= n working on to become a more fully featured and production quality CLI?=C2= =A0

Yes, absolutely!

Cool!
=C2=A0
What would be the next step? I'm all for do= ing a CLI that covers pgAdmin quality standards.

I think there are a few things we'd need to do:
=

1) Agree a standard for command line syntax. CLIs can e= asily become very messy - I've spent time designing such standards in t= he past (unfortunately, I can't just share that), but some thoughts mig= ht be:

=C2=A0 - have a consistent <command> = <command options> <module> <action> <action options>= ; syntax. For example;

=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0pgacli --output-format=3Djson schema-diff generate-diff --source=3D..= . --target=3D...
=C2=A0 =C2=A0 =C2=A0 =C2=A0=C2=A0pgacli=C2=A0cor= e register-server --host=3D... --maintenance-db=3D... ...
=C2= =A0 =C2=A0 =C2=A0 =C2=A0=C2=A0pgacli=C2=A0grant-wizard grant-acl --select -= -type=3Dtables --objects=3Dschema.tb_*

=C2=A0 = =C2=A0 (I've assumed a module name of 'core' for things directl= y related to the heart of pgAdmin)

=C2=A0 =C2=A0 -= Actions are in the form "verb-noun"
=C2=A0 =C2=A0 - Ev= ery common option has both a short and a long form
=C2=A0 =C2=A0 = - Uncommon options have a long form only
=C2=A0 =C2=A0 - Support = various help options, showing appropriate info;

= =C2=A0 =C2=A0 pgacli -h
=C2=A0 =C2=A0 pgacli --help
=C2= =A0 =C2=A0 pgacli core -h
=C2=A0 =C2=A0 pgacli=C2=A0schema-diff -= -help

2) Figure out how to avoid the hacks in your= existing code to do things like:

=C2=A0 =C2=A0 - = Hide initialisation output
=C2=A0 =C2=A0 - Setup the dummy SQLite= database

3) Make the code modular and extensible;= perhaps using a Python module per CLI module.

4) = Allow the use of saved connections as well as arbitrary connection strings.=

I think 2 is the hardest issue. It may be that we= add a flag to the main application code that tells it it's running und= er the CLI, so that it can avoid doing things it doesn't need to in tho= se cases. One example is the module loading; we might want to skip that ent= irely under the CLI, and have the CLI module that's being called load j= ust the modules it needs.
=C2=A0

On Wed, 31 Mar 2021 at 03:08, Dave Page <dpage@pgadmin.org> wrote:
=
Hi

On Tue, Mar 30, 2021 at 3:36 PM Steve Chavez <steve@supabase.io> wrote:
Hey a= ll,

In case anyone is interested, I've managed to enable a CLI = mode for the Schema Diff on this repo:


It basically works by using a Flask test client that inte= racts with the Schema Diff endpoints.
It's a single isolated = file, I haven't patched any of the existing modules.

For a quickstart, there's also a docker image that can be used l= ike:

docker run supabase/pgadmin-schema-diff \
  'postgres://postgres@host/diff_source' \
  'postgres://postgres@host/diff_target' \<=
br style=3D"box-sizing:inherit">  > diff.sql

## the=
 stderr output shows the same messages as the Schema Diff GUI:
Starting schema diff...Comparision started......0%
Comparing FTS Dictionaries ...35%
Comparing Functions ...50%
Comparing =
Trigger Functions ...60%
Comparing Sequence=
s ...70%
Comparing Tables ...80%
Comparing Views ...90%
Done.

That'= s an interesting approach! Obviously the code is just a proof of concept at= the moment (redirecting stdout is masking errors for example), but is this= something you'd be interested in working on to become a more fully fea= tured and production quality CLI?=C2=A0

--
Dave Page
Blog: http://pgsnake.blogspot.com
Twitte= r: @pgsnake

EDB: http://www.enterprisedb.com



--
--000000000000347ce405c04cddf6--