Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rvH3j-00Aw5A-3o for pgsql-announce@arkaria.postgresql.org; Fri, 12 Apr 2024 13:35:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rvH3g-00EBKL-L2 for pgsql-announce@arkaria.postgresql.org; Fri, 12 Apr 2024 13:35:12 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rvH3f-00EBKB-Od for pgsql-announce@lists.postgresql.org; Fri, 12 Apr 2024 13:35:11 +0000 Received: from mahout.postgresql.org ([2001:4800:3e1:1::227]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rvH3b-000YSn-FZ for pgsql-announce@lists.postgresql.org; Fri, 12 Apr 2024 13:35:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=postgresql.org; s=20171124; h=Message-ID:Date:Reply-To:From:To:Subject: MIME-Version:Content-Type:Sender:Cc:Content-Transfer-Encoding:Content-ID: Content-Description:In-Reply-To:References; bh=//2EsIyx9LWdE+EWNoIQfU8tSHTBkYEilMKVI2lq1Ac=; b=DSD+TrN0YdifWsaGnddUsg2Usn bR/A9MvN5oBcDKqbKNniD55KV+dKtVE5mzTPqUhJdBU8NEpTir2gQGdTg93/hewKi4cj9Dmb56q3U C+5HsAMA55Xk5JkO0VcX//NRuMqop2PkXFYzuV1hzUhF4CH+/imnOZTnXqtE1EcNSSCv/eph5XWyE Wbcl47ptKueZ0oDeifgI2uE219iOwZh3ZMQx2LXATMYZYgXRtMWTsgb2g7Rq1szTJEm+132vmaQM9 pevboC1tXq/5WfdR4Y5cAeo1QwUp0V4K3q5WK6qRw3oVJTFBKA8Hp0UkftCe7DO3HC9Yn+X0hOGr/ cdKAmUIQ==; Received: from wrigleys.postgresql.org ([2a02:16a8:dc51::60]) by mahout.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rvH3Y-005cOh-L1 for pgsql-announce@lists.postgresql.org; Fri, 12 Apr 2024 13:35:05 +0000 Received: from localhost ([127.0.0.1] helo=wrigleys.postgresql.org) by wrigleys.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rvH3W-002B0D-Rx for pgsql-announce@lists.postgresql.org; Fri, 12 Apr 2024 13:35:03 +0000 Content-Type: multipart/mixed; boundary="===============1364933765273568868==" MIME-Version: 1.0 Subject: Greenmask v0.1.7 Release To: PostgreSQL Announce From: "Greenmask.io via PostgreSQL Announce" Reply-To: info@greenmask.io Date: Fri, 12 Apr 2024 13:34:04 +0000 Message-ID: <171292884413.677.2742917245185705322@wrigleys.postgresql.org> X-Auto-Response-Suppress: All Auto-Submitted: auto-generated X-pglister-tags: related X-pglister-tagsig: f756f9d22d0b8dd20255f84ea40d57a9f6d2f522265986760232895dedb56020 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --===============1364933765273568868== Content-Type: multipart/alternative; boundary="===============5116009812662550967==" MIME-Version: 1.0 --===============5116009812662550967== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable # Greenmask v0.1.7 Release ## PostgreSQL Dump and Obfuscation Tool This release introduces a new Greenmask command, improvements, bug fixes, a= nd documentation updates. # Greenmask Overview ## Key Features * **Cross-platform** =E2=80=94 can be easily built and executed on any plat= form, thanks to its Go-based architecture, which eliminates platform dependencies. * **Interactive** - allows to check the transformation result and show prob= lems if any using the single command `validate` * **Schema difference check** - validate your current database with the pre= vious dump in storage and show diff * **Database type safe** =E2=80=94 ensures data integrity by validating dat= a and utilizing the database driver for encoding and decoding operations. This approach guarantees the preservati= on of data formats. * **Transformation validation and easy maintainable** =E2=80=94 during obfu= scation development, Greenmask provides validation warnings and a transform= ation diff feature, allowing you to monitor and maintain transformations ef= fectively throughout the software lifecycle. * **Partitioned tables transformation inheritance** =E2=80=94 define transf= ormation configurations once and apply them to all partitions within partit= ioned tables, simplifying the obfuscation process. * **Stateless** =E2=80=94 Greenmask operates as a logical dump and does not= impact your existing database schema. * **Backward compatible** =E2=80=94 it fully supports the same features and= protocols as existing vanilla PostgreSQL utilities. Dumps created by Green= mask can be successfully restored using the pg_restore utility. * **Extensible** =E2=80=94 users have the flexibility to implement domain-b= ased transformations in any programming language or use predefined template= s. * **Declarative** =E2=80=94 Greenmask allows you to define configurations i= n a structured, easily parsed, and recognizable format. * **Integrable** =E2=80=94 integrate Greenmask seamlessly into your CI/CD s= ystem for automated database obfuscation and restoration. * **Parallel execution** =E2=80=94 take advantage of parallel dumping and r= estoration, significantly reducing the time required to deliver results. * **Provide a variety of storage** =E2=80=94 Greenmask offers a variety of = storage options for local and remote data storage, including directories an= d S3-like storage solutions. ### Greenmask is ideally suited for: * Routine backup and restoration tasks, ensuring data integrity and availab= ility. * Anonymization and data masking for staging environments and analytics, pr= otecting sensitive information while maintaining data utility. ## Release Notes ### v0.1.7 ##### New features * Added restoration filtering by `--table`, `--schema` and `--exclude-schem= a` parameters * Validate command without parameters validates only the configuration file * Added the `--schema` parameter, which allows to make a schema diff betwee= n the previous dump and the current. This=20 is useful when you want to check if the schema has changed after the migr= ation. By controlling it we can exclude=20 data leakage after migration * Validate command divided by many stages that can be controlled using para= meters * Configuration validation * Transformer validation * Constraint violation check * Data difference check * Schema difference check ##### Improvements * Improved Hash transformer=20 * Added salt parameter that can be set via config or via `GREENMASK_GLO= BAL_SALT` * Added sha3 functions support in different modes (sha3-224, sha3-256, = sha3-384, sha3-512) * Refactored `Cmd` transformer logic * Json API: Now it allows to use of column names instead of column inde= xes in JSON format * Csv API: Now it can use the column order from config via column remap= ping * The `validate` command was rewritten almost from scratch. * New option `--transformed-only` - displays only columns that are tran= sformed with a primary key (if exists). This allows to reduce the output data and make it more readable * Implemented `json` format for output * Added the `--table-format` parameter which is responsible for the `ve= rtical` and `horizontal` table orientation. This works only when `--format=3Dtext` * Added the `--warnings` parameter, if it is specified then not only fa= tal-warnings will be displayed, but also those with a lower severity ##### Fixes * Fixed `--use-list` option - now it applies to entries according to the or= der in the list file * Fixed `--use-list` option behavior together with `--list-format` option (= `json` or `text`). Now it generates a temporal list file in text format for providing it to the pg_= restore call * Updated documentation according to the latest changes ### v0.1.2 - v0.1.6 * Fixed uncontrolled buffer growth in the restore command * The `Hash` transformer has been completely remastered and now has the `fu= nction` parameter to choose from several hash algorithm options and the `ma= x_length` parameter to truncate the hash tail. * Split information about transformers between the `list-transformers` and = new `show-transformer` CLI commands, which allows for more comprehensible a= nd useful outputs for both commands=20 * Added error severity for the `Cmd` parameter validator * Significantly refactored the structure and content of documentation * Fixed metadata enrichment for validation warnings caused by `RawValueVali= dator` * Fixed a typo in the `credit_card` value for the `type` parameter of the `= Masking` transformer * Fixed Greenmask Playground environment variables and the `cleanup` command * Fixed `list-dump`, `list-transformers`, and `restore` commands exit code = on error * Fixed database connection parameters behavior fields * Fixed Json transformer: now it correctly parses operations field * Fixed bug when raw COPY lines were parsed incorrectly * Fixed `--version` parameter behavior ## Useful Links Change log [greenmask.io](https://greenmask.io/release_notes/greenmask_0_1_= 7/) Explore detailed [Documentation](https://greenmask.io/) Access the [Latest Release on GitHub](https://github.com/GreenmaskIO/greenm= ask/releases/tag/v0.1.7) Contact us for support at [Email Support](mailto:support@greenmask.io) --===============5116009812662550967== Content-Type: text/html; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Greenmask v0.1.7 Release
 

Greenmask v0.1.7 Release

Greenmask v0.1.7 Release

PostgreSQL Dump and Obfuscation= Tool

This release introduces a new Greenmask com= mand, improvements, bug fixes, and documentation updates.

Greenmask Overview

Key Features

  • Cross-p= latform =E2=80=94 can be easily built and executed on any platform= , thanks to its Go-based architecture, which eliminates platform dependencies.
  • Interac= tive - allows to check the transformation result and show problems= if any using the single command validate
  • Schema = difference check - validate your current database with the previou= s dump in storage and show diff
  • Databas= e type safe =E2=80=94 ensures data integrity by validating data an= d utilizing the database driver for encoding and decoding operations. This approach guarantees the preservati= on of data formats.
  • Transfo= rmation validation and easy maintainable =E2=80=94 during obfuscat= ion development, Greenmask provides validation warnings and a transformatio= n diff feature, allowing you to monitor and maintain transformations effect= ively throughout the software lifecycle.
  • Partiti= oned tables transformation inheritance =E2=80=94 define transforma= tion configurations once and apply them to all partitions within partitione= d tables, simplifying the obfuscation process.
  • Statele= ss =E2=80=94 Greenmask operates as a logical dump and does not imp= act your existing database schema.
  • Backwar= d compatible =E2=80=94 it fully supports the same features and pro= tocols as existing vanilla PostgreSQL utilities. Dumps created by Greenmask= can be successfully restored using the pg_restore utility.
  • Extensi= ble =E2=80=94 users have the flexibility to implement domain-based= transformations in any programming language or use predefined templates.
  • Declara= tive =E2=80=94 Greenmask allows you to define configurations in a = structured, easily parsed, and recognizable format.
  • Integra= ble =E2=80=94 integrate Greenmask seamlessly into your CI/CD syste= m for automated database obfuscation and restoration.
  • Paralle= l execution =E2=80=94 take advantage of parallel dumping and resto= ration, significantly reducing the time required to deliver results.
  • Provide= a variety of storage =E2=80=94 Greenmask offers a variety of stor= age options for local and remote data storage, including directories and S3= -like storage solutions.

Greenmask is ideally suited for= :

  • Routine backup = and restoration tasks, ensuring data integrity and availability.
  • Anonymization a= nd data masking for staging environments and analytics, protecting sensitiv= e information while maintaining data utility.

Release Notes

v0.1.7

New features
  • Added restorati= on filtering by --table, --schema and --exc= lude-schema parameters
  • Validate comman= d without parameters validates only the configuration file
  • Added the --schema parameter, which allows to make a schema diff between the = previous dump and the current. This=20 is useful when you want to check if the schema has changed after the migr= ation. By controlling it we can exclude=20 data leakage after migration
  • Validate comman= d divided by many stages that can be controlled using parameters
    • Configuration v= alidation
    • Transformer val= idation
    • Constraint viol= ation check
    • Data difference= check
    • Schema differen= ce check
Improvements
  • Improved Hash t= ransformer
    • Added salt para= meter that can be set via config or via GREENMASK_GLOBAL_SALT<= /li>
    • Added sha3 func= tions support in different modes (sha3-224, sha3-256, sha3-384, sha3-512)
  • Refactored Cmd transformer logic
    • Json API: Now i= t allows to use of column names instead of column indexes in JSON format
    • Csv API: Now it= can use the column order from config via column remapping
  • The valid= ate command was rewritten almost from scratch.
    • New option --transformed-only - displays only columns that are transformed wi= th a primary key (if exists). This allows to reduce the output data and make it more readable
    • Implemented json format for output
    • Added the --table-format parameter which is responsible for the vertica= l and horizontal table orientation. This works only when --format=3Dtext
    • Added the --warnings parameter, if it is specified then not only fatal-warnin= gs will be displayed, but also those with a lower severity
Fixes
  • Fixed --u= se-list option - now it applies to entries according to the order in= the list file
  • Fixed --u= se-list option behavior together with --list-format opt= ion (json or text). Now it generates a temporal list file in text format for providing it to the pg_= restore call
  • Updated documen= tation according to the latest changes

v0.1.2 - v0.1.6

  • Fixed uncontrol= led buffer growth in the restore command
  • The Hash<= /code> transformer has been completely remastered and now has the fun= ction parameter to choose from several hash algorithm options and th= e max_length parameter to truncate the hash tail.
  • Split informati= on about transformers between the list-transformers and new show-transformer CLI commands, which allows for more comprehensi= ble and useful outputs for both commands
  • Added error sev= erity for the Cmd parameter validator
  • Significantly r= efactored the structure and content of documentation
  • Fixed metadata = enrichment for validation warnings caused by RawValueValidator=
  • Fixed a typo in= the credit_card value for the type parameter of = the Masking transformer
  • Fixed Greenmask= Playground environment variables and the cleanup command
  • Fixed lis= t-dump, list-transformers, and restore com= mands exit code on error
  • Fixed database = connection parameters behavior fields
  • Fixed Json tran= sformer: now it correctly parses operations field
  • Fixed bug when = raw COPY lines were parsed incorrectly
  • Fixed --v= ersion parameter behavior

Useful Links

Change log greenmask.io

Explore detailed Documentation<= /a>

Access the Latest Release on GitHub

Contact us for support at Email Support
This email was sent to you from Greenmask.io. It was delivered on their beh= alf by the PostgreSQL project. Any questions about the content of the message shou= ld be sent to Greenmask.io.

You were sent this email as a subscriber of the pgsql-announce mai= linglist, for the content tag Related Open Source. To unsubscribe from further emails, or change which emails you want to receive, please click th= e personal unsubscribe link that you can find in the headers of this email, or visit https://lists.postgresql.org/unsubscribe/.
 
--===============5116009812662550967==-- --===============1364933765273568868==--