Delphix Products

 View Only
Expand all | Collapse all

Masking Sustainability - Profiling, Jenkins, Engine Compare

  • 1.  Masking Sustainability - Profiling, Jenkins, Engine Compare

    Posted 05-25-2021 01:50:00 PM
    Hello All,

    I'm looking for ways to make our masking efforts here easier, scalable and more sustainable.  Is anyone doing automated engine comparisons and promote to production via BitBucket?  Would you be will to share how you're doing it, scripts?

    We have procedures for new schema changes that are supposed to involve me, but like all processes that involve humans, people change roles, forget, etc.  I'm looking to automate profiling into schema reviews using Jenkins.  Is anyone doing this?  Would you be willing to share your experience / scripts?  On the surface, checking columns via regex as a first step doesn't seem hard, but given the size of our organization and the number of systems, it's complex.

    Let me know.
    Thanks!

    ------------------------------
    Lisa Garczynski
    Database Engineer III
    Paychex, Inc.
    ------------------------------


  • 2.  RE: Masking Sustainability - Profiling, Jenkins, Engine Compare

    Posted 05-31-2021 03:16:00 AM
    Hi Lisa
    I've been playing with a scripts from Delphix that basically creates a brand new Masking Env and connectors from list defined ( matching schemas from proper masking env ).
    All our masking algorithms have a flag USER so we just run profiling and call API to download results in json form and load into database from both PROD and DISCOVER env.
    All auto profiling results are saved as AUTO.
    I can then use a simple minus queries to find differences comparing what columns AUTO are not in USER ser. T
    his is still very much in flight but that's the high level idea.

    Cheers
    Lukasz

    ------------------------------
    Lukasz Wasylo
    Community Member
    Baillie Gifford & Co,
    ------------------------------