Delphix Products

Expand all | Collapse all

Masking multiple columns and filters

Jump to Best Answer
  • 1.  Masking multiple columns and filters

    Posted 12-02-2020 11:05:00 PM
    Hello All,

    As I'm relatively new to Delphix masking and I recently discovered behavior that is working as designed but not sure how to resolve the issue that it creates.

    I have a table which stores FEINs and SSNs in the same column.  The table was setup with a filter looking for just type SSN.  This sounds good; a filter is a WHERE clause so it provides the results that I need to just mask SSNs.

    However, on the same table is also DOB and DOD columns.  I want to mask ALL records for these columns.

    What is the best way to accomplish this?  If I have a filter for SSN, but it ignores DOB and DOD rows that are not of type SSN.

    Thanks for the help!

    ------------------------------
    Lisa Garczynski
    Database Engineer III
    Paychex
    ------------------------------


  • 2.  RE: Masking multiple columns and filters
    Best Answer

    Posted 12-03-2020 12:33:00 AM

    Hi Lisa,

    The easiest way to apply two separate filters and algorithms on the same table is by creating two separate rule sets.

    On the fist one apply the SSN filter and the DOB filter will be applied to the second one, also notice this will result in creating two jobs as there's a 1-1 relation between job and rule set.

    BR,

    Mouhssine



    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 3.  RE: Masking multiple columns and filters

    Posted 12-03-2020 06:55:00 AM
    Mouhssine,

    This is what I expected; thank you for the confirmation.  How do you prevent those two jobs from locking each other?  I'd like to run more than one stream at a time.

    Best Regards,
    Lisa

    ------------------------------
    Lisa Garczynski
    Database Engineer III
    Paychex
    ------------------------------



  • 4.  RE: Masking multiple columns and filters

    Posted 12-03-2020 07:57:00 AM
    Hi Lisa,

    I'm not for running tow different jobs on the same table at the same time, as you can end up with one job locking the other one as you stated.

    It's better to run them serially to prevent any issue.

    BR,

    Mouhssine

    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 5.  RE: Masking multiple columns and filters

    Posted 12-04-2020 09:43:00 AM
    Hi Lisa,

    We had similar issue, what we do is we created Pre-Script and extract the SSN or FEIN from the same column and copy to new column. Once the masking is completed, the post script will copy the masked data to the original column and drop the column eventually.

    The benefit of this is, we will have single masking job where we had an opportunity to run directly from Virtual Engine.

    ------------------------------
    Srinivasan Sarman
    Delphix Community Members
    ------------------------------



  • 6.  RE: Masking multiple columns and filters

    Posted 12-04-2020 09:59:00 AM
    Edited by Mouhssine SAIDI 12-04-2020 04:20:26 PM

    Hi Srinivasan,

    Right, this is also a valid option.

    Personally, I think the only advantage of this option is using only one job. But from performance perpective I don't think it is.

    You have to do 4 operations (copy,mask,update,drop) compared to 1 (update) with 2 separate jobs.

    Someone have raised a request to allow selecting the same table many times at the rule set level.

    So going with 2 jobs will perform better.
    regards,

    Mouhssine




    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 7.  RE: Masking multiple columns and filters

    Posted 12-04-2020 10:04:00 AM
    Yes, that's the only benefit compromising the performance.

    ------------------------------
    Srinivasan Sarman
    Test Data Manager
    The Toronto-Dominion Bank
    ------------------------------