Delphix Products

 View Only
Expand all | Collapse all

Having issue in Pipe(|) delimited with "(Quotes) enclosed data masking

  • 1.  Having issue in Pipe(|) delimited with "(Quotes) enclosed data masking

    Posted 04-21-2021 02:12:00 PM
    The source file is a pipe(|) delimited and data having "(double quotes) enclosed.
    Source file like this:

    Source file:
    "ID" | "Name" | "DOB"
    "111111111" | "AAAAAAA" | "1999-08-22"

    After masking Name by using SL algorithm like below:

    "ID" | "Name" | "DOB"
    "999999999" | BBBBBBBBB | "1999-08-01"

    For the Name column it is removing Quotes but after masking also want to be "BBBBBBBBB" instead BBBBBBBBB.
    Could you please suggest what exactly i need to do?



    ------------------------------
    THIMMA REDDY BOGATHI
    DATA ENGINEER
    Capgemini America

    ------------------------------
    THIMMA REDDY BOGATHI
    DATA ENGINEER
    Capgemini America
    ------------------------------


  • 2.  RE: Having issue in Pipe(|) delimited with "(Quotes) enclosed data masking
    Best Answer

    Posted 04-21-2021 02:56:00 PM
    Hi Thimma,

    First off, it looks like you want to use " as the enclosure character. This is done on a per-file basis by selecting the ruleset, then for each file, click the edit button, and enter " as in the Text Enclosure box. I suspect you may be aware of this, but just wanted to mention it to be certain.

    The second issue might be your data. In particular, is there whitespace between your delimiter characters and the enclosures? For enclosure detection and preservation to work correctly, there must be no intervening whitespace:
    Correct:
    "value1"|"value2"|"value3"
    
    Incorrect:
    "value1"| "value2"|"value3"
    "value1"|"value2" |"value3"
    "value1" | "value2" | "value3"
    ​

    If your data looks like any of the "incorrect" cases, it's likely you will need to preprocess it to correct the formatting before masking.

    Regards,

    ------------------------------
    John Leser
    Staff Engineer
    Delphix
    ------------------------------