APIPythonOpen-Source Solutions

 View Only

  • 1.  (data profiling) Searching for personal data with weak-defined formats using profiling.

    Posted 03-02-2018 07:05:00 AM
    Dear Delphix Community,

    When sensitive data to be discovered  do have pre-defined format, then REGEX-based profiling is working fine. Example of pre-defined-format data: email address or IBAN accout number, etc. 

    When sensitive data to be discovered doesn't have any defined format (like human related info's: first name, last name, etc) then regex usually is not giving optimal results. 

    My questions:
    • Is there a option to use valid-list-lookup data profiling please  (e.g. list of 500 city names)?
    • Is there an option to write custom data profile plugin in some programming language please ?

    Data profiling is planned be executed on enterprise-scale systems, with thousands of tables, etc. High-quality of profiling will save a lot of time on manual reviews of results.


    Looking forward for yours suggestions.

    Many thanks in advance,

    best regards,

    Adam Przybyslawski




    #Masking


  • 2.  RE: (data profiling) Searching for personal data with weak-defined formats using profiling.
    Best Answer

    Posted 03-02-2018 07:58:00 AM
    Hi Adam,
    Thank you for your suggestions.

    Like you, I see comprehensive profiling as a fundamental feature of data protection and the recent release of our Masking API allows a much more automated approach to profiling and masking, which assists large enterprises to profile large data estates.

    The option to use list lookups is not currently available but has been requested in the past and is accepted onto our feature enhancement backlog.

    I like the idea of a custom profiling plug-in.  This marries with the ability to create custom masking algorithms also.  I did not see this in our enhancement request catalogue so I will discuss this feature with product management.

    Regards,
    Gary


  • 3.  RE: (data profiling) Searching for personal data with weak-defined formats using profiling.

    Posted 03-02-2018 08:43:00 AM
    Dear Garry,

    Thanks for prompt and professional answer.  

    have a nice day,

    best regards,
    Adam



  • 4.  RE: (data profiling) Searching for personal data with weak-defined formats using profiling.

    Posted 03-02-2018 09:26:00 AM
    Dear Garry,

    Thanks for prompt and professional answer.  

    have a nice day,

    best regards,
    Adam



  • 5.  RE: (data profiling) Searching for personal data with weak-defined formats using profiling.

    Posted 03-26-2018 04:23:00 PM

    Hello Adam,

    We are also looking for the same thing since our goal is to reduce as much as possible the manual intervention. Is this just a dream or it is feasible: execute the profiler on tables and get near 100% associated fields and accurate?


    Thanks

    John