search for sensitive data on plain txt files

  • 0
  • 1
  • Problem
  • Updated 1 year ago
  • Solved
  • (Edited)

Hello,


Can Delphix profile unstructured .txt files?

There is documentation about profiling structured files.

In addition there is documentation about virtualize unstructured files.

Maybe does it has to do with profiling unstructured files?

Luigi

Photo of luigidep

luigidep

  • 622 Points 500 badge 2x thumb

Posted 1 year ago

  • 0
  • 1
Photo of Hims

Hims, Employee

  • 2,096 Points 2k badge 2x thumb
Official Response
Hi Luigi,
Unstructured files are hard to process, however we can redact them using some creativity.
Approach is below:
1. We will treat the file as delimited, we have to find a character which does not exist in file to be used as field delimiter.
2. This will ensure every row is treated as a whole single field.
3. We will use Multi PHI checkbox on field to keep finding sensitive fields ( the default algorithm to be applied is in a property file), this will find out all patterns existing in the file.
4. We apply the "Free Text Redaction" Algorithm with values and patterns to be redacted to the one field.
5. Execute masking job.

User will have to do a few iterations to get this accurate though.
--Hims