Delphix Products

Back to discussions
Expand all | Collapse all

(data profiling) Option to exclude incompatible DATA TYPES (column-types) types for given search rule ?

  • 1.  (data profiling) Option to exclude incompatible DATA TYPES (column-types) types for given search rule ?

    Posted 03-12-2018 09:41:00 AM
    Dear Community,

    Is there any option available to EXCLUDE particular data-types from being searched for each of search rules please ? (for single search rule, not globally)

    For example, when searching for email addresses (both in column names and data), then option to exclude ALL columns with data type NUMBER or DATE or BINARY or TIMESTAMP would be helpful.

    Each of search rules may work faster and produce better results, if incompatible DATA TYPES can be excluded. (improvement in quality and performance).

    As each sensitive attributes have different data-types to be excluded, then this option should be available separately for each search rule.

    Is there any existing option to enable this functionality for separate search rule please ?

    Looking forward for yours suggestions,

    thanks in advance,

    Adam Przybyslawski


    #Masking


  • 2.  RE: (data profiling) Option to exclude incompatible DATA TYPES (column-types) types for given search rule ?
    Best Answer

    Posted 03-12-2018 11:57:00 AM
    Hi Adam,
    Most data types which will not have PII/PHI inside are already excluded from the profiling scope. E.g. 2 or 3 width Varchar/char, Boolean/ Flag data types.

    Any data type which may have sensitive data like DATE/ TIMESTAMP ( for birthdates) or BINARY data ( PDF/JPEG) are NOT excluded.

    These properties are currently global and not configured at the job level. In future, they should be user-configurable.

    Hope this helps.

    Hims