Delphix Products

How to create a Secure Lookup algorithm with our own listed values in a specific column, For example, a custom Email Secure Lookup algorithm with a list of email addresses that we create.
#Masking

Hi Rajesh,
You can create a list of values by extracting the unique values from the column into a file and then use it as a lookup file for creating the algorithm. Please ensure the file's character-set is compatible with your target database.  Preferably no nulls and special/non-printable characters.
For more details refer here - https://maskingdocs.delphix.com/Securing_Sensitive_Data/Configuring_Your_Own_Algorithms/

Thanks
Hims

Hi Hims,

Thank you very much,
As I have access to target  database (sql server), Are you saying to extract unique values from the column from sql server(In my perspective)?

Thanks!

yes

HI Rajesh,

Most of the time emails need to be unique in table. In that case you cannot use simple Secure Lookup Algorithm, but you need to choose Mapping Algorithm with lookup file(s) which has unique values (emails). Emails you can generate, or take from number of free web services and generate file(s) with equal or more emails (rows) that are in your DB. You can split Mapping Algorithms lookup files in to 1mio rows files, Mapping Algorithms can be appended.

Be aware when you choose to use Mapping Algorithm on multi milion tables, because Masking Engine first need to load large (multi milion) values from Mapping Algorithm, them mask your data, (probably it takes additional 15-30mins in masking process). Whole process takes more time than Segment Mapping Algorithm scenario, but you are 100% sure that masked data has required by DB data uniqueness and they will look like real data.

See Masking Engine docs for Mapping Algorithm Framework
https://maskingdocs.delphix.com/Securing_Sensitive_Data/Configuring_Your_Own_Algorithms/?_ga=2.26432...">https://maskingdocs.delphix.com/Securing_Sensitive_Data/Configuring_Your_Own_Algorithms/?_ga=2.264328911.1937216059.1550220065-336661834.1524486625#mapping-algorithm-framework">https://maskingdocs.delphix.com/Securing_Sensitive_Data/Configuring_Your_Own_Algorithms/?_ga=2.26432...

Regards
Marcin

Thank you Hims/Marcin,

But in the table which I am using have identical email Id's, so that I can't be able to use mapping algorithm. Also when I used SL algoritm, the values are replaced by character "m" which is not a realistic value. Can you suggest which is the best algorithm to use for Email Id's with repeated values with list of own selected values.

Thanks!

HI Rejesh,

1. Search google for free WEB Service which can generate realistic emails.
2. Create .txt file and fill .txt file with emails, one per line, can be comma separated, lookup file should contains with not more than 500k emails.
3. Create new SL Algorithm and import .txt Lookup file with generated emails.
4. Optionally create your own Domain for your algorithms.

Here is a basic configuration:
https://community.delphix.com/delphix/topics/tip_of_the_day_adding_a_secure_lookup_algorithm">https://community.delphix.com/delphix/topics/tip_of_the_day_adding_a_secure_lookup_algorithm">https://community.delphix.com/delphix/topics/tip_of_the_day_adding_a_secure_lookup_algorithm

Regards
Marcin

Hi Rajesh, 

This just to complete Marcin's answer and clarify some keys regarding SL and SM algorithms.

You are looking to generate you're own masked values for email column, taking in consideration that this should provide the ability to preserve the uniqueness.

SM : is the algorithm that preserve natively the uniqueness condition, but you can't use it because you hope to generate your own list.

Good news you have a possibility to do it with SL algorithm under some conditions.

SL : can preserve the uniqueness only if the number of values defined in you input file are superior or equal (prefer to have superior values) to the number of unique values you have to mask in your column.

So to achieve you need you can use the SL with the condition mentioned; generate you algorithm with superior unique values form what you have inside the column to mask.

P.S : Ensure that number of values defined on the SL algorithm are always superior the once in the column you want to mask to get some kind of uniqueness value preservation

 

Regards,

Mouhssine

Hi Mouhssine,

If you don't mind, can you please explain this by giving an example?
Actually what I did is, just exported 50 list of email id's from email table which I wanted to mask them. Then I used the same 50 email id's as lookup file with SL algorithm. After done with masking, the whole email column getting replaced by the character 'm' irrespective of listed email id's in lookup file. Can you suggest me on this?

Thanks!

Hi Rajesh,

Do you mine to upload the file you used to create your algorithm to the forum, if there is sensitive data and can't communicate theme.

I will provide you a dummy file with a procedure to create the algorithm, i suspect the input file is misconfigured reason why you are ending with this results.

By the way can you confirm please that your input file is saved in ASCII or UTF8 format and that you have an email address per line without space or return at the end of it.

In this KB note under "UI - CREATION AND MODIFICATION" you will find the implementation way once you have your input file ready with correct format and number of records 

Regards,

Mouhssine

Hi Mouhssine,

I think I cannot share the file to you..
But yes I am encoding the lookup file with UTF-8 format

Thanks!

Rajesh,

Undestand.

Can you try creating a new SL algorithm with the provided attachment via this link https://ufile.io/b1hrf and try the masking please.

Regards,

Mouhssine