APIMaskingSidekickPythonOpen-Source Solutions

Hi,
I'm facing errors during execution of the same masking job repeatedly. The log is stating deadlock  and unique constraint violated errors though the job succeeded for the first time.

Kindly help me what would be the problem.

Regards,
Swathy
#Masking

Hello Swathy,
                   Could you please confirm database type, Oracle/Sql Server? 

About deadlock, How many streams/threads you using in masking job?
Try using 1 stream and 1 thread while creating masking job.

Are you trying to mask the table again, which is already masked with first masking job successfully?

Please check in log file, which table/column is throwing unique constraint violated error. How many rows in the column and which algorithm is applied to mask that column?

Thanks

Hello Jatinder,

Thanks a lot for the response. My Database type is Oracle. I'm using 4 streams and 4 threads and my commit size is 100. I'll try using 1 stream and 1 thread for my job and execute again. I've done on-the-fly masking and yes, I'm using the same job again. 

Can you tell me what problem will occur if I use the same Masking job repeatedly? I thought it is fine because I'm doing on-the-fly masking.

The errors are around a single table in Ruleset, where I have masked a column using Segmented Algorithm for a unique column. Though my range of the algorithm is wide enough, I'm facing issue with Segmented algorithm often. Can you help me with that?

Thanks,
Swathy

Hello Swathy,
                   You can use same masking job. My question was, are you trying to mask table second time, which was already masked first time, OR you have unmasked data in table, which you tried to mask using same job?

Could you please help with details of Segmented Mapping algorithm(Segments, Range etc) you created and what type of column you trying to mask using that algorithm?

For troubleshooting purposes, you can also try to remove unique constraint from the column and tried to mask it. After masking, check out the masked values which violate the unique constraint. This will be helpful to understand, which unmasked values are getting masked to same value. 

Just wondering, if it works first time for same data, it should work again. Please also check, if you get any new data in table, which is getting masked to same value.

Thanks

Hi Jatinder,
I'm masking the same set of tables and provisioning to another environment. Does it create problem too?

I have an account number column like 1470026592 where I'm preserving 147 and splitting the rest into 2 segments 0026 and 592.

original range/masked minimum and maximum value
0026/8000 to 9999
592/111 to 999

I have 1000 records in my Source and the column is the primary key column. Also, I have this column as the Foreign key in another table.

Sure Jatinder, I'll try disabling the constraints to mask.

We're trying to Provision and Mask a set of tables through API. My jobs are executing successfully in DMsuite but failing when I run them through API.

Thanks for the response :)
Swathy

Hello Swathy,
                   For your first question, it doesn't create problem.

Is there any particular reason you have minimum and maximum value, assigned? You can just create two alphanumeric segments with lengths 4 and 3, while preserving first three digits.

If jobs are executing successfully from GUI and failing through API, it might be some issue with API call.  Could you please share API call you made to run masking job?

Please refer to below link for masking API calls to run a masking job.

https://docs.delphix.com/docs/delphix-masking/delphix-masking-quick-start-guide/masking-api-calls-to...


Thanks

Hi Jatinder,

I'm using this algorithm for key column masking. I need 1000 unique masked values wherease I'm able to mask only around 980 records with this. Around 20 records are masked with the same values. 

I split the algorithm as you said with segments 3-4-4 (alphanumeric) and preserved 3 characters apart from this ( in the beginning). The records look like:

16559668748654
16559668742596
16559668741475
16559668743542
16559668745429

I mean the the segments 3-4 are also displaying one record for all the 1000 values and only the last segment values are changing. 

Can you please explain the logic behind this algorithm.

Thanks

Hi,
Segmented mapping will always generate 1:1 unique values and maintains numeric/Alphanumeric types.

But, to ensure uniquness you haven't to  specify any range nor min-max values. so this couls explaine the deadlock and constraint violation errors.

Segmented Mapping Algorithm segments target values and then masks those segments


Regards,

Mouhssine 

Hello Swathy,
                    As @Mouhssine mentioned in the comment, segmented mapping algorithm target the values in each segment, and to ensure uniqueness you haven't to specify any range nor min-max values.

Looks like in this column values, first two segments have same unmasked value while the last segment has a different value for each record. To maintain consistency, masking engine provides same masked value, if unmasked value is same.

I did a similar test as per your data and you could check the results below, preserving first three digits(165) in the record. First two segments in yellow(3) and blue(4) are masked to the same value as their unmasked value is same, while the last segment in green(4) got different masked values.

This might be the reason, you got same masked values in first two segments.

Hope this helps.

Thanks

Thanks a lot Mouhssine  :)

Thanks a lot Jatinder for clarifying my doubts :)