Delphix Products

 View Only

  • 1.  Maximum number of Algorithms for one single table in a masking job

    Posted 01-12-2023 02:05:00 PM
    Hello, I'm trying to perform masking for a table having 60 PHI columns. I have created an "On the fly" job for that one single table and mapped 60 algorithms in the inventory. Upon executing the masking job, it masks the records only for first 54 PHI columns and remaining 6 columns are not masked. Is there a limitation for number of algorithms that can be assigned in a SINGLE table in a masking job?
    Is it like Delphix can handle only the maximum of 54 algorithms in one SINGLE table?
    Please advise and help!

    ------------------------------
    Ramesh Kumar S A
    Community Member
    Cognizant Technology Solutions
    ------------------------------


  • 2.  RE: Maximum number of Algorithms for one single table in a masking job

    Posted 01-13-2023 02:58:00 AM
    Hello Ramesh,
    there is no fixed limit of algorithms allowed to be used on one table .. however 60 algorithms is a significant number.
    More algorithms you assign to a table more CPU power and memory is required, means the performance might be slower than average.
    In no case you should get a situation that 6 out of 60 columns do not mask. Based on what did you get this impression?
    By comparing values before/after masking? Getting any errors?
    What did the monitoring table level log tell you about that execution for this specific table? Any errors or warnings?
    Please feel free to raise a support ticket and provide a support bundle so our support org can investigate this.
    Best regards,
    Tino

    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------

    Original Message


  • 3.  RE: Maximum number of Algorithms for one single table in a masking job

    Posted 01-17-2023 12:20:00 PM
    Hi Tino,

    Thanks for responding to my clarifications.

    To answer your question. My team did added the algorithms one by one and tested this. 
    EG: Ran the job by mapping 1 algorithm in the inventory...it was success
    Ran the job by mapping 2 algorithms in the inventory...it was success
    Ran the job by mapping 3 algorithms in the inventory...it was success
    Ran the job by mapping 4 algorithms in the inventory...it was success........Ran the job by mapping 54 algorithms in the inventory...it was success
    Now, When we run the job by mapping 55 algorithms in the inventory, the job failed and below is the error which we took from the admin log.  Please help!

    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.util.Arrays.copyOf(Arrays.java:3181)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.util.ArrayList.grow(ArrayList.java:267)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:241)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:233)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.util.ArrayList.add(ArrayList.java:464)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.lang.String.split(String.java:2353)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at java.lang.String.split(String.java:2422)
    2023-01-11 04:51:08,237 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at com.masking.SegmentMapping.SegmentMappingClass.getMaskedValue(SegmentMappingClass.java:207)
    2023-01-11 04:51:08,238 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at com.masking.SegmentMapping.SegmentMapping.createMaskedString(SegmentMapping.java:371)
    2023-01-11 04:51:08,238 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] at SC.eval0(SC.java:1)
    2023-01-11 04:51:08,238 [pool-5-thread-7] INFO c.d.d.m.t.RunServiceImpl - [JOB_ID_3310_8573] ... 9 more



    ------------------------------
    Ramesh Kumar S A
    TDM Engineer
    Cognizant Technology Solutions
    ------------------------------

    Original Message


  • 4.  RE: Maximum number of Algorithms for one single table in a masking job
    Best Answer

    Posted 01-17-2023 01:35:00 PM
    Hello Ramesh,
    this is fairly easy and understandable. The memory configured for the job is NOT sufficient > you need to increase it, fairly simple.
    Depending on what which version of Delphix engine you are running you get some statistics in the log about memory consumption.
    I have confirmed that there is no software designed restriction > but the requirements regarding required memory need to be fulfilled and more algorithms you assign more memory you need. When you edit the job memory I further advise to set the MIN and MAX to the same value.
    BR .. Tino

    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------

    Original Message


  • 5.  RE: Maximum number of Algorithms for one single table in a masking job

    Posted 01-25-2023 10:07:00 AM
    Thank you Tino, We have increased the memory and able to get through beyond 55 Algorithms for a table.

    ------------------------------
    Ramesh Kumar S A
    TDM Engineer
    Cognizant Technology Solutions
    ------------------------------

    Original Message


Related Content