Delphix Products

 View Only

  • 1.  Masking best practice for the Partitioned table

    Posted 12-05-2022 06:55:00 AM
    Hello All , 

    in oracle database -- CDR table which have 291 partitioned and more than 2900M records in it.
    While masking speed of job very low like 200 to 400K  rows /min and taking more than 48 hrs to mask one table.
    Currently using MOD approach to split the table.


    Can any one suggest how i can increase the masking speed for partitioned table ?
    How to create rule set using partitioned key?
    Number of masking job depends on number of partitioned ?
    is there any regex that we can use to iterate masking job on all partitioned ?

    Please help me with best practice to mask the huge partitioned table

    ------------------------------
    PANKAJ ASEGAONKAR
    Delphix
    Infosys
    ------------------------------


  • 2.  RE: Masking best practice for the Partitioned table
    Best Answer

    Posted 12-06-2022 12:42:00 AM
    You need to add to your custom SQL a subselect to get the partition details so you can limit them.

    select column1,column2,... from table where partition_key between p1 and p2
    (where p1 and p2 are the wanted boundary of partitions)
    The advantage is that you do then only a partition scans to identify which rows are to be processed in that specific job.

    If using the modulo on rowid each job executes first a full table scan to identify the rows belonging to the job which takes time on very large tables.

    mod(DBMS_ROWID.ROWID_BLOCK_NUMBER(rowid),2) =0



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------

    Original Message


  • 3.  RE: Masking best practice for the Partitioned table

    Posted 12-07-2022 11:31:00 PM
    Hi  Pankaj,

    Long time no talk!  Please reach out to your CS representative and arrange for a demo of our relatively new Hyperscale product, which is specifically designed to handle very large workloads. 

    Best,
    David

    ------------------------------
    David Wells
    Product Management, Sr. Director
    Delphix
    ------------------------------

    Original Message


  • 4.  RE: Masking best practice for the Partitioned table

    Posted 01-03-2023 03:55:00 AM
    Hello David,

    Thanks for your reply ..!!
    Sure i will reach out to jithen for the demo ..!!

    As per suggestion i have created the 1 masking job for one partition like that i have 296 masking jobs( as per number of partition).
    Execution time is 52 to 70 Hrs for 296 masking jobs sequentially, but not getting the expected speed.
    Even after running 5 jobs in parallel speed of masking jobs are getting reduced and execution time is getting extended ( no extra outcome from parallel execution).
    Kindly suggest is there work around or solution to mask the partitioned table with 3K M records. ( no primary key present in table , there no trigger present in table ).





    ------------------------------
    PANKAJ ASEGAONKAR
    Delphix
    Infosys
    ------------------------------

    Original Message


Related Content