Delphix Toolkits (dxToolkit and dxmToolkit)

 View Only

  • 1.  How to run masking job parallel sequentially using dxmc?

    Posted 12-29-2022 08:01:00 AM
    Hello Team, 

    I have 290 Masking jobs which needs to be run parallels of 5 masking jobs but sequentially.
    However when i am executing by putting below commands  in unix script
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_21 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_22 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_23 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_24 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_25 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_26 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_27 --parallel 5 &
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_28 --parallel 5 &

    OR 
    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_28 --parallel 5 --nowait
    Execution order is not sequentially it is launching job randomly.
    and while some masking job getting below error 
    Masking job JO_CDR_36 finished.
    0 rows masked



    Masking job JO_CDR_32 finished.
    0 rows masked



    Masking job JO_CDR_39 finished.
    0 rows masked
    Job JO_CDR_33 is processing rows



    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'




    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'




    Masking job JO_CDR_27 finished.
    0 rows masked
    Masking job JO_CDR_21 finished.
    0 rows masked






    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'

    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'







    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'




    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'

    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'

    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'

    Exception in thread Thread-1:
    Traceback (most recent call last):
    File "threading.py", line 916, in _bootstrap_inner
    File "threading.py", line 864, in run
    File "dxm/lib/DxJobs/jobs_worker.py", line 551, in job_selector
    File "dxm/lib/DxJobs/jobs_worker.py", line 757, in do_start
    File "dxm/lib/DxJobs/DxJob.py", line 568, in start
    File "dxm/lib/DxJobs/DxJob.py", line 606, in wait_for_job
    AttributeError: 'GenericModel' object has no attribute 'rows_total'




    ------------------------------
    PANKAJ ASEGAONKAR
    Delphix
    Infosys
    ------------------------------


  • 2.  RE: How to run masking job parallel sequentially using dxmc?
    Best Answer

    Posted 01-04-2023 02:08:00 AM
    Hello Pankaj,
    the correct syntax would be :

    ./dxmc job start --envname PREM --engine va0023 --jobname JO_CDR_21 --jobname JO_CDR_22 --jobname JO_CDR_23 --jobname JO_CDR_24 --jobname JO_CDR_25 --jobname JO_CDR_26 --jobname JO_CDR_27 --parallel 3

    About the error/exception you are getting I assume two things:
    > the job just finishes regular but the "reporting" only throws this error
    > you are either masking files or the engine version somehow not having "row_total" inside the json response.

    Please raise a ticket in GITBUB for this DXM issue so it can be tracked/resolved (DXM is opensource)

    Best regards,
    Tino

    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------

    Original Message


  • 3.  RE: How to run masking job parallel sequentially using dxmc?

    Posted 01-04-2023 02:38:00 AM
    Thanks a lot Tino for the valuable reply ..!!

    ------------------------------
    PANKAJ ASEGAONKAR
    Delphix
    Infosys
    ------------------------------

    Original Message


Related Content