APIPythonOpen-Source Solutions

 View Only
  • 1.  initial installation / use of delphixpy

    Posted 03-10-2022 04:23:00 PM
    Sorry for the long intro post. Summing up a couple weeks worth of learning, trial & error.

    I'm new as it gets. Our delphix appliances and delphixpy are my introduction to API's and python.
    My initial goals are to snapshot and refresh vdb's from windows command prompt.

    From my current point of progress. My primary question is with which set of examples should I continue forward. I've tried scripts from both delphixpy-examples-master, and delphixpy-examples-python3-api-1-10-2. Pretty close to success with the former, the latter seems to be still under construction??

    Within Delphix we're currently provisioning Sybase ASE on AIX. We have ~10 dSources but replicated ~20 times over. So ~20 datasets each containing the same vdb sets.

    Our Delphix version is 6.0.10.0
    On the client side We're a windows shop. I've installed python 3.10.2 on a windows 2019 server instance.

    Pip list is as follows:
    Package Version
    --------------- ---------
    delphixpy 1.11.12.0
    docopt 0.6.2
    pip 22.0.4
    python-dateutil 2.8.2
    setuptools 60.9.3
    six 1.16.0
    untangle 1.1.1


    Running vdb snapshot from delphixpy-examples-master, the operations are successful, but some errors are thrown afterward:

    (venvdx2) C:\python\venv\venvdx2\delphixpy-examples-master>python dx_snapshot_db.py --group AIX8_RPRT02_SYBASE --name bbtool_db

    INFO: Welcome to dx_snapshot_db.py, version v.0.0.100
    INFO: Executing against the default Delphix Engine in the dxtools.conf: PDELPHIX01
    INFO: PDELPHIX01: Syncing bbtool_db
    INFO: PDELPHIX01: bbtool_db: RUNNING
    INFO: PDELPHIX01: 1 jobs running. 0 jobs waiting to run
    INFO: PDELPHIX01: bbtool_db: COMPLETED
    Exception in thread Thread-1 (main_workflow):
    Traceback (most recent call last):
    File "C:\Program Files\Python310\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
    File "C:\Program Files\Python310\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
    File "C:\python\venv\venvdx2\delphixpy-examples-master\dx_snapshot_db.py", line 486, in main_workflow
    i = update_jobs_dictionary(engine, server, jobs)
    File "C:\python\venv\venvdx2\delphixpy-examples-master\dx_snapshot_db.py", line 687, in update_jobs_dictionary
    for j in jobs.keys():
    RuntimeError: dictionary changed size during iteration
    INFO: script took 0.2 minutes to get this far.


    Running from delphixpy-examples-python3-api-1-10-2:
    initially unable to pass both group and name parameters...

    (venvdx2) C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2>python dx_snapshot_db.py --name bbtool_db --group AIX8_RPRT02_SYBASE
    Usage:
    dx_snapshot_db.py (--group <name> |--name <name> | --all_dbs )
    [--engine <identifier>]
    [--usebackup --bck_file <name> --parallel <n>]
    [--poll <n> --create_bckup --single_thread <bool>]
    [--config <path_to_file> --logdir <path_to_file>]
    dx_snapshot_db.py -h | --help | -v | --version


    Tried group/name combo, as I've seen this format in the CLI :

    (venvdx2) C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2>python dx_snapshot_db.py --name AIX8_RPRT02_SYBASE/bbtool_db

    INFO: Executing against default Delphix Engine
    Exception in thread Thread-1 (main_workflow):
    Traceback (most recent call last):
    File "C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2\dx_snapshot_db.py", line 109, in snapshot_database
    db_source_info = get_references.find_obj_by_name(
    File "C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2\lib\get_references.py", line 79, in find_obj_by_name
    raise dlpx_exceptions.DlpxObjectNotFound(f"Object {obj_name} not found.")
    lib.dlpx_exceptions.DlpxObjectNotFound: Object AIX8_RPRT02_SYBASE/bbtool_db not found.

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
    File "C:\Program Files\Python310\lib\threading.py", line 1009, in _bootstrap_inner
    self.run()
    File "C:\Program Files\Python310\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
    File "C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2\dx_snapshot_db.py", line 195, in main_workflow
    snapshot_database(
    File "C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2\dx_snapshot_db.py", line 116, in snapshot_database
    raise dlpx_exceptions.DlpxException from err
    TypeError: DlpxException.__init__() missing 1 required positional argument: 'error'
    INFO: script took 0.0 minutes to get this far.



    Running on the group only:
    In this case it ran, but ran on vdb's in groups other than the one specified by the --group parameter.

    (venvdx2) C:\python\venv\venvdx2\delphixpy-examples-python3-api-1-10-2>python dx_snapshot_db.py --group AIX8_RPRT02_SYBASE
    INFO: Executing against default Delphix Engine
    INFO: checking running jobs on engine: PDELPHIX01
    INFO: Engine: PDELPHIX01: JOB-44896 is 100% COMPLETE
    INFO: Engine: PDELPHIX01: JOB-44898 is 100% COMPLETE
    INFO: Engine: PDELPHIX01: JOB-44900 is 100% COMPLETE
    INFO: Engine: PDELPHIX01: JOB-44902 is RUNNING and 0.0% complete
    INFO: Engine: PDELPHIX01: JOB-44897 is 100% COMPLETE
    INFO: Engine: PDELPHIX01: JOB-44901 is RUNNING and 5.0% complete
    INFO: Engine: PDELPHIX01: JOB-44902 is 100% COMPLETE
    INFO: Engine: PDELPHIX01: JOB-44899 is 100% COMPLETE
    INFO: script took 0.8 minutes to get this far.


    From the 'System>Events' page of the management console (showing where ran on groups other than AIX8_RPRT02_SYBASE):
    DB_SYNC job for ""AIX8_RPRT02_SYBASE/extract_db"" completed successfully."
    DB_SYNC job for ""AIX4_TRNG02_SYBASE/BCIFacets_dbxc"" completed successfully."
    DB_SYNC job for ""AIX7_TRNG01_SYBASE/BCIFacets_dbar"" completed successfully."
    DB_SYNC job for ""AIX6_TEST07_SYBASE/BCIFacets_db"" completed successfully."
    DB_SYNC job for ""AIX9_TEST12_SYBASE/bcidaho_db"" completed successfully."
    DB_SYNC job for ""AIX9_QUAL03_SYBASE/bbtool_db"" completed successfully."

    ------------------------------
    Vance Hill
    Database Administrator
    Blue Cross of Idaho Health Service, Inc.
    ------------------------------


  • 2.  RE: initial installation / use of delphixpy

    Posted 03-11-2022 07:08:00 PM
    Hi Vance, 
    Trying to get a better understanding of what you're trying to achieve. Can you give me a brief rundown of the use case here? 

    Arun

    ------------------------------
    Arun Saju
    Technical Services
    Delphix
    ------------------------------



  • 3.  RE: initial installation / use of delphixpy

    Posted 03-15-2022 10:59:00 AM
    Short term goal is to run snapshots, vdb refreshes and masking jobs from windows command prompt, with valid return codes.  Longer term is to run from a job scheduler, a series of scripts to refresh vdb's, run masking jobs on the vdb's, snapshot, then refresh downstream vdb's from the completed masked copies. 

    --vh

    ------------------------------
    Vance Hill
    Database Administrator
    Blue Cross of Idaho Health Service, Inc.
    ------------------------------



  • 4.  RE: initial installation / use of delphixpy

    Posted 03-16-2022 02:26:00 PM
    Hi Vance, 
    Thank you for clarifying the use cases for me.
    There may be a few options you can consider. However, to make sure we're setting you up with an option that's future proof, I will discuss this internally and have someone from your Account team @ Delphix reach out. 

    Thanks
    Arun​

    ------------------------------
    Arun Saju
    Technical Services
    Delphix
    ------------------------------