APIPythonOpen-Source Solutions

Good Morning,

     We are automating the data masking portion of our Test Data Management application.  Currently we are able to login, generate the authorization token, kick off a masking job with a job and Connector ID, check its status then logout.

     The purpose of the connector will stay the same with rulesets, inventory, schema, etc with every masking iteration or execution.  Challenge is the hostname will change dynamically with a request # which will be provided to my function.  I only want to update the host name in this API but requiring me to feed a bunch of other attributes such as username, schema name, port, database type, connector name, etc.  We were hoping not to have to feed that many variables to update a connector with unchanged information.

     Behind the scenes it looks like Delphix Masking engine views the details of the job, ruleset and connector, creates a new one on the fly then deletes the old connector.  Would we have to do the same thing?

PUT /database-connectors/{databaseConnectorId}

​

------------------------------
Robert L. Henning
Enterprise Platforms - Technology Services - Senior Associate
Enterprise Databases
------------------------------

Hi Robert,
My understanding is that you are in contact with our Technical Services team and working toward a solution. When you've worked through it, if possible, please consider posting back how you were able to get this solved. This is great work, and I'm sure others would find value in how it was done.

Thank you,
Michael

------------------------------
Michael Torok
Director of Knowledge and Community Management
Delphix
------------------------------

Original Message:
Sent: 10-08-2020 05:52:59 AM
From: Robert Henning
Subject: Updating Masknig Connector / API

Good Morning,

     We are automating the data masking portion of our Test Data Management application.  Currently we are able to login, generate the authorization token, kick off a masking job with a job and Connector ID, check its status then logout.

     The purpose of the connector will stay the same with rulesets, inventory, schema, etc with every masking iteration or execution.  Challenge is the hostname will change dynamically with a request # which will be provided to my function.  I only want to update the host name in this API but requiring me to feed a bunch of other attributes such as username, schema name, port, database type, connector name, etc.  We were hoping not to have to feed that many variables to update a connector with unchanged information.

     Behind the scenes it looks like Delphix Masking engine views the details of the job, ruleset and connector, creates a new one on the fly then deletes the old connector.  Would we have to do the same thing?

PUT /database-connectors/{databaseConnectorId}

​

------------------------------
Robert L. Henning
Enterprise Platforms - Technology Services - Senior Associate
Enterprise Databases
------------------------------

Original Message:
Sent: 10/8/2020 11:24:00 AM
From: Michael Torok
Subject: RE: Updating Masknig Connector / API

Hi Robert,
My understanding is that you are in contact with our Technical Services team and working toward a solution. When you've worked through it, if possible, please consider posting back how you were able to get this solved. This is great work, and I'm sure others would find value in how it was done.

Thank you,
Michael

------------------------------
Michael Torok
Director of Knowledge and Community Management
Delphix
------------------------------

Original Message:
Sent: 10-08-2020 05:52:59 AM
From: Robert Henning
Subject: Updating Masknig Connector / API

Good Morning,

     We are automating the data masking portion of our Test Data Management application.  Currently we are able to login, generate the authorization token, kick off a masking job with a job and Connector ID, check its status then logout.

     The purpose of the connector will stay the same with rulesets, inventory, schema, etc with every masking iteration or execution.  Challenge is the hostname will change dynamically with a request # which will be provided to my function.  I only want to update the host name in this API but requiring me to feed a bunch of other attributes such as username, schema name, port, database type, connector name, etc.  We were hoping not to have to feed that many variables to update a connector with unchanged information.

     Behind the scenes it looks like Delphix Masking engine views the details of the job, ruleset and connector, creates a new one on the fly then deletes the old connector.  Would we have to do the same thing?

PUT /database-connectors/{databaseConnectorId}

​

------------------------------
Robert L. Henning
Enterprise Platforms - Technology Services - Senior Associate
Enterprise Databases
------------------------------

Hi Michael,

     I was able to use python to code the following sequence like Delphix does behind the scenes only having the Job ID, peeling the onion per se to uncover the ruleset ID, which is assigned to a variable and passed to the base connector information.  That response are all assigned to variables to build a new connector on the fly leveraging the same job, ruleset, inventory, using the uuid module for the name.  From there we kick off the same original job using the new connector built on the fly, check its status with the sequential execution ID created, then delete the connector created at run time and logout.  A base connector is required to build upon as a template and to keep the rulesets and inventory in place.  There is a bunch of code in between all of these below but just showing the core pillars of how we achieved it.

     There are 2 attachments to share, that with the 2 green highlights is my job showing its symmetry with a standard Delphix masking job (no highlights).

1.) Generate Auth Key - response = requests.post(MASKING_ENGINE+'/masking/api/login', headers=headers, data=data)
2.) View Job                 - response = requests.get(MASKING_ENGINE+'/masking/api/masking-jobs/'+maskingJobID, headers=headers)
3.) View Ruleset           - response = requests.get(MASKING_ENGINE+'/masking/api/database-rulesets/'+str(rulesetId), headers=headers)
4.) View Connector      - response = requests.get(MASKING_ENGINE+'/masking/api/database-connectors/'+str(databaseConnectorId), headers=headers)
5.) Create Connector   -
    newid = uuid.uuid4().hex
    data = {"connectorName":newid,
            "databaseType":getdatabaseType,
            "environmentId":getenvironmentId,
            "host":gethost,
            "password":MASKING_PWD,
            "port":getport,
            "schemaName":getschemaName,
            "sid":getsid,
            "username":getusername
           }
    response = requests.post(MASKING_ENGINE+'/masking/api/database-connectors', headers=headers, json=data)
 
6.) Start Masking Job - data = {"jobId":maskingJobID, "targetConnectorId":newconnectorID}
    response = requests.post(MASKING_ENGINE+'/masking/api/executions', headers=headers, json=data)

7.) Get Job Status with Execution ID
    def get_job_status_by_id(headers, getexid):
    status = f'{MASKING_ENGINE}/masking/api/executions/{getexid}'

8.)  Delete Runtime Connector-  response = requests.delete(MASKING_ENGINE+'/masking/api/database-connectors/'+str(newconnectorID), headers=headers)



------------------------------
Robert L. Henning
Enterprise Platforms - Technology Services - Senior Associate
Enterprise Databases
703-833-5709 (Desk) 215-764-0815 (Mobile)
------------------------------

Original Message:
Sent: 10-08-2020 10:39:47 AM
From: Robert Henning
Subject: Updating Masknig Connector / API

Hi Michael,

 

• I appreciate this feedback we worked hard on it and see the light at the end of the tunnel.  Small hiccup so I wanted to tap the Delphix community to see if anyone had the same experience.  I was able to programmatically get the job to run, provide a status but wanted to see if we need to pass all of the static variables again.

 

• Behind the scenes via audit log, we see that the masking engine is viewing the masking job, the ruleset, connector information, creating the new connector, masking then deleting the on the fly connector created.  I know this is proprietary software so wanted to code leveraging the API's we have and handle this unique requirement.

 

Robert L. Henning

Enterprise Platforms - Technology Services - Senior Associate

Enterprise Databases

703-833-5709 (Desk) 215-764-0815 (Mobile)

 

Original Message:
Sent: 10/8/2020 11:24:00 AM
From: Michael Torok
Subject: RE: Updating Masknig Connector / API

Hi Robert,
My understanding is that you are in contact with our Technical Services team and working toward a solution. When you've worked through it, if possible, please consider posting back how you were able to get this solved. This is great work, and I'm sure others would find value in how it was done.

Thank you,
Michael

------------------------------
Michael Torok
Director of Knowledge and Community Management
Delphix

Original Message:
Sent: 10-08-2020 05:52:59 AM
From: Robert Henning
Subject: Updating Masknig Connector / API

Good Morning,

     We are automating the data masking portion of our Test Data Management application.  Currently we are able to login, generate the authorization token, kick off a masking job with a job and Connector ID, check its status then logout.

     The purpose of the connector will stay the same with rulesets, inventory, schema, etc with every masking iteration or execution.  Challenge is the hostname will change dynamically with a request # which will be provided to my function.  I only want to update the host name in this API but requiring me to feed a bunch of other attributes such as username, schema name, port, database type, connector name, etc.  We were hoping not to have to feed that many variables to update a connector with unchanged information.

     Behind the scenes it looks like Delphix Masking engine views the details of the job, ruleset and connector, creates a new one on the fly then deletes the old connector.  Would we have to do the same thing?

PUT /database-connectors/{databaseConnectorId}

​

------------------------------
Robert L. Henning
Enterprise Platforms - Technology Services - Senior Associate
Enterprise Databases
------------------------------