APIMaskingSidekickPythonOpen-Source Solutions

We are moving into VM servers and planning to virtualize all our databases. In this case, we need to be able to provision vdbs but also expire them after a certain time because we dont want them to be tracked forever. What is the best way to achieve this ? Is there a way we can set the vdb retention when it is getting provisioned ?

Hi Srividya,
I'd be interested to hear a little more detail on your use case, however I'm not aware of a way of setting an expiry/delete time for a VDB.

We do have a REST API which will allow you to automate the VDB deletion after a specified period.  There is also an open source Perl wrapper to the API here (remove VDB Perl script here).

Web Service and CLI guides are here">https://docs.delphix.com/docs/reference">here.

Regards,
Gary

The use case is this – lets say we have a demand to provision a vdb dbA and the application team tells us they need this database for 15 days. We want to be able to set that timeframe during creation of the vdb because we will have a number of these databases and don’t want to be housekeeping them. does that help ?

Hi Srividya, One question in return to you. What happens if your dev didn’t end up his project in the window of 15 days and need more time. Regards, Mouhssine

Thanks for the clarification, I understand the use case and also Mouhssine's response which hints at some of the issues that might be faced.
The way forward is to use the API.
In fact you're probably better to disable VDBs in the first instance - send out a warning and then subsequently delete the VDB after a further time period.  Or just run a periodic job that whacks aged VDBs after a prescribed period.
This is probably a good enhancement request for a UI for managing VDBs.
It would be good to hear from the community how others have managed this scenario using the API or some other method.  Perhaps there's scope for a dxToolkit executable where you enter a number of days and the job disables or deletes all VDBs older than the entry.  Rather a dangerous utility though.
Regards,
Gary

Mouhssine - I was thinking if there is a way to specify a retention period then if any of the VDBs need to be retained longer (which should be a minimal number) then we can extend the retention period. I am not sure if that is possible but that is how I am envisioning.

Gary - We don't mind disabling the VDBs and then deleting them. Are you suggesting we do this manually ? If so, the issue is we will have a huge number (more than 50) of virtual databases and each of them would be from different time periods. I thought it will be cumbersome to keep track of each of them manually.

No I was figuring that you would need to write a small script, perhaps using the dxtoolkit to expire aged VDBs, perhaps taking a parameter for a list of VDBs you might want to keep. 

Hi Srividya, Gary is right. I think you have run a job on scheduled time (every morning) that compares the creation time of vdb with the actual timestamp if it exceeds 15 days it will disable/drop vdb. Regards, Mouhssine

Ok that makes sense. I will have to dig in more. to see how to get details on the vdb. will reach out if I need more info. thanks a lot

I was referencing the CLI cookbook and I am able to connect to the delphix engine and run manual commands to list the vdb creationtime but how do we automate and get that value in a script. Are there examples of something similar ?

Srividya,
You could use the dxtoolkit to create a script. [dxtoolkit is here]

You just need to download the executable version of the files (there's a zip file for Windows and one for Linux/Mac).

One you download and unzip the executables you then need to update the dxtools.config file with the details of your Delphix Engine.

After that it's a matter of checking the documentation and executing a command with appropriate parameters.  Or you can download the source and modify to your needs.

For instance I can use the following command:

./dx_ctl_db -engine LandsharkEngine -name vdb21 -type vdb -action disable

to disable a VDB called "vdb21" on my Delphix Engine called "LandsharkEngine"

You can use the executable "dx_remove_db" to delete the VDB.  See description below.

./dx_remove_db -help

SYNOPSIS

     dx_remove_db.pl [ -engine|d <delphix identifier> | -all ] < -group group_name | -name db_name | -host host_name | -type dsource|vdb > 

     [-force] [-skip] [ --help|? ] [ -debug ] [-parallel p]

DESCRIPTION

    Delete all database(s) selected by filter on selected engine(s)

ARGUMENTS

  Delphix Engine selection - if not specified a default host(s) from dxtools.conf will be used.

    -engine|d Specify Delphix Engine name from dxtools.conf file

    -all Display databases on all Delphix appliance

  Filters

    -group Group Name

    -name Database Name

    -host Host Name

    -type Type (dsource|vdb)

OPTIONS

    -skip Skip confirmation

    -force Force mode

    -help Print this screen

    -debug Turn on debugging

Regards,
Gary

Thanks Gary. I get that part. But I have to first check if the vdb can be disabled or deleted based on the timestamp. For eg., I see this on my vdb

delphengine database 'vtestdb'> get creationTime
    2018-03-05T18:28:59.722Z

Is there a way to get this creationtime through a script so I can compare and see if it is eligible to be deleted

Okay, Srividya
You can create a script something like:

ssh delphix_admin@LandsharkEngine "cd database; select vdb21; get creationTime"

Or this will get you a list of creation times (unless your DB names have spaces):

DELPHIX_ENGINE=LandsharkEngine
SSH_CMD="ssh delphix_admin@${DELPHIX_ENGINE}"
env_array=('${SSH_CMD}  "cd database; list display=name" | grep -v NAME')
for i in "${env_array[@]}"
do
  ${SSH_CMD} "cd database; select $i; get creationTime; back"
echo $i
done

In my system this script did not work because my dataset names contain spaces.  In any case you may want  to use the REST API.

To get around the fact that you will be prompted for a password when you use ssh you'll need to create a key pair on your machine.  To do that on a Mac you simply type the following: 

ssh-keygen -t rsa

This puts a public and private key in the folder ~/.ssh
There may already be one there, in which case use it.
Open the public key (usually named id_rsa.pub) with an editor and copy the text.
You then have to place that public key into the Delphix Engine against the user that you want to log in as from your client machine - let's say delphix_admin.  This is done by logging onto the engine as delphix_admin and using the following commands:

delphix> user current
delphix user "delphix_admin"> update
delphix user "delphix_admin" update *> set publicKey
Enter publicKey: AAAAB3NzaC1yc2EYYYYDAQABAAABAQCt6Gkq3sxtv//OzIl1tPTbHNblo7y+rLMwcPOkpra6iwmcIL+7pVbSSWLGYOXuW/keLOCey4r3LqsTdpwL2F2vvngafCIP9yEhqfzsZ4zhTOaDioBnZ1jc/gQiM9lk8bEO6qpg8Snk/OZ71BxJK1tszuleOBd6uHfD36P6d3DjjvAOGi0C9A/+QnWNW2+SRUzfJZryijuyfYfe46FziVM0HwAPxJiB8h8n8DBN8d1GOuvBljPNYSEu5wpphiK/QvW7c8d0LW2jonbJ3X/FJaOA4d1xG8Fz43SJHCMq9by5Czp6jPdXtVEmQRoCIY7SBAr+2/2LNxDtw2YzR srividya@Srividyas-MacBook-Pro.local

Some documentation around the CLI:
CLI Automation
Generating SSH Key

Regards,
Gary

We do have a feature request for functionality similar to this, however it is not yet on the roadmap at this point in time.  I have added your request to the list - if you are working on behalf of a specific customer then I can add that customer name also.
Regards,
Gary