Refresh a Self Service Container from a Template Snapshot to reduce refresh time

  • 1
  • 4
  • Article
  • Updated 1 month ago
  • (Edited)
By default Self Service Refreshes on a container will refresh the container database or databases to the latest point in time available. We have seen that some of these refreshes may take some time. In the Oracle case, all archivelogs created after the backup have to be applied.


Our clients have been requested a way to refresh a container to the point in time where the snapshot happened in the template, but unfortunately the snapshots are not visible on the template timeline.


Here we present a workaround to be able to see snapshots in the template timeline on the Self Service screen.


On this example, the template has only one datasource.

The solution is very simple and based on the fact that bookmarks on templates are always shared, never private. So what I did is just create a post sync hook on the source (the template), that every time we do a snapsync, it will create a bookmark in the template that will be visible for all Self Service users. The hook is very simple and I used DxToolkit for it:



So every time a snapsync runs, this post sync hook will create a bookmark on the template timeline called snapsync + date:



With this simple hook, all Self Service users will have visibility to all the snapsyncs that are running on the source.

If you are not familiar with DxToolkit you must check it out. It's available here: https://github.com/delphix/dxtoolkit


If you don't want to use the DxToolkit script, as you will have to put it in all the targets, we can use a script like the one I pasted below. The good news about this approach is that all the logic to create the bookmark is stored in Delphix. This script has been tested in Red Hat Linux, may require minor changes for other Linux flavors or Unix.



On my example you can see delphix_admin is being used. If this is just not good enough in terms of security, we can just create a user with ownership on the dsource.



## This hook will create a bookmark in the template that contains this dsource

# Below Variables may change, make sure you have the proper values:
ENGINEURL=http://192.168.92.220
export URL
USR=delphix_admin
export USR
PASS=delphix
export PASS
BRANCHNAME=JS_BRANCH-1
export BRANCHNAME
SOURCEDATALAYOUT=JS_DATA_TEMPLATE-1
export SOURCEDATALAYOUT




#Below variables are fixed, no need to modify
BNAME=snapsyc-$(date +%Y-%m-%d-%H-%M-%S)
export BNAME




## Connect to the Delphix Engine
echo
echo
echo "Create session"
curl -s -X POST -k --data @- ${ENGINEURL}/resources/json/delphix/session \
    -c ~/cookies.txt -H "Content-Type: application/json" <
{
    "type": "APISession",
    "version": {
        "type": "APIVersion",
        "major": 1,
        "minor": 4,
        "micro": 0
    }
}
EOF
echo
echo
echo "Logon as:" ${USR}/${PASS}
curl -s -X POST -k --data @- ${ENGINEURL}/resources/json/delphix/login \
    -b ~/cookies.txt -H "Content-Type: application/json" <
{
    "type": "LoginRequest",
    "username": "${USR}",
    "password": "${PASS}"
}
EOF1




# Create Bookmark on Template
echo
echo
echo
echo
curl -s -X  POST -k --data @- ${ENGINEURL}/resources/json/delphix/jetstream/bookmark \
 -b ~/cookies.txt -H "Content-Type: application/json" <
{
    "type": "JSBookmarkCreateParameters",
    "bookmark": {
        "type": "JSBookmark",
        "name": "$BNAME",
        "branch": "$BRANCHNAME"
    },
    "timelinePointParameters": {
        "type": "JSTimelinePointLatestTimeInput",
        "sourceDataLayout": "$SOURCEDATALAYOUT"
    }
}
EOF1


Photo of Diego Loureda

Diego Loureda, Employee

  • 394 Points 250 badge 2x thumb

Posted 6 months ago

  • 1
  • 4
Photo of Jatinder Luthra

Jatinder Luthra, Employee

  • 2,996 Points 2k badge 2x thumb
This Powershell script can be used as a post hook to create bookmark after every snapshot.

############################### Delphix Masking and Snapshot API Calls ################################
# File: ss_bookmark_template.ps1
# Type: Powershell script
# Author: Delphix Professional Services (Created by - Jatinder Luthra)
# Date: 17-Oct-2018
#
# Copyright and license:
#
#       Licensed under the Apache License, Version 2.0 (the "License"); you may
#       not use this file except in compliance with the License.
#
#       You may obtain a copy of the License at
#     
#               http://www.apache.org/licenses/LICENSE-2.0
#
#       Unless required by applicable law or agreed to in writing, software
#       distributed under the License is distributed on an "AS IS" basis,
#       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#
#       See the License for the specific language governing permissions and
#       limitations under the License.
#     
#       Copyright (c) 2018 by Delphix.  All rights reserved.
#
# Description:
#
# Powershell script to create bookmark on Self Service Template
#   
#   Parent Script: ss_bookmark_template.ps1
#   
#   Notes: This script can be called as a hook which will create a bookmark in the template that contains this dSource                           
#________________________________________________________________________________________
######################################################################################################
#########################################################
#                   DELPHIX CORP                        #
#########################################################
########################################################################################
#########################       SET THE SCRIPT PARAMETERS        #######################
########################################################################################
$SS_TEMPLATE=$args[0]
If ([string]::IsNullOrEmpty($SS_TEMPLATE)){
echo 'Please pass the Self-Service Template Name'
exit 1
}

$VIRTUALIZATION_ENGINE="http://172.16.98.25/resources/json/delphix";
$VIRT_USERNAME="delphix_admin"
$VIRT_PASSWORD="delphix"
#$SS_TEMPLATE = "PerformanceMasking"
$BOOKMARK_TIME=(Get-Date -Format yyyy-MM-dd-HH-mm-ss).ToString()
$SS_BOOKMARK = 'snapsync-'+$BOOKMARK_TIME
#####################################################################################
################         NO CHANGES REQUIRED BELOW THIS POINT        ################
#####################################################################################
#####################################################################################
####################       FUNCTION TO PARSE JSON OUTPUT         ####################
#####################################################################################
function ConvertFrom-Json20([object] $item){ 
    add-type -assembly system.web.extensions
    $ps_js=new-object system.web.script.serialization.javascriptSerializer
    #The comma operator is the array construction operator in PowerShell
    return ,$ps_js.DeserializeObject($item)
}
#########################################################
#
######## Creating Session to Virtualization Engine #######
$cookies = "cookies.txt"
$json = @"
{
    "type": "APISession",
    "version": {
        "type": "APIVersion",
        "major": 1,
        "minor": 7,
        "micro": 0
    }
}
"@
write-output $json | Out-File "session.json" -encoding utf8
$Session = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -c $cookies -sX POST --header 'Content-Type: application/json' -d "@session.json" -k "$VIRTUALIZATION_ENGINE/session")
$getsesstat = ConvertFrom-Json20 $Session
$sesstatus=$getsesstat.status
echo "##### Session Status ##### - $sesstatus"
if ("${sesstatus}" -ne "OK") {
   echo "Session to DELPHIX Virtualization Engine failed with ${sesstatus}"
   exit 1
}

################# Login to Virtualization Engine #################### 
echo " "
echo "########### Login into Virtualization Engine ############"
echo " "
$json = @"
{
    "type": "LoginRequest",
    "username": "${VIRT_USERNAME}",
    "password": "${VIRT_PASSWORD}"
}
"@
write-output $json | Out-File "login.json" -encoding utf8
$Login = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -b $cookies -sX POST --header 'Content-Type: application/json' -d "@login.json" -k "$VIRTUALIZATION_ENGINE/login")
$getloginstat = ConvertFrom-Json20 $Login
$loginstatus=$getloginstat.status
echo "##### Login Status ##### - $loginstatus"
if ("${loginstatus}" -ne "OK") {
   echo "Login to DELPHIX Virtualization Engine failed with ${loginstatus}"
   exit 1
}
#####################################################################################
################        GET Self Service TEMPLATE REFERENCE        ##################
#####################################################################################

$fetch_templates = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -b $cookies -sX GET --header 'Content-Type: application/json' -k "$VIRTUALIZATION_ENGINE/jetstream/template")
# write-output "Databases API Results: ${dbsList}"
$get_temp_list = ConvertFrom-Json20 $fetch_templates
$temp_status=$get_temp_list.status
if ("${temp_status}" -ne "OK") {
   echo "Fetch Jet Stream Templates from DELPHIX Virtualization Engine failed with ${temp_status}"
   exit 1
}
$templist = $get_temp_list.result
#### Extract Template #####
$extractemplate = $templist | where { $_.name -eq "${SS_TEMPLATE}" -and $_.type -eq "JSDataTemplate"} | Select-Object
$TEMPLATE_REFERENCE=$extractemplate.reference
echo "#### TEMPLATE REFERENCE FOR ${JS_TEMPLATE} IS #### " $TEMPLATE_REFERENCE
#####################################################################################
################        GET Self Service BRANCH REFERENCE        ##################
#####################################################################################
$fetch_branches = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -b $cookies -sX GET --header 'Content-Type: application/json' -k "$VIRTUALIZATION_ENGINE/jetstream/branch")
$get_br_list = ConvertFrom-Json20 $fetch_branches
$branch_status=$get_br_list.status
if ("${branch_status}" -ne "OK") {
   echo "Fetch Jet Stream Branches from DELPHIX Virtualization Engine failed with ${branch_status}"
   exit 1
}
$branchlist= $get_br_list.result
#### Extract Branch #####
$extractbranch = $branchlist | where { $_.name -eq 'master' -and $_.dataLayout -eq ${TEMPLATE_REFERENCE}} | Select-Object
$BRANCH_REFERENCE=$extractbranch.reference
echo "#### BRANCH REFERENCE FOR ${SS_BRANCH} IS #### " $BRANCH_REFERENCE
#####################################################################################
################         Bookmark Template and Track Progress        ################
#####################################################################################
echo " "
echo "########### Creating Bookmark on Self Service Template - ${SS_TEMPLATE} ############"
echo " "
$bookmark_param = @"
{
    "type": "JSBookmarkCreateParameters",
    "bookmark": {
        "type": "JSBookmark",
        "name": "$SS_BOOKMARK",
        "branch": "$BRANCH_REFERENCE"
    },
    "timelinePointParameters": {
        "type": "JSTimelinePointLatestTimeInput",
        "sourceDataLayout": "$TEMPLATE_REFERENCE"
    }
}
"@
write-output $bookmark_param | Out-File "bookmark.json" -encoding utf8
$bookmark_template = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -b $cookies -sX POST --header 'Content-Type: application/json' -d "@bookmark.json" -k "$VIRTUALIZATION_ENGINE/jetstream/bookmark")
$getbmstat = ConvertFrom-Json20 $bookmark_template
$bm_status=$getbmstat.status
echo "##### GET Bookmark Status ##### - $bm_status"
if ("${bm_status}" -ne "OK") {
   echo "Bookmark of Template "${SS_TEMPLATE}" failed with ${bm_status}"
   exit 1
}
$bm_jobID = $getbmstat.job
echo " "
echo "###### Bookmark Creation JOBID ##### - $bm_jobID "
echo " "
################ Tracking Bookmark Creation Progress ################

echo "############### Tracking Bookmark Status of Template - $SS_TEMPLATE ####################"
$sleepTimer=1
DO {
   Start-Sleep $sleepTimer
$bm_job = $(C:\Users\dtully\Downloads\curl-7.58.0\src\curl.exe -b $cookies -sX GET --header 'Content-Type: application/json' -k "$VIRTUALIZATION_ENGINE/job/$bm_jobID")
$getjobstat = ConvertFrom-Json20 $bm_job
$jobState=$getjobstat.result.jobState
echo "####### Current Status of Bookmark ${SS_BOOKMARK} Creation ####### - $jobState"
} WHILE ($jobState -eq'RUNNING')
echo " "
echo "####### Self Service Bookmark, ${SS_BOOKMARK} created on Template, ${SS_TEMPLATE} ####### - $jobState"
exit 0

################ END OF SCRIPT ###############

~Thanks
Jatinder Luthra