Delphix Toolkits (dxToolkit and dxmToolkit)

 View Only
Expand all | Collapse all

dx_ctl_js_container: Not all VDBs mapped to sources.

  • 1.  dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 10:09:00 AM
    We frequently have to onboard new users, adding them into Delphix as Self-Service users. I'm trying to write a script using the dxtoolkit that will create the user, build the required VDBs (with hooks), and create a container for those VDBs. The first two steps I've already figured out, but I cannot seem to get the syntax right for the last part. After looking at the dxtoolkit, I've been trying to use the dx_ctl_js_container command. The documentation of this command only gives one example of creating a new container and that's based on a template with 2 sources.

    Notes
    • Delphix 5.3.7.1
    • dxtoolkit: 2.4.5
      • Running on a Windows 2012 server, using the compiled scripts from the Releases page.
    • I'm writing in my script in PowerShell; but when I had problems running dx_ctl_js_container, I switched to calling dx_ctl_js_container.exe directly.
    • Our local, best practices for Delphix include:
      • The Data Template should be named the same as the Dataset Group, and should only contain the Containers for the VDBs in the Dataset Group.
      • Containers should be named the same as their VDB.
      • Dataset Groups/Data Templates are named to show which server they're for (i.e. SERVER01Targets).
      • Containers/VDBs are named for the users they belong to (i.e. VDBmmckibben).
    Below is an example of the command with all the options that I'm trying to run, I'm hoping somebody can point out how I've messed up the syntax as I keep getting this error message "Not all VDBs mapped to sources. Exiting"

    dx_ctl_js_container.exe -d DelphixEngine -action create -container_def "SERVER01Targets,VDBmmckibben" -container_name VDBmmckibben -template_name SERVER01Targets -container_owner mmckibben -dontrefresh

    In this example:
    • DelphixEngine is the Delphix Engine as defined in dxtools.conf.
    • SERVER01 is a Microsoft SQL Server.
    • SERVER01Targets (container_def) is an existing Dataset Group.
    • SERVER01Targets (template_name) is an existing Data Template.
    • VDBmmckibben (container_def) is an existing VDB.
    • VDBmmckibben (container_name) is the name we would want to use for this container.
    • mmckibben (container_owner) is an admin user.
    Any ideas on what I'm doing wrong?

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------


  • 2.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 10:23:00 AM
    Hi Mark,

    Can you please add the flag -debug to your command and past the result to the community.

    Regards,

    Mouhssine

    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 3.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 10:39:00 AM
    Here's the debug output:
    Dxtoolkit version 2.4.5
    Loading engines from C:/Scripts/dxtoolkit2/dxtools.conf
    Can't secure cookie. Windows machine
    connecting to: DelphixEngine ( IP/name : 192.168.1.17 )
    GET: http://192.168.1.17:80/resources/json/delphix/session
    Session exists.
    GET: http://192.168.1.17:80/resources/json/delphix/user/current
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/template
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/container?template=JS_DATA_TEMPLATE-42
    GET: http://192.168.1.17:80/resources/json/delphix/role
    GET: http://192.168.1.17:80/resources/json/delphix/authorization
    GET: http://192.168.1.17:80/resources/json/delphix/user
    GET: http://192.168.1.17:80/resources/json/delphix/user/current
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/datasource?dataLayout=JS_DATA_TEMPLATE-42&
    GET: http://192.168.1.17:80/resources/json/delphix/source
    GET: http://192.168.1.17:80/resources/json/delphix/sourceconfig
    GET: http://192.168.1.17:80/resources/json/delphix/repository
    GET: http://192.168.1.17:80/resources/json/delphix/environment
    GET: http://192.168.1.17:80/resources/json/delphix/environment/oracle/clusternode
    GET: http://192.168.1.17:80/resources/json/delphix/environment/windows/clusternode
    GET: http://192.168.1.17:80/resources/json/delphix/environment/user
    GET: http://192.168.1.17:80/resources/json/delphix/environment/oracle/listener
    GET: http://192.168.1.17:80/resources/json/delphix/host
    GET: http://192.168.1.17:80/resources/json/delphix/namespace
    GET: http://192.168.1.17:80/resources/json/delphix/toolkit
    GET: http://192.168.1.17:80/resources/json/delphix/database
    GET: http://192.168.1.17:80/resources/json/delphix/namespace
    GET: http://192.168.1.17:80/resources/json/delphix/group
    Not all VDBs mapped to sources. Exiting

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------



  • 4.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 10:46:00 AM
    HI Marc,

    Please try to skip double quotes on -container_def values.

    Regards
    Marcin

    ------------------------------
    Marcin Kwasninski
    Technical Principal
    Spica Solutions
    ------------------------------



  • 5.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 10:55:00 AM
    Running the command without the quotes around the values on container_def gave the same results.
    Dxtoolkit version 2.4.5
    Loading engines from C:/Scripts/dxtoolkit2/dxtools.conf
    Can't secure cookie. Windows machine
    connecting to: DelphixEngine ( IP/name : 192.168.1.17 )
    GET: http://192.168.1.17:80/resources/json/delphix/session
    Session exists.
    GET: http://192.168.1.17:80/resources/json/delphix/user/current
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/template
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/container?template=JS_DATA_TEMPLATE-42
    GET: http://192.168.1.17:80/resources/json/delphix/role
    GET: http://192.168.1.17:80/resources/json/delphix/authorization
    GET: http://192.168.1.17:80/resources/json/delphix/user
    GET: http://192.168.1.17:80/resources/json/delphix/user/current
    GET: http://192.168.1.17:80/resources/json/delphix/jetstream/datasource?dataLayout=JS_DATA_TEMPLATE-42&
    GET: http://192.168.1.17:80/resources/json/delphix/source
    GET: http://192.168.1.17:80/resources/json/delphix/sourceconfig
    GET: http://192.168.1.17:80/resources/json/delphix/repository
    GET: http://192.168.1.17:80/resources/json/delphix/environment
    GET: http://192.168.1.17:80/resources/json/delphix/environment/oracle/clusternode
    GET: http://192.168.1.17:80/resources/json/delphix/environment/windows/clusternode
    GET: http://192.168.1.17:80/resources/json/delphix/environment/user
    GET: http://192.168.1.17:80/resources/json/delphix/environment/oracle/listener
    GET: http://192.168.1.17:80/resources/json/delphix/host
    GET: http://192.168.1.17:80/resources/json/delphix/namespace
    GET: http://192.168.1.17:80/resources/json/delphix/toolkit
    GET: http://192.168.1.17:80/resources/json/delphix/database
    GET: http://192.168.1.17:80/resources/json/delphix/namespace
    GET: http://192.168.1.17:80/resources/json/delphix/group
    Not all VDBs mapped to sources. Exiting

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------



  • 6.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 11:01:00 AM
    Have you created a template before running  dx_ctl_js_container ?

    dx_ctl_js_template -d DE_SERVER -action create -template_name TEMPL_NAME -source FOLDER,dSource,template_name,1


    Regards
    Marcin

    ------------------------------
    Marcin Kwasninski
    Technical Principal
    Spica Solutions
    ------------------------------



  • 7.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 11:14:00 AM
    Yes, the SERVER01Targets specified in the template_name parameter is a Data Template that already exists in Self-Service (Jetstream) on this DelphixEngine.

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------



  • 8.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 02:12:00 PM
    I have an idea, this is only my assumption...
    How you create a template ?
    Maybe you are Ponting to wrong parent name of VDB/dSource which has no child VDB.


    Regards
    Marcin

    ------------------------------
    Marcin Kwasninski
    Technical Principal
    Spica Solutions
    ------------------------------



  • 9.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 03:09:00 PM
    @Marcin Kwasninski I'm not sure I'm understanding your ques​tion. The data template in Self-Service already exists and has other containers in it. Is there some other template I'm forgetting about that needs to be created here? Or are you maybe saying something in the command should be pointing at the dSource of the VDB that I'm trying to create the container for?

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------



  • 10.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-02-2020 11:05:00 AM
    Edited by Mouhssine Saidi 03-02-2020 11:07:49 AM
    Hi,

    I'm aligned with the first replay from Marcin, because I think that it looks for the the named container_def including double quotes in the name.

    You have to skip them or try without double quotes

    Regards,

    Mouhssine

    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 11.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.

    Posted 03-03-2020 03:09:00 AM
    Hi Mark,

    To allow us dig into the root cause can you run the following commands and past the result to the community website please (if there is no sensitive information)

    dx_get_hierarchy -d DelphixEngine
    dx_get_db_env -d DelphixEngine
    dx_get_js_templates -d DelphixEngine


    Regards,

    Mouhssine

    ------------------------------
    Mouhssine SAIDI
    Community Member
    Delphix Community Members
    ------------------------------



  • 12.  RE: dx_ctl_js_container: Not all VDBs mapped to sources.
    Best Answer

    Posted 03-03-2020 09:29:00 AM
    After a good night's sleep, and some stronger coffee, I found I'd made a mistake in setting up the VDB I was using to test this process. My test VDB was setup to use DSOURCE2, but that dSource is not listed on the Sources page for my Data Template. Once I rebuilt my test VDB using the correct dSource; then the dx_ctl_js_container started working for me.

    Thank you both for all your help in troubleshooting this!

    ------------------------------
    Mark McKibben
    Senior Database Administrator
    United Fire Group
    ------------------------------