Delphix Products

 View Only
  • 1.  Gitlab YAML sample for masking job execution

    Posted 11-29-2023 02:43:00 PM

    Are there any sample GitLab YAML scripts for executing Delphix masking jobs? Looking for plugging in masking process in the gitlab pipeline



    ------------------------------
    Mahesh Chandran
    Delphix Community Members
    ------------------------------


  • 2.  RE: Gitlab YAML sample for masking job execution

    Posted 11-30-2023 08:36:00 AM

    I have not called masking jobs from gitlab, but we do have CICD setup for custom plugin/algorithm build and deployment.  Here's some general thoughts on the subject.

    Gitlab Runner Executor

    First thing to understand is what "Executor" your Gitlab Runner is configured with.  Shell or Docker will work - but each have their own configuration steps required.  You may not have a 

    Shell Executor

    If you already have a Linux host with `dxmc toolkit` setup, then adding a shell executor to those host may be easier than building a docker image.  Your job would specify the script to pass in shell commands to call dxmc (example at the bottom)

    RunMasking:
      script:
        - uname -a
        - dxmc job start --engine ${DXENGINE} --envname ${DXENVIRONMENT} --jobname ${DXJOBNAME}

    Docker Executor

    OK, if you can't use a Shell executor, then Docker executor is likely what you have.  For this, you will need to build a docker image with all the required tools and scripts.  I'm not great with Docker, so I will defer to documentation in your organization for creating a new image from a base image.  When building this image, you can use the curl command to pull the dxmc code from the public github repository.  You may need to install additional prerequisites while building the image.  

    Calling the dxmc command should be the same with the container as it is above in the shell executor example.  

    Skipping dxmc

    If for whatever reason you can't use dxm toolkit, then you will need to get familiar with the API.  You can use shell + curl to access the REST API, but I highly recommend python with the "requests" package.  At this point, you're writing a custom script.  You can execute it the same way as above, but obviously you will dictate what parameters are needed to be passed in, or maybe you choose to utilize environment variables instead.  

    I hope that helps!  



    ------------------------------
    Kevin Bott
    Sr. Database Architect
    Northwestern Mutual Life Insurance Company
    ------------------------------