Delphix Products

Expand all | Collapse all

Delphix Virtualization - subsetting data during VDB creation

  • 1.  Delphix Virtualization - subsetting data during VDB creation

    Posted 09-08-2021 06:32:00 AM
    Hi Team,

    Is it possible to subset data when a VDB is created from a dSource snapshot ?

    For example: My dSource snapshot has millions of products, however I would like to have only active products in the VDB.

    One way I can think of, is using an on-the-fly masking job and setting the filter criteria within the ruleset. However can I link on-the-fly masking jobs to dSource ? If yes, will it consider the dSource snapshot as source and VDB as target ?

    If the above approach of using on-the-fly masking job is not possible, is there any other way to set up the filter criteria ?

    Thanks,

    ------------------------------
    Rajat Solanki
    TCS
    Delphix Community Members
    ------------------------------


  • 2.  RE: Delphix Virtualization - subsetting data during VDB creation

    Posted 09-09-2021 02:31:00 AM
    Just went through an other post which has some great information and details regarding this topic:
    https://community.delphix.com/communities/communityhome/digestviewer/viewthread?GroupId=97&MID=4504&CommunityKey=4e2d7f5c-8842-432b-86ec-1bd8b6f628d7&tab=digestviewer

    Basic concept is as follows:
    Storage space which VDB occupies depends on changes made to it.
    So if VDB data is exactly same as source - It will occupy minimum storage.
    If lots of data is removed from VDB by applying data subsetting - The storage will be huge.

    So the ideal thing is not to do any subsetting on VDB, if the aim is to save storage space.


    ------------------------------
    Rajat Solanki
    TCS
    Delphix Community Members
    ------------------------------



  • 3.  RE: Delphix Virtualization - subsetting data during VDB creation

    Posted 09-09-2021 03:17:00 AM
    Yes, indeed. That's the caveat, if huge subset of data has to be purged.

    ------------------------------
    Yogesh Tiwari
    Senior Principal Technical Services
    Delphix
    ------------------------------



  • 4.  RE: Delphix Virtualization - subsetting data during VDB creation

    Posted 09-09-2021 02:14:00 PM
    It's true that if you delete large amounts of data, you make large amounts of changed blocks which consumes storage on Delphix.  However, there are valid business reasons to subset:  test cases might expect it, batch jobs in testing might run faster.  If you need to subset, we recommend you create a "master/golden" VDB and perform your subsetting and other transformations (masking, add synthetic data, upgrade, etc etc etc).  Then the magic:  create other VDBs from the master VDB.  All the other ones will inherit all the transformations with no extra work or storage penalties.  These VDBs are then passed out to devs, testers, analysts, etc.

    The above approach across a healthy number of systems will not inflict large storage penalties, and will yield huge efficiencies for your org.

    Ranzo

    ------------------------------
    Ranzo Taylor
    Technical Services Director
    Delphix
    ------------------------------