Delphix Products

Expand all | Collapse all

Masking In-Memory DBs

  • 1.  Masking In-Memory DBs

    Posted 02-15-2018 10:06:00 AM
    Delphix does not support In-Memory DBs. We have an In-Memory DB called as Mnesia which is written in Erlang, do we have any past experiences supporting that ? Or any approaches that were developed ?
    #Masking


  • 2.  RE: Masking In-Memory DBs

    Posted 02-15-2018 11:13:00 AM
    Hi Mayank,
    Our general approach to masking unsupported data sources is to extract, mask and then reload.  So you use whatever extract and load tool is available for your data source and extract to a supported file system - such as a csv file.  Mask the file and then load the masked data back.  We have used this technique with unsupported mainframe type files (Non-stop Tandem, Base24 for instance).  You can combine this with the virtualisation capability to provide storage savings and instant provisioning of masked files.
    Regards,
    Gary


  • 3.  RE: Masking In-Memory DBs

    Posted 02-16-2018 06:57:00 AM
    Thanks Gary. The challenge here is that we get a CSV file with JSON's inside. There are about 1200 unique JSON structures and most of them are highly nested with the nesting going up to 100 in certain cases. Can Delphix handle JSON data inside a CSV ? Also, is there a challenge that we should expect with parsing by Delphix given the high amount of nesting ?


  • 4.  RE: Masking In-Memory DBs

    Posted 02-16-2018 07:35:00 AM
    Hi Mayank,
    I imagine that JSON inside a CSV might be fraught with challenges.  We do not support JSON generally at the moment but it is something we are investigating for inclusion onto the roadmap.
    I would look at ways of pre-processing the JSON to get into a supported format.  Of course, it is worth testing to see if the CSV idea works.
    Regards,
    Gary


  • 5.  RE: Masking In-Memory DBs

    Posted 02-16-2018 07:45:00 AM
    Thanks Gary. We are also looking at translating it into XML files, where each unique JSON structure becomes a XML file. How about high nesting within XML ? You foresee any challenges in handling of complex XMLs ?


  • 6.  RE: Masking In-Memory DBs

    Posted 02-16-2018 07:52:00 AM
    Hi Mayank,
    It's probably worth testing this.  Are you working with anyone in Delphix on this particular project?  We could take this off-line and get someone to look at your file structure.  I'm not aware of any challenges with Delphix XML masking; there are certainly a lot of tools out there that do XML-JSON mapping but I haven't used these with complex structures.  Let me know if you'd like me to connect you with one of our partner managers to take this forwards.
    Regards,
    Gary


  • 7.  RE: Masking In-Memory DBs

    Posted 02-16-2018 08:47:00 AM
    Thanks Gary, we have just started our implementation and for now have been talking to Roy during the sales and pre-sales and now Peter Majeed.  Will explore this further and see