Delphix Products

 View Only
Expand all | Collapse all

Masking Cassandra database using Delphix

  • 1.  Masking Cassandra database using Delphix

    Posted 10-31-2024 04:11:00 PM

    I am trying to mask a Cassandra Database. Currently I have masked Oracle, SQL databases using Delphix. Please let me know if I can mask Cassandra database using delphix. If yes, what is the difference in steps from the usual Oracle or SQL databases



    ------------------------------
    Dimple AB
    TCS
    ------------------------------


  • 2.  RE: Masking Cassandra database using Delphix

    Posted 11-01-2024 07:55:00 AM

    Hi Dimple,

    My notes say that it is possible, but it requires a Custom Connector and Professional Services are required for support.  You should talk with your Delphix account partners (Sales Engineer, CSM, RSM, etc). 

    In general, there's two options:

    • Buy 3rd party driver to make Cassandra look like a Relational database.  See CData: Cassandra
    • Unload the data to delimited files, mask, reload

    If this is a large Cassandra database, talk to your CSM to see if Cassandra is on the roadmap for Delphix Hyperscale Compliance

    When you do have success, please come back and post your experience!  It will be valuable for customers in the future (like myself) :-)

     



    ------------------------------
    Kevin Bott
    Sr. Database Architect
    Northwestern Mutual Life Insurance Company
    ------------------------------



  • 3.  RE: Masking Cassandra database using Delphix

    Posted 11-01-2024 09:21:00 AM

    There is Delphix support available for us, but I am not aware of what level of support they can provide. But before checking with them I wanted to know about the custom connector. Will the Delphix team be able to provide a connector which can connect to Cassandra DB and then I can mask it similar to other DBs?



    ------------------------------
    Dimple AB
    TCS
    ------------------------------



  • 4.  RE: Masking Cassandra database using Delphix

    Posted 11-01-2024 02:57:00 PM

    I think Kevin's response is the correct approach. If professional services can provide a custom connector, your support for that connector would be Professional services. Delphix support would not be involved, and there would be a cost to that solution. Cassandra is not supported  for masking, but you can try to use a 3rd party driver like CData, this would be an extended driver from the point of view of delphix support, which also does not provide support but may get you at least some ability to mask.

    The likely best solution would be to unload the data and mask the files, the masking for this would be supported

    I am not aware of any current project to mask data natively for cassandra



    ------------------------------
    Donald Kyle
    Member of Technical Staff
    Delphix
    ------------------------------



  • 5.  RE: Masking Cassandra database using Delphix

    Posted 11-04-2024 04:10:00 AM

    We have heard about accessing Cassandra using Hive or Spark .. but that might be restricted to queries (not update/inserts). Further there are some commercial JDBC vendors like CData that offer custom drivers that translate between SQL and the targeted DB native protocol .. but those drivers may have functional restrictions means I would first get a test version to verify functionality.

    My suggestion is to use the ExtractMaskLoad approach, in CQL language (CassandraQueyLanguage) you can use COPY command to export to CSV/import from CSV > and mask the extracted CSV files in Delphix using the Delimited connector. This approach possibly offers the best performance and no costs.



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------



  • 6.  RE: Masking Cassandra database using Delphix

    Posted 11-04-2024 08:36:00 AM

    Thanks for the suggestion.

    So if I proceesd with the ExtractMaskLoad approach, the masking using delimited connector would be different from other usual masking approach?



    ------------------------------
    Dimple AB
    TCS
    ------------------------------



  • 7.  RE: Masking Cassandra database using Delphix

    Posted 11-04-2024 09:27:00 AM

    It will be good to learn how file masking works, even if you don't use if in this case.  It's great for testing something out when you don't have a database you can use.  

    Setup for File Masking is a bit more involved than database masking, as you have to provide the File Format: https://masking.delphix.com/docs/latest/constructing-file-formats-for-upload

    You need to setup a file connector: https://masking.delphix.com/docs/latest/managing-connectors#id-(28.0.0.0)Managingconnectors-Fileconnectors

    Once you have those, you can create a ruleset.  You will select the file accessible via the connector and assign the format, choose a delimiter, set header or footer rows, etc.  

    Then you can configure your algorithms similar to how its done for databases.  



    ------------------------------
    Kevin Bott
    Sr. Database Architect
    Northwestern Mutual Life Insurance Company
    ------------------------------



  • 8.  RE: Masking Cassandra database using Delphix

    Posted 11-04-2024 08:39:00 AM

    Is this the right documentation where it is showing to mask a file - Masking Whole File - Delphix Masking Documentation



    ------------------------------
    Dimple AB
    TCS
    ------------------------------



  • 9.  RE: Masking Cassandra database using Delphix

    Posted 11-04-2024 09:30:00 AM

    You don't need that for masking delimited files.  That is an edge case where you need to pass the entire file into an algorithm.  This was needed to mask JSON before JSON support was added.  



    ------------------------------
    Kevin Bott
    Sr. Database Architect
    Northwestern Mutual Life Insurance Company
    ------------------------------



  • 10.  RE: Masking Cassandra database using Delphix

    Posted 11-05-2024 06:21:00 AM

    Thanks for the response. So when masking cassandra as delimited files is it possible for In-place masking or on the fly would be the best option? 



    ------------------------------
    Dimple AB
    TCS
    ------------------------------



  • 11.  RE: Masking Cassandra database using Delphix

    Posted 11-05-2024 07:06:00 AM

    You can do either :)  I would choose OTF because if anything goes wrong with the masking you can easily apply changes and re-run the job.



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------



  • 12.  RE: Masking Cassandra database using Delphix

    Posted 11-05-2024 03:42:00 PM

    Is it possible to share a delphix document link explaining this on the fly masking using the delimited connector - from creating connector to masking as I need to refer and define a process which can be suitable for our customer.



    ------------------------------
    Dimple AB
    TCS
    ------------------------------



  • 13.  RE: Masking Cassandra database using Delphix

    Posted 11-06-2024 02:27:00 AM

    This is basic knowledge, have you had any kind of training ?

    For OTF you need two environments, one to read from and one to write to.

    In each environment a delimited connector connecting to the files using either ftp/sftp/nfs/cifs/blockStorage.

    On the TARGET environment you create a ruleset, add to ruleset the files either by selecting them or using a regex pattern to match the files.
    On the ruleset per FILE or PATTERN you assign the file-format.

    The file-format for delimited is very simple, just one horizontal line per COLUMN of the data, this is used to render the UI where you enter the domain/algorithms. Column names must not contain space or special characters!


    Assign algorithms in ruleset as required and run masking job. Done.



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------



  • 14.  RE: Masking Cassandra database using Delphix

    Posted 11-06-2024 08:17:00 AM

    Yes, I have done Oracle and SQL DB In place masking, but not file masking. But for these everything like creating connector to ruleset and job is direct. But for files is it different. I got some idea referring the above, I will check if I can test with some files to be clearer before proceeding with actual files.



    ------------------------------
    Dimple AB
    TCS
    ------------------------------