Masking JSON Files

  • 0
  • 1
  • Question
  • Updated 3 weeks ago
  • Acknowledged
Does Delphix support masking JSON files ? Based on the online documentation it doesn't, but is there a plan to introduce this soon enough ?
Photo of Mayank Ahluwalia

Mayank Ahluwalia

  • 708 Points 500 badge 2x thumb

Posted 9 months ago

  • 0
  • 1
Photo of Gary Hallam

Gary Hallam, Official Rep

  • 2,008 Points 2k badge 2x thumb
Hi Mayank,
Masking JSON is certainly on our roadmap and a hot topic item that is being actively investigated by our engineering team.  We see a lot of demand here.  We do not support this yet and would suggest at this point in time, some pre-processing into a supported format for masking.
Regards,
Gary
Photo of Mayank Ahluwalia

Mayank Ahluwalia

  • 708 Points 500 badge 2x thumb
Thanks Gary
Photo of santosh kumar

santosh kumar

  • 430 Points 250 badge 2x thumb
Hi Gary,

I am another customer who is looking for masking capability of JSON format as most of our files in AWS S3 are JSON. As it is old post, so wanted to check is there any capability added now or tentative dates for near future.

Thanks,
Santosh
Photo of Alexandros

Alexandros

  • 80 Points 75 badge 2x thumb
Hi Santosh, 

Masking JSON directly is on our roadmap and is currently being investigated by our team. We have seen a growing demand over the last few years as NoSQL sources have become more popular. If there is any JSON data you need to mask right now, the best way to accomplish that is to convert it to a file format Delphix supports (see maskingdocs.delphix.com), mask it, and then convert it back. 
Photo of santosh kumar

santosh kumar

  • 430 Points 250 badge 2x thumb
Thanks Alex. In our scenario, we are trying for all our S3 objects which are primarily 80% of data. Is there preferred approach to convert or some examples to highlight. So, i can have more insight and look at the feasibility. 

Appreciate your response !! 
Photo of Hims

Hims, Employee

  • 2,216 Points 2k badge 2x thumb
@Santosh, 

AN S3 bucket can be accessed via a CLI or by mounting on an EC2 instance. A common way to mask objects in S3 is via a script with workflow as below:
  1. Use AWSCLI to move objects to EC2 instance (https://docs.aws.amazon.com/cli/latest/reference/s3/).
  2. Host an SFTP server on an EC2 instance, connect and mask the files via SFTP connectivity in the instance.
  3.  Move masked files back to S3 using CLI.

Hope this helps.

Thanks
Hims
Photo of santosh kumar

santosh kumar

  • 430 Points 250 badge 2x thumb
Thanks Hims. I have taken exact approach as you mentioned. the S3 bucket  is mounted using s3Fuse on EC2 and use SFTP connection to browse files. But, getting lost when it comes to mask 100 of GBs of Json and other formatted files. Couple of challenges to list down:

- JSON is not supported
- Not easy way to browse files from different folder structures and apply masking rules
- Files are grouped by defining the patterns, but  jobs doesn't execute successfully all the time
- It doesn't load more than 200 files at a time.

Trying to explore how other customers are handling File masking (JSON, XML) of S3 bucket.

Thanks,
Santosh