Delphix Products

Does Delphix support masking JSON files ? Based on the online documentation it doesn't, but is there a plan to introduce this soon enough ?
#Masking

Hi Mayank,
Masking JSON is certainly on our roadmap and a hot topic item that is being actively investigated by our engineering team.  We see a lot of demand here.  We do not support this yet and would suggest at this point in time, some pre-processing into a supported format for masking.
Regards,
Gary

Thanks Gary

Hi Gary,

I am another customer who is looking for masking capability of JSON format as most of our files in AWS S3 are JSON. As it is old post, so wanted to check is there any capability added now or tentative dates for near future.

Thanks,
Santosh

Hi Santosh, 

Masking JSON directly is on our roadmap and is currently being investigated by our team. We have seen a growing demand over the last few years as NoSQL sources have become more popular. If there is any JSON data you need to mask right now, the best way to accomplish that is to convert it to a file format Delphix supports (see maskingdocs.delphix.com), mask it, and then convert it back. 

Thanks Alex. In our scenario, we are trying for all our S3 objects which are primarily 80% of data. Is there preferred approach to convert or some examples to highlight. So, i can have more insight and look at the feasibility. 

Appreciate your response !! 

@Santosh, 

AN S3 bucket can be accessed via a CLI or by mounting on an EC2 instance. A common way to mask objects in S3 is via a script with workflow as below:

1. Use AWSCLI to move objects to EC2 instance (https://docs.aws.amazon.com/cli/latest/reference/s3/).
2. Host an SFTP server on an EC2 instance, connect and mask the files via SFTP connectivity in the instance.
3.  Move masked files back to S3 using CLI.

Hope this helps.

Thanks
Hims

Thanks Hims. I have taken exact approach as you mentioned. the S3 bucket  is mounted using s3Fuse on EC2 and use SFTP connection to browse files. But, getting lost when it comes to mask 100 of GBs of Json and other formatted files. Couple of challenges to list down:

- JSON is not supported
- Not easy way to browse files from different folder structures and apply masking rules
- Files are grouped by defining the patterns, but  jobs doesn't execute successfully all the time
- It doesn't load more than 200 files at a time.

Trying to explore how other customers are handling File masking (JSON, XML) of S3 bucket.

Thanks,
Santosh