Delphix Products

Hi,

Does Delphix support AWS File Gateway with LiteSpeed integration to pull in backup files?

Thanks

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company
------------------------------

Hi Megan,

I have not heard of any customers attempting this. Delphix natively supports LiteSpeed ( https://docs.delphix.com/docs/datasets/sql-server-environments-and-data-sources/sql-server-support-and-requirements/sql-server-support-matrix#overview ).

If I understand this correctly ( https://aws.amazon.com/storagegateway/file/ ), it "offers SMB or NFS-based access to data in Amazon S3 with local caching. It can be used for on-premises applications, and for Amazon EC2-based applications that need file protocol access to S3 object storage.".

So it sounds like it would just appear like any old SMB share that you could configure your dSource's "Load Backup Path" to point to ( \\my_aws_smb_share\backups ). My concern would be with performance. From what I understand, S3 uses a lower tier of storage so it might not be performant. Amazon mentions a feature:

• Local applications also benefit from a low-latency local cache of frequently used content. 

However, I don't think Delphix would benefit from this because the dSource would load the database dump or transaction log once and that would be the last time it tried to access that file. It wouldn't fall into the frequently used category.

In addition to the lower tier of storage, you would be contending with added network latency between the Windows host on premises and the S3 bucket in the cloud.

If your dSource is loading a single full or differential database dump once a day, the lesser performance may be OK. However, if the dSource is configured to stay synchronized by loading transaction logs and the source database is dumping new transaction logs every 15 minutes and they are large, the added latency could prevent Delphix from loading the transaction logs fast enough to stay synchronized with the source database.

It sounds like it would work but I just don't know if the performance impact would meet your business requirements for keeping Delphix synchronized with the source database.

Thanks,
  Neal

------------------------------
Neal Stack
Senior Principal Technical Support Engineer
Delphix
------------------------------

Original Message:
Sent: 04-07-2020 01:50:05 PM
From: Meagan Howe
Subject: AWS File Gateway Support

Hi,

Does Delphix support AWS File Gateway with LiteSpeed integration to pull in backup files?

Thanks

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company
------------------------------

Thanks for all of the information! You mentioned S3 may impact performance... what is Delphix's preferred source backup location in AWS?

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company
------------------------------

Original Message:
Sent: 04-07-2020 02:54:13 PM
From: Neal Stack
Subject: AWS File Gateway Support

Hi Megan,

I have not heard of any customers attempting this. Delphix natively supports LiteSpeed ( https://docs.delphix.com/docs/datasets/sql-server-environments-and-data-sources/sql-server-support-and-requirements/sql-server-support-matrix#overview ).

If I understand this correctly ( https://aws.amazon.com/storagegateway/file/ ), it "offers SMB or NFS-based access to data in Amazon S3 with local caching. It can be used for on-premises applications, and for Amazon EC2-based applications that need file protocol access to S3 object storage.".

So it sounds like it would just appear like any old SMB share that you could configure your dSource's "Load Backup Path" to point to ( \\my_aws_smb_share\backups ). My concern would be with performance. From what I understand, S3 uses a lower tier of storage so it might not be performant. Amazon mentions a feature:

• Local applications also benefit from a low-latency local cache of frequently used content. 

However, I don't think Delphix would benefit from this because the dSource would load the database dump or transaction log once and that would be the last time it tried to access that file. It wouldn't fall into the frequently used category.

In addition to the lower tier of storage, you would be contending with added network latency between the Windows host on premises and the S3 bucket in the cloud.

If your dSource is loading a single full or differential database dump once a day, the lesser performance may be OK. However, if the dSource is configured to stay synchronized by loading transaction logs and the source database is dumping new transaction logs every 15 minutes and they are large, the added latency could prevent Delphix from loading the transaction logs fast enough to stay synchronized with the source database.

It sounds like it would work but I just don't know if the performance impact would meet your business requirements for keeping Delphix synchronized with the source database.

Thanks,
  Neal

------------------------------
Neal Stack
Senior Principal Technical Support Engineer
Delphix

Original Message:
Sent: 04-07-2020 01:50:05 PM
From: Meagan Howe
Subject: AWS File Gateway Support

Hi,

Does Delphix support AWS File Gateway with LiteSpeed integration to pull in backup files?

Thanks

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company
------------------------------

Hi Megan,

We don't have documented requirements for backup storage on any platform including Windows.

For the Delphix Engine itself, we do document its requirements here:
   https://docs.delphix.com/docs/deployment/deployment-for-aws-ec2#DeploymentforAWSEC2-EBSConfiguration

Checking with one of my colleagues, he said that given a reasonable compression ratio (~5:1) and AWS's advertised cache miss throughput for AWS File Gateway reads of 100MB/sec, it's possible that this could perform well as long as the network between the staging server and the file gateway was suitable.

Examples of AWS iops limitations for different storage types:
  https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html

So it may work just fine. I am just suggesting that this is an area I would keep an eye on. Speaking from experience, in technical support we have seen that the Delphix Engine is unable to keep up with production when the network throughput/performance to read a file from the SMB share into the SQL Server staging database is poor.

This is a difficult thing to estimate:
* How many databases will be restoring from the SMB share?
* How frequently does each source database create a database dump or transaction log?
* How large are the database dumps and/or transaction logs?
* What is the peak iops necessary to restore backups/logs?

------------------------------
Neal Stack
Senior Principal Technical Support Engineer
Delphix
------------------------------

Original Message:
Sent: 04-08-2020 09:18:22 AM
From: Meagan Howe
Subject: AWS File Gateway Support

Thanks for all of the information! You mentioned S3 may impact performance... what is Delphix's preferred source backup location in AWS?

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company

Original Message:
Sent: 04-07-2020 02:54:13 PM
From: Neal Stack
Subject: AWS File Gateway Support

Hi Megan,

I have not heard of any customers attempting this. Delphix natively supports LiteSpeed ( https://docs.delphix.com/docs/datasets/sql-server-environments-and-data-sources/sql-server-support-and-requirements/sql-server-support-matrix#overview ).

If I understand this correctly ( https://aws.amazon.com/storagegateway/file/ ), it "offers SMB or NFS-based access to data in Amazon S3 with local caching. It can be used for on-premises applications, and for Amazon EC2-based applications that need file protocol access to S3 object storage.".

So it sounds like it would just appear like any old SMB share that you could configure your dSource's "Load Backup Path" to point to ( \\my_aws_smb_share\backups ). My concern would be with performance. From what I understand, S3 uses a lower tier of storage so it might not be performant. Amazon mentions a feature:

• Local applications also benefit from a low-latency local cache of frequently used content. 

However, I don't think Delphix would benefit from this because the dSource would load the database dump or transaction log once and that would be the last time it tried to access that file. It wouldn't fall into the frequently used category.

In addition to the lower tier of storage, you would be contending with added network latency between the Windows host on premises and the S3 bucket in the cloud.

If your dSource is loading a single full or differential database dump once a day, the lesser performance may be OK. However, if the dSource is configured to stay synchronized by loading transaction logs and the source database is dumping new transaction logs every 15 minutes and they are large, the added latency could prevent Delphix from loading the transaction logs fast enough to stay synchronized with the source database.

It sounds like it would work but I just don't know if the performance impact would meet your business requirements for keeping Delphix synchronized with the source database.

Thanks,
  Neal

------------------------------
Neal Stack
Senior Principal Technical Support Engineer
Delphix

Original Message:
Sent: 04-07-2020 01:50:05 PM
From: Meagan Howe
Subject: AWS File Gateway Support

Hi,

Does Delphix support AWS File Gateway with LiteSpeed integration to pull in backup files?

Thanks

------------------------------
Meagan Howe
Database Admin.
New Jersey Manufacturers Insurance Company
------------------------------