DB_SYNC job failed due to an error during execution

  • 0
  • 1
  • Problem
  • Updated 3 years ago
  • Acknowledged
Hello,

After launching the first snapshot from a sqlserver source (VM), on Landshark 2.0, we get this error:
"DB_SYNC job for ... failed due to an error during execution:
Failed to discover and mount LUNs from the Delphix Engine through the iSCSI initiator on target host ... .
Make sure that the iSCSI initiator is running and that the user ... has privileges to perform disk management on the host."

We double-checked that:
delphix user is a domain user with local administrator privileges on both source and target,
it has sysadmin role on both SQL Server instances (source and target),
and has "Logon as Batch job" rights on target.
Delphix Connector and iSCSI initiator services are up and running on target machine.
(target machine is used as staging too)

Could you help us to solve this issue?
Thanks.

Note: This conversation was created from a reply on: Tip of the Day: SQL Server Set Up and a Windows User.
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb

Posted 4 years ago

  • 0
  • 1
Photo of Neal Stack

Neal Stack, Employee

  • 2,762 Points 2k badge 2x thumb
Hello,

Can you execute the "iscsicli.exe" command from the command line without getting an error? If you get an error while trying to execute this command, you will get the same error you reported from Delphix:

C:\Windows\system32>iscsicli
Microsoft iSCSI Initiator Version 6.1 Build 7601

Error getting node name: Access is denied.
If you are able to execute it without error, you should see the IQN returned which is what Delphix is looking for:
C:\Users\delphixaduser>iscsicli
Microsoft iSCSI Initiator Version 6.1 Build 7601

[iqn.1991-05.com.microsoft:10-43-3-155.ad.delphix.com]

Thanks,
  Neal
(Edited)
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Hello,

thanks for your reply.

Running "iscsicli.exe" as delphix user on target machine will result in something like your example:

[iqn.1991-05.com.microsoft:delphix]

so, it seems ok.

What do you suggest now?

Thanks
Giorgio
Photo of Neal Stack

Neal Stack, Employee

  • 2,762 Points 2k badge 2x thumb
Hello,

Did you execute the "iscsicli.exe" as the same user defined as the "Environment User"? In other words, you did not try executing it from a different user account such as "Administrator"?

Could there be a firewall between Landshark and SQL Server blocking port 3260? There are certain ports that need to be accessible:
   http://docs.delphix.com/display/DOCS42/Network+and+Connectivity+Requirements+for+SQL+Server+Environm...

Thanks,
  Neal
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Hello,

I executed the "iscsicli.exe" as the same user defined as the "Environment User" and also with "Administrator" getting same result. ([iqn.1991-05.com.microsoft:delphix])

Source, Target and Landshark machines are on same network without any firewall / network limitations.

Just to be sure I turned off (temporarly) windows firewall on source and target machines.

Unfortunately still not works!

May I do some testing activities on landshark?

Thanks,
Giorgio
Photo of Neal Stack

Neal Stack, Employee

  • 2,762 Points 2k badge 2x thumb
Hello,

I work in Delphix technical support and would normally request that a technical support case be opened and a support bundle uploaded to it. It is my understanding that the Landshark environment is supported by a different group here at Delphix so I would expect one of them to follow on soon with a request or suggestion.

Thanks,
  Neal
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Giorgio, what happens if you change the environment user to the Administrator account? Does the error persist? This is just for troubleshooting.
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Hello Adam,

the error persists even using the Administrator account instead of delphix as the environment user (for both source and target/staging Environment).

Thanks,
Giorgio
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
What version of Windows is this?
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Can you post the iqn's from all the Windows environments attached to this Delphix engine?
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Also, can you post a screenshot of this from your target?  "iSCSI Initiator" GUI ->Targets -> Discovered Targets -> Status field
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Hello Adam,

source and target/staging are VM both equipped with:
Windows Server 2008 R2 SP1 64bit
SQLServer Enterprise 11 (2012) 64bit

source iqn (iSCSI initiator GUI > Configuration): iqn.1991-05.com.microsoft:delphix1.delphix.delphix
target iqn (iSCSI initiator GUI > Configuration): iqn.1991-05.com.microsoft:delphix2.delphix.delphix

here is the screenshot taken from target machine:


Hope this helps.

Thanks,
Giorgio
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Do you get any errors when refreshing your source or target environment in the Delphix Console?
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
I did source and target environment refresh many times.
Never gotta an error.
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Then I am even more perplexed as the refresh actually detects and builds these iSCSI LUNs as part of the refresh process, I believe. That is why you have the discovered Target in the iSCSI initiator. 

When you first created the dSource, did it have a time card entry? Or was it blank, and then you tried to take a snapshot?
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
It was blank.
This issue occurs trying to get the very first snapshot
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Ok. A snapshot should be there at the end of successfully adding the dSource (look in the dashboard). Can you post the entire job log from adding the dSource? If not, can you please delete and re-add it, then post the entire log? Then create and download a support bundle and email it to hsic@delphix.com.
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Hello,

I delete and re-created dSource.

Here is job logs:

DB_LINK job started for "sqlserver_sources/test_source".
DB_LINK job for "sqlserver_sources/test_source" completed successfully.
DB_SYNC job started for "sqlserver_sources/test_source".
Starting sync from the source database "test_source"
Creating new TimeFlow for dSource "test_source"
Exporting storage containers.
Generating scripts needed for validated sync
Validating user environment settings on the staging host "10.80.3.221"
Mounting datasets on the staging host "10.80.3.221"
Failed to discover and mount LUNs from the Delphix Engine through the iSCSI initiator on target host "10.80.3.221".

Thank you so much,
I appreciate your helps.

Giorgio
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Can you go back to that target in the iSCSI initiator and disconnect it? Then refresh the target in the Delphix engine, then delete and re-add the dSource. Post the logs
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
iSCSI initiator  disconnected
target environment refreshed
dSource deleted and re-created (with same settings)

job logs are IDENTICAL to the previous try.

DB_LINK job started for "sqlserver_sources/test_source".
DB_LINK job for "sqlserver_sources/test_source" completed successfully.
DB_SYNC job started for "sqlserver_sources/test_source".
Starting sync from the source database "test_source"
Creating new TimeFlow for dSource "test_source"
Exporting storage containers.
Generating scripts needed for validated sync
Validating user environment settings on the staging host "10.80.3.221"
Mounting datasets on the staging host "10.80.3.221"
Failed to discover and mount LUNs from the Delphix Engine through the iSCSI initiator on target host "10.80.3.221".
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Can you download and email the support bundle to me?
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
I sent you an email with a Dropbox link to download the support bundle.

Thanks,
Giorgio
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Got it Giorgio, I have found the root error message, but I am not sure what is causing it yet. I will update you when I know a little more.
Photo of Giorgio Segnato

Giorgio Segnato

  • 332 Points 250 badge 2x thumb
Great. I look forward to get some news from you soon (hope).
Thanks again.
Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Can you send me the screen shots from here?
  1. In Control Panel, open Regional and Language Options.

  2. The Languages tab, 

  3. The Regional Options tab

  4. The Advanced tab

Photo of Adam Bowen

Adam Bowen, Official Rep

  • 17,420 Points 10k badge 2x thumb
Giorgio, we believe it is a localization issue since you are on an Italian locale in Windows. It is something we are looking into and will address some time in the future. For now, there are a few things we can try.

1) Set the user locale for the environment user for the Target to us_EN

2) Set the system locale for the Target to us_EN

3) we have better support for languages in 4.2, I can send you a link for the download and you can try with that