Virtualization Plugins

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

Hi Paul,

The short answer is you can use the OwnershipSpecification object to do this.

More generally, a plugin controls where and how mounts are mounted for virtual databases by implementing the virtual.mount_specification() operation and for staging databases (if applicable) by implementing the linked.mount_specification() operation. These operations return a MountSpecification object that dictates where the mount is created. A MountSpecification object has an optional OwnershipSpecification. In your mount specification operation, you can create an OwnershipSpecification object with the correct user and group.

It sounds like you are trying to preserve the user and group from the source database. There are two ways to do this. The first is during discovery you could store the permissions in the RepositoryDefinition object. You'll have access to this object during the mount specification operations. Alternatively, you could ask the user for the permissions they want in either the LinkedSourceDefinition and/or the VirtualSourceDefinition and they will be prompted at link or provision time respectively.

One last note, the reason you're getting permission issues while executing as root is because root squash is enabled by default so root is being mapped to an anonymous user. 

Hope this helps! Let us know if you have any more questions.

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix
------------------------------

Original Message:
Sent: 06-28-2019 03:39:48 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

I was able to get the resulting VDB mount point to the owner and group that I need.  Thanks!

ownership_spec = OwenershipSpecification(repository.uid, repository.gid)

Question now is how to change the mode? 
The staging mount point (i.e. /staging/mongo) is defaulting to 750 so when I create the VDB mount point (i.e. /target/mongo-vdb1) and set the uid/gid the resulting VDB mount is also 750 and owned by mongod:mongod.  With this root cannot read it so the service fails to start.  Note that the original source directory is set to 755 so I'm basically trying to get the mount point to the same permissions as the original mongod data directory.

I do not see a corresponding repository.mod under OwnershipSpecification.
I tried to do a run_bash to do a chmod 755 after mounting the staging mount but it is failing.
I even tried to manually change the staging mount point after the dSource was created and it changed, but then not reflected when creating the VDB.
How can I get it so that the resulting VDB mount point has a mode of 755?  

Thanks!




------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

Original Message:
Sent: 06-28-2019 04:25:04 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

The short answer is you can use the OwnershipSpecification object to do this.

More generally, a plugin controls where and how mounts are mounted for virtual databases by implementing the virtual.mount_specification() operation and for staging databases (if applicable) by implementing the linked.mount_specification() operation. These operations return a MountSpecification object that dictates where the mount is created. A MountSpecification object has an optional OwnershipSpecification. In your mount specification operation, you can create an OwnershipSpecification object with the correct user and group.

It sounds like you are trying to preserve the user and group from the source database. There are two ways to do this. The first is during discovery you could store the permissions in the RepositoryDefinition object. You'll have access to this object during the mount specification operations. Alternatively, you could ask the user for the permissions they want in either the LinkedSourceDefinition and/or the VirtualSourceDefinition and they will be prompted at link or provision time respectively.

One last note, the reason you're getting permission issues while executing as root is because root squash is enabled by default so root is being mapped to an anonymous user. 

Hope this helps! Let us know if you have any more questions.

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 06-28-2019 03:39:48 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

Hi Paul,

Glad you got it working.

We currently do not expose a way for plugins to set the actual permissions on those files. Could you explain in some more detail why you need 755 and why it needs to be read by root? Can the mongod user not be used to run the service?

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix
------------------------------

Original Message:
Sent: 07-02-2019 08:40:44 AM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I was able to get the resulting VDB mount point to the owner and group that I need.  Thanks!

ownership_spec = OwenershipSpecification(repository.uid, repository.gid)

Question now is how to change the mode? 
The staging mount point (i.e. /staging/mongo) is defaulting to 750 so when I create the VDB mount point (i.e. /target/mongo-vdb1) and set the uid/gid the resulting VDB mount is also 750 and owned by mongod:mongod.  With this root cannot read it so the service fails to start.  Note that the original source directory is set to 755 so I'm basically trying to get the mount point to the same permissions as the original mongod data directory.

I do not see a corresponding repository.mod under OwnershipSpecification.
I tried to do a run_bash to do a chmod 755 after mounting the staging mount but it is failing.
I even tried to manually change the staging mount point after the dSource was created and it changed, but then not reflected when creating the VDB.
How can I get it so that the resulting VDB mount point has a mode of 755?  

Thanks!




------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC

Original Message:
Sent: 06-28-2019 04:25:04 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

The short answer is you can use the OwnershipSpecification object to do this.

More generally, a plugin controls where and how mounts are mounted for virtual databases by implementing the virtual.mount_specification() operation and for staging databases (if applicable) by implementing the linked.mount_specification() operation. These operations return a MountSpecification object that dictates where the mount is created. A MountSpecification object has an optional OwnershipSpecification. In your mount specification operation, you can create an OwnershipSpecification object with the correct user and group.

It sounds like you are trying to preserve the user and group from the source database. There are two ways to do this. The first is during discovery you could store the permissions in the RepositoryDefinition object. You'll have access to this object during the mount specification operations. Alternatively, you could ask the user for the permissions they want in either the LinkedSourceDefinition and/or the VirtualSourceDefinition and they will be prompted at link or provision time respectively.

One last note, the reason you're getting permission issues while executing as root is because root squash is enabled by default so root is being mapped to an anonymous user. 

Hope this helps! Let us know if you have any more questions.

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 06-28-2019 03:39:48 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

The root user needs to have read access to the mongo data directory in order to start the mongod service.  The /var/lib/mongo directory on the source that we are replicating has permissions 755 mongod:mongod so root can read it and thus can start the service.  When it is mounted on the target host it has permissions 750 and since it is mongod:mongod root can no longer read it so the service fails to start.  Since it's a Delphix mount I cannot change it to 755.  I may be able to find a work around (i.e. add the mongod group to the root user), but am hesitant to do this since it would not be standard, and may be questioned by customer system administrators.

I there a reason that Delphix does not support changing the mode of the mount point?

Thanks!

Paul

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

Original Message:
Sent: 07-02-2019 03:26:13 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

Glad you got it working.

We currently do not expose a way for plugins to set the actual permissions on those files. Could you explain in some more detail why you need 755 and why it needs to be read by root? Can the mongod user not be used to run the service?

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 07-02-2019 08:40:44 AM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I was able to get the resulting VDB mount point to the owner and group that I need.  Thanks!

ownership_spec = OwenershipSpecification(repository.uid, repository.gid)

Question now is how to change the mode? 
The staging mount point (i.e. /staging/mongo) is defaulting to 750 so when I create the VDB mount point (i.e. /target/mongo-vdb1) and set the uid/gid the resulting VDB mount is also 750 and owned by mongod:mongod.  With this root cannot read it so the service fails to start.  Note that the original source directory is set to 755 so I'm basically trying to get the mount point to the same permissions as the original mongod data directory.

I do not see a corresponding repository.mod under OwnershipSpecification.
I tried to do a run_bash to do a chmod 755 after mounting the staging mount but it is failing.
I even tried to manually change the staging mount point after the dSource was created and it changed, but then not reflected when creating the VDB.
How can I get it so that the resulting VDB mount point has a mode of 755?  

Thanks!




------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC

Original Message:
Sent: 06-28-2019 04:25:04 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

The short answer is you can use the OwnershipSpecification object to do this.

More generally, a plugin controls where and how mounts are mounted for virtual databases by implementing the virtual.mount_specification() operation and for staging databases (if applicable) by implementing the linked.mount_specification() operation. These operations return a MountSpecification object that dictates where the mount is created. A MountSpecification object has an optional OwnershipSpecification. In your mount specification operation, you can create an OwnershipSpecification object with the correct user and group.

It sounds like you are trying to preserve the user and group from the source database. There are two ways to do this. The first is during discovery you could store the permissions in the RepositoryDefinition object. You'll have access to this object during the mount specification operations. Alternatively, you could ask the user for the permissions they want in either the LinkedSourceDefinition and/or the VirtualSourceDefinition and they will be prompted at link or provision time respectively.

One last note, the reason you're getting permission issues while executing as root is because root squash is enabled by default so root is being mapped to an anonymous user. 

Hope this helps! Let us know if you have any more questions.

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 06-28-2019 03:39:48 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------

Hi Paul,

Interesting. Is the mongod service being ran by root or the mongod user? I would have figured if the service was being ran by the mongod user only that user would need access.

There isn't a reason we don't support changing the mode other than none of our other data platforms have ran into this issue. I'll file a ticket to track this.

Another work around is to disable root squash in the plugin config file. With root squash disabled, you should be able to use the root user to change the permissions to whatever you want.

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix
------------------------------

Original Message:
Sent: 07-02-2019 04:21:39 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

The root user needs to have read access to the mongo data directory in order to start the mongod service.  The /var/lib/mongo directory on the source that we are replicating has permissions 755 mongod:mongod so root can read it and thus can start the service.  When it is mounted on the target host it has permissions 750 and since it is mongod:mongod root can no longer read it so the service fails to start.  Since it's a Delphix mount I cannot change it to 755.  I may be able to find a work around (i.e. add the mongod group to the root user), but am hesitant to do this since it would not be standard, and may be questioned by customer system administrators.

I there a reason that Delphix does not support changing the mode of the mount point?

Thanks!

Paul

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC

Original Message:
Sent: 07-02-2019 03:26:13 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

Glad you got it working.

We currently do not expose a way for plugins to set the actual permissions on those files. Could you explain in some more detail why you need 755 and why it needs to be read by root? Can the mongod user not be used to run the service?

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 07-02-2019 08:40:44 AM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I was able to get the resulting VDB mount point to the owner and group that I need.  Thanks!

ownership_spec = OwenershipSpecification(repository.uid, repository.gid)

Question now is how to change the mode? 
The staging mount point (i.e. /staging/mongo) is defaulting to 750 so when I create the VDB mount point (i.e. /target/mongo-vdb1) and set the uid/gid the resulting VDB mount is also 750 and owned by mongod:mongod.  With this root cannot read it so the service fails to start.  Note that the original source directory is set to 755 so I'm basically trying to get the mount point to the same permissions as the original mongod data directory.

I do not see a corresponding repository.mod under OwnershipSpecification.
I tried to do a run_bash to do a chmod 755 after mounting the staging mount but it is failing.
I even tried to manually change the staging mount point after the dSource was created and it changed, but then not reflected when creating the VDB.
How can I get it so that the resulting VDB mount point has a mode of 755?  

Thanks!




------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC

Original Message:
Sent: 06-28-2019 04:25:04 PM
From: Grant Magdanz
Subject: How to change owner:group on VDB mount?

Hi Paul,

The short answer is you can use the OwnershipSpecification object to do this.

More generally, a plugin controls where and how mounts are mounted for virtual databases by implementing the virtual.mount_specification() operation and for staging databases (if applicable) by implementing the linked.mount_specification() operation. These operations return a MountSpecification object that dictates where the mount is created. A MountSpecification object has an optional OwnershipSpecification. In your mount specification operation, you can create an OwnershipSpecification object with the correct user and group.

It sounds like you are trying to preserve the user and group from the source database. There are two ways to do this. The first is during discovery you could store the permissions in the RepositoryDefinition object. You'll have access to this object during the mount specification operations. Alternatively, you could ask the user for the permissions they want in either the LinkedSourceDefinition and/or the VirtualSourceDefinition and they will be prompted at link or provision time respectively.

One last note, the reason you're getting permission issues while executing as root is because root squash is enabled by default so root is being mapped to an anonymous user. 

Hope this helps! Let us know if you have any more questions.

- Grant

------------------------------
Grant Magdanz
Senior Member of Technical Staff II
Delphix

Original Message:
Sent: 06-28-2019 03:39:48 PM
From: Paul Jauquet
Subject: How to change owner:group on VDB mount?

I am developing a SDK to virtualize a MongoDB database on CentOS.  Original files on the source are under /var/lib/mongo and have perms mongod:mongod.  Using rsync to a staging host where the ownership is changes to delphix_os:delphix. Then when we virtualize it creates a mount of the staging directory on the target host as expected.  Once the VDB is mounted I need to chown -R mongod:mongod <vdb nfs mount>  to get the ownership back to the original, or perhaps even to a different user, but it is giving me Operation not permitted even though I am running as root via sudo.  Looks like I cannot change permissions on a NFS mount.  Is there a way to tell Delphix what the owner/group should be as part of the VDB mount?  It would need to be done as part of the mount, or as a post step, since the host/group may vary based on VDB and target host.  Any insight on how to accomplish this would be greatly appreciated.

------------------------------
Paul Jauquet
Senior Data Services Consultant
Axis Technology, LLC
------------------------------