Virtualization Plugins

 View Only
  • 1.  VDB status not returning up immediately

    Posted 01-24-2020 11:10:00 AM

    When I run a VDB snapshot and I start the underlying database processes at the end of virtual.post_snapshot and return... the status in the UI immediately shows as running.

    However, when I run a VDB refresh and I start the underlying database processes at the end of virtual.post_snapshot and return.. the status in the UI remains down for a period of time (i.e. minutes), until the next status is ran which then turns it to running.  How can I force the status to be checked/updated at the end of the refresh.  I'd actually think Delphix should automatically kick off a status update of a VDB after any user action completes, whether it be refresh, snapshot, rollback, etc so as to ensure the status is current.

    Also, is there a way in an operation (i.e. pre-snapshot, post-snapshot, configure, etc) to find the action/workflow being performed (i.e. refresh, snapshot, rollback, ) so that I can determine if/when to start the underlying processes.  i.e. if a refresh I want the post refresh operation to start it, but if it's an initial configure or reconfigure/rollback I want to start as part of configure/reconfigure since post snapshot will not be ran.

    Thanks!  Paul

    ------------------------------
    Paul Jauquet
    Senior Data Services Consultant
    Axis Technology, LLC
    ------------------------------


  • 2.  RE: VDB status not returning up immediately

    Posted 01-24-2020 02:12:00 PM
    Edited by Filip Drozdowski 01-24-2020 02:12:41 PM
    Hi Paul,


    We run the status operation after a stop, start, refresh, provision, delete, enable, and disable, and every 5 min for every dSource and VDB. There is no way to force a status update. We do not run the status operation after a snapshot so the behavior you're seeing is odd. Are you logging in your status script to see if and when it is running? 


    There's also no way to find the workflow being performed from the plugin context. Could you elaborate on the reason why you start the database process in post_snapshot operation? Also, based on the scenario you described, it seems that you would like to change the behavior of a plugin operation depending on the user workflow that is being executed? For instance, you would like your virtual.configure() plugin operation to do different things depending on whether it's running as part of a provision or refresh workflow? Could you describe your use case in a little bit more detail?


    -Filip


    ------------------------------
    Filip Drozdowski
    Senior Member of Technical Staff I
    Delphix
    ------------------------------



  • 3.  RE: VDB status not returning up immediately

    Posted 01-24-2020 03:49:00 PM
    Thanks for the insight as to when the status is checked.   Curious then why the VDB status seems to be inconsistent?   Some times it shows running as soon as the engine operation completes?  Other times it takes a few minutes, as if it was only being changed when the next 5 minute check it as being up.  I'm started it from the same virtual.post_snapshot operation so I'm not sure why things would be inconsistent?

    The reason I want to know the workflow is to know if/when virtual.post_snapshot should start the instance or not.  If the VDB is being created it should start automatically.  If it's being refreshed or a snapshot was taken it should leave the instance in the same state as when it was started (i.e. only start if it had been running before the user made the request).  I have worked around this by setting global variables to track if it was stopped or not, but was curious as to whether there was a setting somewhere that I could check instead, which would simplify things. 

    Knowing the workflow would also be nice for unconfigure to know if it's actually being removed in which case I want to do total cleanup, or just part of a refresh/rollback, in which case I may not want to remove everything.

    ------------------------------
    Paul Jauquet
    Senior Data Services Consultant
    Axis Technology, LLC
    ------------------------------



  • 4.  RE: VDB status not returning up immediately
    Best Answer

    Posted 01-24-2020 06:57:00 PM
    Edited by Michael Torok 02-12-2020 01:11:00 PM
    I just walked through the code and it appears that the status operation should be called after refresh. One thing you could try is to log your plugin operations along with timestamps. This way, you would be able to see whether the virtual.status() operation was triggered after virtual.configure() (run as part of refresh).  Here's more information on how to retrieve plugin logs: https://developer.delphix.com/References/Logging/.


    I think I now have a better understanding of your use case. That said, provisioning and refreshing a VDB are almost identical operations with a refresh just being an unconfigure followed by a provision. These behaviors should be the same as it is effectively a new database being created. Do you have a requirement to maintain state through a refresh operation?


    With regard to virtual.unconfigure() operation and determining whether it's running in the context of disable or delete operation, we've had multiple discussions about it. In the case of disable or delete, the plugin should remove all traces of the VDB on the host. Let me describe one scenario in which leaving VDB traces on a host might result in unexpected errors. 


    It is possible to migrate a VDB from one host to another. For example, if you are running a VDB on host A and need to upgrade this host - which may take a while -you can migrate your VDB from host A to host B. In this case, the VDB would be disabled on host A and enabled on host B. If your enable operation relies on some files written to the previous host (host A), then enable operation will break.


    If you're interested, we can set up a sync up meeting next week and discuss possible solutions further.


    ------------------------------
    Filip Drozdowski
    Senior Member of Technical Staff I
    Delphix
    ------------------------------



  • 5.  RE: VDB status not returning up immediately

    Posted 02-12-2020 12:42:00 PM

    Hi Paul,


    You mentioned you've been using "global variables to track if [a VDB] was stopped or not". I just wanted to point out that using global variables is not a safe solution. The Python interpreter that we use to execute your Python code is stored in memory. It is not guaranteed that the same Python interpreter is used across plugin operations. The Python interpreter can get recreated due to many reasons like a Delphix Engine restart etc. Additionally, the same Python interpreter is used for executing all Delphix operations for given plugin which means that your plugin might end up writing to the same global variables from different threads. That might result in data races unless the access to global variables is synchronized.

    If you want to determine if a VDB is running, it's always safer to run a bash script on the host where the VDB is running. It might be a little slower but certainly a safer solution.

    -Filip



    ------------------------------
    Filip Drozdowski
    Senior Member of Technical Staff I
    Delphix
    ------------------------------