Virtualization Plugins

I need to import the glob, os, and re modules to the SDK plugin_runner.py script.
I need these for discovery in source_config_discovery.
I have tried the following:
   import os
   from dlpx.virtualization import os
   from dlpx.virtualization.platform import os

After trying each I refresh the environment and it is showing a generic error that it is failing.
I remove the import and call and it works fine.
Also, where on the engine can I see more detailed errors to help in my troubleshooting when an operation fails?

I went thru the SDK doco and created the first_plugin fine.  It would help if there were more complex examples to look at for reference, and to get ideas.

#VirtualizationPlugins

How do you import standard Python modules?

You've got the right idea here. It is expected that 'import os' should "just work". There are no special hoops you should have to jump through to import standard Python stuff like this. I just confirmed that I could include 'import os' in my 'plugin_runner.py' file successfully.

(As an aside... plugin code will run with fairly tight security constraints. Many of the 'os' functions will need higher levels of permissions than plugin code is allowed. Depending on what you're trying to do with the 'os' module, you might get security-related errors. But simply importing the module is expected to work just fine.)

Since you're asking about 'os' and 'glob' modules specifically, I also want to point out that Python plugin code runs on the Delphix Engine. So, for most plugins, any OS-related and filename-related code will actually be in Bash or Powershell code, which runs on remote hosts/environments.


How can you debug Python problems when you get an error during discovery?

Unfortunately, you are hitting a known bug here. In the special case, of discovery, we are accidentally not showing the Python error message. Had the error happened as part of another operation (such as dSource syncing), you would see the Python error in the UI. We hope to fix this bug soon.

In the meantime, to workaround this problem, you can see the Python error by looking in the engine's debug logs:

•  In the Engine UI, go to "Help" then "Support Logs", and follow the prompts to download a support bundle.
•  Untar the downloaded support bundle to a scratch directory.
•  Open up the 'logs/mgmt_log/debug.log' file in a text editor. Search for the most recent occurrence of "generic.plugin.problem.detected".
•  Immediately afterward, you should see a Python stack trace and the error as reported by the Python interpreter.

For example, you might see something like this:

com.delphix.server.core.messages.PythonExceptions$PythonGenericPluginProblemDetected: exception.python.python.generic.plugin.problem.detected {Repository Discovery|Traceback (most recent cal
l last):
  File "__pyclasspath__/dlpx/virtualization/platform/_plugin.py", line 226, in _internal_repository
  File "/Users/tomwalsh/code/toolkits/exploit/src/plugin_runner.py", line 69, in repository_discovery
  File "/Users/tomwalsh/code/toolkits/exploit/src/plugin_runner.py", line 69, in repository_discovery
NameError: global name 'bad_name' is not defined
}

This tells me that, at line 69 of my 'plugin_runner.py' file, I've tried to use a variable named 'bad_name', but no such thing exists. Your problem might be completely different, of course, but hopefully the error report will point you to the problem.

Thanks! Very helpful!  I figured out that I needed to do the import within the function, as opposed to the top of the script, but it still did not return anything which makes sense if it is actually running on the engine as opposed to the source host.  So if I need to look for files /etc/mongo*.conf on a linux source host I'd need to write it in bash?  How would I execute that from within  source_config_discovery?  And how would it return and set 1-M source_config values?

Looks like I'd use https://developer.delphix.com/References/Platform_Libraries/#run_bash.
Do you have any examples of how you'd return multiple rows/values?  Or would just just parse the STDOUT?

If I write a bash script that will do most of the parsing and return what I need in STDOUT that can then be parsed by the plugin, is there a way to add the script to the Delphix toolkit so that it is deployed to the environment/host?

Hi Paul,

I figured out that I needed to do the import within the function, as opposed to the top of the script, but it still did not return anything which makes sense if it is actually running on the engine as opposed to the source host. 

I'm curious, what was the issue doing the import at the script level? It shouldn't matter where the import is done.

Looks like I'd use https://developer.delphix.com/References/Platform_Libraries/#run_bash.
Do you have any examples of how you'd return multiple rows/values?  Or would just just parse the STDOUT?

Correct, however you cannot return multiple values and will need to parse stdout. If you want something more structured you could return JSON and the 'json' module in Python will make it easy to deserialize. 

If I write a bash script that will do most of the parsing and return what I need in STDOUT that can then be parsed by the plugin, is there a way to add the script to the Delphix toolkit so that it is deployed to the environment/host?

The bash script is pushed to the remote environment at execution time as part of the call to 'run_bash'. Just out of curiosity, how would it help to have the script pushed as part of the toolkit?

I hope this helps. Let me know if things weren't answered or if you have any other questions.