Delphix Products

 View Only
  • 1.  Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-24-2024 02:20:00 AM

    We have upgraded the Delphix Continuous Compliance from version 6.0.13 to version to 6.0.24 in our Lab. It is fresh setup on the new infrastructure and now we are migrating the objects from old Delphix engine to new one.

    While migrating the environments from old engine to new engine we are facing the below error - "All references to the legacy algorithm 'TelephoneNoLookup' must be updated to an algorithm of type COMPONENT, as this algorithm not available. This includes indirect references that occur when 'TelephoneNoLookup' is the default algorithm for one or more domains. It is no longer possible to import environments that reference this algorithm via engine sync."}.

    We tried to remove the domain from profile sets which were using the same algorithm but still environment migration is failing. Any suggestion on this will be helpful. We do not want to loose the current configuration of the objects in new engine.  

    Thanks.

     



    ------------------------------
    Saurabh Gorhe
    TDM Architect
    Cognizant Technology Solutions
    ------------------------------


  • 2.  RE: Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-24-2024 02:30:00 AM

    Hi Saurabh,

     

    If you have moved from version 6.x engine to version 24 engine, that would mean that there are couple of years worth of upgrades and changes in between engine versions.

     

    For the last two years there were a lot of changes (upgrades, updates, removal of algorithms, addition of algorithms) to the Compliance (Masking) Engine.

     

    The best thing might be to start with checking the changelog of the engine versions.

    https://masking.delphix.com/docs/latest/new-features

     

    When it comes to what to do with a specific algorithm that no longer exists ('TelephoneNoLookup' in this case), there are multiple options.

     

    1. It might be updated in the environment export document manually.
    2. Take the list as a dxmc set algorithm command, then remove it from the environment, and set it anew in the new Engine

     

    One of the first things is to check the changes and then decide which of the new algorithms to be used instead of the removed ones.

     

     

    BR,

     

    Rahim CETINEL 

    Lead DataOps Architect

    Delphix Blackbelt 

    +971509176453 

    +905334112490 


    Accuras DMCC
     
    DMCC Business Centre Unit No: 1518 
    PO Box 41174, Dubai,  UAE
    https://accuras.com  

     






  • 3.  RE: Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-24-2024 09:16:00 AM

    The reason you get those warnings is that in version 6.0.15 we replaced any LEGACY algorithms with Java SDK based algorithms.
    During the upgrade to 6.0.15 old legacy algorithms got converted to SDK based algorithms. So you have two options:

    1) upgrade "old" engine to 6.0.15 and export algorithms again > you should get only SDK algorithms (type=COMPONENT).
    You should be able to import into v24 successfully.
    (the version is called 24 > not  6.0.24, after version 6.0.17 Delphix switched to single number versioning > 7 )


    2) when you export algorithms from old engine exclude the ones where type=LEGACY.  Then you should be able to import into v24.



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------



  • 4.  RE: Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-25-2024 10:31:00 AM
      |   view attached

    Thanks Rahim and Tino for your responses. A one suggestion that we tried is removing the reference to the algorithm manually in environment export file. Example- 

    {

          "objectIdentifier" : {

            "id" : "TELEPHONE_NO"

          },

          "objectType" : "DOMAIN_REFERENCE",

          "revisionHash" : "de84bc0c24435cb289660bac666aeef4b9c33d41"

        },

    {

          "objectIdentifier" : {

            "algorithmName" : "TelephoneNoLookup"

          },

          "objectType" : "ALGORITHM_REFERENCE",

          "revisionHash" : "98ed0b91036b2f7bcb67dc536b6c15497df6237b"

        },

    We removed the references for algorithm as highlighted and tried the environment migration. Migration got through but we do not see any other objects created under the migrated environment in new engine. I have attached a sample environment export for a reference.



    ------------------------------
    Saurabh Gorhe
    TDM Architect
    Cognizant Technology Solutions
    ------------------------------

    Attachment(s)

    docx
    Unconfirmed 424749.docx   32 KB 1 version


  • 5.  RE: Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-25-2024 11:01:00 AM

    You cannot modify the environment export.
     (don't exactly remember if it was XML or JSON based export in 6.0.13 > the JSON you cannot modify!)

    You can use API : /Sync : syncable-objects > choose your object type > USER_ALGORITHM > execute.
    From response you can select only the algorithms you WANT to sync and use them as input for:
    API : /Sync : async-export
    That way you export only the COMPONENT type algorithms > and can import them into the new engine.



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------



  • 6.  RE: Facing error while migrating environment from Delphix Data Compliance Engine to another

    Posted 07-25-2024 11:08:00 AM

    One thing that confuses me is that you wrote :
    << We have upgraded the Delphix Continuous Compliance from version 6.0.13 to version to 6.0.24 in our Lab. >>


    If you upgraded the existing engine you have on the new version everything SDK based .. so you can export from there to another engine based on v24.
    Or did you just install a second engine based on v24 image ?


    You are aware that you can upgrade 6.0.13 to v24 ? It is not a one step upgrade > its a multi step upgrade - Support can tell you which upgrade steps you need to do (for example: first 6.0.15 then v8 then v14.... only support can advise you the required upgrade path because your version is quite old).
    During this upgrades ALL old legacy algorithms get migrated automatically....



    ------------------------------
    Tino Pironti
    Masking SME
    Technical Manager
    Delphix
    ------------------------------