Delphix Products

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hey Mouhssine,

I got the solution. It was a type cast error. tried using cast function in custom sql filter and that worked for me.
:)



------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

I am already using cast function in custom SQL while fetching data from this table.
Ex: SELECT cast ("build_id" as varchar) from <tablename>

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi Indrajeet,

If I understand you correctly I would suggest that this is not really masking the data at all.  If you have a use case where the end user needs to see the unmasked data then just provide them with the original untouched data (and implement the access controls accordingly).  For all other use cases you should mask the data irreversibly in a separate dataset/database/vdb.

Either way I don't think you are achieving anything but simply adding a negative sign to the original data.

------------------------------
Matthew Griffith
Principal Consultant
https://thedatalobby.kuzodata.com
Kuzo Data
------------------------------

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi Matthew,

My core concern here is that the data cleansing algorithm is not working with integer columns without using type casting to varchar.
One special case here is that if a particular column is a primary key then even typecasting will also not work.


------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 03:54:03 AM
From: Matthew Griffith
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi Indrajeet,

If I understand you correctly I would suggest that this is not really masking the data at all.  If you have a use case where the end user needs to see the unmasked data then just provide them with the original untouched data (and implement the access controls accordingly).  For all other use cases you should mask the data irreversibly in a separate dataset/database/vdb.

Either way I don't think you are achieving anything but simply adding a negative sign to the original data.

------------------------------
Matthew Griffith
Principal Consultant
https://thedatalobby.kuzodata.com
Kuzo Data

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Thanks Mouhssine for your suggestion, but I am doing in place masking and not using any VDB for the moment.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 04:45:18 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi,

Do you have only those columns to treat as part of masking or do you have others ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 05:20:16 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Thanks Mouhssine for your suggestion, but I am doing in place masking and not using any VDB for the moment.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 04:45:18 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

I have other columns also

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 05:35:33 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Do you have only those columns to treat as part of masking or do you have others ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:20:16 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Thanks Mouhssine for your suggestion, but I am doing in place masking and not using any VDB for the moment.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 04:45:18 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

Hi,

I purpose the following :

- use update sql order with result of product "value*-1" for the columns (release_id and build_info) as part of pre / post script of the masking job
https://maskingdocs.delphix.com/Securing_Sensitive_Data/Creating_Masking_Job/
- treat the other columns as part of masking job

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 05:40:59 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I have other columns also

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:35:33 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Do you have only those columns to treat as part of masking or do you have others ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:20:16 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Thanks Mouhssine for your suggestion, but I am doing in place masking and not using any VDB for the moment.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 04:45:18 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------

That is a good idea Mouhssine.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members
------------------------------

Original Message:
Sent: 12-18-2019 06:54:58 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

I purpose the following :

- use update sql order with result of product "value*-1" for the columns (release_id and build_info) as part of pre / post script of the masking job
https://maskingdocs.delphix.com/Securing_Sensitive_Data/Creating_Masking_Job/
- treat the other columns as part of masking job

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:40:59 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I have other columns also

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:35:33 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Do you have only those columns to treat as part of masking or do you have others ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-18-2019 05:20:16 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Thanks Mouhssine for your suggestion, but I am doing in place masking and not using any VDB for the moment.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-18-2019 04:45:18 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Now I got the need clearly.

I'm aligned with Matth's answer, but if there still a need to add the negative sign use instead an update for those columns with the product of the "value*-1".

You can add the update script as configure hooks of you vdb.

Regards,

Mouhssine



------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 10:54:59 PM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

My requirement is to replace the positive integer number by its negative equivalent. i.e. 2304 by -2304
This is not possible with segmented mapping algorithm. Algorithm SL and SM provides random data. In this particular case I just want the automation job to de-identify the key column by replacing it with a negative integer equivalent but at the same time user should be able to identify it by removing negative sign infront of it.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-17-2019 09:05:40 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

Only one algorithm can guarantee the uniqueness that delphix recommends to mask PK/FK. (segment mapping)

https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Segment_Mapping_(KBA1775)
https://support.delphix.com/Delphix_Masking_Engine/Masking_Columns_with_PK_and_FK_Constraints_(KBA1776)
https://support.delphix.com/Delphix_Masking_Engine/Algorithm%3A_Casting_Values_Before_Masking_(KBA1580)

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-17-2019 06:13:48 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Now the same algorithm is throwing error with a similar integer column in postgres DB (only difference is that of primary key).
This column Build_Info is a primary key  and Index column.

I just wanted to replace 23649 by -23649

Struggling to resolve this from morning. Below is the error logs:

2019/12/17 08:10:43 - Update.0 - Error updating batch
2019/12/17 08:10:43 - Update.0 - Batch entry 0 UPDATE "azuredevops_reporting"."build_info"
2019/12/17 08:10:43 - Update.0 - SET "build_id" = -23649.0
2019/12/17 08:10:43 - Update.0 - WHERE ( ( "build_id" = '23649' ) ) was aborted: ERROR: operator does not exist: integer = character varying
2019/12/17 08:10:43 - Update.0 - Hint: No operator matches the given name and argument type(s). You might need to add explicit type casts.

------------------------------
indrajeet yadav
Technical Test Lead
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:51:51 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

integer to Varchar

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:35:11 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Great news, so finally it was related to data type :)

For information you casted from what to what ?

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 08:15:44 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi,

Either, try to use segment mapping algorithm or engage with support as they have the tools to look on what's going wrong on masking engine.

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:48:18 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

In postgres database Integer datatype ( INT) is a 4-byte integer that has a range from -2,147,483,648 to 2,147,483,647.
So the range should not be an issue as far as i understood.

------------------------------
indrajeet yadav
Delphix Community Members

Original Message:
Sent: 12-16-2019 07:14:06 AM
From: Mouhssine SAIDI
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

Hi indrajeet,

My interpretation to this error message is.

Lookup value returned is of type long that doens't fit into postgres int type, you can try to change the column type to bigint if you can or change the masking algorithm to use one that gives you more control like "segment mapping".

Regards,

Mouhssine

------------------------------
Mouhssine SAIDI
Community Member
Delphix Community Members

Original Message:
Sent: 12-16-2019 04:25:05 AM
From: indrajeet yadav
Subject: Delphix data cleansing algorithm throwing error with Postgres DB

I was replacing the positive integer with negative integer by using data cleansing algorithm.
This was working correctly in Oracle DB but same algorithm throwing error in postgres DB.
below is the error screenshot.

2019/12/16 06:58:10 - Get All Lookups Values.0 - A String type was specified for field [Col0_lkUpKey], but a [class java.lang.Long] type was returned as a result of formula [release_id]

Can some one suggest me the possible resolution here:
In Oracle I was using data type as Varchar for the particular field while in Postgres DB I was using data type as int.
That is the only difference I understood.

My data cleansing file looks like:

1=-1
2=-2
3=-3
4=-4
.
.
some 1 lakh rows .


------------------------------
indrajeet yadav
Delphix Community Members
------------------------------