SSIS Derived Column Calcuation Issue - sql

I am having an issue with a derived column calculation in SSIS not returning the correct values. To give some background, I had to change the format of my calculation due to an overflow issue (resolved), see here - SSIS - Derived Column Calculation Error
Now, the old and new calculations return different values in SSIS, even though they are both mathematically the same and return the same value when executed in .net, excel and SQL.
In .Net -
decimal test1 = Convert.ToDecimal((415 * 415 * 415) * 0.000000030940000);
decimal test2 = Convert.ToDecimal((0.000000030940000 * 415 * 415 * 415));
both return 2.2113862225
In SSIS -
test1 = ((415 * 415 * 415) * 0.000000030940000)
test2 = (0.000000030940000 * 415 * 415 * 415)
return different values (test1 = 2.2113862225 and test2 = 2.211535)
Interestingly, SSIS sees test1 as a numeric (38,6) and test2 as a numeric (27,15)....
Has anyone encountered this issue before?
I don't really want to have to re-factor the whole package to use .net calculations, however that will be my last resort solution.

In case anyone else has this problem - I didn't find the solution but did implement a workaround -
Instead of doing the calculation in a derived column -
Use the derived column component to reate the variable and set it as 0
Add a data conversion and switch it to the required data type (float in this case)
Add an OLE DB Command to call a stored procedure to do the calculation, with an output parameter filling the variable created at point 1.

Related

PgSQL insert into select after cast

I have a table with many columns stored as text. These columns have information which i am planning to cast and copy the data into a fresh table with the same column names but with correct data types ( float8 )
I tested the SELECT with CAST operator "::" and it works fine. All the columns are being converted as you can see in the picture
However , when I uncomment the INSERT statement ( above it ) to start writing in the target table, it throws an error. The target table has identical column names and only float8 column types.
The expression from the source table is indeed of type text but i am using the cast operator so why does it not work like before when only running the SELECT statement?
My query below:
INSERT INTO "PM"."new_VM_gcell_evolution_hourly_BSC"
(select
"CR3120:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"DL_Mean_Quality"::float8,
"A312Ca:Failed_Assignments_during_MTC_on_the_A_Interface_Includi"::float8,
"nsp_Urban_TA_4number"::float8,
"A3129C:Failed_Assignments_First_Assignment,_Assignment_Timed_Ou"::float8,
"R3120C:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"TSs_Interf_B5"::float8,
"R3120D:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"HO_Out_Internal_Succ_Rate"::float8,
"CH_Req_Protocol_Undefinednbnumber"::float8,
"UL_Drop_Congnumber"::float8,
"Timing_Adv"::float8,
"DL_Qual"::float8,
"A3100C:Assignment_Requests_TCHH_Only"::float8,
"MS_to_BTS_max_distance"::float8,
"TSs_Interf_B2"::float8,
"UL_Qual"::float8,
"UL_Drop_EDGE_Rate"::float8,
"Avg_BTS_Power_Level_AMR"::float8,
"DL_Drop_N3105number"::float8,
"nsp_Urban_TA_2_R"::float8,
"CH_Req_CallReestabnbnumber"::float8,
"TCH_Drops_cause_Timing_Advance"::float8,
"TCH_Drops_per_Erlang"::float8,
"UL_Drop_Flushnumber"::float8,
"A312F:Number_of_Assignment_Failures_No_Abis_Resource_Available"::float8,
"UL_Drop_GPRS_Rate"::float8,
"nsp_Urban_TA_3number"::float8,
"Call_Drop_rate"::float8,
"DL_Fail_Assingnumber"::float8,
"SDCCH_Radio_Failures"::float8,
"HO_Cmd_UL_Qual"::float8,
"UL/DL_RxQualnumber"::float8,
"UL/DL_RxLevnumber"::float8,
"UL_Fail_EDGE_Rate"::float8,
"Call_Drop_Total"::float8,
"HO_Out_Succ_Rate"::float8,
"Date"::text,
"A3127E:Failed_Assignments_during_Call_Reestablishment_on_the_Um"::float8,
"UL_Drop_N3103number"::float8,
"Configured_TCH"::float8,
"SDCCH_Drop_Call_Rate_only_LU"::float8,
"A312M:Failed_Assignments_Reconnection_to_Old_Channels,_No_Chann"::float8,
"SDCCH_Blocking_Rate"::float8,
"TCH_Drops_cause_DL_FER"::float8,
"TCH_Fail_rate"::float8,
"DL_Drop_Preemnumber"::float8,
"FR_Traffic_totalErl"::float8,
"DL_Congnumber"::float8,
"TCH_Assign_Unsuccnumber"::float8,
"nsp_Urban_TA_0_R"::float8,
"DL_Drop_GPRS_Rate"::float8,
"HR_TraficErl"::float8,
"SDCCH_Cong_Rate"::float8,
"Abis_and_Ater_Interface_Anlysistimes"::float8,
"A3100B:Assignment_Requests_TCHF_Only"::float8,
"AS4300D:Mean_Uplink_Level_during_Radio_Link_Failure_SDCCHdB"::float8,
"nsp_Urban_TA_89number"::float8,
"TCH_Drops_cause_DownQual"::float8,
"SDCCH_Drop_RxLev"::float8,
"Max_UL_Pwr_Duration"::float8,
"nsp_Channel_Req_LUnumber"::float8,
"TCH_HR_Initialy_Config"::float8,
"HO_Inc_Cong_Rate"::float8,
"DL_RxLev_avgdBm"::float8,
"TCH_Assign_Requestsnumber"::float8,
"A312K:Failed_Assignments_First_Assignment,_No_Channel_Available"::float8,
"Call_Drop_TCH_Quality_Rate"::float8,
"AMR_HR_TraficErl"::float8,
"nsp_Urban_TA_67_R"::float8,
"UL_Fail_MSnumber"::float8,
"Time"::text,
"TCH_Drops_cause_UpLevel"::float8,
"AMR_FR_TraficErl"::float8,
"TCH_FR_Initialy_Config"::float8,
"UL_Drop_N3101number"::float8,
"DL_FERnumber"::float8,
"SDCCH_Drop_Call_Rate_wo_LU"::float8,
"DL_Mean_StrengthdB"::float8,
"Call_Drop_Abis"::float8,
"SDCCH_Initialy_Confignumber"::float8,
"Call_Drop_TCH_RxLev_Rate"::float8,
"TA_Meannumber"::float8,
"nsp_Urban_TA_67number"::float8,
"AS3240NA:Average_MS_Power_Level_of_NonAMR_Call"::float8,
"Load"::float8,
"A3129Q:Failed_Assignments_Reconnection_to_Old_Channels,_Timer_E"::float8,
"CH_Request_11_bit"::float8,
"AS4340D:Mean_TA_during_Radio_Link_Failure_SDCCH"::float8,
"S4210A:Uplink_Interference_Indication_Messages_SDCCH"::float8,
"UL_FERnumber"::float8,
"A3129P:Failed_Assignments_Reconnection_to_Old_Channels,_Timer_E"::float8,
"K3003A:Successful_SDCCH_Seizures_Call_Type"::float8,
"CH_Req_LAUnbnumber"::float8,
"SDCCH_Drop_Rate"::float8,
"GBSC"::text,
"CS_CSSR_with_SDCCH_blocks_wo_LU"::float8,
"A312Aa:Failed_Assignments_during_MOC_on_the_A_Interface_Includi"::float8,
"Avg_BTS_Power_Level"::float8,
"R3120A:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"A312S:Failed_Assignments_Signaling_Channel"::float8,
"SD_Fail_MoC"::float8,
"UL_RxQual_avg"::float8,
"Call_Drop_HO"::float8,
"AS4330D:Mean_Downlink_Quality_during_Radio_Link_Failure_SDCCH"::float8,
"CH_Req_Emerg_Callsnbnumber"::float8,
"UL_RxQualnumber"::float8,
"UL_Fail_OtherCausenumber"::float8,
"A312Da:Failed_Assignments_during_Emergency_Call_on_the_A_Interf"::float8,
"UL_Drop_Preemnumber"::float8,
"TCH_Drops_cause_UpDown_FER"::float8,
"AS4320D:Mean_Downlink_Level_during_Radio_Link_Failure_SDCCHdB"::float8,
"A3129O:Failed_Assignments_First_Assignment,_Directed_Retry_Time"::float8,
"Max_DL_Pwr_Duration"::float8,
"UL_Drop_Suspendnumber"::float8,
"TCH_Assign_Congestionnumber"::float8,
"Transmission_Resource_Analysistimes"::float8,
"HO_Out_RxQual_rate"::float8,
"nsp_Urban_TA_1013number"::float8,
"A3129I:Failed_Assignments_Invalid_State"::float8,
"DL_Fail_OtherCausenumber"::float8,
"CR3129:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"HO_Inc_Successnumber"::float8,
"A3129B:Failed_Assignments_First_Assignment,_Terrestrial_Resourc"::float8,
"Trafic_totalErl"::float8,
"nsp_urban_TA_1463all_R"::float8,
"HO_Out_RxQualnumber"::float8,
"DL_RxQualnumber"::float8,
"HO_Inc_Succ_Rate"::float8,
"SDCCH_Fail_Rate"::float8,
"SDCCH_Non_Radio_Drops"::float8,
"nsp_Urban_TA_0number"::float8,
"A3129E:Failed_Assignments_CIC_Unavailable"::float8,
"nsp_CH_Request_PSnumber"::float8,
"TCH_Assign_Unsucc_rate"::float8,
"Call_Drop_Radio"::float8,
"EFR_TraficErl"::float8,
"A312L:Failed_Assignments_Reconnection_to_Old_Channels,_No_Chann"::float8,
"TCH_Drops_cause_DownLevel"::float8,
"nsp_Urban_TA_5number"::float8,
"Abnormal_Terminals_Analysistimes"::float8,
"HO_Inc_Congnumber"::float8,
"CSSR_Rate"::float8,
"M3020C:Call_Drops_on_SDCCHQuality"::float8,
"RH333:Handover_Drop_Rate_of_TCH"::float8,
"UL_Drop_EDGE_Abisnumber"::float8,
"A3129J:Failed_Assignments_Invalid_Message"::float8,
"TCH_Availability"::float8,
"UL_Level"::float8,
"nsp_Urban_TA_4_R"::float8,
"UL_Drop_OtherCausenumber"::float8,
"HR_Traffic_Rate"::float8,
"DL_Level"::float8,
"CH_Request_8_bit"::float8,
"UL_Fail_MS_Assingnumber"::float8,
"DL_Fail_BSC_Commandnumber"::float8,
"A3129R:Failed_Assignments_Reconnection_to_Old_Channels,_Reconne"::float8,
"UL_Fail_BSC_Commandnumber"::float8,
"DL_Drop_Suspendnumber"::float8,
"nsp_urban_TA_1463allnumber"::float8,
"Direct_Retry"::float8,
"nsp_Urban_TA_89_R"::float8,
"CH_Req_PSnbnumber"::float8,
"A3129N:Failed_Assignments_Reconnection_to_Old_Channels,_Terrest"::float8,
"A312A:Failed_Assignments_First_Assignment,_No_Channel_Available"::float8,
"nsp_Urban_TA_2number"::float8,
"DL_RxQual_avg"::float8,
"UL_Mean_StrengthdB"::float8,
"DL_Drop_EDGE_Rate"::float8,
"A3129T:Failed_Assignments_No_Ater_Resource_Available"::float8,
"TSs_Interf_B1"::float8,
"A3129D:Failed_Assignments_Reconnection_to_Old_Channels,_Reconne"::float8,
"nsp_Channel_Req_MOCnumber"::float8,
"UL_Fail_GPRS_Rate"::float8,
"S4210B:Downlink_Interference_Indication_Messages_SDCCH"::float8,
"R3120E:Channel_Assignment_Failures_All_Channels_Busy_or_Channel"::float8,
"HO_Inc_Failnumber"::float8,
"DL_Drop_Flushnumber"::float8,
"S3655:Number_of_configured_TRXs_in_a_cell"::float8,
"TCH_Drops_cause_UpDown_Qual"::float8,
"CR3005:Number_of_Initially_Configured_Channels_Static_PDTCH_Sup"::float8,
"UL/DL_FERnumber"::float8,
"SDCCH_Blocking_"::float8,
"HO_Out_InterRAT_Succ_Rate"::text,
"CM30E:Call_Drops_on_SDCCH_Location_Updating"::float8,
"AS4310D:Mean_Uplink_Quality_during_Radio_Link_Failure_SDCCH"::float8,
"UL_Congnumber"::float8,
"TAnumber"::float8,
"Configured_SDCCH"::float8,
"Othernumber"::float8,
"TCH_Assign_Fail_Radionumber"::float8,
"HO_Out_Internal_Req_Nb"::float8,
"SDCCH_Cong_Nbnumber"::float8,
"UL_Mean_Quality"::float8,
"HO_Out_External_Req_Nb"::float8,
"nsp_Urban_TA_1013_R"::float8,
"SDCCH_Drops_wo_LU"::float8,
"Call_Drop_TCH_Qualitynumber"::float8,
"A3100A:Assignment_Requests_Signaling_Channel_TCH"::float8,
"Better_Cell"::float8,
"Call_Drop_TCH_RxLevnumber"::float8,
"nsp_Channel_Req_MTCnumber"::float8,
"DL_Fail_EDGE_Rate"::float8,
"ZTR104B:Call_Drop_Rate_on_SDCCH_Call_Type"::float8,
"Call_Drop_no_MR"::float8,
"DL_Drop_Congnumber"::float8,
"HO_Inc_InterRAT_reqnumber"::float8,
"TCH_Assign_Cong_rate"::float8,
"HO_Inc_Fail_Rate"::float8,
"TCH_Assign_Fail_sp_ver_unav"::float8,
"HO_Inc_InterRAT_unsucc"::float8,
"A3170A:Number_of_Completed_TCH_Assignments_CSFB_MOC"::float8,
"HO_Inc_Unsucc_Rate"::float8,
"TSs_Interf_B4"::float8,
"A3100K:Assignment_Requests_Signaling_Channel_SDCCH"::float8,
"TCH_Drops_cause_UpDown_Level"::float8,
"SDCCH_Fail_Nbnumber"::float8,
"TCH_Non_Radio_Drops"::float8,
"DL_Fail_MSnumber"::float8,
"HO_Cmd_DL_Qual"::float8,
"DL_Drop_OtherCausenumber"::float8,
"SMS_on_SDCCH"::float8,
"FERnumber"::float8,
"HO_Inc_Reqnumber"::float8,
"nsp_Urban_TA_3_R"::float8,
"M3020D:Call_Drops_on_SDCCHOther"::float8,
"HO_Out_External_Succ_Rate"::float8,
"M3020A:Call_Drops_on_SDCCHTA"::float8,
"TCH_Assign_Fail_Radio_rate"::float8,
"CR3001:Number_of_Initially_Configured_Channels_Static_PDCH"::float8,
"HR_Traffic_totalErl"::float8,
"CH_Req_MTCnbnumber"::float8,
"SDCCH_Drop_RxQual"::float8,
"UL_RxLev_avgdBm"::float8,
"CH_Req_MOCnbnumber"::float8,
"SDCCH_Dropnumber"::float8,
"A3169A:Failed_Assignments_Um_Cause"::float8,
"A3129H:Failed_Assignments_Clear_Commands_Sent_By_MSC"::float8,
"TCH_Radio_Drops"::float8,
"HO_Out_InterRAT_Req_Nb"::float8,
"HO_Inc_InterRAT_succnumber"::float8,
"DL_Drop_Abisnumber"::float8,
"TCH_Drops_cause_UL_FER"::float8,
"UL_RxLevelnumber"::float8,
"nsp_CH_Request_CSnumber"::float8,
"A3170B:Number_of_Completed_TCH_Assignments_CSFB_MTC"::float8,
"A3129F:Failed_Assignments_CIC_Allocated"::float8,
"DL_Fail_GPRS_Rate"::float8,
"M3020B:Call_Drops_on_SDCCHReceived_Level"::float8,
"nsp_Urban_TA_1number"::float8,
"SDCCH_Availability"::float8,
"Call_Drop_forced_HO"::float8,
"SDCCH_Drop_Others"::float8,
"Call_Drop_Equip."::float8,
"CH_Req_PS"::float8,
"SDCCH_Drop_Call_Rate"::float8,
"TCH_Drops_cause_UpQual"::float8,
"CH_Req_LMU_Reservednbnumber"::float8,
"DL_RxLevnumber"::float8,
"nsp_Urban_TA_1_R"::float8,
"A312Ea:Failed_Assignments_during_Call_Reestablishment_on_the_A_"::float8,
"TSs_Interf_B3"::float8,
"SDCCH_Drop_TimingAdvance"::float8,
"Avg_BTS_Power_Level_NAMR"::float8,
"TCH_Drops_cause_Other"::float8,
"FR_TraficErl"::float8,
"Avg_MS_Power_Level"::float8,
"HO_Outgoing_Requestsnumber"::float8,
"A3129G:Failed_Assignments_A_Interface_Failure"::float8,
"CH_Req_LAU"::float8,
"nsp_Urban_TA_5_R"::float8,
"HO_Inc_Unsuccessnumber"::float8,
"AS3240A:Average_MS_Power_Level_of_AMR_Call"::float8
from "PM"."VM_gcell_evolution_hourly_BSC_recovered")
It says "column is of type double but expression is of type text" so what's really happening is that it's trying to insert one of the expressions where you cast to ::text into a column that is of type double.
If the problem was converting from text to double, you'd get a different message. Besides, the SELECT worked fine, which means it didn't encounter any text data that it couldn't convert to double.
Since you don't specify the target table columns in your INSERT, and you have so many of them, most likely you missed a column or got the order wrong.
Honestly if the table has 270 columns and they have the same name in the source and destination tables, you should really generate the query using something like python from a list of columns, that will be faster than proofreading the 270 lines...

Amount Field Length Extension: Code Adaptations (2 decimals / 3 decimals)

While preparing for the conversion to S/4HANA, our custom code check produces following error message:
Old Arithmetic type conflict (Type DMBTR, Note: 0002610650) P(13,3)
I have recreated the problem in a simple demo program.
DATA: punit TYPE dmbtr, "curr(23,2)
two TYPE dmbtr VALUE '12.55', "curr(23,2)
three TYPE menge_d VALUE '5.123'. "quan(13,3)
CATCH SYSTEM-EXCEPTIONS arithmetic_errors = 4.
punit = two / three.
ENDCATCH.
WRITE (26) punit.
The error is in the line punit = two / three.
I've already checked the SAP Note 2610650 but can not find any useful information in it. Hope you can help me.
With the new data type dmbtr_cs it worked fine. (see SAP Note 2628040)
DATA: punit TYPE dmbtr_cs,
two TYPE dmbtr_cs VALUE '12.55',
three TYPE menge_d VALUE '5.123'.
CATCH SYSTEM-EXCEPTIONS arithmetic_errors = 4.
punit = two / three.
ENDCATCH.
WRITE (26) punit.

Passing parameters in HIVE query

I need to pass a parameter into an HQL at execution time. For this we have updated the .param file with the required value. Now I am facing issue in using it.
SELECT * FROM temp_table A
WHERE PRODT_CTGRY_CD = 'CAR' AND
(
(DATE(A.CUST_BRTH_DT) BETWEEN concat(${hiveconf:PRODYEAR} - 13 ,"-02","-28") and concat(${hiveconf:PRODYEAR} - 7 ,"-03","-01")
AND DATE(A.TERM_DT) >= concat(${hiveconf:PRODYEAR} - 1 ,"-03","-31")
)
In the above query if I don't include the condition of term_dt the query is running fine. However when I include that I am getting an error as below:
FAILED: ClassCastException org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableIntObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspector.primitive.BooleanObjectInspector (state=42000,code=40000)
Can someone let me know what needs to be done here?

Execute SQL Task in SSIS string parameter

I created two string variables (tot and tot_value) and assigned a value (tot = MyVal) for testing. Then I created an Execute SQL Task which takes (tot) as parameter and the value returned is saved in tot_value.
In the General Tab, i set:
ResultSet to Single row. SQL Source Type to Direct input Query is listed below.
In Parameter Mapping I selected my tot variable with Input DirectionSQL_VARCHAR Data Type 1 as Parameter Name (Since i am using ODBC)Size set to default -1.
In Result SetResult Name to 1Variable Name to tot_value.
If in the query I hard code 'MyVal' i get the correct result, however when I use ? to use my variable as a parameter, I always get a 0 returned.
Note that my tot variable is set to MyVal
Any clue of what I might be missing? Thanks in advance
select TOP 1 CAST('' + ISNULL((SELECT distinct type_of_transfer_code
FROM SYSTEM.history_program_transfer
WHERE type_of_transfer_value = ?),'') AS VARCHAR(100)) as type_of_transfer_code
FROM SYSTEM.table_facility_defaults

Querying Netezza via SquirrelSQL returns WKT geometry in unknown encoding

I am using SquirrelSQL to write and execute SQL queries on a Netezza database. Using Netezza's spatial capabilities (which are essentially the same as those of PostGIS) I've executed a query and returned a single result that contains a geometry. Here's the query, for reference:
SELECT t.SHAPE
FROM (SELECT * FROM OS_AB_PLUS..E12_ADDRESSBASE WHERE POSTCODE = 'RH1 6NE'
AND PAO_START_NUMBER = '14') as a, OS_TOPO..TOPOGRAPHICAREA as t
WHERE inza..ST_Within(a.shape, t.shape) = TRUE
My issue is that the geometry field, which should contain the polygon coordinates represented as Well-Known Text (WKT), looks instead like this:
g¹ AË Affff¬0AÍÌÌÌî0AÒ 3333Ê AÍÌÌÌî0A» Aë0Afffæ» AffffÒ0A¹ AÒ0A333³¹ A3333¿0AŒ AffffÀ0AÍÌÌLŒ Affff¬0AË A¯0AëQ8Ê A3333í0A3333Ê AÍÌÌÌî0A
I can't seem to find anywhere in SquirrelSQL to specify the encoding of VARCHAR columns, and I've seen the column returned without encoding issues in Aginity (another SQL client). Any suggestions on how to proceed would be much appreciated.
Turns out my issue was not really related to encoding at all. The human-readable version of the geometry in a PostGIS-like database will only be returned when ST_AsText is used in the select statement. So my SQL query becomes:
SELECT inza..ST_AsText(t.SHAPE)
FROM (SELECT * FROM OS_AB_PLUS..E12_ADDRESSBASE WHERE POSTCODE = 'RH1 6NE'
AND PAO_START_NUMBER = '14') as a, OS_TOPO..TOPOGRAPHICAREA as t
WHERE inza..ST_Within(a.shape, t.shape) = TRUE
Which returns. as intended:
POLYGON ((526696.15 148931.9, 526703.94 148932.34, 526703.8 148935.2, 526705.5 148935.3, 526705.4 148937.8, 526695.9 148937.35, 526696.15 148931.9))