I have a PowerQuery connected to DB2 Data Source, but because of some kind of load balancing the DB Server changes periodically (ip also changes), There is no way for me to know beforehand which data source (IP), I should use until I try it and see that it errors out, then I have to use the other one, I googled for error handling in PowerQuery and found some examples of error handling but this examples did not apply to my case, most of them handled errors AFTER the connection was made or were preventing errors on missing columns or files not found, I tried to adjust the examples to my case but was not able to.
What I want is just try one IP and if it fails then use the other one.
let
//fParametros("ParamQuery",1) is the "standard" Server/Ip address (provider=IBMDADB2.IBMDBCL1;data source=CP3;location=pn8us7ldbcp3.us.mycompany.com:5912)
dbSource = fParametros("ParamQuery",1),
//fParametros("ParamQuery",5) is the "Alternate" Server/Ip address (provider=IBMDADB2.IBMDBCL1;data source=CP3;location=pn8us7ldbcp3h.us.mycompany.com:5912)
AltdbSource = fParametros("ParamQuery",5),
pOrden = Text.From(fParametros("ParamQuery",2)),
//Create the query
dbQuery = "SELECT SAPCP3.vbak.VBELN SO , SAPCP3.vbap.posnr PoLine , SAPCP3.vbep.ETENR Sch_Line , SAPCP3.vbap.matnr Part_Number,SAPCP3.makt.maktx Description,SAPCP3.vbap.kwmeng Qty ,SAPCP3.vbep.BMENG Conf_qty ,SAPCP3.vbap.vrkme UOM ,SAPCP3.vbap.netpr SalesPrice ,SAPCP3.vbap.kpein LotSize FROM SAPCP3.vbak JOIN SAPCP3.vbap ON SAPCP3.VBAp.VBELN = SAPCP3.VBAK.VBELN JOIN SAPCP3.vbep ON SAPCP3.vbep.vbeln = SAPCP3.vbak.vbeln AND SAPCP3.vbap.posnr = SAPCP3.vbep.posnr JOIN sapcp3.makt ON sapcp3.vbap.matnr=sapcp3.makt.matnr WHERE SAPCP3.VBAK.VKORG = '4000' AND (SAPCP3.vbep.edatu >= '20190701') AND SAPCP3.vbak.VBELN ="& pOrden & " ORDER BY SAPCP3.vbak.VBELN",
//Get the data
Source = OleDb.DataSource(dbSource, [Query=dbQuery]),
//Failed Attempt to handle the error:
TestForError= try Source,
//next line does not work, I get error saying Source is already defined/declared
Source = if TestForError[HasError] then OleDb.DataSource(AltdbSource, [Query=dbQuery]) else OleDb.DataSource(dbSource, [Query=dbQuery])
in
Source
I also Tried the following:
.
.
.
//Get the data
Source = OleDb.DataSource(dbSource, [Query=dbQuery]),
//Failed Attempt to handle the error:
TestForError= try Source,
Output = if TestForError[HasError] then OleDb.DataSource(AltdbSource, [Query=dbQuery]) else OleDb.DataSource(dbSource, [Query=dbQuery])
in
Output
//This last part works if the dbSource is correct, but if it is not it doesnt catch the error and gives me the Connection error shown below:
DataSource.Error: OLE DB: SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "172.16.0.1". Communication function detecting the error: "connect". Protocol specific error code(s): "10061", "", "". SQLSTATE=08001
Details:
DataSourceKind=OleDb
DataSourcePath=data source=CP3;location=pn8us7ldbcp3.us.mycompany.com:5912;provider=IBMDADB2.IBMDBCL1
Message= SQL30081N A communication error has been detected. Communication protocol being used: "TCP/IP". Communication API being used: "SOCKETS". Location where the error was detected: "172.16.0.1". Communication function detecting the error: "connect". Protocol specific error code(s): "10061", "", "". SQLSTATE=08001
ErrorCode=-2147467259
I want to connect to dbSource (172.16.0.1) and if it is not possible then connect to AltdbSource (172.16.0.2)
Currently I have 2 identical excel sheeets and user needs to open one or the other if he gets the connection error, I want it to change automatically.
I'm guessing that TestForError = try OleDb.DataSource(BadSource) isn't working because OleDb.DataSource(...) returns a table value than only produces an error when trying to enumerate rows.
What if you change TestForError to drill into the first row if it exists:
TestForError= try Source{0}?,
If that still doesn't work, next try indexing into a table cell in the first row.
Related
I have a SQL job (4 steps job consist of SSIS packages) which runs on daily basis and extract data from various sources (source1, source2, source3) then it loads data to warehouse. Now my job fails due to 'Communication Link failure' with source1 at step 1.
Is there any way I can set up retry attempt SQL job based on this above error only.
For example, if I get error 'primary key violation' or some other data related issue then we should directly get notification that job failed but if we have error 'Communication Link failure' then step1 should do retry attempt.
Any suggestion would be appreciated.
Short answer: No, not with the SQL agent.
Longer answer: Maybe you can build up som logic where the package checks if the previous error was that specific error you're looking for, if then, execute again. Cumbersome but possible.
You can create an Event Handler for the OnError event with a Script Task which will check for this specific error and execute msdb.dbo.sp_start_job if this error occurred. Since I'm not sure the exact error code you're getting, this only checks the #[System::ErrorDescription] system variable for the specific text using the StringComparison.CurrentCultureIgnoreCase option to make this match case insensitive. However I would strongly recommend finding the exact error code and using the #[System::ErrorCode] variable to verify this instead. I'd also suggest only retrying the job a certain number of times or within a given time-frame to avoid excessive failures if this issue persists as well.
string errorMsg = Dts.Variables["System::ErrorDescription"].Value.ToString();
if (errorMsg.IndexOf("Communication Link failure", 0, StringComparison.CurrentCultureIgnoreCase) >= 0)
{
string connString = #"YourConnectionString;";
string startJobCmd = #"EXEC MSDB.DBO.SP_START_JOB N'NameOfJobToRetry;";
using (SqlConnection conn = new SqlConnection(connString))
{
SqlCommand sql = new SqlCommand(startJobCmd, conn);
conn.Open();
sql.ExecuteNonQuery();
}
}
Are there any examples on the net how to process errors when downloading data from Interactive Brokers using the IBrokers package? I've had a look at the package details and eWrapper and twsCALLBACK seem to handle this but I can't get them to work. For example the code below produces an error and R hangs, the error msg isn't processed. Thanks for any suggestions.
contract <- twsContract(0,
symbol="SPI",
sectype="XXX", #bad sectype
exch="SNFE",
primary="",
expiry= "20181220",
strike="",
currency="AUD",
right="",
local="",
multiplier = "25",
combo_legs_desc = "",
comboleg = "",
include_expired = "",
secIdType = "",
secId = "")
tws <- twsConnect()
data <- reqMktData(tws,contract,snapshot = TRUE)
You should append a "Disconnect" command to you code. Otherwise your program try to build to connections on the same port, that's not possible and it will not terminate.
I don't know the IBroker package very well, please check the command for disconnecting and append it to your code. Refresh your command line and rerun your code.
In addition, connect to IB Gateway instead of TWS by using that port number (check API settings of your IB Gateway application). In the settings choose a detailed Log.
Run your code again (after changing port number) and send your log file. Then I will try to help more. It's hard to help without any error message.
Trying to use the mv.NET by bluefinity tools. Made some integration packages with it for importing data from a d3 multi-value database into MS SQL 2012 but seem to be having some trouble with the mapping.
For the VOYAGES table have some commentX fields in the D3 application that are acting quite unwieldy and the INSERT fails after a certain number of rows with the following message
>Error: 0xC0047062 at INSERT, mvNET Source[354]: System.Exception: Error #8: dataReader[0] = LTPAC002 ci.BufferColumnIndex = 52, ci.ColumnName = COMMGROUP(Error #8: dataReader[0] = LTPAC002 ci.BufferColumnIndex = 52, ci.ColumnName = COMMGROUP(The value is too large to fit in the column data area of the buffer.))
at mvNETDataSource.mvNETSource.PrimeOutput(Int32 outputs, Int32[] outputIDs, PipelineBuffer[] buffers)
at Microsoft.SqlServer.Dts.Pipeline.ManagedComponentHost.HostPrimeOutput(IDTSManagedComponentWrapper100 wrapper, Int32 outputs, Int32[] outputIDs, IDTSBuffer100[] buffers, IntPtr ppBufferWirePacket)
Error: 0xC0047038 at INSERT, SSIS.Pipeline: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.The PrimeOutput method on mvNET Source returned error code 0x80131500.The component returned a failure code when the pipeline engine called PrimeOutput().The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.There may be error messages posted before this with more information about the failure.
The value is too large to fit in the column data area of the buffer. -> tried changing the input / outputs types but can't seem to get it right.
In the SQL table the columns are of type ntext.
In the .dtsx job the data type for the columns are of type Unicode String [DT_WSTR] with length 4000 , I guess these are auto-detected.
The import worked for other D3 files like this not sure why it fails for these comment fields.
Running the query on the mv.NET Data Manager ( on the d3 server) times out after 240 seconds so maybe this is the underlying issue?
Any ideas how to proceed? Thank you ~
Most like reason is column COMMGROUP does not have correct data type or some record in source do not fit in output type
To find error record (causing) you have to use on redirect row (property of component failing component ) and get the result set in some txt.csv /or tsv file .
then check data
The exception is being thrown from mv.NET so I suggest you call (or ask your reseller) to call Bluefinity support and ask them about this. You're paying for support, might as well use it. Those programs shouldn't be allowed to throw exceptions like that.
D3 doesn't export Unicode, that might be one issue. But if the Data Manager times-out then I suspect something is wrong in the connectivity into D3. Open a Connection Monitor from the Session Monitor and watch the connection when you make the request. I'm guessing it's either hanging or more probably it's falling into BASIC Debug.
Make sure all D3-side programs related to this are either all Flash-compiled, or all Not Flashed. Your app code will fall into Debug if it's not Flashed but MVNET.BP is.
If it's your program that's in Debug, fix it. If you're not sure which program it is, LIST-RUNTIME-ERRORS in DM.
If it's a MVNET.BP program, again work with Bluefinity. If you are using MVSP for connectivity then the Connection Monitor may be useless, you'll need to change that to an IP (Telnet) connection to see the raw data exchange.
I am running a very simple query and trying to extract the results to a text file. The entire query is essentially what is below, I am selecting everything from one single table with one piece of where criteria which is limiting the data to one month's worth. After it has extracted around 1.2 gig this error shows up. Is there any way that I can work around this other than extracting smaller date ranges? I am trying to pull a couple of years worth of data so if I can only get it a few days at a time it will take a lot of manual work.
I am currently using the free trial of a DB2 query tool - Razor SQL if that makes a difference, I can probably purchase different software if it would help. I am trying to get IBM's tool but for some reason it freezes during the download so I am still working on that. I have searched about this error but everything I see seems much more complex than what I am doing and I can't tell if it applies or not. Thanks in advance.
select *
from MyTable
where date_col between date '2014-01-01' and date '2014-01-31'
I stumbled at this error too, found out it is related to db2jcc.jar (type 4) driver.
Excerpt: If there are no items in the result set left (or to begin with), the Result set is closed automatically and therefore the Exception. Suggestion is to handle it in the application, perhaps in my case, I started checking if(rs.next()) but otherwise, there is a work around. Check out the source link below for how you can set some properties to Data source and avoid exception.
Source :
"Invalid operation: result set is closed" error with Data Server Driver for JDBC
In my case, i missed some properties in WAS, after add allowNextOnExhaustedResultSet the issue is fixed.
1.Log in to the WebSphere Application Server administration console.
2.Select Resources > JDBC > Data sources > Application Center DataSource name > Custom properties and click New.
3.In the Name field, enter allowNextOnExhaustedResultSet.
4.In the Value field, type 1.
5.Change the type to java.lang.Integer.
6.Click OK.
Sometimes you need also check whether resultSetHoldability properties exists. Details refer to here.
I encountered this failure also when ugrading from JDBC Type 2 driver (db2java.zip) JDBC type 4 driver (db2jcc4.jar)
Statement statement = results.getStatement();
if (statement != null)
{
connection = statement.getConnection(); // ** failed here
statement.close();
}
Solution was to check if the statement is closed or not as follows.
Changed to:
Statement statement = results.getStatement();
if (statement != null && !statement.isClosed()) {
{
connection = statement.getConnection();
statement.close();
}
Creating property bellow with type Integer it's worked for me:
allowNextOnExhaustedResultSet:
I had the same issue on WAS 7 so i had to add and change few this on Admin Console.
This TeamWorksRuntimeException exception should be fixed by applying APAR JR50863 which is available on top of BPM V8.5.5 or included on BPM V8.5 refresh pack 6.
For the case that the APAR does not solve the problem, try following workaround:
Log in to the WebSphere Application Server admin console
Select Resources > JDBC > Data sources > DataSource name (TeamWorksDB) > Custom properties and click New
In the Name field, enter downgradeHoldCursorsUnderXa
In the Value field, type true
Change the type to java.lang.Boolean
Click OK to save your changes
Select custom property resultSetHoldability
In the Value field, type 1
Click OK to save your changes
Source of the Answer : https://developer.ibm.com/answers/questions/194821/invalid-operation-result-set-is-closed-errorcode-4/
Restarting the app may fix the problem if connection pool lost session to Db2. If using Tomcat then connection pool property of 'testonBorrow' may reestablish the connection to Db2.
One of my primary SSIS import packages has been showing the following error the last few times it's been run. The data loads exactly how it should, just with this one error which doesn't seem to make a real difference. I would like to get it cleared up though.
Here is the error:
Error 1 Error loading Update.dtsx: The connection type "OLEDB" specified for connection manager "SQL.Staging" is not recognized as a valid connection manager type. This error is returned when an attempt is made to create a connection manager for an unknown connection type. Check the spelling in the connection type name. D:\SSIS Projects\Update.dtsx 1 1
Error 2 Error loading Update.dtsx: Error loading value " 0 SQL.Staging {925404E7-27AA-4C4E-337B-D6058341" from node "DTS:ConnectionManager". D:\SSIS Projects\Update.dtsx 1 1
Error 3 Error loading 'Update.dtsx' : The package failed to load due to error 0xC0010014 "One or more error occurred. There should be more specific errors preceding this one that explains the details of the errors. This message is used as a return value from functions that encounter errors.". This occurs when CPackage::LoadFromXML fails. . D:\SSIS Projects\Update.dtsx 1 1
As I said, the package runs fine. It doesn't seem to have any issues with the OLEDB source or destination's, which is what the error seems to indicate. Any other recommendations? Thanks!