we load a CSV from the web via Power Query. Among other columns there are three money columns with the data type DT_NTEXT as output column. I have already tried to use a data conversion task as first thing to convert the columns to DT_WSTR(4000) and then convert them to numeric(17,5) in a following Derived Column Task with the following expression:
TRIM(price_conv) == "" ? NULL(DT_NUMERIC,17,5) : ((DT_NUMERIC,17,5)TRIM(price_conv))
Unfortunately, this still fails.
Error messages:
[Derived column [2]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Derived column" failed because error code 0xC0049063 occurred, and the error row disposition on "Derived column.Output[Output of the Derived column].column[price_derived]" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "erived column" (2) failed with error code 0xC0209029 while processing input "Inputof the derived column" (3). The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure.
PS: Error messages partly translated from German.
I also tried to set DT_WSTR in the advanced editor of the Power Query source, but it failed as well.
Does anyone have an idea?
I had to use the replace function for the separator (. and ,), then it worked.
#1 Convert to DT_WSTR using the data conversion task or in the derived column task
#2 Pay attention to the separator and replace with the replace function if necessary
Example:
TRIM(origprice_conv) == "" ? NULL(DT_NUMERIC,17,5) : ((DT_NUMERIC,17,5)REPLACE(TRIM(origprice_conv),".",","))
origprice_conv is already converted to DT_WSTR.
It's a pity that SSIS does not give a meaningful error message here.
My problem was two fold but the Data Viewer helped identify the problems. The first was that I could have blank values for origprice_conv and the default conversion did not work. I solved this by casting a null to the correct DT_NUMERIC specification. The second problem was that the source data used US style decimal places in the data. Being in Germany, the conversion expected the string to be "123,45" instead of "123.45". Internationalization note, the period is the thousands separator for Germany.
The problem is that there is not implicit or explicit conversion from ntext to numeric, reference: https://learn.microsoft.com/en-us/sql/t-sql/functions/cast-and-convert-transact-sql?view=sql-server-ver15
The trim will fail in the first place so what you may need to do, as suggested by billinkc, is a double conversion, to wstr, trim, and then to numeric :
(DT_NUMERIC,17,5)TRIM((DT_WSTR,20)(price_conv))
UPDATE
It now seems that anything put in a vector breaks. I have tried char and u/int/8/16/32 and they all generate some kind of error. I'm a bit perplexed. There may be an error in my code, but I'm not sure what the YAML should look like, so I'm probably not doing a very a very good job of looking for when the data becomes incorrect.
Is YAML::Binary from Yaml-Cpp finished yet? I've tried serializing my data as int's, but Yaml-Cpp seems to be confused about ints and chars, and this generally never works. Instead, now I'm trying to use Yaml::Binary, but I get an error on the other side when I try to recover the YAML::Binary node on the other end. Specifically, this chunk fails:
3: 0\n6: WAUAAAAAAABYBQAAAAAAAP////84UBspV0FVQUFBQUFBQUJZQlFBQUFBQUFBUC8vLy84NFVCc3BWMEZWUVVGQlFVRkJRVUpaUWxGQlFVRkJRVUZCVUM4dkx5ODRORlZDYzNCV01FWldVVlZHUWxGVlJrSlJWVXBhVVd4R1FsRlZSa0pSVlVaQ1ZVTTRka3g1T0RST1JsWkRZek5DVjAxRldsZFZWbFpIVVd4R1ZsSnJTbEpXVlhCaFZWZDRSMUZzUmxaU2EwcFNWbFZhUTFaVlRUUmthM2cxVDBSU1QxSnNXa1JaZWs1RFZqQXhSbGRzWkZaV2JGcElWVmQ0UjFac1NuSlRiRXBYVmxoQ2FGWldaRFJTTVVaelVteGFVMkV3Y0ZOV2JGWmhVVEZhVmxSVVVtdGhNMmN4VkRCU1UxUXhTbk5YYTFKYVpXczFSRlpxUVhoU2JHUnpXa1phVjJKR2NFbFdWbVEwVWpGYWMxTnVTbFJpUlhCWVZteG9RMkZHV2xkYVJGSlRUVlZhZWxWdGVHRlZNa1YzWTBaT1YySkdXbWhWVkVaaFZteFNWVlZ0ZEdoTk1tTjRWa1JDVTFVeFVYaFRiazVZWVRGS1lWcFhjekZTUmxweFVWaG9VMkpIVW5wWGExcGhWakpLUjJORmJGZFdiVkV3VldwR1lXTXhUblZUYkZKcFVsaENXVlp0ZUc5Uk1rWkhWMnhrWVZKR1NsUlVWbFpoWld4V2RHVkhSbFpOYTFZeldUQmFUMVl5U2tkWGJXaFdWa1ZhYUZadGVGTldWbFowWkVkb1RrMXRUalJXYTFKRFZURlZlRlZZYUZSaWF6VlpXVlJHUzFsV2NGaGpla1pUVW14d2VGVldhRzlWTWtwSVZXNXdXR0V4Y0doV2FrcExVakpPUm1KR1pGZGlWa1YzVmxkd1IxbFhUWGhVYmxaVVlrWktjRlZzYUVOWFZscDBaVWM1VWsxcldraFdNbmhyV1ZaS1IxTnNVbFZXYkZwb1dsZDRWMlJIVmtoU2JGcE9ZVEZaZWxkVVFtRlVNVmw1VTJ0a1dHSlhhRmRXYTFaaFlVWmFkR1ZHVGxkV2JGb3dXa1ZrYjFSck1YUlVhbEpYWVRGS1JGWlVSbFpsUmxaWllVWlNhV0Y2VmxwWFZsSkhVekZzVjJOR2FHcGxhMXBVVlcxNGQyVkdWbGRoUnpsV1RXdHdTVlpYTlhkWFIwVjRZMGRvVjJGcmNFeFZha3BQVW0xS1IxcEdaR2xXYTFZelZteGtkMUl4YkZoVVdHaFZZbXhhVlZscldrdGpSbFp6WVVWT1dGWnNjREJhVldNMVZXc3hjbGRyYUZkTmJtaHlWMVphUzFJeA==\n7: /USER_NAMES/src/sockets/rsc/atkrscs.tar.gz\n1: 0\n4: 0\n5: 651633\n2: 0
As:
terminate called after throwing an instance of 'YAML::ParserException'
what(): yaml-cpp: error at line 7, column 7: unknown escape character:
What should I do? Is there another way to send/receive binary? Did I do something wrong?
Short: I am looking for a way to get the text of the script that was evaluated and caused a syntax error from within the context of window.onerror.
Long:
The full scenario includes a phone gap application and the PushNotifications plugins.
When a push message is sent to the device a javascript error is caught using window.onerror.
with the text "SyntaxtError: Expected token '}'"
the reported line number is 1 (is it is usually when dealing with EVALuated code.
The way the plugin executs its code is by using:
NSString * jsCallBack = [NSString stringWithFormat:#"%#(%#);", self.callback, jsonStr];
[self.webView stringByEvaluatingJavaScriptFromString:jsCallBack];
I belive but not 100% sure that this is the code PhoneGap Build are pushing
more code can be seen in here https://github.com/phonegap-build/PushPlugin/blob/master/src/ios/PushPlugin.m#L177
the self.callback is a string passed by me to the plugin and jsonStr is (supposed to be) an object describing the push message.
when I tried to pass as the parameter that ends up being self.callback the string alert('a');// then I did get the alert and no syntax error. ad now I am trying to understand what does jsonStr gets evaluated to so that maybe I can find a way around it or figure out if its my fault somehow (maybe for the content I am sending in the push notification....)
I also tried to look at the last item of the $('script') collection of the document hopeing that maybe stringByEvaluatingJavaScriptFromString generates a new script block but that does not seem to be the case.
further more in the window.onerror I also tried to get the caller
using var c=window.onerror.caller||window.onerror.arguments.caller; but this returns undefined.
As I stated before - I am looking for ideas on how to determine what exactly is causing the syntax error possibly by getting a hold of the entire block of script being evaluated when the syntax error happened.
When an an NSXMLParser encounters an invalid character, it calls the delegate method
parser:parseErrorOccurred:
from the documentation:
When this method is invoked, parsing is stopped. For further
information about the error, you can query parseError or you can send
the parser a parserError message. You can also send the parser
lineNumber and columnNumber messages to further isolate where the
error occurred. Typically you implement this method to display
information about the error to the user.
my question:
is there a way to use the line number and column number to remove the the invalid characters? (perhaps by somehow stripping everything from the current location up until the next closed bracket)? I am in search of a device side solution to receiving invalid XML.
Any thoughts?
When i send:
Result = CInt(NetApiBufferFree(pBuffer))
I receive (SOME TIMES) this error:
Arithmetic operation resulted in an overflow.
What exactly means that? and how i can resolve it?
It means that CInt argument is out of range of Integer, -0x80000000 to 0x7FFFFFFF
And it happens when NetApiBufferFree returns an error: error codes are bigger than 0x80000000.
There is no unsigned int32 type, so use CLng instead of CInt.
About source of error. You should find out code of error which you get: call MsgBox or log it to file, or use breakpoint. Next find its description. If it won't help you (for example error would be E_FAIL), add code to check that pBuffer value is valid - that it wasn't modified by something, and wasn't already freed. Add logging for NetApiBuffer* calls.
Best answer is replace 'double' in place of 'Int16/Int32/Int64'
some times file conversion takes huge numbers.. double never has maximum range.