I need to compare two varbinary columns using SSIS Conditional Split component.
I cannot use the == operator with varbinary.
What is the best way to compare two varbinary columns.
Have tried converting to TEXT datatype but this gives me various problems.
Two Columns are compared in Script Transformation Component , But do not know how to columns with Binary DataTypes are compared.
If anybody answers how binary values can be compared in Scrit component..you will have ur answer.
Could you use a Script Component, and compare the varbinary columns using VB.NET or C#? You could then assign a value to a new column in the data flow that lets you identify if they were identical. The Conditional Split could then use the new column as the condition.
Related
I currently have this sql
CREATE TEMPORARY VIEW binary_input_table
AS
SELECT binary(CONCAT(column_1, column_2, column_3)) AS binary_input_str
FROM input_table;
where I need binary_input_str as an input to a custom UDF I made. However, this solution isn't scalable in case there are thousands of columns, which I would then have to CONCAT manually. I've also tried SELECT binary(*)... but fails as binary only expects one argument.
Is there an easy way to convert all the columns into binary and store it into a variable?
simple solution
SELECT binary(CONCAT(*)) AS binary_input_str
Need to split column value into multiple columns based on delimiter,
Also need to create columns dynamically based on no. of delimiters, delimiter could be comma or so. Thanks,
You might be better off working with an array in this case. You haven't specified whether each record will have the same number of delimiters, so dynamically creating columns for tables will take some scripting. If they are, you could potentially use a SPLIT, LATERAL FLATTEN, and PIVOT, but the pivot needs static column names, so you'll likely need a stored procedure to deal with that.
To answer your first question, you can use SPLIT and/or SPLIT_PART to split the column into values. The SPLIT function creates an array for you, while the SPLIT_PART function creates the array, but outputs a single value from the array.
https://docs.snowflake.com/en/sql-reference/functions/split.html
https://docs.snowflake.com/en/sql-reference/functions/split_part.html
https://docs.snowflake.com/en/sql-reference/functions/flatten.html
https://docs.snowflake.com/en/sql-reference/constructs/pivot.html
In my SQL, I would need to compare data between two tables in SQLServer 2008R2 to return the rows where mismatch is present (using EXCEPT) and likewise matching rows in other cases (INTERSECT). The problem is, some of the columns have NTEXT datatype (SQLServer), and SQLServer gives error when such tables having columns with NTEXT are present.
Example:
SELECT * FROM table_pre
EXCEPT
SELECT * FROM table_post
The above operation gives an error -
'The ntext data type cannot be selected as DISTINCT because it is not comparable.'
I believe that tables (table_pre, table_post) have at least one column of datatype = NTEXT that is causing the comparison to fail.
Question -
1. Is there some way to exclude these NTEXT columns from the above comparison, without me having to explicitly list out the column names and excluding the problem column? There's a large number of columns involved and explicitly listing is not easy.
2. Can I just explicitly cast/convert the NTEXT column alone to say VARCHAR, and still go by not having to list down the rest of the columns?
3. Or, in general, can I somehow exclude certain columns by listing those out during such comparisons?
Any suggestions, really appreciated! Thanks.
Question - 1. Is there some way to exclude these NTEXT columns from the above comparison,
Yes, use explicitly the column names.
without me having to explicitly list out the column names and excluding the problem column?
Using * is a bad habit, you well deserve the error for abusing it.
There's a large number of columns involved and explicitly listing is not easy
Is actually trivial, build the statement dinamycally
Can I just explicitly cast/convert the NTEXT column alone to say VARCHAR
No. You have to convert to NVARCHAR, the N is very important. But, yes you can convert.
Or, in general, can I somehow exclude certain columns by listing those out during such comparisons
Fortunately no. SQL does not randomly decide what columns are or are not part of a result, so you get the predictability you desire.
So, in conclussion:
never use *
build complex statements dynamically. SELECT ... FROM sys.columns is your friend, you can easily build it in a few seconds
ditch the deprecated TEXT, NTEXT and IMAGE types
I have a column of varchar(10) called TITLE. Except for the first row, which contains the column header, the rest of the column happens to be all integers so I wanted to change the datatype to int.
ALTER TABLE X
ALTER COLUMN TITLE int
I get an error when converting the first row, which is the column header: "Conversion failed when converting the varchar value 'TITLE' to data type int.
So, how do I convert the data type for all rows, except the column header?
The short answer is No, you cannot mix data types in a single SQL column. Data types need to be consistent for things like sorting, building indexes, etc.
You could possibly use another table to store various column headers, or another column in the same table.
Using a NoSQL solution such as MongoDB might be an approach, depending on the type of data you're storing. These solutions allow you to be a lot more flexible with the schema, which can even differ per document.
Nope sorry you cannot have multiple data types on the same column.
Data types
You don't. A column isn't a collection of independent variables that can each have their own type. Everything in a column has the same type. If you're trying to do this, then your schema isn't likely what it should be. If you post a little more detail, you can likely get some answers with an improved schema.
You can't mix'n'match data types within a column. You can use fuzzy data types like VarBinary or XML and interpret them as you please.
OTOH, you can use sp_addextendedproperty to store column titles and other extraneous bits of fluff.
I have a column containing the strings 'Operator (1)' and so on until 'Operator (600)' so far.
I want to get them numerically ordered and I've come up with
select colname from table order by
cast(replace(replace(colname,'Operator (',''),')','') as int)
which is very very ugly.
Better suggestions?
It's that, InStr()/SubString(), changing Operator(1) to Operator(001), storing the n in Operator(n) separately, or creating a computed column that hides the ugly string manipulation. What you have seems fine.
If you really have to leave the data in the format you have - and adding a numeric sort order column is the better solution - then consider wrapping the text manipulation up in a user defined function.
select colname from table order by dbo.udfSortOperator(colname)
It's less ugly and gives you some abstraction. There's an additional overhead of the function call but on a table containing low thousands of rows in a not-too-heavily hit database server it's not a major concern. Make notes in the function to optomise later as required.
My answer would be to change the problem. I would add an operatorNumber field to the table if that is possible. Change the update/insert routines to extract the number and store it. That way the string conversion hit is only once per record.
The ordering logic would require the string conversion every time the query is run.
Well, first define the meaning of that column. Is operator a name so you can justify using chars? Or is it a number?
If the field is a name then you will use chars, and then you would want to determine the fixed length. Pad all operator names with zeros on the left. Define naming rules for operators (I.E. No leters. Or the codes you would use in a series like "A001")
An index will sort the physical data in the server. And a properly define text naming will sort them on a query. You would want both.
If the operator is a number, then you got the data type for that column wrong and needs to be changed.
Indexed computed column
If you find yourself ordering on or otherwise querying operator column often, consider creating a computed column for its numeric value and adding an index for it. This will give you a computed/persistent column (which sounds like oxymoron, but isn't).