Data - from table (58 columns), in this table 4 columns will use in conditional filter.
1 column - bit data type(Color_Ind)
1 column - bit data type(Type_ind)
2 columns - varchar data type(Region,State)
process steps
1) OLE DB connection to extract all data
2) use Data Conversion to convert bit type columns into string data type.
3) use conditional filter to filter the data into 5 different category.
Probelm:-
- If I use color_ind and Type_ind it will not create my desired output, but If I use it without these columns and only use Region and state it will create my desired output.
Could you please tell me how to use bit data type into condtional filter in SSIS?
Appreciate your help.
Thanks
I got the answare from http://social.msdn.microsoft.com/Forums/en-US/sqlintegrationservices/thread/a0c4dbe0-bbe1-421f-b2fe-b72dce4d224e post...
Thanks for your efforts.
Related
I have a dataset that in simple representation looks like:
Col1
Col2
1
[A,B]
2
[C]
I want to denormalize the data and create another column while flattening, which would be the index of the elements in the array. The desired result set would look like:
Col1
Col2
Col3
1
A
1
1
B
2
2
C
1
I was able to achieve the requirement using mapindex, keyvalues and mapassociation expression functions.
Somehow I feel like this is not the right way to do it and there must be a better and easier way to do it. I read the microsoft documentation and couldnt find it.
Can someone help/guide me to a better solution?
Edit 1:
Source is Azure Blob Storage. I have access to only ADF. Data is a complex XML document. All transformations are to be performed only with ADF.
Edit 2:
Target is SAP BW . But I don't have control on it. I can only write to it.
You can use flatten transformation to flatten the array values and Window transformation to get the RowNumber, partition by Col1.
Flatten transformation: Unroll by array column (Col2).
Window transformation: Connect the output of flatten to Windows transformation.
Set a partition column in the Over clause.
Set a sort column to sort the data ordering.
In window columns setting, you can define the aggregation rowNumber() to get the index value based on col1.
Output of Window transformation:
We have used array aggregation method and loaded the data in BigQuery
Clarification :
Is it possible to retrieve the specific value in array aggregation method? What are the methods available for retrieving the data from the field which have multiple records?
Query Clarification
We tried to find out the value of all data from the particular field which has multiple values in the screenshots [image.png] using below query but we got an error.
Sample Query
select fv,product.productSKU,product.productVariant,product.productBrand
from dataset.tablename
where hn=9 and product.productBrand='Politix'
You should use UNNEST as in below example
#standardSQL
SELECT
fv,
product.productSKU,
product.productVariant,
product.productBrand
FROM `dataset.tablename`,
UNNEST(product) product
WHERE hn=9
AND product.productBrand='Politix'
You can also check Working with Arrays in Standard SQL
I have one table with rows and each row has a column that contains a field name (say raw1 - 'Number001', raw2-'ShortChar003', etc). In order for me to get that value of these fields I have to use a second table; this table has 1 raw with many columns (number001, Number002, ShortChar003, etc).
How can I extract the value?
Good Question..You can use lookup function
=Lookup(Fields!CityColumn.Value, Fields!CityColumn.Value, Fields!CountColumn.Value, "Dataset1")
Or you might have to use string functions..ex LEFT, Substring, Right same like SQL.If possible pls post some data of both tables, I will explain in detail
I am facing an issue related to olap and kettle. Thing is I am using pentaho DI(kettle) to export data from my database to olap star schema(facts & dimensions). Now when it comes to timestamp I am able to retreive months,years and days from aw timestamp using calculator in kettle . Now What I need is to convert month numeric value to its coresponding month name (like replace 1 with JAN and so on). How this can be acheived with kettle . Please suggest me .
In PDI/Kettle, I've found it tricky to set the value of one field based on the value of another field that is a different data type. A JavaScript or Formula step will do it, but in this case I think I'd use a Stream Lookup because your lookup values are few and fixed.
Use a Data Grid step with an Integer column for the month number (1, 2, 3 ...) and a string column with the month name (JAN, FEB, MAR ...). Use this step as the 'Lookup step' in your Stream Lookup. Then just retrieve the month name into your data flow. This should be pretty fast, which is good if you're working with typical data warehouse volumes.
As Brian said you can also use the Modified JavaScript step to perform the conversion. Here is how you can do it with that:
var month_names=new Array();
month_names[1]="Jan";
month_names[2]="Feb";
month_names[3]="Mar";
month_names[4]="Apr";
month_names[5]="May";
month_names[6]="Jun";
month_names[7]="Jul";
month_names[8]="Aug";
month_names[9]="Sep";
month_names[10]="Oct";
month_names[11]="Nov";
month_names[12]="Dec";
var month_short = month_names[month_num];
I'm working with a database, where one of the fields I extract is something like:
1-117 3-134 3-133
Each of these number sets represents a different set of data in another table. Taking 1-117 as an example, 1 = equipment ID, and 117 = equipment settings.
I have another table from which I need to extract data based on the previous field. It has two columns that split equipment ID and settings. Essentially, I need a way to go from the queried column 1-117 and run a query to extract data from another table where 1 and 117 are two separate corresponding columns.
So, is there anyway to split this number to run this query?
Also, how would I split those three numbers (1-117 3-134 3-133) into three different query sets?
The tricky part here is that this column can have any number of sets here (such as 1-117 3-133 or 1-117 3-134 3-133 2-131).
I'm creating these queries in a stored procedure as part of a larger document to display the extracted data.
Thanks for any help.
Since you didn't provide the DB vendor, here's two posts that answer this question for SQL Server and Oracle respectively...
T-SQL: Opposite to string concatenation - how to split string into multiple records
Splitting comma separated string in a PL/SQL stored proc
And if you're using some other DBMS, go search for "splitting text ". I can almost guarantee you're not the first one to ask, and there's answers for every DBMS flavor out there.
As you said the format is constant though, you could also do something simpler using a SUBSTRING function.
EDIT in response to OP comment...
Since you're using SQL Server, and you said that these values are always in a consistent format, you can do something as simple as using SUBSTRING to get each part of the value and assign them to T-SQL variables, where you can then use them to do whatever you want, like using them in the predicate of a query.
Assuming that what you said is true about the format always being #-### (exactly 1 digit, a dash, and 3 digits) this is fairly easy.
WITH EquipmentSettings AS (
SELECT
S.*,
Convert(int, Substring(S.AwfulMultivalue, V.Value * 6 - 5, 1) EquipmentID,
Convert(int, Substring(S.AwfulMultivalue, V.Value * 6 - 3, 3) Settings
FROM
SourceTable S
INNER JOIN master.dbo.spt_values V
ON V.Value BETWEEN 1 AND Len(S.AwfulMultivalue) / 6
WHERE
V.type = 'P'
)
SELECT
E.Whatever,
D.Whatever
FROM
EquipmentSettings E
INNER JOIN DestinationTable D
ON E.EquipmentID = D.EquipmentID
AND E.Settings = D.Settings
In SQL Server 2005+ this query will support 1365 values in the string.
If the length of the digits can vary, then it's a little harder. Let me know.
Incase if the sets does not increase by more than 4 then you can use Parsename to retrieve the result
Declare #Num varchar(20)
Set #Num='1-117 3-134 3-133'
select parsename(replace (#Num,' ','.'),3)
Result :- 1-117
Now again use parsename on the same resultset
Select parsename(replace(parsename(replace (#Num,' ','.'),3),'-','.'),1)
Result :- 117
If the there are more than 4 values then use split functions