I'm trying to get a percentage to display as a decimal in my database.
I have the following set up to convert the percentage columns into decimals:
---------------- ---------------- ------------
excel source ---------> data conversion ----------> db output
---------------- ---------------- ------------
I've tried to strictly convert the input to decimal and numeric.
Neither of these have changed my results.
In my columns in the database I'm getting just 0's and 1's.
Forgive my crude drawing; I do not have enough rep to post pictures yet.
Hope this is what you are looking for
Excel sheet like this is the source.
I just tested it in my system.It is working fine. This is what I did.
Created an SSIS package with just 1 DFT.
Data flow is given below. Please note that the value which appeared as 40% in Excel sheet is visible as 0.40. So I added two derived columns. One converting as such and the next which multiplies with 100.
the derived column structure is shown below.
The destination table structure be
Create table Destination
(
id int,
name varchar(15),
hike decimal(8,2)
)
I am getting the result as expected.
Select * from Destination
There are many ways to accomplish this. Here's one:
1) Save your excel file as a tab delimited text file.
2) Create a New Flat File Connection in SSIS
a) Set File Name = .txt file
b) Go to Advanced tab and click on the column with the percentages
c) Set the Data Type to match the target field in your database (e.g., numeric(10,5)
3) In the SSIS workflow, create a derived column of your percent field to convert from percent to decimal(e.g., newfield = oldfield/100). Make sure to check the data type has not changed in the Derived Column Transformation Editor.
Related
I want to find out if there is a better way to present a data the way I need using T-SQL and generating a TEXT file in SSIS from SQL command(Exec stored proc).
So I need to present a data in a text file with fixed width for each column, like col1 - 10 alphacharacters, col2 numeric with zero pads, col3 blanks etc.
the Total number of characters in a line cannot exceed and must be 275.
However Each Row is going to have different data and different column requirements.
So if
First row: '1', col1 - 22 alphacharacters, col2 numeric with zero pads 10 characters, col3 blanks(fill up to 275)
Second row:'2', col1 - date 6 characters,col2 3 blanks
col3 30 alphacharacters, col4 numeric, col 5 blanks(fill up to 275)
What I come up with is to concat the row into 1 big string and then Union ALL the rows.
And in SSIS I do Ragged right without columns headers and the text file is coming up exactly as I want, but I wonder if there is a better way to do that
I Figured how to manipulate data with different functions, so i'll just present a code without them to make everything simple
SELECT CONCAT('1',COL1, COL2, REPLICATE(' ',n...)) as 123
FROM MYTABLE
UNION ALL
SELECT CONCAT('2',COL1,REPLICATE(' ',3),COL2, COL3, REPLICATE (' ',n..)) as 123
FROM MYTABLE2
The Layout of results should look like that in a text file (I just put random data for this example to make visualization better)
1Microsoft SQL Server 0000002017
208202019 John Doe 00000015
208202019 Jane Doe 00000109
208202019 Will Smith 00001996
Rephrasing your task - you need to set a custom header for your Flat File, and output sows in Ragged format.
The alternative approach is the following. Create a string variable and fill it with header row data as desired. On the Data Flow task which fills in the Flat File Destination, locate [Flat File Destination].[Header] property and set its expression to the string variable from above. This will create a file with defined header. Then on the data flow itself - create a string computed column where you format your output string, and later - save this column into the Flat File Destination.
This is more SSIS approach, since you do not have to do complex SQL statements.
I am trying to use a cell as a parameter in Excel powerquery. The query works without this, but I have to manually input the values, which I need to constantly change them in the query in other to get the results that I want.
Query (Advanced Editor):
let
Criteria01 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{0},
Criteria02 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{1},
Criteria03 = Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{2},
Source = Sql.Database("SERVERNAMEHERE", "DATABASENAMEHERE", [Query="SELECT DISTINCT [...........] AND (TABLEPREF.COLUMNHERE like '%MANUALVALUE01%' OR#(lf)TABLEPREF.COLUMNHERE like '%MANUALVALUE02%' OR#(lf)TABLEPREF.COLUMNHERE like '%MANUALVALUE03%' OR#(lf)TABLEPREF.COLUMNHERE like Criteria01)#(lf)#(lf)#(lf)order by 1 asc"])
in
Source
"Servers" is the table name and "ServerSearch" is the column header. If I check the step for Criteria01/etc it will show me the correct value of that table that I need to use.
Original query done in Sql-Server. I have no problems when running the query with only LIKE '%MANUALVALUES%' lines.
My main goal is to automatically get N values of "MANUALVALUES" from a table in a sheet, which will be used as an input for comparing WHERE TABLEPREF.COLUMNHERE like '%VALUEHERE%'. I must use this and I can't get the whole table/database because there are way too many results besides the ones that I want.
However for test purposes at this moment, I am trying to use only 1-3 values, the first 3 of this table (Criteria{0}{1}{2} in the query above). However, if I try to do something like TABLEPREF.COLUMNHERE like Criteria01 I get the following error:
DataSource.Error: Microsoft SQL: Invalid column name 'Criteria01'.
Details:
DataSourceKind=SQL
DataSourcePath=dalsql390;itdw
Message=Invalid column name 'Criteria01'.
Number=207
Class=16
So my questions are:
I am getting the table cell value by the right way? Meaning:
Excel.CurrentWorkbook(){[Name="Servers"]}[Content][ServerSearch]{0}.
How do I refer this value in my query? Since the way that I wrote
that query bought me that error.
Also please note that if change TABLEPREF.COLUMNHERE like
Criteria01 to CHG1.CI_Name like "Criteria01" I get the
following error:
Expression.SyntaxError: Token Comma expected.
After fixed 1 and 2, how can I use this dynamically? For
example, instead of getting values of index 1 2 3, what if I want to
use a whole table? I know that using
Excel.CurrentWorkbook(){[Name="Servers"]}[Content] will bring me the whole table of values (1 column, unknown number of rows), but
how do I use this table content 1 by 1 in my query?
That will get the value, but you can't refer to steps inside of text values by putting the step name inside of it.
You have a couple options for doing this dynamically.
Use Value.NativeQuery to create a parameterized query where you can pass in other values as parameters. For example, Value.NativeQuery(Sql.Database("SERVERNAMEHERE", "DATABASENAMEHERE"), "select #a, #b", [a = 1, b = "x"]) will return the table [1, x]. You can put in the step name in the record value to pass that it (e.g. replace "x" with Criteria01).
Add the text values directly in the query field, e.g. [Query = "select " & Criteria01 ";"]. This is highly discouraged since this can lead to SQL injection issues.
For the third question, it depends what you want to do with the list of values. At some point you will likely need List.Accumulate to turn them all into a single text value which can be placed in the query value, and maybe to turn them into a record to place into the parameters value.
I am working with data from the hospital, and when I add the .csv extension to my text files they output in the following way:
It would be much easier to manage if there were a way to only include the numbers in the first column once, and also transpose them as column headers. And go through the first ten in the second column, add and transpose them underneath, then do the next ten. The final product looking like this:
I have tried transposing them manually, but since there are millions of files, the CSV's are quite extensive. I have looked for a way in Excel to do it, but I have found nothing.
Could someone help me with a macro for this?
An excel formula could be used, if the numbers are repeated exactly.
If the data is in Columns A & B, the following formula could be placed in C2:
=INDEX($B:$B,(ROW(C1)-1)*10+COLUMN(A$1))
And then copied to the right and down as far as needed.
You didn't mention whether the sequence of row numbers (1,90,100,120...) is always the same for each "row". From your sample, I will assume that the numbers repeat the same way, ad infinitum.
First, import the CSV into Microsoft Access. Let's assume your first column is called RowID, and your second is called Description. RowID is an integer, and Description is a memo field.
Add a third column, an Integer, and call it "Ord" (no quotes).
In Access's VBA editor add a new module with this GroupIncrement function:
Function GroupIncrement(ByVal sGroup)
Static Num, LastGrp
If (StrComp(LastGrp, Nz(sGroup, "")) <> 0) Then
Num = 0
End If
Num = Num + 1
GroupIncrement = Num
LastGrp = sGroup
End Function
Create a new query, replacing MyTable with the name of your Access table containing the CSV data:
UPDATE (SELECT * FROM [MyTable]
ORDER BY [RowID]) SET [Ord]=GroupIncrement([RowID])
Create a third query:
TRANSFORM First([Description])
SELECT [Ord]
FROM [MyTable]
GROUP BY [Ord]
PIVOT [RowID]
This should put the data into the format you want (with an extra column on the left, Ord).
In Access, highlight that query and choose External Data, and in the Export section, choose Excel. Export the query to Excel.
Open the file in Excel and delete the Ord column.
I would like to know a thing if thst is possible.. let's put that I have a database with 3 column (name, last name and age) that obviously will be type: varchar, varchar and int.
then on visual studio I create a datagridrow with a dataset for automatically retrieve values from database so in my grid I will see this:
Mark //Name
Wally //Last name
38 //Age
And untill here all is easy, but if for some reason I change the value of age from int to varchar and add a vale like: "December" when I open my datagrid I should see
Mark //Name
Wally //Last name
December //Age converted from int to varchar only for this example
Instead I get an exception that says: "impossible to store a String in the column 'Age' because type is int", So for fixing this problem a person has got only 2 chances
1) Delete the Dataset on the form designer and create back another one;
2) Going in the code behind of the dataset (DataSet.Designer) and manually change all the values of the column Age from int to varchar.
Both are boring solutions anyway, is it possible isn't there a way to automatically update the dataset after that a changed is made inside the database?
Ty very much guys and Merry Christmas.
I import Excel files via SSIS to SQL-Server. I have a temp table to get everything in nvarchar. For four columns I then cast the string to money type and put in my target table.
In my temp table one of those four columns let me call it X has a comma as the delimiter the rest has a dot. Don't ask me why, I have everything in my SSIS set the same.
In my Excel the delimiter is a comma as well.
So now in my target table I have everything in comma values but the X column now moves the comma two places to the right and looks like this:
537013,00 instead of 5370,13 which was the original cell value in the temp and excel column.
I was thinking this is a culture setup problem but then again it should or shouldn't work on all of these columns.
a) Why do I receive dot values in my temp table when my Excel displays comma?
b) how can I fix this? Can I replace the "," in the temp table with a dot?
UPDATE
I think I found the reason but not the solution:
In this X column in excel the first three cells are empty - the other three columns all start with 0. If I fill these three cells of X with 0s then I also get the dot in my temp table and the right value in my target table. But of course I have to use the Excel file as is.
Any ideas on that?
Try the code below. It checks whether the string value being converted to money is of numeric data type. If the string value is of numeric data type, then convert it to money data type, otherwise, return a NULL value. And it also replaces the decimal symbol and the digit grouping symbol of the string value to match the expected decimal symbol and digit grouping symbol of SQL Server.
DECLARE #MoneyString VARCHAR(20)
SET #MoneyString = '$ 1.000,00'
SET #MoneyString = REPLACE(REPLACE(#MoneyString, '.', ''), ',', '.')
SELECT CAST(CASE WHEN ISNUMERIC(#MoneyString) = 1
THEN #MoneyString
ELSE NULL END AS MONEY)
As for the reason why you get comma instead dot I have no clue. My first guess would be cultural settings but you already checked that. What about googling, did you get some results?
First the "separator" in SQL is the decimal point: its only excel that is using the comma. You can change the formatting in excel: you should format the excel column as money and specify a decimal point as the separator. Then in the SSIS import wizard split out the transformation of the column so it imports to a money data type. Its a culture thing, but delimiter tends to be used in the context of signifying the end of one column and the start of the next (as in csv)
HTH
Well thats a longstanding problem with excel. It uses the first 30 or so rows to infer data type. It can lead to endless issues. I think your solution has to be to process everything as a string in the way Yaroslav suggested, or supply an excel template to have data predefined and formatted data type columns, which then have the values inserted. Its a pita.