Need some help please ..
I have excel data that is formatted. Example 33.257615111 shows as 33.258 (thirty three million two hundred and fifty eight thousand)
I am trying to import data file into SQL using SSIS ETL. Is there a way I can convert this during ETL process as a number with 2 decimal places? i.e. 33257615.11
Thank you
This can be done. Use a Derived Column and the ROUND() expression.
Inside your Dataflow, between the Excel File Source component and the OLE DB Destination component, add a Derived Column component. Inside that Derived Column component create a new column that rounds the value.
(DT_NUMERIC,25,2)ROUND(MyColumnName, 2)
Map this new Derived Column above to your SQL destination and we are all set.
But! Are you SURE you want to store rounded numbers in your database?
Will you ever need to sum this column? If so, the rounded values are going to yield a very different total than the non-rounded values. Maybe, don't do this rounding in SSIS and instead store the full numbers. Let applications reading this data do the formatting. It's just as easy.
SELECT ROUND(MyColumnName,2) AS MyColumnName
FROM MyTable
In multitier architecture, the formatting of numbers should be left up to the display layer, not the data layer. This is exactly what Excel is doing. It's showing you a rounded number, but behind the scenes is the full value. Don't drop precision for the sake of formatting.
Related
My issue is similar to this one Multiple data types in a Power BI matrix but I've got a bit of a different setup that's throwing everything off.
What I'm trying to do is create a matrix table with several metrics that are categorized as Current (raw data values) and Prior (year over year percent growth/decline). I've created some dummy data in Excel to get the format the way I want it in PowerBI (see below):
Desired Format
As you can see the Current values are coming in as integers and the Prior % numbers as percentages which is exactly what I want; however, I was able to accomplish this through a custom column with the following formula:
Revenue2 = IF(Scorecard2[Current_Prior] = "Current", FORMAT(FIXED(Scorecard2[Revenue],0), "$#,###"), FORMAT(Scorecard2[Revenue], "Percent"))
The problem is that the data comes from a SQL query and you can't use the FORMAT() function in DirectQuery. Is there a way I can have two different datatypes in the same column of data? See below for how the SQL data comes into PowerBI (I can change this if need be):
SQL
Create 2 separate measures, one for the Current second for Prior, and format these measures.
Probably you can also use a case in SQL query to format your data to bring it as STRING.
What I wound up doing was reformatting the SQL code to look like this:
Solution
That way Current/Prior are have two separate values and the "metric" is categorical.
I got the idea from this post:
Simple way to transpose columns and rows in SQL?
I have an ETL process set up to take data from an Excel spreadsheet and store it in a database using SSIS. However, one of the columns in the the Excel file is formatted as a percent, and it will sometimes erroneously be stored as a NULL value in the database, as if there was some sort of translation error.
Pictured is the exact format being used for the column in Excel.
Interestingly, these percent values do load properly on some days, but for some reason one particular Excel sheet I was given as an example of this issue will not load any of them at all when put through the SSIS processor.
In Excel, these values will show up like "50.00%", and when the SSIS processor is able to translate them properly it will display as the decimal equivalent in the database, "0.5", which is what I want instead of the NULL values. The data type I am using in SSIS for this is Unicode string [DT_WSTR], and it is saved as an NVARCHAR in the database.
Any insight as to why these values will sometimes not display/translate as intended? I have tried messing around with the data types in SSIS/SQL Server, but it has either resulted in no change or error. When I put test values in the Excel sheet, such as "test" to see if it is importing anything at all from this column, it does seem to work (just not for the percent numbers that I need).
The issue was caused by the "mixed data types" that were present in the first few rows of my data (the "mixed" part being blank fields), which would explain why some sheets would work and others wouldn't.
https://stackoverflow.com/a/542573/11815822
Setting the connection string to accommodate for this fixed the issue.
ok so I have this report I have to write in SSRS with a very specific format. It looks like the screenshot below at the bottom. Ignore the arrows and colors. It's pulling from an Oracle database. Each number value cell in this table/matrix has a different sql query to pull it in because they come from different tables, etc.
the top half of the numbers in the table are each from a query. the bottom half of the table is all calculations from the numbers in the top half. I already have the queries for the top half and was trying to figure out how I could just use those to make this table in SSRS with just those and then creating calculations in the bottom half for the report. I can't use a table or a matrix because each query is a different dataset and you can only have one dataset per tablix.
I was thinking maybe doing textboxes and drawing the grid manually, which would be a huge pain. I get errors about not having an aggregate and being out of scope or something and haven't figured out the reason for this yet as it is not my ideal solution.
My current solultion that will eventually get me there is unioning every single query and then creating columns with static values for the rows and columns in the grid and turning it into a matrix. Problem with this is it continues to increase in complexity as I create each further down the table calculation, and the code becomes larger and larger, and takes a long time to create, and I have to do like 6 reports similar in nature to the format of this one. Will probably be a thousand lines of sql and force me to have to make a stored proc because of the ssrs character limitation.;
So my question in a more simple way is, how can I take multiple sql queries that return a static value and make them a single value in a tablix that doesn't repeat, then create more blank rows in that tablix that are calculations of other cells values, i.e. Textbox1 - textbox2, textbox3/textbox4 ?
I got it figured out using expressions with multiple datasets. The answer seemed too easy once I found it. Basically just created a table tablix using my first dataset. Created more detail rows with insert row inside group below. Then I went to the expression builder for each one and found the other dataset and double clicked it to get the expression to pull from the other dataset. For example the bac_labor dataset value would look like this. =Sum(Fields!BAC_LABOR.Value, "BAC_Labor")
Then for calculations can use either same thing like =Sum(Fields!BAC_LABOR.Value, "BAC_Labor") + Sum(Fields!BAC_LABOR_OVERHEAD.Value, "BAC_Labor") or could do something like this =ReportItems!Textbox2.Value - ReportItems!Textbox1.Value to reference a cell value. This saves a ton of time, development effort, and reduction of code for calculations, compared to adding together 500 character select statements to make calculations. Also no need to use stored procs and union or join every select statement together with this method.
I am exporting a file from a system as .csv. My aim is to link to this file as a table (which matches the output field for field) and then run the queries and export.
The problem I am having is that, upon import, all the fields are 255 bytes wide rather than what they need to be.
Here's what I've tried so far:
I've looked at ALTER TABLE but I cannot run multiple ALTER TABLE statements in one macro.
I've also tried appending the table into another table with the correct structure but it seems to overwrite the structure.
I've also tried using the Left function with the appropriate field length, but when I try to export, I pretty much just see 5 bytes per column.
What I would like is a suggestion as to what is the best path to take given my situation. I am not able to amend the initial .csv export, and I would like to avoid VBA if possible, as I am not at all familiar with it.
You don't really need to worry about the size of Text fields in an Access linked table that is connected to a CSV file. Access simply assigns each Text field the largest possible maximum size: 255. It does not mean that every value is actually 255 characters long, it just means that any values in those fields can be at most 255 characters long.
Even if you could change the structure of the linked table (which you can't), it wouldn't make any real difference except to possibly truncate longer Text values, and you could easily do that with a String function. For example, if a particular field had to be restricted to 15 characters then you could simply use Left([fieldName], 15) as a query column or as the control source in a report.
In the end, as the data set is not that large, I have set this up to append from my source data into a table with the correct structure. I can now run my processes against this table as per normal.
I have a database extract in excel and want to create a custom value in Tablue using their create calculation, which I believe is SQL based.
Basically I have a large number of feeds which all show up different amounts in a column. For example:
feed 1
feed 1
feed 2
feed 3
feed 4
feed 4
feed 4
And I want to have a sum for feed 1, feed 2, and feed 4. But in my actual DB there's about 100 feeds all with different number of appearances. I'm having troubles finding a good way to do this. If there even is one. Any help or direction would be appreciated!
I'm assuming that your list is a single column and you need a count of the number of occurrences of each feed. For the sake of example, since a column or table names were not supplied, let's call them colname and tablename.
select colname, count(*) as Ct from tablename group by colname
It would be easier to give an exact answer if you posted a small simplified subset of your spreadsheet. But assuming you have a column called "feed_name" which takes on values like "feed 1", "feed 2" etc depending on the row. Then the feed_name column should be a discrete dimension in Tableau.
Then just put the feed_name pill on a shelf, say the row shelf. And put the "Number of Records" field on another shelf, say the column shelf.
You don't need to write SQL to do this (or most tasks) in Tableau. It helps to understand SQL concepts and its very helpful to drop down to the SQL level when needed to solve tricky issues. But for most situations, you can just interactively explore the data by moving fields around and writing some simple calculations -- and let Tableau take care of generating the SQL necessary to retrieve the data needed to build the visualization you requested.
Tableau supports SQL and some NO-SQL data sources, along with some cubes too. It does that quite well and in multiple ways. You just can work more quickly and efficiently by using Tableau's visual based manipulations in most cases, and then drop to the lower level detail when needed. It just takes getting used to how Tableau operates.