I am working on Excel file to load the data into sql table using SSIS VS2013.
How do I extract the column names into a temptable?
In the image attached,there is the "2017 Marketing Sales - Actual" column in the first row, I want to extract the YYYY from the column name and pass that value to a variable and update the table field with YYYY info.
Can you anyone help me how to do this?
In your Excel Connection Manager, uncheck the "First row has column names" check box. This should allow you to access that first row. You'll need to setup a Data Flow Task using this Excel Connection Manager, followed by a derived column to extract the left 4 characters from that 1st row. Unless you somehow limit the rest of the Excel data source, you'll probably also get a lot of extraneous rows importing into your destination. Might need to do some clean up to get it down to just that year.
Related
I am very new to SQL, I want to import data from SQL Server to Excel using this query
SELECT
Model, Factory, TargetTime, TotalEvalMins
FROM
AMSView
WHERE
WeekNumber = 45 AND WeekYear = 2021
I want to change the week number & year dynamically by taking user input from a cell.
Can anyone please suggest how to change the query?
Let's say the user values week & year in worksheet sample in A1,A2 , how can I write that query?
Since the amount of data is huge I must apply where while querying the data instead of applying filters in Excel.
Sorry for my bad English
Name each of your cells that you will use as parameters. This page describes the process.
Name a cell
1. Select a cell.
2. In the Name Box, type a name.
3. Press Enter.
For each cell containing a parameter for your query:
Select the cell
Use Data>Get & Transform Data>From Table/Range. This will open the PowerQuery Editor. You will see something like this:
Right-click the cell in row 1 in the grid in the Power Query Editor and select 'Drill Down'. This converts the query on your parameter cell to a named value which can be used in other queries. It looks like this:
Now in Excel, use Data>Get Data and create your query from the database. I created a sample table in a local SQL Server database called AMSView, then connected to it with the query text in your post. When finishing the query connection, select 'Transform' so the query opens in the PowerQuery Editor.
Now, use Home>Advanced Editor and edit as follows by replacing the fixed values in the WHERE clause with concatenated names of your parameter cells, converted to text. For brevity, I have only used one parameter. If you've used capital letters in your cell names, remember, the M language is case-sensitive, so the concatenated parameter name must have identical casing to the named value.
let
Source = Sql.Database("localhost", "StackOverflowTest", [Query="SELECT #(lf) Model, Factory, TargetTime, TotalEvalMins #(lf)FROM #(lf) AMSView #(lf)WHERE #(lf) WeekNumber = " & Number.ToText(week_number)])
in
Source
Once your query is finished, use Home>Close & Load to load the results to the workbook. Now, when your parameter cells change, you need only refresh the query (right-click, refresh) and the data will be filtered as required.
I am creating table using google drive as a source and google sheet as a format.
I have selected "Drive" as a value for create table from. For file Format, I selected Google Sheet.
Also I selected the Auto Detect Schema and input parameters.
Its creating the table but the first row of the sheet is also loaded as a data instead of table fields.
Kindly tell me what I need to do to get the first row of the sheet as a table column name not as a data.
It would have been helpful if you could include a screenshot of the top few rows of the file you're trying to upload at least to see the data types you have in there. BigQuery, at least as of when this response was composed, cannot differentiate between column names and data rows if both have similar datatypes while schema auto detection is used. For instance, if your data looks like this:
headerA, headerB
row1a, row1b
row2a, row2b
row3a, row3b
BigQuery would not be able to detect the column names (at least automatically using the UI options alone) since all the headers and row data are Strings. The "Header rows to skip" option would not help with this.
Schema auto detection should be able to detect and differentiate column names from data rows when you have different data types for different columns though.
You have an option to skip header row in Advanced options. Simply put 1 as the number of rows to skip (your first row is where your header is). It will skip the first row and use it as the values for your header.
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
I'm trying to remove manual data entry and duplicate from a process. I have a report that is exported to excel and I'm trying to merge it with a prebuilt data set. I've tried the basic vlookup and if formulas. Where I am running into trouble is Column D lists an activity which may or may not be leave. When it is leave, I need the value returned to Column N and the Row corresponding to date and ID for that employee. Is this something that is even possible?
Sample Data
I have a table with a bunch of different fields. One is named period.
The period is not part of the raw data but I run a query when I import new data to the database that gives each record a period.
Now I need a delete query that will delete all the records that have the same period as what is selected in a combobox.
The values in the combobox come from a calendar table that contain all the possible values that could be in that period column at any time.
This is the basic query i thought would solve this issue but it tells me it is going to delete 0 rows every time I run it:
DELETE *
FROM PlanTemp
WHERE PlanTemp.period = Forms![Plan Form]!Combo163;
If you don't need the key field, just remove it.
Look at the "PROPERTIES" section and look at the column names.
Ether remove it there, or from your QUERY source.
You can also look at the Data section of the properties, and change your BOUND column, to Column 2... or whatever holds the data you want to use.