I have got an upload button on my webportal through which the customers can upload a csv file. After uploading the .csv file the csv data will be displayed on datagrid view on the screen, After uploading, the customer is suppose to verify data by clicking on verify button which basically checks the uploaded data have got the right data in particular columns against the existing database.What I want to do is to verify particular column like (productid) against the dbo.existingcustomers*. If the respective productid is present in thedbo.existing customer then status(column in the dbo.exisitingcustomers) should be populated with "ok" If the productid is notpresent in the dbo.exisiting then the Status(column in the dbo.exisitingcustomers) should be populated with "Please Check Your productid". After checking each row of the uploaded .csv file the system should display the data with an extra column named Status which will show the above messages for each row.Any help will be highly appreciated.I can upload the csv file and insert the data into sql server but I am having problems with checking the existing productid. As I want to make sure that the customers enterthe right productid which is present on my sql server, If they upload the wrong one which does not exist on the live database, they should amend it in order to proceed to next step. At present the customers have all the respective productid, I just want to check it if by mistake they typed the wrong productid which will save a huge amount of time*dbo.existingcustomers = table(sql server
I have got an upload button on my webportal through which the customers can upload a csv file. After uploading the .csv file, the csv data will be displayed on datagrid view on the screen, After uploading, the customer is suppose to verify data by clicking on verify button which basically checks the uploaded data have got the right data in particular columns against the existing database.
What I want to do is to verify particular column like (productid) against the dbo.existingcustomers*. If the respective productid is present in the dbo.existing customer then status(column in the dbo.exisitingcustomers) should be populated with "ok" If the productid is not present in the dbo.exisiting then the Status(column in the dbo.exisitingcustomers) should be populated with "Please Check Your productid". After checking each row of the uploaded .csv file the system should display the data with an extra column named Status which will show the above messages for each row.
Any help will be highly appreciated.
I can upload the csv file and insert the data into sql server but I am having problems with checking the existing productid. As I want to make sure that the customers enter the right productid which is present on my sql server, If they upload the wrong one which does not exist on the live database, they should amend it in order to proceed to next step . At present the customers have all the respective productid, I just want to check it if by mistake they typed the wrong productid which will save a huge amount of time.
If any one can help me to write a query which can sort out this issue.
*dbo.existingcustomers = table(sql server)
In general, you'd need to write code to cycle through the datagrid and execute a query to check each field - something like
select * from [dbo.existingcustomers] where productid = ?
Without further information about your database it's tough to be clearer.
Share and enjoy.
How many product id's are exist in the dbo.existingcustomers table? How many rows are exist in a typical csv file?
If user upload a file which contains more than 5-10 rows and you have not more than 100-200 product IDs in the table, the better way is select all IDs at once then search through collection of IDs in your code, because multiple SQL queries in loop is not a good idea.
Moreover, even if you have 1000-2000 product IDs in the table, you can use this way, but need to use more intelectual algorithm for looking ID in a collection of IDs, like binary search (for example, Array.BinarySearch in .net framework).
Related
I'm new to PDI and Kettle, and what I thought was a simple experiment to teach myself some basics has turned into a lot of frustration.
I want to check a database to see if a particular record exists (i.e. vendor). I would like to get the name of the vendor from reading a flat file (.CSV).
My first hurdle selecting only the vendor name from 8 fields in the CSV
The second hurdle is how to use that vendor name as a variable in a database query.
My third issue is what type of step to use for the database lookup.
I tried a dynamic SQL query, but I couldn't determine how to build the query using a variable, then how to pass the desired value to the variable.
The database table (VendorRatings) has 30 fields, one of which is vendor. The CSV also has 8 fields, one of which is also vendor.
My best effort was to use a dynamic query using:
SELECT * FROM VENDORRATINGS WHERE VENDOR = ?
How do I programmatically assign the desired value to "?" in the query? Specifically, how do I link the output of a specific field from Text File Input to the "vendor = ?" SQL query?
The best practice is a Stream lookup. For each record in the main flow (VendorRating) lookup in the reference file (the CSV) for the vendor details (lookup fields), based on its identifier (possibly its number or name or firstname+lastname).
First "hurdle" : Once the path of the csv file defined, press the Get field button.
It will take the first line as header to know the field names and explore the first 100 (customizable) record to determine the field types.
If the name is not on the first line, uncheck the Header row present, press the Get field button, and then change the name on the panel.
If there is more than one header row or other complexities, use the Text file input.
The same is valid for the lookup step: use the Get lookup field button and delete the fields you do not need.
Due to the fact that
There is at most one vendorrating per vendor.
You have to do something if there is no match.
I suggest the following flow:
Read the CSV and for each row look up in the table (i.e.: the lookup table is the SQL table rather that the CSV file). And put default upon not matching. I suggest something really visible like "--- NO MATCH ---".
Then, in case of no match, the filter redirect the flow to the alternative action (here: insert into the SQL table). Then the two flows and merged into the downstream flow.
Before I begin I found this to be most relevant with the research I have done.
How to split the data from one column into separate columns using the contents of another column in SQL
Attached are pictures of my progress so far. How can I display this information such as it is shown in the excel file without disrupting the GROUP BY filter in my Query?
It's a Fishbowl Database, newest version. I am running the queries through Flamerobin which you see in the picture. Trying to organize the query to display correctly so I can format it into 'iReports' and export it into an excel spreadsheet like the one shown. Maybe there is some part of this that would better be done in excel?
Notice the numbers for Qty are different, that's ok right now.
My reputation is too low to post pictures I am sorry. Here are the two JPGs in my Dropbox. I really appreciate the help.
https://www.dropbox.com/sh/r2rw5r2awsyvzs9/AAAXXg27CMPOYtZFqPX3Dx6la?dl=0
I have a table in Access linked to a SharePoint list. The table is comprised of about 15 fields whose contents are originally pulled from another data source (in Excel format). There are an additional 10 or so fields after the original 15 that make up a questionnaire (added via SharePoint) that contain answers to questions about the first 15 fields.
The data in the first 15 fields needs to be updated periodically when new data from my external source is available to download. A lot of the information will remain the same, however some of the fields within each of the rows will change and need to be updated. It is also important that the 10 fields that contain the questionnaire are not modified at all during this process.
Is there a way for me to easily update the cells that have changed using an Update query or something similar? The data does have a unique identifier column (ID NUMBER) that is present on the current SharePoint list and the external data source.
I was thinking from a logical standpoint to put the new external data into a table, find the ID Number in the SP list and new external data, compare the values in the rest of the row on the SP list to the row of the external data, and if a value is different update the cell with the value from the external data. Not sure how to accomplish this using Access queries though.
I really appreciate any help at all! If you need more information, please let me know. If you think there's a more logical way to do this, please let me know your feedback!!
Here's how to get started:
http://workerthread.wordpress.com/2009/02/03/using-access-2007-to-update-sharepoint-lists/
After you get the connection set up, it's just a matter of writing the queries correctly. If you need to run multiple queries periodically, you can setup a form with buttons, and attach some VBA code to the buttons that runs the queries.
MS Access - execute a saved query by name in VBA
I need to set up a new company for automated data import. The utility has provided the data in a spreadsheet. (Image 1)
Based on this data, I need to create a stored procedure that will identify the correct meter, if it exists, and perform either an insert or update to the monthly data table. For automated utility data import, I want to make sure I restrict everything to a particular utility company.
The steps are the following ( I am having a hard time converting this to SQL)
1- I just want a script that identify the correct meter to see if it exists, basically check the Meter# column in the excel with the MeterNumber column in the Meters table.
2- The next step is perform either an insert or update to the MonthlyData table. This is a screen shot of all its columns.
3- Then I just want to make sure that I am restricting everything to the particular company which in this case Site1 since 2 different companies might have the same meter#. The UtilityCompany table contains 3 columns: ID, Name, UtilityType
I honestly do not know from where to get started, would anybody help me with the script? Thank you
You will want to:
perform a Bulk Insert operation to take your data from the excel file into a staging table.
write a query to select ALL rows for the corresponding utility company (notice I didn't see iterate over each row...). This select could be an update where you update an additional column to mark the row as an INSERT, or an UPDATE.
Then the last step (2 parts), retrieve all of the rows that were marked as INSERT, and insert those into your table. Then grab all rows that were marked with an UPDATE, and update their corresponding values based on your matching criteria.
I'm hoping someone might be able to help me out with this one - I have 24 files in CSV format, they all have the same layout and need to be joined onto some pre-existing data. Each file has a single column that needs to be joined onto the rest of the data, but those columns all have the same names in the original files. I need the columns automatically renamed to the filename as part of the join.
The final name of the column needs to be: Filename - data from another column.
My current approach is to use a foreach container and use the variable generated by the container to name the column, but there's nowhere I can input that value in the join, and even if I did, it'd mess up the output mappings, because the column names would be different.
Does anyone have any thoughts about how to get around these issues? Whoever has an idea will be saving my neck!
EDIT In case some more detail helps with this... SSIS version is 2008 and there are only a few hundred rows per file. It's basically a one time task to collect a full billing history from several bills which are issued monthly.
The source data has three columns, the product number, the product type and the cost.
The destination needs to have 24*3 columns, each of which has a monthly cost for a given product category. There are three product categories, and 24 bills (in seperate files) hence 24*3.
So hopefully I'm being a bit clearer - all I really need to know how to do, is to change the name of a column using a variable passed in from the foreach file container.
I think the easiest is to create a tmp database (aka staging db)
to load data from xls file to it and to define stored procedures where you can pass paramas (ex file names etc) and to build your won logic ...
Cheers Mario