in my everyday work, I am receiving data in Excel spreadsheets, which I need to insert into relational database.
To accomplish this, I prepare formulas which generate "insert" statement (I am using both insert and select statement for example to choose ID of all elements with specific label).
Because those spreadsheets are complex, they contain SQL commands in more than one column.
This is the point where problems begin - I cannot simply select all cells, copy them and paste into SQL Server (it will concatenate information from cells in the same row).
In most cases I'm preparing additional sheet where I'm collecting all statements in one column
(using simply formula which rewrite text from other cells). Unfortunately preparing such sheet is time consuming and might causing an error (for example if I forgot about column or I add rows).
Is there any more convenient way to do it?
I thought about writing a macro which collect all values from selected range.
Is it good idea or can I use something better?
You can do all that using VBA.
You know what are the rules so you have the business logic in your head. Now, just type the code to do it :)
If you want you can do the insert in the Excel using something like this.
Related
I have a large table in my SQL database with a few million rows so I can't just load it into excel. I'd like to setup a data connection in Excel to the database which I can do now, but I want to be able to open the excel sheet, type my search parameter in a cell and then have the matching rows returned from the database table into the excel sheet.
Does anyone know if that's possible? I can't find a way to do it without manually adjusting the search term in the connection properties because I don't know how to pass along what I typed in the cell to be the search term.
Ok figured it out, microsoft has a great guide for it here:
https://support.microsoft.com/en-us/office/create-a-parameter-query-in-microsoft-query-c67d9af7-c8a0-4bf7-937c-087cb25f7ad3
I have a collection of excel spreadsheets that are formatted... less than ideally.
I'm testing out some solutions involving SQLBulkCopy and OleDB, but I'm a bit concerned about how to handle the format of this sheet.
I was considering writing a custom Insert statement, but would like to see if there may be some easier way to implement a heuristic.
Below is a sample of the data I will be parsing:
The highlighted columns are the ones I'll be loading into the two tables. One table will hold order #s, and the other table will hold all the lines below that order number.
Any suggestions on tackling this would be lovely. The excel sheets are hand entered, so some weird cases exist (one order number with multiple carriers, which imposes the question of whether I should treat the first row with the order number as a line in the database structure I designed.
I'm implementing this importer within VB.net, to my dismay, to avoid being looked at funny by my coworkers :).
One approach would be to save the worksheet to a text file (e.g., CSV) and then use AWK to split it at the empty row. Some examples are in this SO answer: Bash how to split file on empty line with awk
You could then import the CSV files directly into the database.
Amusingly , if I wrote anything in VB.NET I'd definitely get looked at funny by my coworkers
So I'd use a library called EPPlus to read the excel and not have to worry about converting it. How you do the blank line detection is an open question- checking that the Value of ten cells on the row is Nothing or Empty would suffice. Then take the next row as your parent, and proceed with subsequent rows as children until the next blank
Take a look at this answer for more info on how to detect blank rows in Excel- if you get stuck turning any of the c# into vb shoot us a question. Online converters exist because the two languages are the same thing under the hood
I'm trying to run a duplicate check In which varying data is pulled from a website and compared to a master list, the master list being stored in Excel. The information from the website is read from a table in which has line breaks. These breaks are translated over to the data collection they are initially stored in. Some of the data from the website us eventually written to the master list in Excel. So when I read the master list back into Blue Prism to run a duplicate check, the rows that have line breaks are written into a collection as multiple rows (ex. I should have on 7 rows in my collections but am getting 42). Since the rows are not EXACTLY the same between the 2 collections, when it runs the automation does not recognize the duplicates.
The easiest way to solve this would be if I could make the collection rows have no line breaks as soon as the data is read. I've attempted to use the calculation stage to do so with no luck. I'm not sure if it is actually possible to do this, but would appreciate any direction.
Record an Excel macro to do the data sorting/cleaning in Excel (possibly Text To Columns, etc..) and then include the running of the macro as part of your Blue Prism process by using an action stage and the MS Excel VBO - Run Macro. Get the process to create an Excel instance (and create a handle data item from that stage), then use Open Workbook (whatever workbook you store your Macro in) and then use the MS Excel VBO - Run Macro (use the same handle created earlier and type in the name of the "macro").
It sounds like what is happening is that the MS Excel VBO is grabbing the data from the Excel Worksheet wholesale.
This is to say that it's accessing your Worksheet table, copying the cell values BUT not the cell formatting data, and then dumping the values into a BP collection.
Since it did not bring along any of the original cell formatting data to reference when it went to populate the collection it's just breaking up the values based on crturn/line breaks. Thus, your collection is organized based on that, and not on the original Worksheet cell.
So, with that said, on to a solution!
Solution 1
Brute force the organization of the incoming Excel cell data to the collection by looping over the Excel Worksheet cell-by-cell.
Run a loop, and in that loop have BP go into the Excel Worksheet and grab the first populated cell it comes across. Run a formatting/cleanup Calculation stage over the data. Dump the cell value into a single collection field.
Repeat.
This is...inelegant, expensive at best, and not at all recommended for any medium to large dataset. But it's definitely the best way to do string manipulation and value comparisons before it hits your collection. Since it sounds like your using a Master template then you as-well know what the expected format of your data should be.
This method will enable you implement Trim(), Concat(), or Split() in a Calculation stage to better organize your incoming data before you dump it into a collection.
This is also basically what I think you're already trying to do, but cell-by-cell instead of Worksheet row-by-row or table-by-table.
Solution 2
Clean up the table data you grab from the website before you dump it into the Excel Worksheet.
This is basically Solution 1, but in reverse. Simply format/cleanup your data before it hits you Excel Worksheet.
I'm not sure this is any better than Solution 1, but, you know, it's something...
Solution 3
Format the cell data IN the MS Excel Worksheet itself.
Basically rearrange the cells and cell data in the Excel Worksheet into a more predictable format by using the Split, Trim, Merge, or other actions included in the MS Excel VBO. You can also do this using the Data - OLEDB utility object, but that requires some pretty solid understanding of SQL syntax.
This would look like this using the MS Excel VBO:
Grab the Excel Worksheet data wholesale and dump into a collection
Count the rows/fields of the collection
Is that number consistent with the desired/expected format of your data?
If not, have the bot go back into the Excel Worksheet and reformat the cells by removing any carriage returns/line breaks/whatever else
Repeat.
However, I'm always reluctant to reformat any original source, as it's then hard to figure out what wrong and where it went wrong when you've changed the original structure of your data. So it's best to always make a copy of the Worksheet before you make any manipulation.
Unfortunately I don't have access to my BP environment at the moment or I'd provide you with the act object actions you'd need to do any of this, my bad. Once I do I'll update this answer.
you most probably going to think "what an idiot" but remember i never done any type of coding before so this is all new to me,
My problem are that i'm working on a HUGE excel sheet with loads of data that is not needed. i need to sort the data into a few columns, i only need column "A,K,AN,AQ" but in column "AS" i only need certain values (yes,no,blank) i only want the yes and blank values. like i said never done any coding before but i know that you can use an macro to do it so please help, how do i go about this?
before trying to get into macros, try to use functions with if else statements. They are quite easy to handle. Like: If (yes) then put it into X. Later, you could select all needed. Also, check the, how the dollar sign is used
use this links to see, if it is something for you.
One quick and dirty way of getting this job done would be to:
Delete the columns you don't need.
Select all cells in the range you're interested in, click the Insert menu, and choose "Table". If your columns have titles, select the box for "My Table has Headers."
-This turns your data into an array so that Excel recognizes that each row is an entry (instead of thinking that the cells are unrelated).
Now you can use the filter icon in the column headers to select and display only the rows containing the values in column X that you're interested in.
Note that there are some limitations to what the table feature is good for, so, as always, whether this is a good solution for you depends on what you want to do with the data.
to export data from an Access 2010 Database I would like to use an Excel template as seen below.
In the first row I can define columns that are in the database. In the second line I can define exactly 1 filter which will be used in the "where" statement.
When pressing a button in Access, a query that is based on the defined columns and filters should be executed and the result will be exported to the Excel.
I am honest, I am not that skilled in VBA. What would be the best point to start from.
Your question is way too broad and as such it should not even be here, but let me give you some points to look at.
First you'll need to get your data from the Excel file. If you are not skilled with VBA, then the easiest way would be opening it through VBA using an Excel.Application object and Workbooks.Open method, and get your values from your Range.
Create an SQL Select statement from your values.
Do not use just a SELECT query, make it a SELECT INTO query (a "make table query" as Access likes to call it). That way your results will be inserted into a (new) table, making it easier to output.
Export your new table to Excel. There are several ways to do this, search for DoCmd.OutputTo or DoCmd.TransferSpreadsheet.