I am looking for any tips or resources on importing from excel into a SQL database, but specifically when the information is NOT in column and row format.
I am currently doing some pre-development work for a project, and in most ways I would like to use SSIS for the project. The one area that my research and googling is leaving a big question mark over is the import of the excel sheet.
In short our users are using excel as a simple calculator / application. Values are entered into specific cells in the sheet and then in other cells formulas come up with the final answers. The sheet has been "beautified" to make it easier for users to use (i.e. white space, merged cells, pretty colours, etc). I need a mechanism to get both the raw values and the final answers and import them into a SQL database.
There is a ton of information on the trials and tribulations of importing column and row info into Excel, but have any fellow stack-overflowers had experience with this? Is SSIS appropriate for this (the other viable option would have to be integration into a .NET service).
Thanks!
What about adding a sheet to the workbook that pulls the answers from the calculation sheet and shows them in a tabular format, and then use the magic of SSIS to suck in the values from that new sheet?
If you can't modify the worksheet, you might want to look into Interop services or something where you can specify the exact cells you want to pull data from.
I'd try the first route if possible, though another option might be to simply recreate their Excel calculator as a .NET app...
A SSIS Excel data source uses worksheet or range -- think table with column names. You could also consider using VBA from within Excel to push data into your DB.
SpreadsheetGear for .NET will let you load an Excel workbook, put values in cells, calculate, get results as raw values or formatted text and more. This might be your best option if the data is not suitable for SSIS - but it will require that you use .NET (C#, VB or any .NET language) to automate your process. The advantage of SpreadsheetGear is that you then have a solution which does not depend on anything other than .NET (of course it depends on SpreadsheetGear, but SpreadsheetGear can be deployed royalty free with your application using xcopy deployment or any other deployment method).
You can see live samples here and download the free trial here.
Disclaimer: I own SpreadsheetGear LLC
Related
I've created an Excel 2010 workbook with a number of sheets. VBA code that performs various data manipulations is in a couple of modules, and also attached to a couple of forms. The workbook is being distributed to a couple dozen people in different departments to use. They will be populating their workbook with their own department-specific data.
If I need to distribute an update to the code (either a bug fix or some new function), how can that be done? I don't want the users to have to reenter or copy/paste all their data into the 'new' workbook - I'm essentially looking for a method to update the VBA Project that's inside their existing workbook.
You could create an additional helper workbook Help.xlsm, that has its file attributes set to read-only.
Move all the vba code, that you might need to change in the future, into Help.xlsm
The file that you distribute then needs a reference adding to Help.xlsm
Now in the future changes can be made to Help.xlsm and they should appear in the client's files.
Above assumes all your customers are on the same network so that you can store Help.xlsm somewhere that is accessible to all their files.
This article explains it better than me:
http://www.excelguru.ca/content.php?152-Deploying-Add-ins-in-a-Network-Environment
You would need to export the modules and forms and manually import them into the existing workbooks. I used to have to do this for some projects i worked on.
Alternatively you would need to write some helper code to import the old data into a newly published workbook, but this depends on how the data is organised of course. Again this is another approach I took for a different project.
You can also do this procedurally. Ive used this for small patches.
http://www.cpearson.com/excel/vbe.aspx
The managing director at our company wants me to produce an automated monthly document that saves to a certain place on our system so that he doesn't have to manually input all of the data. I have set up so that the document can save to the correct place in the correct format but my knowledge of VBA is not great.
Tackling this from a 1 question at a time point of view I suppose my question would be is it possible to create 1 very long macro that will accomplish many different tasks over several workbooks. For example we have a report that comes from our ERP (Baan) and shoves all of the data into one cell. Is it possible to create a macro that will accomplish formatting text to columns, then copy data from a cell based on a row reference and then take said data and paste it in to a different workbook? Would it then be able to save the workbook all from just running one macro and if so how long will all of that take once the macro is executed?
Yes I believe this should be do-able, keeping in mind that the file names + location remain the same (otherwise you'll have to edit each month). Create different Subs/Functions and call them in one main macro.
The easiest way is probably to do it step by step. Record macro's and see whether that already helps you out and if not use google & stackoverflow for help! :)
it is entirely possible - but in my Opinion VBA is not well suited to the task. The editor is atrocius at best and it is easy to produce highly specific "spaghetti code".
File operatione are possible, but are not nice. Error handling is 80s style with lots of goto.
So if you want to build something maintainable, build an external Application using Interop or epplus (.net package for reading /writing to excel documents) or an .net addin for office.
I have an Excel workbook that I want to use as a template. It has several worksheets setup, one that produces the pretty graphs and summarizes the numbers. Sheet 1 needs to be populated with data that is generated by another program. The data comes in a tab delimited file.
Currently the user imports the tab delimited file into a new Workbook, selects all and copies. Then goes to the template and pastes the data into sheet1.
This is a large amount of data, 269 columns and over 135,000 rows. It’s a cumbersome process and the users are not experienced Excel users. All they really want is the pretty graphs.
I would like to add a step after the program that generates the data to programmatically automate the process the user currently must do manually.
Can anyone suggest the best method/programming language that could accomplish this?
POI is the answer. Look at the Apache website. You can use java to read the data and place it in cells. The examples are very easy.
You can can solve this, for example, by a simple VBA macro. Just use the macro recorder to record the steps the user does manually now, this will give you something to start with (you probably will have to add a function to let the user choose the import file).
You said you have some data generated by another program. What kind of program? A program that you have developed by yourself and where you can add the excel-import functionality? Or a third party program with a GUI that cannot be automated easily?
And if you really want to create an external program for this task - choose whatever programming lanuguage you like as long as it can use COM objects. In .NET, you have the option of using VSTO, but I would only suggest that for this task if you have already some experience with that (but than you would not ask this kind of question, I think :-))
Look here:
Create Excel (.XLS and .XLSX) file from C#
There's NPOI (.NET Framework version of POI) so that you can code in C# if you want.
If you use two workbooks - one for data and one for graphs - and don't update links automatically you can use a macro to get the data (maybe an ODBC connection if the file is in a format it can read - long shot) and then link the charts to the data workbook.
Use a macro to update the links and generate the charts and then send them out and hope no one updates the links.
In my app, I'm downloading a spreadsheet from FTP, moving the data read from the spreadsheet to a DataTable, and, depending on certain conditions, emailing a new spreadsheet (one that contains certain rows from the 1st spreadsheet).
My problem is creating the spreadsheet that will be mailed. I can't seem to work out how to add the row from the DataTable (originally from the spreadsheet that was downloaded) into the spreadsheet that's going to be mailed.
DS.Tables(0).Rows.Add(ObjSheet.Range(workTable.Rows(i)) - 1)
Is what I've got at the moment, but it's not working. workTable is the DataTable where the rows from the spreadsheet were copied to. DS is the DataSet where I'm putting the rows from the specific selection.
I'm pretty sure I can simplify the flow of data by eliminating either the DataTable or the DataSet.
To eliminate any confusion, I'm using VB2005 - Wish I could use 2008 for this, LINQ simplifies so much...
SpreadsheetGear for .NET can help you do this.
See the live ASP.NET (C# & VB) "Excel to Datagrid" samples here. There are more samples - some of which might prove useful - here.
Disclaimer: I own SpreadsheetGear LLC
One of the answers may be to not program with office - if you can use the Office 2007 file formats (and given the existence of the Office Compatability pack there's no fundamental reason why you shouldn't although there may be issues within a specific environment) then you can use .NET code, System.IO.Packaging and the Open XML Format SDK 1.0 - that's .NET 3.0 which is just libraries and therefore useable from VS2005.
More pragmatically, don't be so keen (at least in the first instance) to do away with the intermediate steps - something I need to remind myself at regular intervals.
If you start with the following:
1) Load the data from the source
2) Process the data from the source
3) Write the result out to the destination
Then you'll be able to demonstrate at steps 1 and 2 that you have what you want, and step 3 becomes more manageable in that you're just trying to move known data from app to file.
Hmm, that's more "thoughts" than "answer" - but I hope it helps.
We have built an Excel 2003 template that asks the user to select an XML file which is then imported into an XML Map and used to populate a worksheet.
Unfortunately it turns out that the users have Excel Standard Edition, which does not include the XML import functionality - namely the Workbook.XmlImport function.
Users are able to open the XML file via File -> Open etc, however this opens it up as a new workbook.
Does anyone know of a way to get around this? How can we get the XML data to populate the XML Map when the users have Excel 2003 Standard Edition?
Quick workaround:
Let them select the file to open
open that file into a new workbook using vba,
copy the contents of the new worksheet (or read it all in to vba recordsets to parse)
paste it back into your active worksheet (maybe after you've done stuff to it in VBA)
close the xml worksheet, not saving any changes.
That would get around it, but it won't be quite as flexible. Depends how much you need to do with it.
Any help?
The best course may be to buy a used version of 2003 Professional. You can get one for about $50. If you have a ton of users, this obviously doesn't scale. If you have four users, this would be a $200 fix to your problem.
If you're on the verge of upgrading, you might accelerate that process and make sure the 2007 flavor has XML support. I want to say that all of 2007 has XML, but definitely investigate it before you take my word.
Option 3 involves a lot of time and code. You can write VBA to import and refresh XML (or CSV or something else). Obviously you'd have to weigh the cost of coding to finding and buying enough used licenses.