I am running into an issue, while I import an excel file into sql server 2005 using OpenRowSet, it works fine when excel file is closed, If excel file is open it gives an error message.
I have an excel file which is being updated 8 to 10 times in a minute by a third party software, I have to import this excel file into sql server 2005 very 10 seconds.
Any help would be higly appriciated.........
Thanks, Yogi
How do you know which rows you want to import if the file is constantly being updated? Do you have some kind of sequence number so that you can detect gaps?
If you really want to run with something like you have suggested then why not set up a process to copy the Excel spreadsheet file periodically and then get your OPENROWSET code to read from this copy.
I think it's a general principle that a file cannot, or should not, be accessed while it's open and being written to by another program.
How is the conflict resolved if a part of the file is simultaneously written and read?
That being said, I've come across a situation on one of the computers I use (not Excel, not Windows, not even a PC, so it's hardly relevant) where I could download open files by FTP.
But I use another practically identical computer where this is not possible. I don't know why, but it seems to me to be the normal situation.
This idea of copying a file, and operating on the copy fits what pjp has said. If you can get away with it ... The frequency of access you require, Yogi, seems to invite conflict between reading and writing.
Is it bad if you just stop the import when you get the error and wait 10 seconds until the next import?
Related
We have wrote an application with C++/Qt which reads some data from multiple excel files and then generate a word report. We have used benefits of Office automation server to read/find and copy data from excel files and write them into a single word file.
The application works fine without any significant problem in my office computers. Yesterday I've decided to copy the app into my laptop which has same OS and office version. After running the app, it seems that somethings wrong my OS/Office. None of the data are copied into word file. After debugging I've figured out that the app actually can read/write office files but failed to call CopyPicture method of automation!!!
I wanted to copy the range as picture, so I have used "CopyPicture" method. but it seems that the vba fails to call copyPicture on my laptop.
I'll be very appreciated for any hints. Thank you
I've found the problem and it's with excel docs which are used to copy data from. In these files, there are some embeddedCharts. after removing these charts the code works fine!!!! (very odd). Anyway, I've tried to make excel file visible in order to see if the operation goes well or not. After running (with embeddedCharts) the code worked fine (more odd!!)
excel->dynamicCall("Visible", true );
I don't know the reason but with making excel file visible, the error with copyPicture is gone!
I have been working on building a new database. I began by building the structure within the database it is replacing and populating this as I created each set of tables. Once I had made additions I would drop what had been created and execute the code to build the structure again and a separate file to insert the data. I repeated this until the structure and content was complete to ensure each stage was as I intended.
The insert file is approximately 30mb with 500,000 lines of code (I appreciate this is not the best way to do this but for various reasons I cannot use alternative options). The final insert completed and took approximately 30 minutes.
A new database was created for me, the structure executed successfully but the data would not insert. I received the first error message shown below. I have looked into this and it appears I need to use the sqlcmd utility to get around this, although I find it odd as it worked in the other database which is on the same sever and has the same autogrow settings.
However, when I attempted to save the file after this error I received the second error message seen below. When I selected OK it took me to my file directory as it would if I selected Save As, I tried saving in a variety of places but received the same error.
I attempted to copy the code into notepad to save my changes but the code would not copy to the clipboard. I accepted I would lose my changes and rebooted my system. If I reopen this file and attempt to save it I receive the second error message again.
Does anyone have an explanation for this behaviour?
Hm. This looks more like an issue with SSMS and not the SQL Server DB/engine.
If you've been doing few times, possibly Management Studio ran out of RAM?
Have you tried breaking INSERT into batches/smaller files?
I came by this page and was thinking about the best method to distribute my VbaProject.OTM file (located into %appdata%\Microsoft\Outlook\) to a bunch of ~30 users at my office. Is it better to simply copy/paste the OTM file onto the network and then copy/paste it back to all users' computers (manually or with a .bat) OR would it be better to use the method described in the link above to generate a OPS file and import it back with Proflwiz.exe? What's the difference?
We are all on Microsoft Office Outlook 2003 actually, we might upgrade to 2007 one day but still years from now.
Finally came up with some elements to deploy a Outlook VBA Project. There are a lot of ways to do this, but the easiest way to do so without installing anything and keeping the same methodology would be to run a OTM file directly from a server. I found out that the process outlook.exe has a parameter altvba that allows to specify another path to run the OTM file from. Here is en example:
outlook.exe /altvba "\\myServer\myFolder\myFile.otm"
This allows me to update only one file to get all computers updated. Obviously, if the file is big and the server's ping is on the high side, it may delay the launch of Outlook. The other problem with this method is that everybody will have to shut down Office if you want to update the OTM file on the server (and if you do work in an office where everyone uses Outlook, you do know that it is impossible to get everyone to shut it down at the same time, except if you code a macro to do so eventually). To prevent both those problems, I could setup a batch file to copy the server OTM file clientside everytime there is a new version (just have to check the NTFS last-modify attribute). This way, Outlook will boot with a local file, the batch file take 2-3 seconds to copy the file if needed (or will launch Outlook instantaneously) and there will be no problem updating the OTM file on the server. Users will have to start Outlook with the batch file (or with the slightly different outlook.exe path with the altvba parameter, so either way they need a different shortcut/file to start off the first time). One other advantage of the altvba is that it's still easy for the user to run Outlook without it (to see if the VBA is problematic or not in case Outlook is sluggish) and the file will remain unchanged after a Outlook reinitialization.
Others solutions include a COM complement that can be developed in a lot on languages including VB6 (no conversion needed from VBA). There is also a bunch of tools included into Microsoft Office XP Developer that could help getting the job done (not free however, especially if you need the most up-to-date version).
I'm creating a program in VB.net that does the following:
At a high level I receive a file in email, put the attachment in a monitored folder, import the text file to excel, format the excel, and then email the excel file to a list of recipients.
Here is my plan:
Completed: Outlook VBA to monitor all incoming email for specific message. Once message is received drop attached .txt file in a specific network folder.
Completed: (VB.net) Monitor folder, when text file is added begin processing
Not Complete: (VB.net) Import text file to Excel
Not Complete: (VB.net) Format Excel Text file. (add in a row of data, format column headers with color/size, add some blank columns, add data validation to some of the blank columns that allow drop down selections)
Completed: (VB.net) Save file.
Completed: (VB.net) Send file to list of recipients.
Obviously the items above that are not complete are the bulk of the work, but I wanted to get some advice on what some of you think would be the best way to approach something like this. The import and formatting of the file are causing me some problems because I just can't decide what would be the most efficient way to do this.
What I've thought of so far:
The way stated above. Import to excel -> format
Having a template excel that contains all of the formatting already done for me and attempting to transition the data to this document (no clue if/how I can do this). Is it even feasible? Have the template already created and then import the text file to a new excel file, then transition that data to the excel template?
Something I thought about, in terms of formatting the document, was to record a macro of me doing all of the formatting that I'm going to need and then attempt to convert that macro into my vb.net code, but I'm not sure if that will work. I will need to verify that the text file comes in the EXACT format every time correct?
I really appreciate any advice/suggestions that anyone is willing to give.
You will want to use http://epplus.codeplex.com/
It allows you to create an Excel file from scratch, without having to start Excel itself. Automating Excel will make the process slow and it lacks robustness (Excel process can hang or not close properly).
In addition, using a .Net library allows you to run it on a server or so where no Excel is installed. (Next step would be to inspect the mailbox via POP, IMAP or the Exchange API, so that part doesn't have to be run on a client machine either)
http://msdn.microsoft.com/en-us/library/kh3965hw(v=vs.100).aspx
You can also just use the Interops from MS to interact with Excel, Outlook, Word, etc. They're not difficult at all to use. I'm not familiar with CodePlex, so that may be a better route or an easier one. I just wanted to provide you with an alternative.
With Microsoft Office 2010 Interops you can not generate Office files from .net applications anymore.
You can manipulate data from existing Excel files so you need templates(your 4th point). Then Excel allows you to query some databases. You may be able to simulate one with your folder, otherwise I suggest to convert your .txt files into some databases. (3rd point)
If you do use an older version, you can crate your Excel files by loading them into an instance of Excel and manipulating them as you wish.
By the way I supposed your attached files would have some sort of format.
If you want to manipulate Excel files, I can recommand you the NPOI library found on CodePlex. It has several advantages over OLE-automation:
NPOI is not dependent on a specific Excel version.
Excel (or any other Office component) need not to be installed.
It is faster.
It works with both .XLS and .XLSX files.
We are using a third party software called excel writer. May not be what you are looking for becauseit needs to be license, but it is very fast and the clients does not have to wait for a chart or a data output. Because we have that tool, have not try anything else.
I've created a small application that basically reads and writes to a single Excel.exe process. It's basically a timer that records the time I use on projects and then store it in an Excel sheet. This works great, however, I've noticed that if I open Excel manually, work on some sheets and whatnot, save and exit etcetc, the process my software use gets broken or something. The same thing that happens if I manually close the excel.exe process and my software doesn't "know".
So I was wondering if it's possible to protect the excel.exe process somehow? To make sure it can't be closed or tampered with in the meantime?
Let me suggest an alternative approach that does not require you to have an Excel process running all the time (after all, this also consumes a lot of system resources):
Let your application record your information. Every now and then -- for example, after a work entry has been finished or a specific time has elapsed -- open the Excel sheet, write the data, and close it again (also closing the Excel process that you are automating). This save operation should not take more than a few seconds and it will (mostly) prevent the problem you are experiencing.
In fact, since Office automation is always a bit painful, an even better way would be to output your data without requiring an Excel process. To do this, you could use
one of the third-party Excel libraries available for .net,
a CSV or HTML file, which can be opened by Excel, or
open the Excel file as a database with ADO.NET.
You cannot protect a process, but you can check the process.HasExited property to find out whether the process has terminated and take action based on that.
Add an exception handler. Either call non-throw methods if possible.