Importing data from Excel to SQL through webpage - Searching for phrases - sql

Very low-level programmer tasked with handling something I don't really understand, here.
My company has a webpage that takes a customer's Excel document, reads the data, and moves it to a SQL database. It isn't too sophisticated: it apparently looks for data in a particular cell (e.g., "The cell below the column named "OrderNumber") using Excel's Name Manager as a guide.
If IsDBNull(xlRS.Fields("OrderNumber").Value) OrElse IsNothing(xlRS.Fields("OrderNumber").Value) Then
strPartNumber = ""
Else
strPartNumber = Trim(xlRS.Fields("OrderNumber").Value)
However, each of the customers that will be using this page uses a slightly different Excel form. Although every one will have an "Order Number" column, its location on the form will vary from customer to customer. Most of them can't be persuaded to use our standardized template, so I need to find out if there's a better way to do this.
I'm not sure whether I'm putting this correctly, but using VB.net, is it possible to locate an Excel form cell by searching for a phrase (e.g., "Order"), instead of providing an exact location? If not, what could be used to get around that limitation?

You could you ADO.NET with the appropriate data provider if the data is in a tabular format. I do this sort of thing all the time and it works really well but only if the data is a simple table.

Related

Word Automation (VBA): Mail Merge Rich Text Format

I'm trying to do a Word MailMerge via VBA from my Access project. I created a clsWordMerge class so I could declare the Word application WithEvents, and take advantage of Word's MailMerge events, mainly the AfterMerge event.
Everything works fine, and I get the finished Word documents created, except that the source fields containing RTF data end up in the document not as formatted text, but instead the RTF codes and data:
<div><font face="Times New Roman" size=3 color=black>This is my <strong><em>test </em></strong>paragraph.</font></div>
Where I would expect to see:
This is my test paragraph
This happens whether I do a mail merge using a CSV file for my data source or an Access table.
So is there any way to correct this, and show the formatted data? I have access to all of the MailMerge events that Word provides.
Thanks..
No, there's no way to merge in RTF and have it display as Word content. RTF is not Word's native file format - a converter is required to display RTF as Word content.
Mail merge literally displays the data text, as it appears in the data source. There are no "advanced features" that enable selectively formatting the mail merge result.
Also, based on painful experience, relying on MailMergeAfterMerge is not advisable. When it was introduced, I tried it, was enthusiastic... until it started failing. The event is unpredictable and not reliable.
Given your requirements, a fully VBA-driven data transfer from Access to Word is a better investment of time and energy.
It probably can be done in certain circumstances, but I agree with Cindy Meister that the Mail Merge Events have not proven reliable (unless they have been fixed - I haven't actually used them for years). The following description of real and likely problems that I have previously encountered when trying this may help:
Not sure any of it can be done if you are merging to Email.
AFAICR the event you are likely to need (MailMergeBeforeRecordMerge) only fires each time Word processes the Main Document, not each time it processes a record in the data source. So if your Mail Merge Main Document "consumes" more than one Data Source record, e.g. because it uses { NEXT } or { NEXTIF } fields, it may be very difficult to get MaiMergeBeforeRecordMerge to do what you need. If I am right about that, that would be enough to put me off making the attempt.
in order to insert your "RTF", you must either
a. Have code that can interpret the "RTF" encoding and do all the right things necessary to insert it in your document, or
b. Have code that saves the "RTF" to an external file, then uses (say) Range.InsertFile to insert it and have Word interpret its contents, or perhaps
c. Use the clipboard to help you do the conversion.
If any of your rich text fields actually contained RTF, (a) would be difficult unless you could find a suitable library to help you. But in fact your sample shows a typical Access rich text field value, which is HTML-like. In fact, I think it is all standard HTML tagging that Word can interpret, but I don't know for sure. That could be much easier to interpret, especially if you only need the plain text (at its simplest, you might be able to throw away the tagging and insert the result.
If your rich text is longer than 255 characters (including the markup), Word's Document.MailMerge.DataSource.DataFields("the case-sensitive field name as Word sees it").Value will be truncated. So if you need the whole of the text or more of it, you'll have to get it somewhere else
The value inserted in the document using a { MERGEFIELD } field is not truncated to 255 characters so you may be able to get the value from the document. Word MailMerge may impose another limit (can't remember, perhaps 64Kb for an OLE DB connection, perhaps less, or perhaps there is a length limit for the data as a whole.
If you can't get the data from the document, you can get it directly from Access. Probably rather easily if your code is running in Access, but it can be done by using ADODB or perhaps ADO from Word VBA code. Your Mail Merge Data Source will need to retrieve the key fields of the record if you want to do that reliably. During development, if your application is running from Access but you are using VBA code in Word, you will probably also need to make sure that you save your Access database each time you modify your Access VBA code, otherwise Access opens the database exclusively and Word won't be able to retrieve data from it.
If you need to use (b) or (c) to save your HTML to a file then you may need to surround the HTML that you get from Access with tags and possibly tags to get Word to recognise the HTML. You could use Scripting.FileSystemObject to save the text, or perhaps ADODB.Stream if you are already using ADODB to retrieve Access data.
You should be able to use VBA Range.InsertFile to insert it, as long as you have some placeholder that tells you what to put it. Or you could use an INCLUDETEXT field and ensure that your Event code updates that field. A snag with the INCLUDETEXT approach is that if you merge to a new document, the INCLUDETEXT fields remain in the document so if you update them, they will all end up with the same result if you do not also create a new file for each source record.
i.e. quite a lot to think about!

How to get info from a listbox populated by an SQL query?

The title says the vague question, but, I will be more specific. I'm more of a hardware oriented person with some experience in VB, Java, and C++. I've been asked to modify a form in Access which uses SQL queries to take information from a database. The edit I am trying to make is to a form that has selections to narrow down the data it queries from to appear in a listbox in the center of the form (so, like radio buttons to specify a certain height or weight of an object in the database). The selections, as they are selected, concatenate additional specifications to an string that is then run as an SQL query. What I am trying to accomplish is to take a column of the narrowed-down data and find the maximum number in the column and the average of the numbers in the column. It seems really simple and know how to do that with regular lists and arrays, but I'm at a loss with my limited knowledge. I've considered making another SQL string, but in another part of the form, it was done with two strings (so, one to put the data in the listbox from the database and one to pull the specific column of the narrowed down data). That being said, it doesn't seem logical to me to pull the same sets of data each time, so I'm wondering if I can just pull the info from the already pulled info. At the bottom of the code set is this:
ItemList.RowSource = vSQLStock
ItemList = Null
ItemList.Requery
QuoteList.RowSource = vSQLNonStock
QuoteList = Null
QuoteList.Requery
Both vSQLStock and vSQLNonStock have their own sets of SQL strings that are run by this function, or as far as I can tell (I'm still studying up on SQL) put the narrowed down items in the QuoteList listbox. What I'm asking is if there is a way with any function that can be used to go through this information? Specifically, take a column and do some calculations on the data. I don't want to run another SQL query and bog down the server, but rather manipulate the data on the user's computer.
An example of what happens is the user selects whether the item is stock/nonstock, searches by the name/code/ID, and then puts all the matches on the listbox on the form. The listbox shows lines of data with columns like Name, Cost, Size, Weight, etc. of which I am looking to do calculations on the Cost column. Hopefully this is enough information; I appreciate any advice.
It looks from your question like you are attempting to get the Maximum and the Average from a SQL query.
the syntax:
SELECT MAX(myColumn) as MyMax, AVG(myColumn) as MyAverage
FROM myTable
WHERE . . .
Will return the Maximum and Average Values within SQL Server or within MS Access
If you're really bound and determined to do this locally with the data you've already retrieved from the server, then you probably need to look into working with adodb.recordsets. You can kind of treat it like an array. Kind of.
I really recommend querying the server for the data you need though. It's a world simpler.
dim rs as ADODB.Recordset
Set rs = QuoteList.Recordset
rs.MoveFirst
Do Until rs.EOF
' loop through recordset doing something
rs.MoveNext
Loop

Get column name & table name from value

Actually I have a new client & their Database has no standard naming conventions & the application is in classic asp.I have a form in which a form there are many values in the different textboxes, it it very difficult to trace the value come from which table.& also there is no erd.
I need a query from which I can get the table name with column name by giving Value.
Let's suppose I have a value having label name abc#= '6599912268'
& the new project has no ERD no standard of naming conventions... I need a fast way to know the abc# ='6599912268' is taking from which table & which column name.... like this the UI has many values which is time taken to trace manually
Is there any way to trace it?
The simple answer is no. There is no way to trace table/column it comes from by mere inspection of the value.
I suggest the following.
Find out what type of db your product is using. Where it is situatede, do you have access to it.
If you have access to the database, get to know the db structure. What each table is meant to store, the relationships etc. Speak to the db administrator or the business analayst to increase your knowledge on the product domain.
Once you have the db structure, try and compare the table to the page. Eg. The user details will most like be stored in a db table named 'Users' or 'Membership' Catch my drift?
Then have a look at the web sites source code. Look at the specific page you are at. Is the sql code embedded in the source code (asp page) or does it call a COM server or something similar? If you are "lucky" (and I say lucky for on the purpose of your problem that you are having) you fill find the sql code in the asp page.
If it calls a COM object or something similar, then you will have to dig up the source code for that, and that is most likely where you sql will reside.
There is no easy way to do this, you have to use a stored procedure to loop over all the tables in the database and search for the value, and it will probably take a while.
There's a stored procedure and examples here: Search all columns in all the tables in a database for a specific value. You'll see there are stored procedures for finding dates, strings, numbers.
Not possible, and If you search the column with the value, there is a possible chance that you get multiple columns with the same value, so how would you differentiate them and the same case is for the table.

#DBColumn in Lotus Notes

I've been tasked with learning Lotus Domino Designer - not sure what I did in a previous life, but it must have been pretty bad... - and was wondering how to do a lookup on a database to get some values for selections. As this information could potentially be used in a lot of the applications, I'd prefer it only to be in the one place.
I gather I can use #DBColumn, but what happens if an entry in that lookup changes? If the unique value of the lookup is the text, then the relationship would be broken, wouldn't it? Is there any way of mimicing the idea of relational lookups?
I'm assuming I'm looking at Lotus development from the wrong angle, as this seems to be a real limitation of look ups.
I haven't found any decent learning material on the interwebs, so would appreciate any help.
Ta
You would want to store a unique ID along with the textual value in the source database (not unlike what you would do in an RDBMS). Then, only store that ID in any referencing documents, and use a computed-for-display field to lookup the display value. (There is a performance consideration here - and you could "de-normalize" the data and store the ID and text value in the referencing documents, and do some asynchronous work to keep the values in sync - eg: using a scheduled agent that runs every night or every week).
If DB1 has the key values and DB2 has the documents which will reference these values, then in the form in DB2, you would still do a #DbColumn to lookup your value list. In the lookup view in DB1, concat the text value and ID with a pipe separator (textField + "|" + ID) in the first column. That will tell Notes to store only the ID value (what follows the pipe is the "alias" and is what will be stored).
Note: I would avoid using #DocumentUniqueID as the unique ID for these values, as the Document Unique ID will change if the documents are copied and pasted, or the entire database is copied, etc. You can use the #unique formula function in a computed-when-composed field to generate something close to a unique ID (almost like an identity column in sql).
If you need relational properties, look for non-Notes solutions. It is possible to get some relational behavior using document UNIDs and update agents, but it will be harder than with a proper relational backend.
Your specific problem with referencing to a piece of text that might change can to some extent be resolved by using aliases in the choice fields. If a dialog list contains values on the form...
Foo|id1
Bar|id2
...the form will display Foo but the back-end document will store the value id1 - (and this is what you will be able to show in standard views - although xpages could solve that). Using the #DocumentUniqueID for alias can be a good idea under some circumstances.
It depends on where your using the data. The #DBLookup or #DBColumn will work in Lotus Notes fields if the fields are set to be computed for display. That way they always get the most up to date information when you open the form etc.
If you make it so the data is saved on to the document then you will have to write some update code when you need to refresh the values.
The Lotus Notes help files for designer are pretty good, have a look at that.
SM
You could use a key or alias to store the relationship to your lookup value so if the value itself changes, the connection remains because the alias is intact. For example, if your lookup values were being stored as a collection of documents, I'd have the #DBColumn retrieve Document UNID|lookup value pairs. When in display mode, you could then retrive the value using #GetDocField. If the lookup values are in a different database, then you'd have to retrieve them for display using #DBLookup and construct a view that is keyed off of the UNID or whatever key you decide to use.The only drawback to this technique is that you wouldn't be able to display the field value in views as the actual value isn't stored in the document, just a reference to it. Using XPages, though, you COULD map the relationship into a dynamic datatable just like you would in a truly relational system.
It's tricky, but using LEI, you could also use Notes to front-end a relational backend system, also giving you the dynamic relationship you desire in your lookups.
Hope this helps!
The content of the lookup can change freely. A problem only arises (as it would on any other platform in the same circumstances) if the lookup key changes. You need to use a key that won't change. Human-readable text is an advantage, but if you want to be able to change your key description from, say, "Divisions" to "Business Units" and still have lookups work, you need to use an alias of some kind, which will presumably be mapped to your text description and only used internally. #Unique is pretty good for this, and gives a shortish key, if that is important to you. #DocumentUniqueID is most reliable, but as Ed pointed out, will change (must change - it's a new document) if you copy/paste or make a non-replica copy. This is easy to get around, though. Create a Computed-when-composed field (called, say, "LookupRef") on the form you are using for your reference document with the formula "#DocumentUniqueID". That will capture the ID at the time of creation, and it will not change on copy/paste etc. Use that as your key.

how can you parse an excel (.xls) file stored in a varbinary in MS SQL 2005?

problem
how to best parse/access/extract "excel file" data stored as binary data in an SQL 2005 field?
(so all the data can ultimately be stored in other fields of other tables.)
background
basically, our customer is requiring a large volume of verbose data from their users. unfortunately, our customer cannot require any kind of db export from their user. so our customer must supply some sort of UI for their user to enter the data. the UI our customer decided would be acceptable to all of their users was excel as it has a reasonably robust UI. so given all that, and our customer needs this data parsed and stored in their db automatically.
we've tried to convince our customer that the users will do this exactly once and then insist on db export! but the customer can not require db export of their users.
our customer is requiring us to parse an excel file
the customer's users are using excel as the "best" user interface to enter all the required data
the users are given blank excel templates that they must fill out
these templates have a fixed number of uniquely named tabs
these templates have a number of fixed areas (cells) that must be completed
these templates also have areas where the user will insert up to thousands of identically formatted rows
when complete, the excel file is submitted from the user by standard html file upload
our customer stores this file raw into their SQL database
given
a standard excel (".xls") file (native format, not comma or tab separated)
file is stored raw in a varbinary(max) SQL 2005 field
excel file data may not necessarily be "uniform" between rows -- i.e., we can't just assume one column is all the same data type (e.g., there may be row headers, column headers, empty cells, different "formats", ...)
requirements
code completely within SQL 2005 (stored procedures, SSIS?)
be able to access values on any worksheet (tab)
be able to access values in any cell (no formula data or dereferencing needed)
cell values must not be assumed to be "uniform" between rows -- i.e., we can't just assume one column is all the same data type (e.g., there may be row headers, column headers, empty cells, formulas, different "formats", ...)
preferences
no filesystem access (no writing temporary .xls files)
retrieve values in defined format (e.g., actual date value instead of a raw number like 39876)
My thought is that anything can be done, but there is a price to pay. In this particular case, the price seems to bee too high.
I don't have a tested solution for you, but I can share how I would give my first try on a problem like that.
My first approach would be to install excel on the SqlServer machine and code some assemblies to consume the file on your rows using excel API and then load them on Sql server as assembly procedures.
As I said, This is just a idea, I don't have details, but I'm sure others here can complement or criticize my idea.
But my real advice is to rethink the whole project. It makes no sense to read tabular data on binary files stored on a cell of a row of a table on database.
This looks like an "I wouldn't start from here" kind of a question.
The "install Excel on the server and start coding" answer looks like the only route, but it simply has to be worth exploring alternatives first: it's going to be painful, expensive and time-consuming.
I strongly feel that we're looking at a "requirement" that is the answer to the wrong problem.
What business problem is creating this need? What's driving that? Try the Five Whys as a possible way to explore the history.
It sounds like you're trying to store an entire database table inside a spreadsheet and then inside a single table's field. Wouldn't it be simpler to store the data in a database table to begin with and then export it as an XLS when required?
Without opening up an instance Excel and having Excel resolve worksheet references I'm not sure it's doable at all.
Could you write the varbinary to a Raw File Destination? And then use an Excel Source as your input to whatever step is next in your precedence constraints.
I haven't tried it, but that's what I would try.
Well, the whole setup seems a bit twisted :-) as others have already pointed out.
If you really cannot change the requirements and the whole setup: why don't you explore components such as Aspose.Cells or Syncfusion XlsIO, native .NET components, that allow you to read and interpret native Excel (XLS) files. I'm pretty such with either of the two, you should be able to read your binary Excel into a MemoryStream and then feed that into one of those Excel-reading components, and off you go.
So with a bit of .NET development and SQL CLR, I guess this should be doable - not sure if it's the best way to do it, but it should work.