VBA with sql query - optimization - sql

All,
I have two tabels: Cases(IdCase, Name) and Alerts(IdAlert, refIdCase, Profile).
One case can have multiple alerts connected by refIdCase.
I'm dipslaying the list in VBA listbox which shows case name and profiles assigned to this case.
Firstly I download 50 cases. Then for each recordset I'm finding profile names. Unfortunately it takes some time :( Is there any faster way to achieve that?
set rsCases = cn.Execute("SELECT TOP 50 * FROM Cases")
Do Until rsCases.EOF
Set rsProfiles = cn.Execute("SELECT DISTINCT TOP 50 Profile FROM Alert WHERE refIdCase = " & rsCases.Fields("IdCase").value & ";")
rsCases.movenext
UPDATE: I believe the problem is with our connection to sql server. We are located in Poland and the server is in North America. I performed the same action from computer located in NA and it took only 4 sec, but here from Poland it takes around 45 sec.
Thank you,
TJ

The problem is that you are sending 51 requests to the database. Send 1:
set rstCases = cn.Execute("SELECT c.IdCase, c.Name, a.IdAlert, a.Profile
FROM Cases c
INNER JOIN
(SELECT TOP 50 IdAlert, Profile
FROM Alerts
ORDER BY ???) a
ON c.IdCase=a.refIdCase
ORDER BY ???")
(Linebreaks are for clarity - don't put then in your code)

Related

How to show results by Category in SQL and display in classic ASP

I have a table from where I want to display records on the page grouped by categories in Classic ASP in such order that the Latest records are displayed on top. I am using MS SQL 2005 server.
For example table is like
ID Data1 Data2 Category
1 abc def test1
2 rrr 344 test2
3 ttt edf test1
4 www ghj test2
sql="select * from TABLE where Category='test1" and Category='test2' order by ID desc"
I want to display the all the data from category Test1 and Test2 on the page at different places. How to separate results by category?
Thanks,
If you want to display the results in different places, just run both SQL statements and store the results into 2 different Recordsets.
<%
set rs1=Server.CreateObject("ADODB.recordset")
set rs2=Server.CreateObject("ADODB.recordset")
SQL1="select * from TABLE where Category='test1' order by ID desc"
SQL2="select * from TABLE where Category='test2' order by ID desc"
rs1.Open SQL1, ConnectionString
rs2.Open SQL2, ConnectionString
Test1_ID = rs1("ID")
Test1_Data1 = rs1("Data1)
Test1_Data2 = rs1("Data2")
Test1_Category = rs1("Category")
Test2_ID = rs2("ID")
Test2_Data1 = rs2("Data1")
Test2_Data2 = rs2("Data2")
Test2_Category = rs2("Category")
%>
This code assumes you already have set your connection string = ConnectionString and have already opened it, and side note, you can share the connection, or you can make one for each recordset, if its not that much data it won't matter, but if its a larger chunk of data, it wouldn't hurt to have 2 connection objects.
Now when you go to display the results you can just rs1.MoveNext to move to the next record in the first set, and rs2.MoveNext to move to the next record in the second recordset. Your question was how to be able to display the records in different places, so now you have the variables names and a way to move to the next record, if you know how to move through a loop advancing the recordsets until you reach the last record, you just do that for both recordsets, and as each one is being done and displayed, they will be separate until you write the part to display the results from rs2 as well.
Closing Notes - In case you need to be specific in your SQL, you can specify the source in the format of database.tablename. So in the example you would do something like this for SQL1:
select * from DATABASE.TABLE where Category='test1' order by ID desc

LEFT JOIN is killing my query - how to speed up?

I have two tables:
a list of documents created in our system (offers and invoices) when something is logged as "done".
BELEG_ART
BELEG_NR
DATUM
BELEG_TYPE
160
337691
11.01.2021
Invoice
10
195475
04.01.2021
Offer
20
195444
04.01.2021
Confirmation
A list of transactions (sales etc) with article information AND the document info
ANUMMER
KDNR
NAME1
REC_LIST
181557
59301205
Fred
332240
195973
59306391
John
338225
189661
59304599
Steve
335495
189718
49302475
Ed
196483
59303491
Mark
338204
190021
49302595
Jones
You can see that the Offers and Confirmation... they start with a "1". Invoices with "3".
I need everything to be linked and identified by it's ANUMMER with "1". Later on, I'll pull other tables based on this number, thus it's the critical point for me.
The problem is - in the documents table, when you see the invoice, you don't see the ANUMMER. You only see the "3".
So, I have created a join as below to pull everything together.
SELECT
DAB700.BELEG_ART,DAB700.BELEG_NR,DAB700.DATUM,DAB700.BUCH_DATUM,
case // rename the documents to something more meaningful
when DAB700.BELEG_ART = 10 then 'Angebote'
when DAB700.BELEG_ART = 20 then 'Auftrag'
when DAB700.BELEG_ART = 60 then 'Lieferschein'
when DAB700.BELEG_ART = 160 then 'Rechnung'
else 'not defined'
end as "BELEG_TYPE",
DAB050.ANUMMER,
case // if the document is an offer, then copy it to order_number. If it's a invoice, copy that number to order_number.
when DAB700.BELEG_ART = 10 then DAB700.BELEG_NR
when DAB700.BELEG_ART = 160 then DAB050.ANUMMER
else 'NA'
end as "order_number"
FROM "DAB700.ADT" DAB700
// if the document is an invoice, then join to the table DAB050 and reference the same key field REC_LIST.
left join "DAB050.ADT" DAB050 on
Case
When DAB700.BELEG_ART = 160 then DAB700.BELEG_NR = DAB050.REC_LIST
End
WHERE (DAB700.DATUM={d '2021-01-12'})
So - to my problem and question: when running this join query, it's much slower than I'd like (even with small datasets). Is there a way to restructure this, so it's faster?
Long story short - to simplify:
I want to add some columns to table 1
when table 1-BELEG_NR starts with a "1", good - just move the number into a new column
when table 1-BELEG_NR starts with a "3", then I have to link to table 3, and pull in the ANUMMER
thanks for your help
The problem is that the join condition is not optimizable because 1. it is a complex expression, and 2. it involves a memo/clob columen (REC_LIST is probably a memo based on your other question). What that mean is that the joint condition has to be evaluated for each combination of rows in DAB700 and DAB050.
It seems to me that the join condition can be simplified to:
DAB700.BELEG_ART = 160 And DAB700.BELEG_NR = DAB050.REC_LIST
There is no need for the Case because the default case is null or false. That will cut down the number of rows in DAB700 participating in the join. However, the second part of the join is still not optimal if REC_LIST is MEMO or CLOB data type.
In any case, I would also suggest checking to see if REC_LIST should be a fixed length string column (CHAR, NCHAR, VARCHAR or NVARCHAR) instead of MEMO which cannot be indexed and thus not useful for optimization.

Visual Basic 6 and dbf: issues with 'join' and 'where'

I need to modify an old vb6 application that needs to import some data from a dBase IV database.
In the past the selection query involved a single table (dbf file) and used to work perfectly.
Now i need to edit this query to introduce a join on a second table, using multiple fields.
This is a simplified version of my code:
Dim cnn As New Connection
Dim rs As New Recordset
Dim sql As String
cnn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Extended properties=dBase IV;Data source=d:\100\db;"
sql = "..." 'see below!
rs.CursorLocation = adUseClient
rs.Open sql, cnn, adOpenDynamic, adLockOptimistic
debug.print rs.RecordCount
rs.Close
cnn.Close
These two tables have a typical master-detail structure; I checked the db documentation and inspected TABFAT01 and TABFAT02, so I can assume that:
Join between these tables is [TABFAT01] 1 <-> n [TABFAT02], and is done on
TIPDOC (text), ANNDOC (text) and NUMDOC (numeric) fields.
Each row in TABFAT01 has at least 1 joined row in TABFAT02.
Each row in TABFAT02 has 1 joined row in TABFAT02.
TABFAT01 has 63 records.
TABFAT02 has 907 records.
First issue
My first query is:
select t.TIPDOC, t.NUMDOC, t.ANNDOC, t.DATDOC, t.LIBER03, c.LIBER04
from TABFAT01 t inner join
TABFAT02 c on t.TIPDOC = c.TIPDOC and t.ANNDOC = c.ANNDOC and t.NUMDOC = c.NUMDOC
This query returns 0 records.
If I change conditions order this way:
select t.TIPDOC, t.NUMDOC, t.ANNDOC, t.DATDOC, t.LIBER03, c.LIBER04
from TABFAT01 t inner join
TABFAT02 c on t.ANNDOC = c.ANNDOC and t.NUMDOC = c.NUMDOC and t.TIPDOC = c.TIPDOC
the query returns 907 records.
I don't understand how and why conditions order has impact on query results.
Second issue
If I add a where clause:
select t.TIPDOC, t.NUMDOC, t.ANNDOC, t.DATDOC, t.LIBER03, c.LIBER04
from TABFAT01 t inner join
TABFAT02 c on t.ANNDOC = c.ANNDOC and t.NUMDOC = c.NUMDOC and t.TIPDOC = c.TIPDOC
where c.LIBER04 = 'a'
the query returns 0 records.
However, if I run this query:
select * from TABFAT02 c where LIBER04 = 'a'
it returns 1 record, with TIPDOC = 'F2', ANNDOC = '2018', NUMDOC = 1854.
A subsequent query:
select * from TABFAT01 t where t.TIPDOC = 'F2' and t.ANNDOC = '2018' and t.NUMDOC = 1854
returns 1 record, as expected.
This happens for every field I tried to put in where clause with joined tables, except TIPDOC.
If I filter by TIPDOC, results are correct.
Third issue
The first time I run my code after opening vb6 IDE it gives the following error:
Run-time error '-2147467259 (80004005)': Selected Collating sequence Not Supported by the operating system.
(I actually use the Italian version of vb6, and the original error message says "Sequenza di ordinamento selezionata non supportata dal sistema operativo.".
I guess that the message I written above is the right match for the English version.)
This error is not displayed on subsequent runs, until I close and reopen vb6.
I agree with thx1138v2 about not specifically using the Jet OleDB driver, but one more specific to the source. You mention dBASE IV. Is this accurate or is it really with Visual FoxPro. You could confirm if the table has indexes and note (memo) fields the file names would be suffixed as
YourTable.dbf (actual table)
YourTable.cdx (compound index file)
YourTable.fpt (notes/memo file content is table has such columns).
Having confirmed, and if it IS Visual Foxpro, I would get the Microsoft driver directly from Microsoft
If the data table is being updated from a source outside VFP, then it is possible that the indexes may be out of sync as the other drivers won't necessarily open the index file (cdx) correctly to keep the new records in synch. So when trying to do the join based on one series of join clauses it does use the non-synched index. But changing the order forces to default to the natural order, thus seeing all possible records to resolve.
See and hopefully this makes sense and helps to tune-in on the missing link in your querying issue.
First clear up your error message. "Selected Collating sequence Not Supported by the operating system." probably means you didn't specify a collating sequence or the Jet.OLEDB driver doesn't support whatever collating sequence you specified.
I never got the Jet.OLEDB drivers to work with my application. My guess is because the files I was dealing with were created by FoxPro. So I used the Visual Fox Pro ODBC drivers. Here's an example connection.
"DSN=BCLVariance;UID=;SourceDB=C:\AMSI\BCLTemp;SourceType=DBF;Exclusive=Yes;BackgroundFetch=No;Collate=Machine;Null=Yes;Deleted=Yes;"
You will see that it allows you to control various parameters that you aren't controlling in your Jet.OLEDB connection string. The ODBC data source is setup under System Data Sources tab in ODBC Data Source Administrator.
I never understood exactly why but the sequence you state the table names in a SQL query does make a difference. It has to do with how the internal query interpreter handles them. You need to specify the detail table first and the master is joined to the detail. Like this:
select t.TIPDOC, t.NUMDOC, t.ANNDOC, t.DATDOC, t.LIBER03, c.LIBER04
from TABFAT02 c inner join
TABFAT01 t on t.TIPDOC = c.TIPDOC and t.ANNDOC = c.ANNDOC and t.NUMDOC = c.NUMDOC
That may also clear up your other problems. Give it a try.
BTW, you'll also need to change your initial code to:
if Not rs.BOF then
rs.MoveLast
debug.print "rs.RecordCount=";rs.RecordCount
else
debug.print "rs.BOF = True"
end if

Defaulting missing data

I have a complex set of schema that I am trying to pull data out of for a report. The query for it joins a bunch of tables together and I am specifically looking to pull a subset of data where everything for it might be null. The original relations for the tables look as such.
Location.DeptFK
Dept.PK
Section.DeptFK
Subsection.SectionFK
Question.SubsectionFK
Answer.QuestionFK, SubmissionFK
Submission.PK, LocationFK
From here my problems begin to compound a little.
SELECT Section.StepNumber + '-' + Question.QuestionNumber AS QuestionNumberVar,
Question.Question,
Subsection.Name AS Subsection,
Section.Name AS Section,
SUM(CASE WHEN (Answer.Answer = 0) THEN 1 ELSE 0 END) AS NA,
SUM(CASE WHEN (Answer.Answer = 1) THEN 1 ELSE 0 END) AS AnsNo,
SUM(CASE WHEN (Answer.Answer = 2) THEN 1 ELSE 0 END) AS AnsYes,
(select count(distinct Location.Abbreviation) from Department inner join Plant on location.DepartmentFK = Department.PK WHERE(Department.Name = 'insertParameter'))
as total
FROM Department inner join
section on Department.PK = section.DepartmentFK inner JOIN
subsection on Subsection.SectionFK = Section.PK INNER JOIN
question on Question.SubsectionFK = Subsection.PK INNER JOIN
Answer on Answer.QuestionFK = question.PK inner JOIN
Submission on Submission.PK = Answer.SubmissionFK inner join
Location on Location.DepartmentFK = Department.PK AND Location.pk = Submission.PlantFK
WHERE (Department.Name = 'InsertParameter') AND (Submission.MonthTested = '1/1/2017')
GROUP BY Question.Question, QuestionNumberVar, Subsection.Name, Section.Name, Section.StepNumber
ORDER BY QuestionNumberVar;
There are 15 total locations, with this query I get 12. If I remove a relation in the join for Location I get 15 total locations but my answer data gets multiplied by 15. My issue is that not all locations are required to test at the same time so their answers should default to NA, They don't get records placed in the DB so the relationship between Location/Submission is absent.
I have a workaround almost in place via the select count distinct but, The second part is a query for finding what each location answered instead of a sum which brings the problem right back around. It also has to be dynamic because the input parameters for a department won't bring a static number of locations back each time.
I am still learning my SQL so any additional material to look at for building this query would also be appreciated. So I guess the big question here is, How would I go about creating default data in this query for anytime the Location/Submission relation has a null value?
Edit: Dummy Data
QuestionNumberVar | Section | Subsection | Question | AnsYes | AnsNo | NA (expected)
1-1.1 Math Algebra Did you do your homework? 10 1 1(4)
1-1.2 Math Algebra Did your dog eat it? 9 3 0(3)
2-1.1 English Greek Did you do your homework? 8 0 4(7)
I have tried making left joins at various applicable portions of the code to no avail. All attempts at left joins have ended with no effect on info output. This query feeds into the Dataset for an SSRS report. There are a couple workarounds for this particular section via an expression to take total Locations and subtract AnsYes and AnsNo to get the true NA value but as explained above doesn't help with my next query.
Edit: SQL Server 2012 for those who asked
Edit: my attempt at an isnull() on the missing data returns nothing I suspect because the query already eliminates the "null/missing" data. Left joining while doing this has also failed. The point of failure is on Submissions. if we bind it to Locations there are locations missing but if we don't bind it there are multiplied duplicates because Department has a One-To-Many with Location and not vice versa. I am unable to make any schema changes to improve this process.
There is a previous report that I am trying to emulate/update. It used C# logic to process data and run multiple queries to attain the same data. I don't have this luxury. (previous report exports to excel directly instead of SSRS). Here is the previous logic used.
select PK from Department where Name = 'InsertParameter';
select PK from Submission where LocationFK = 'Location.PK_var' and MonthTested = '1/1/2017'
Then it runs those into a loop where it processes nulls into NA using C# logic
EDIT (Mediocre Solution): I ended up doing the workaround of making a calculated field that subtracts Yes and No from the total # of Locations that have that Dept. This is a mediocre solution because I didn't solve my original problem and made 3 datasets that should have been displayed as a singular dataset. One for question info, one for each locations answer and one for locations that didnt participate. If a true answer comes up I will check its validity but for now, Problem psuedo solved.

INSERT INTO increment control field

I'm using MS-ACCESS database.
From the prject I use and made some other questions the table NOEUDS and INFRA (that should be updated):
Table INFRA:
RECNO - NOEUD - SECURISE
00000008 C002 F
00000005 C009 F
00000001 C035 F
00000002 C001 F
00000003 C036 F
00000006 C012 F
00000007 C013 F
TABLE NOEUDS:
NOEUD TYPE_MAT N_AMONT
C021 COF 100
C022 COF 229
C023 COF 130
C002 COF 111
I want to create a query that checks on NOEUDS the nodes C* that are missing inside INFRA table, if not should be inserted a new one.
The problem is the RECNO field that works as a control and can not be duplicated (not primary key because all the DB is only a repositoty for the program that controls it).
All the fields are text so RECNO is a consecutive counting using HEX numbers as shown.
I used the query to select:
SELECT (SELECT MAX(CINT(INFRA.RECNO))+1 AS N FROM INFRA),
NOEUDS.NOEUD, "F" AS Expr2
FROM NOEUDS
WHERE (((NOEUDS.NOEUD) Like "C*"
And (NOEUDS.NOEUD) Not In (SELECT NOEUD FROM INFRA)));
The result was:
9 C021 F
9 C022 F
9 C023 F
SHOULD BE:
9 C021 F
A C022 F
B C023 F
I need some help on this one so I can insert the correct RECNO in hexadecimal counting after 00000019 passes to 0000001A and so on.
thanks in advance
UPDATE 1:
The program we use uses a Access database as storage. When I add a noeud using the program I have to insert some more info using the menus needed for the maps and as built information. The problem is that a lot info is redundant and the program can not handle it automatically. I am trying to work lees and insert the possible information using querys.
Every time I insert a noeud in noeuds table, is needed to insert a line in INFRA table only with RECNO (sequential counting from the last one), the NOEUD and some other info (to complete the autocad table tag). Since I have hundreds of Cxxx, Bxxx, Pxxx, Gxxx equipments I sabe for each project some hour of boring work.
I need help on counting a sequential way of adding RECNO for each NOEUD found in NOEUDS table that will be inserted in INFRA table.
UPDATE 2:
I'm inserting each noeud by hand. Is it possible to join in a way that it takes the list from the noeuds that I want to insert and insead of doing 1 by 1 it takes the list and does in a sequence?
the 2 queries are these:
Equipes I want to add at table INFRA:
SELECT NOEUDS.NOEUD
FROM NOEUDS
WHERE (((NOEUDS.NOEUD) Like "C*" And (NOEUDS.NOEUD) Not In (SELECT NOEUD FROM INFRA)));
Insertion by hand:
INSERT INTO INFRA ( recno, NOEUD, SECURISE )
SELECT (SELECT Right(String(8, "0") & Hex(Max(Val("&H" & RECNO)) + 1), 8) AS N FROM INFRA), NOEUDS.NOEUD, "F" AS Expr2
FROM NOEUDS
WHERE (NOEUDS.NOEUD=[INSERT CHAMBRE?]);
I think a VBA solution should be better than trying to do what you want with only SQL. If you don't have much VBA experience, it could still be achievable because the required VBA should be fairly basic. See if this code outline is enough to get you started.
Public Sub AddToInfra()
Const cstrQuery As String = "qryUnmatchedNoeuds" ' Note 1 '
Dim db As DAO.Database ' Note 2 '
Dim fld As DAO.Field
Dim rsFrom As DAO.Recordset
Dim rsTo As DAO.Recordset
Set db = CurrentDb
Set rsFrom = db.OpenRecordset(cstrQuery, dbOpenSnapshot)
Set rsTo = db.OpenRecordset("infra", dbOpenTable, dbAppendOnly)
Do While Not rsFrom.EOF
rsTo.AddNew
For Each fld In rsFrom.Fields ' Note 3 '
If Not fld.Name = "RECNO" Then
rsTo.Fields(fld.Name).Value = fld.Value
End If
Next fld
rsTo!RECNO = Next_InfraRecno ' Note 4 '
rsTo!SECURISE = "F" ' Note 5 '
rsTo.Update
rsFrom.MoveNext
Loop
rsTo.Close
rsFrom.Close
Set fld = Nothing
Set rsFrom = Nothing
Set rsTo = Nothing
Set db = Nothing
End Sub
Notes:
I used a saved query based on my best guess as to what you want. See the SQL below.
DAO.Database requires a reference to Microsoft DAO Object Library. If your Access version is 2000 or maybe Access XP, you may need to set that reference (from VBE main menu, Tools->References).
I decided the destination table would include fields which match the name and data type of the fields in the source recordset. If that doesn't work for you, substitute something like this for each of the common fields: rsTo!YourFieldNameHere.Value = rsTo!YourFieldNameHere.Value (And drop .Value if you prefer.)
Create a Next_InfraRecno() function to return the next RECNO value. Translate the approach we used earlier into a function. Post a new question if you run into trouble ... show us your code, error message and line which triggers the error (if any), and anything else we need to know. :-)
I got the impression you want SECURISE = "F" for each of the inserted rows.
In a comment you mentioned "Use field ANCIEN for storage of counting". I don't know what's involved for that and hope, whatever it is, you can integrate it into this code outline. If not, sorry. :-(
Here is the SQL for my qryUnmatchedNoeuds query:
SELECT n.DELETED, n.NOEUD
FROM
noeuds AS n
LEFT JOIN infra AS i
ON n.NOEUD = i.NOEUD
WHERE
(((n.NOEUD) Like "c*")
AND ((i.NOEUD) Is Null));
Although I don't understand your question very well, I hope this answer provides something you can use.
My INFRA table has a text column named RECNO. The table contains one row.
RECNO
00000019
This query give me "1A" as N.
SELECT Hex(Max(Val("&H" & RECNO)) + 1) AS N
FROM INFRA;
To pad N with zeros to a width of 8, I can use this query which gives me "0000001A" as N.
SELECT Right(String(8, "0") & Hex(Max(Val("&H" & RECNO)) + 1), 8) AS N
FROM INFRA;
Regarding the rest of your question, my instinct would be to open a recordset containing the unmatched NOEUDS.NOEUD values, then move through the recordset rows and insert each NOEUD value, your custom RECNO sequence number, and the "other info" into the INFRA table.