I am currently having problems trying the run a query for some tables.
Below is what I am trying to do, I just can't seem to get it to work.
I have also duplicated constituent and gifts to show you the relations between soft credit and gifts/constituents
Link to Table Relations Image
SELECT
Constituent.lookup_id,
Constituent.name,
SplitGifts.amount,
SplitGifts.giftaidamount
FROM
dbo.Gifts Gifts
INNER JOIN dbo.Constituent Constituent
ON Constituent.id = Gifts.constituent_id
INNER JOIN dbo.SplitGifts SplitGifts
ON SplitGifts.giftid = Gifts.id
LEFT JOIN dbo.SoftCredit SoftCredit
ON SoftCredit.giftid = Gifts.id
INNER JOIN dbo.Constituent Constituent_1
ON Constituent_1.id = SoftCredit.constituentid
INNER JOIN dbo.Gifts Gifts_1
ON Gifts_1.id = SoftCredit.giftid
INNER JOIN dbo.Package Package
ON Package.id = SplitGifts.packageid
WHERE
Package.lookup_id = N'CORPCHAL'
Basically, I want the
amount and gift_aid_amount from [SplitGifts]
Constituent Name & lookup_id from [constituent] to show up for all Gifts however if a soft credit exists for that gift I need it to get the same fields via the [SoftCredit] table -> Gifts -> SplitGifts -> Fields
You could try if the query below works.
SELECT
Constituent.lookup_id,
Constituent.name,
SplitGifts.amount,
SplitGifts.giftaidamount
FROM
dbo.Gifts Gifts
LEFT JOIN dbo.SoftCredit SoftCredit ON SoftCredit.giftid = Gifts.id
INNER JOIN dbo.Gifts Gifts_1 ON
Gifts_1.id = SoftCredit.giftid OR
(SoftCredit.giftid IS NULL AND Gifts_1.id = Gifts.id)
INNER JOIN dbo.Constituent Constituent ON
Constituent.id = SoftCredit.constituentid OR
(SoftCredit.constituentid IS NULL AND Constituent.id = Gifts_1.constituent_id)
INNER JOIN dbo.SplitGifts SplitGifts ON SplitGifts.giftid = Gifts_1.id
INNER JOIN dbo.Package Package ON Package.id = SplitGifts.packageid
WHERE
Package.lookup_id = N'CORPCHAL'
It joins back to table Gifts (using alias Gifts_1) on the gift reference in SoftCredit or to itself if there is no SoftCredit.
Table Constituent is joined in a similar fashion: it joins on the value of SoftCredit.constituentid and when NULL, it falls back to Gifts_1.constituent_id.
All next joins regarding the gift should refer to Gifts_1 then.
I have not tested it though. But it might give you a hint in a possible solution direction.
Related
I have an oracle SQL query and a slight problem. I need to check if an item has a PO# that it has at least 1 line item. The query below works however it returns a result for each line of transaction_lines and I need only une result. PS I tried DISTINCT but get an ODBC error.
SELECT ITEMS.NAME, INVENTORY_NUMBER.INVENTORY_NUMBER, INVENTORY_NUMBER.ON_HAND_COUNT, ITEMS.SALESDESCRIPTION, CONDITION.LIST_ITEM_NAME,
BRAND_PARTNER.LIST_ITEM_NAME, PPROGRAM.LIST_ITEM_NAME, ENTITY.NAME, PO.TRANSACTION_NUMBER, INVENTORY_NUMBER.RECEIVED_COST, ITEMS.SALESPRICE, IR.TRANSACTION_NUMBER,
INVENTORY_SOURCE.LIST_ITEM_NAME, LOCATIONS.NAME, INVENTORY_NUMBER.RECEIPT_DATE, PO.INTERNAL_MEMO, INVENTORY_NUMBER.REFERENCE_, TEST_RESULTS.LIST_ITEM_NAME,
INVENTORY_NUMBER.TEST_FILE_LINK, INVENTORY_NUMBER.CONNECT_TRADE_ID, INVENTORY_NUMBER.SOLD_DATE, INVENTORY_NUMBER.SOLD_PRICE, INVENTORY_NUMBER.MEMO, ITEMS.UPC_CODE, ITEMS.MPN,
ITEMS.ITEM_ID, INVENTORY_NUMBER.CLEI, INVENTORY_NUMBER.CERTIFICATION_REF_ID
FROM INVENTORY_NUMBER
INNER JOIN ITEMS ON INVENTORY_NUMBER.ITEM_ID = ITEMS.ITEM_ID
INNER JOIN TRANSACTIONS AS PO ON INVENTORY_NUMBER.PURCHASE_ORDER_ID = PO.TRANSACTION_ID
INNER JOIN TRANSACTIONS AS IR ON INVENTORY_NUMBER.ITEM_RECEIPT_ID = IR.TRANSACTION_ID
INNER JOIN TRANSACTION_LINES ON PO.TRANSACTION_ID = TRANSACTION_LINES.TRANSACTION_ID
INNER JOIN ENTITY ON TRANSACTIONS.ENTITY_ID = ENTITY.ENTITY_ID
INNER JOIN CONDITION ON INVENTORY_NUMBER.CONDITION_ID = CONDITION.LIST_ID
INNER JOIN BRAND_PARTNER ON INVENTORY_NUMBER.BRAND_PARTNER_ID = BRAND_PARTNER.LIST_ID
INNER JOIN PPROGRAM ON INVENTORY_NUMBER.PROGRAM_ID = PPROGRAM.LIST_ID
INNER JOIN INVENTORY_SOURCE ON INVENTORY_NUMBER.INVENTORY_SOURCE_ID = INVENTORY_SOURCE.LIST_ID
INNER JOIN LOCATIONS ON INVENTORY_NUMBER.LOCATION_ID = LOCATIONS.LOCATION_ID
INNER JOIN TEST_RESULTS ON INVENTORY_NUMBER.TEST_RESULTS_ID = TEST_RESULTS.LIST_ID
WHERE INVENTORY_NUMBER.ON_HAND_COUNT IS NOT NULL AND ((INVENTORY_NUMBER.PURCHASE_ORDER_ID IS NULL) OR (INVENTORY_NUMBER.PURCHASE_ORDER_ID IS NOT NULL AND TRANSACTION_LINES.TRANSACTION_LINE_ID IS NOT NULL))
you could also remove the join to the transaction_lines and instead of tl.TRANSACTION_LINE_ID IS NOT NULL use an exists clause
and exists (select 1 from transaction lines tl
where tl.transaction_id = po.transaction_id)
I would suggest using a GROUP BY to help limit your results. You could also if you are interacting with transactions in your query you must always remember that without limiting results based on the "main line" you will receive the header record and then a record for each individual line item.
If you were doing this with a saved search you could put the criteria as "main line = true". Since I don't understand your query entirely I can't advise where to put this limitation in.
So here's a challenge. I have a webpage that needs to show data from 5 different tables in a database. Here's the query:
'SELECT ORGANIZATIONS.ORG_NAME, ORG_SECTIONS.SectionName, ATTORNEYS.NAME, ATTORNEYS.LASTNAME, ATTORNEYS.EMAIL, ATTORNEYS.TEL, ATTY_TITLES.ATT_TITLE, FIRM_PRACTICE_GRPS.PRACTICE_GRP
FROM (ORGANIZATIONS INNER JOIN (Org_Sec_Atty INNER JOIN ATTORNEYS ON Org_Sec_Atty.Atty_ID = ATTORNEYS.ATTY_ID) ON ORGANIZATIONS.ID = Org_Sec_Atty.OrgID) INNER JOIN ORG_SECTIONS ON Org_Sec_Atty.SecID = ORG_SECTIONS.ID
WHERE ATTORNEYS.LASTNAME LIKE #LASTNAME;'
The 5 tables are Org_Sec_Atty, ORGANIZATIONS, ORG_SECTIONS, ATTORNEYS, ATTY_TITLES, and FIRM_PRACTICE_GRPS. First, I didn't come up with the table names, lol.
Second, ATTORNEYS is the table that has ATTY_ID, TITLE_ID, PRACTICE_GRP_ID, which is what I'm sure I need to join to FIRM_PRACTICE_GRPS and ATTY_TITLE, and then it needs to connect with the Org_Sec_Atty and ORG_SECTIONS table.
I already have a page where the three tables ATTORNEYS, Org_Sec_Atty, and ORG_SECTIONS are joined by the JOIN statement you see up there. I'm trying to figure out where the other INNER JOIN statements would go and in what order.
It seems like I need to replace "ATTORNEYS" in the "Org_Sec_Atty INNER JOIN ATTORNEYS" with the subordinate INNER JOIN statements, but, again, I'm not sure of the order. Is this possible?
You didn't specified foreign keys on last two tables. Change appropriately if necessary:
SELECT ORGANIZATIONS.ORG_NAME ,
ORG_SECTIONS.SectionName ,
ATTORNEYS.NAME ,
ATTORNEYS.LASTNAME ,
ATTORNEYS.EMAIL ,
ATTORNEYS.TEL ,
ATTY_TITLES.ATT_TITLE ,
FIRM_PRACTICE_GRPS.PRACTICE_GRP
FROM (((( ORGANIZATIONS
INNER JOIN Org_Sec_Atty ON ORGANIZATIONS.ID = Org_Sec_Atty.OrgID)
INNER JOIN ATTORNEYS ON Org_Sec_Atty.Atty_ID = ATTORNEYS.ATTY_ID)
INNER JOIN ORG_SECTIONS ON Org_Sec_Atty.SecID = ORG_SECTIONS.ID)
INNER JOIN ATTY_TITLES ON ORG_SECTIONS.ATTY_ID = ATTY_TITLES.ID)
INNER JOIN FIRM_PRACTICE_GRPS ON ATTY_TITLES.FIRM_ID = FIRM_PRACTICE_GRPS.ID
WHERE ATTORNEYS.LASTNAME LIKE #LASTNAME;
I am trying to execute the following sql query but it takes 22 seconds to execute. the number of returned items is 554192. I need to make this faster and have already put indexes in all the tables involved.
SELECT mc.name AS MediaName,
lcc.name AS Country,
i.overridedate AS Date,
oi.rating,
bl1.firstname + ' ' + bl1.surname AS Byline,
b.id BatchNo,
i.numinbatch ItemNumberInBatch,
bah.changedatutc AS BatchDate,
pri.code AS IssueNo,
pri.name AS Issue,
lm.neptunemessageid AS MessageNo,
lmt.name AS MessageType,
bl2.firstname + ' ' + bl2.surname AS SourceFullName,
lst.name AS SourceTypeDesc
FROM profiles P
INNER JOIN profileresults PR
ON P.id = PR.profileid
INNER JOIN items i
ON PR.itemid = I.id
INNER JOIN batches b
ON b.id = i.batchid
INNER JOIN itemorganisations oi
ON i.id = oi.itemid
INNER JOIN lookup_mediachannels mc
ON i.mediachannelid = mc.id
LEFT OUTER JOIN lookup_cities lc
ON lc.id = mc.cityid
LEFT OUTER JOIN lookup_countries lcc
ON lcc.id = mc.countryid
LEFT OUTER JOIN itembylines ib
ON ib.itemid = i.id
LEFT OUTER JOIN bylines bl1
ON bl1.id = ib.bylineid
LEFT OUTER JOIN batchactionhistory bah
ON b.id = bah.batchid
INNER JOIN itemorganisationissues ioi
ON ioi.itemorganisationid = oi.id
INNER JOIN projectissues pri
ON pri.id = ioi.issueid
LEFT OUTER JOIN itemorganisationmessages iom
ON iom.itemorganisationid = oi.id
LEFT OUTER JOIN lookup_messages lm
ON iom.messageid = lm.id
LEFT OUTER JOIN lookup_messagetypes lmt
ON lmt.id = lm.messagetypeid
LEFT OUTER JOIN itemorganisationsources ios
ON ios.itemorganisationid = oi.id
LEFT OUTER JOIN bylines bl2
ON bl2.id = ios.bylineid
LEFT OUTER JOIN lookup_sourcetypes lst
ON lst.id = ios.sourcetypeid
WHERE p.id = #profileID
AND b.statusid IN ( 6, 7 )
AND bah.batchactionid = 6
AND i.statusid = 2
AND i.isrelevant = 1
when looking at the execution plan I can see an step which is costing 42%. Is there any way I could get this to a lower threshold or any way that I can improve the performance of the whole query.
Remove the profiles table as it is not needed and change the WHERE clause to
WHERE PR.profileid = #profileID
You have a left outer join on the batchactionhistory table but also have a condition in your WHERE clause which turns it back into an inner join. Change you code to this:
LEFT OUTER JOIN batchactionhistory bah
ON b.id = bah.batchid
AND bah.batchactionid = 6
You don't need the batches table as it is used to join other tables which could be joined directly and to show the id in you SELECT which is also available in other tables. Make the following changes:
i.batchidid AS BatchNo,
LEFT OUTER JOIN batchactionhistory bah
ON i.batchidid = bah.batchid
Are any of the fields that are used in joins or the WHERE clause from tables that contain large amounts of data but are not indexed. If so try adding an index on at time to the largest table.
Do you need every field in the result - if you could loose one or to you maybe could reduce the number of tables further.
First, if this is not a stored procedure, make it one. That's a lot of text for sql server to complile.
Next, my experience is that "worst practices" are occasionally a good idea. Specifically, I have been able to improve performance by splitting large queries into a couple or three small ones and assembling the results.
If this query is associated with a .net, coldfusion, java, etc application, you might be able to do the split/re-assemble in your application code. If not, a temporary table might come in handy.
I posted a query yesterday (see here) that was horrible (took over a minute to run, resulting in 18,215 records):
SELECT DISTINCT
dbo.contacts_link_emails.Email, dbo.contacts.ContactID, dbo.contacts.First AS ContactFirstName, dbo.contacts.Last AS ContactLastName, dbo.contacts.InstitutionID,
dbo.institutionswithzipcodesadditional.CountyID, dbo.institutionswithzipcodesadditional.StateID, dbo.institutionswithzipcodesadditional.DistrictID
FROM
dbo.contacts_def_jobfunctions AS contacts_def_jobfunctions_3
INNER JOIN
dbo.contacts
INNER JOIN
dbo.contacts_link_emails
ON dbo.contacts.ContactID = dbo.contacts_link_emails.ContactID
ON contacts_def_jobfunctions_3.JobID = dbo.contacts.JobTitle
INNER JOIN
dbo.institutionswithzipcodesadditional
ON dbo.contacts.InstitutionID = dbo.institutionswithzipcodesadditional.InstitutionID
LEFT OUTER JOIN
dbo.contacts_def_jobfunctions
INNER JOIN
dbo.contacts_link_jobfunctions
ON dbo.contacts_def_jobfunctions.JobID = dbo.contacts_link_jobfunctions.JobID
ON dbo.contacts.ContactID = dbo.contacts_link_jobfunctions.ContactID
WHERE
(dbo.contacts.JobTitle IN
(SELECT JobID
FROM dbo.contacts_def_jobfunctions AS contacts_def_jobfunctions_1
WHERE (ParentJobID <> '1841')))
AND
(dbo.contacts_link_emails.Email NOT IN
(SELECT EmailAddress
FROM dbo.newsletterremovelist))
OR
(dbo.contacts_link_jobfunctions.JobID IN
(SELECT JobID
FROM dbo.contacts_def_jobfunctions AS contacts_def_jobfunctions_2
WHERE (ParentJobID <> '1841')))
AND
(dbo.contacts_link_emails.Email NOT IN
(SELECT EmailAddress
FROM dbo.newsletterremovelist AS newsletterremovelist))
ORDER BY EMAIL
With a lot of coaching and research, I've tuned it up to the following:
SELECT contacts.ContactID,
contacts.InstitutionID,
contacts.First,
contacts.Last,
institutionswithzipcodesadditional.CountyID,
institutionswithzipcodesadditional.StateID,
institutionswithzipcodesadditional.DistrictID
FROM contacts
INNER JOIN contacts_link_emails ON
contacts.ContactID = contacts_link_emails.ContactID
INNER JOIN institutionswithzipcodesadditional ON
contacts.InstitutionID = institutionswithzipcodesadditional.InstitutionID
WHERE
(contacts.ContactID IN
(SELECT contacts_2.ContactID
FROM contacts AS contacts_2
INNER JOIN contacts_link_emails AS contacts_link_emails_2 ON
contacts_2.ContactID = contacts_link_emails_2.ContactID
LEFT OUTER JOIN contacts_def_jobfunctions ON
contacts_2.JobTitle = contacts_def_jobfunctions.JobID
RIGHT OUTER JOIN newsletterremovelist ON
contacts_link_emails_2.Email = newsletterremovelist.EmailAddress
WHERE (contacts_def_jobfunctions.ParentJobID <> 1841)
GROUP BY contacts_2.ContactID
UNION
SELECT contacts_1.ContactID
FROM contacts_link_jobfunctions
INNER JOIN contacts_def_jobfunctions AS contacts_def_jobfunctions_1 ON
contacts_link_jobfunctions.JobID = contacts_def_jobfunctions_1.JobID
AND contacts_def_jobfunctions_1.ParentJobID <> 1841
INNER JOIN contacts AS contacts_1 ON
contacts_link_jobfunctions.ContactID = contacts_1.ContactID
INNER JOIN contacts_link_emails AS contacts_link_emails_1 ON
contacts_link_emails_1.ContactID = contacts_1.ContactID
LEFT OUTER JOIN newsletterremovelist AS newsletterremovelist_1 ON
contacts_link_emails_1.Email = newsletterremovelist_1.EmailAddress
GROUP BY contacts_1.ContactID))
While this query is now super fast (about 3 seconds), I've blown part of the logic somewhere - it only returns 14,863 rows (instead of the 18,215 rows that I believe is accurate).
The results seem near correct. I'm working to discover what data might be missing in the result set.
Can you please coach me through whatever I've done wrong here?
Thanks,
Russell Schutte
The main problem with your original query was that you had two extra joins just to introduce duplicates and then a DISTINCT to get rid of them.
Use this:
SELECT cle.Email,
c.ContactID,
c.First AS ContactFirstName,
c.Last AS ContactLastName,
c.InstitutionID,
izip.CountyID,
izip.StateID,
izip.DistrictID
FROM dbo.contacts c
INNER JOIN
dbo.institutionswithzipcodesadditional izip
ON izip.InstitutionID = c.InstitutionID
INNER JOIN
dbo.contacts_link_emails cle
ON cle.ContactID = c.ContactID
WHERE cle.Email NOT IN
(
SELECT EmailAddress
FROM dbo.newsletterremovelist
)
AND EXISTS
(
SELECT NULL
FROM dbo.contacts_def_jobfunctions cdj
WHERE cdj.JobId = c.JobTitle
AND cdj.ParentJobId <> '1841'
UNION ALL
SELECT NULL
FROM dbo.contacts_link_jobfunctions clj
JOIN dbo.contacts_def_jobfunctions cdj
ON cdj.JobID = clj.JobID
WHERE clj.ContactID = c.ContactID
AND cdj.ParentJobId <> '1841'
)
ORDER BY
email
Create the following indexes:
newsletterremovelist (EmailAddress)
contacts_link_jobfunctions (ContactID, JobID)
contacts_def_jobfunctions (JobID)
Do you get the same results when you do:
SELECT count(*)
FROM
dbo.contacts_def_jobfunctions AS contacts_def_jobfunctions_3
INNER JOIN
dbo.contacts
INNER JOIN
dbo.contacts_link_emails
ON dbo.contacts.ContactID = dbo.contacts_link_emails.ContactID
ON contacts_def_jobfunctions_3.JobID = dbo.contacts.JobTitle
SELECT COUNT(*)
FROM
contacts
INNER JOIN contacts_link_jobfunctions
ON contacts.ContactID = contacts_link_jobfunctions.ContactID
INNER JOIN contacts_link_emails
ON contacts.ContactID = contacts_link_emails.ContactID
If so keep adding each join conditon on until you don't get the same results and you will see where your mistake was. If all the joins are the same, then look at the where clauses. But I will be surprised if it isn't in the first join because the syntax you have orginally won't even work on SQL Server and it is pretty nonstandard SQL and may have been incorrect all along but no one knew.
Alternatively, pick a few of the records that are returned in the orginal but not the revised. Track them through the tables one at a time to see if you can find why the second query filters them out.
I'm not directly sure what is wrong, but when I run in to this situation, the first thing I do is start removing variables.
So, comment out the where clause. How many rows are returned?
If you get back the 11,604 rows then you've isolated the problems to the joins. Work though the joins, commenting each one out (remove the associated columns too) and figure out how many rows are eliminated.
As you do this, aim to find what is causing the desired rows to be eliminated. Once isolated, consider the join differences between the first query and the second query.
In looking at the first query, you could probably just modify that to eliminate any INs and instead do a EXISTS instead.
Consider your indexes as well. Any thing in the where or join clauses should probably be indexed.
Alright, so I'm putting together a path to select a revision of a particular novel:
SELECT Catalog.WbsId, Catalog.Revision, NovelRevision.Revision
FROM Catalog, BookInCatalog
INNER JOIN NovelMaster
INNER JOIN HasNovelRevision
INNER JOIN NovelRevision
ON HasNovelRevision.right = NovelRevision.obid
ON HasNovelRevision.Left=NovelMaster.obid
ON NovelMaster.obid = BookInCatalog.Right
WHERE Catalog.obid = BookInCatalog.Left;
This returns all revisions that are in the Novel Master for each Novel Master that is in the catalog.
The problem is, I only want the FIRST revision of each novel master in the catalog. How do I go about doing that? Oh, and btw: my flavor of sql is hobbled, as many others are, in that it does not support the LIMIT Function.
****UPDATE****
So using answer 1 as a guide I upgraded my query to this:
SELECT Catalog.wbsid
FROM Catalog, BookInCatalog, NovelVersion old, NovelMaster, HasNovelRevision
LEFT JOIN NovelVersion newRevs
ON old.revision < newRevs.revision AND HasNovelRevision.right = newRevs.obid
LEFT JOIN HasNovelRevision NewerHasNovelRevision
ON NewerHasNovelRevision.right = newRevs.obid
LEFT JOIN NovelMaster NewTecMst
ON NewerHasNovelRevision.left = NewTecMst.obid
WHERE Catalog.programName = 'E18' AND Catalog.obid = BookInCatalog.Left
AND BookInCatalog.right = NewTecMst.obid AND newRevs.obid = null
ORDER BY newRevs.documentname;
I get an error on the fourth line:
"old"."revision": invalid identifier
SOLUTION
Well, I had to go to another forum, but I got a working solution:
select nr1.title, nr1.revision
from novelrevision nr1
where nr1.revision in (select min(revision) from novelrevision nr2
where nr1.title = nr2.title)
So this solution uses the JOIN mentioned by the OA, along with the IN keyword to match it to a revision.
Something like this might work, it's called an exclusive left join:
....
INNER JOIN NovelRevision
ON HasNovelRevision.right = NovelRevision.obid
LEFT JOIN NovelRevision as NewerRevision
ON HasNovelRevision.right = NewerRevision.obid
AND NewerRevision.revision > NovelRevision.revision
...
WHERE NeverRevision.obid is null
The where clause filters out rows for which a newer revision exists. This effectively limits the query to the newest revisions.
In response to your comment, you could filter out only revisions that have a newer revision in the same NovelMaster. For example:
....
LEFT JOIN NovelRevision as NewerRevision
ON HasNovelRevision.right = NewerRevision.obid
AND NewerRevision.revision > NovelRevision.revision
LEFT JOIN HasNovelRevision as NewerHasNovelRevision
ON NewerHasNovelRevision.right = NewerRevision.obid
LEFT JOIN NovelMaster as NewerNovelMaster
ON NewerHasNovelRevision.left = NewerNovelMaster.obid
AND NewerNovelMaster.obid = NovelMaster.obid
....
WHERE NeverNovelMaster.obid is null
P.S. I don't think you can group JOINs and follow them with a group of ON conditions. An ON must directly follow its JOIN.
You can use CTE
Check this
WITH NovelRevesion_CTE(obid,RevisionDate)
AS
(
SELECT obid,MIN(RevisionDate) RevisionDate FROM NovelRevision Group by obid
)
SELECT Catalog.WbsId, Catalog.Revision, NovelRevision.Revision
FROM Catalog, BookInCatalog
INNER JOIN NovelMaster
INNER JOIN HasNovelRevision
INNER JOIN NovelRevesion
INNER JOIN NovelRevesion_CTE
ON HasNovelRevision.[right] = NovelRevision.obid
ON HasNovelRevision.[Left]=NovelMaster.obid
ON NovelMaster.obid = BookInCatalog.[Right]
ON NovelRevesion_CTE.obid = NovelRevesion.obid
WHERE Catalog.obid = BookInCatalog.[Left];
First it select the first revision written for each novel (assuming obid is novel foriegn key) by taking the smallest date and group them.
then add it as join in your query