I have a table,
Contacts:
Contact_ID (int)
ContactName (nvarchar)
I am given a list of contact IDs to select from. Usually, i would just do
SELECT *
FROM Contacts
WHERE IN (List of contact ID)
The problem is, the list of contact IDs could potentially get very big, like 50k or more.
So my question is, Is there a way to deal with a large list of contact IDs without having to use the IN clause?
EDIT: I'm using Microsoft sql server. The query and the contact IDs are built during runtime and passed on to sqlCommand class (c#) to be executed.
I'd create a table type with a single column and a clustered primary key.
CREATE TYPE dbo.ContactId AS TABLE
(
ContactId INT NOT NULL PRIMARY KEY
);
Pass the values into the query using a table valued parameter.
Change your query to
SELECT *
FROM Contacts
WHERE contactID IN (SELECT y.contactID FROM #yourtablevaluedparameter y)
OPTION (RECOMPILE)
The OPTION (RECOMPILE) is there to get the number of rows taken into account as the optimal plan for 50K may well be different than for 1.
You can find some example C# code for populating a TVP here
If you want performance, I would use the EXISTS clause.
SELECT c.Contact_ID, c.ContactName
FROM Contacts c
WHERE EXISTS (List of contact ID)
Create a temp table and populate your contact id in the form of rows.
Do an inner join between your table and temp table like the below.
SELECT c.*
FROM Contacts c
join #temptable t
on c.id=t.id
If you introduce index on the Join column in your temp table then your query will be more faster.
Related
I am trying to make it so all the users have the same items because I am doing an experiment with my app and need the experimental control of flattened data.
I used the following SQL statement in my last attempt:
insert into has (email,id,qty,price,item_info,image)
select 'b#localhost.com',id,qty,price,item_info,image
from
(
select * from has
where email != 'b#localhost.com'
) as o
where o.id not in
(
select id from has
where email = 'b#localhost.com'
);
This should add all items which 'b#localhost.com' does not already have but other users do have, to 'b#localhost.com's inventory. (the 'has' table)
However, I get the following error:
The statement was aborted because it would have caused a duplicate key value in a unique or primary key constraint or unique index
I understand what this error means, but I do not understand why it is occurring. My statement inserts all records which that email doesn't already have, so there should not be any duplicate id/email pairs.
The database structure is shown below, circles are attributes, squares are tables, and diamonds are relationship sets. The HAS table is a relationship set on USER_ACCOUNT and ITEM, where the primary keys are email and id respectively.
Please try the following...
INSERT INTO has ( email,
id,
qty,
price,
item_info,
image )
SELECT a.email,
b.id,
a.qty,
a.price,
a.item_info,
a.image
FROM has a
CROSS JOIN has b
JOIN
(
SELECT email,
id
FROM has
) c ON a.email = c.email AND
b.id <> c.id;
The CROSS JOIN appends a copy of each row of has to each row of has. Please see http://www.w3resource.com/sql/joins/cross-join.php for more information on CROSS JOIN's.
Each row of the resulting dataset will have two id fields, two email fields, two qty fields, etc. By giving each instance of has an alias we create the fields a.id, b.id, a.email, etc.
We then compare each combination of "old" email address and "new" id to a list of existing email / id combinations and insert the old values with the new id replacing the old one into has
If you have any questions or comments, then please feel free to post a Comment accordingly.
Further Reading
http://www.w3resource.com/sql/joins/cross-join.php for more information on CROSS JOIN's
https://www.w3schools.com/sql/sql_in.asp for more information on WHERE's IN operator
https://www.w3schools.com/sql/sql_groupby.asp for more information on GROUP BY
I think the issue here is not that the code is trying to insert something which already exists, but that it's trying to insert more than one thing with the same PK. In lieu of an response to my comment, here is one way to get around the issue:
INSERT INTO has (email,id,qty,price,item_info,image)
SELECT
'b#localhost.com',
source.id,
source.qty,
source.price,
source.item_info,
source.image
FROM
(
SELECT email, id, qyt, price, item_info, image FROM has
) as source
JOIN
(
SELECT min(email) as min_email, id FROM has GROUP BY by id)
) as filter ON
filter.min_email = source.email
WHERE
source.id not in
(
SELECT id from has WHERE email = 'b#localhost.com'
);
The key difference from your original code is my extra join to the subquery I've aliased as filter. This limits you to inserting the has details from a single email per id. There are other ways to do the same, but I figured that this would be a safe bet for being supported by Derby.
I removed the WHERE clause from the source sub-query as that is handled by the final WHERE.
It would be something like:
SELECT * FROM users ORDER BY id ORDER("abc","ghk","pqr"...);
In my order clause there might be 1000 records and all are dynamic.
A quick google search gave me below result:
SELECT * FROM users ORDER BY case id
when "abc" then 1
when "ghk" then 2
when "pqr" then 3 end;
As I said all my order clause values are dynamic. So is there any suggestion for me?
Your example isn't entirely clear, as it appears that a simple ORDER BY would suffice to order your id's alphabetically. However, it appears you are trying to create a dynamic ordering scheme that may not be alphabetical. In that case, my recommendation would be to use a lookup table for the values that you will be ordering by. This serves two purposes: first, it allows you to easily reorder the items without altering each entry in the users table, and second, it avoids (or at lest reduces) problems with typos and other issues that can occur with "magic strings."
This would look something like:
Lookup Table:
CREATE TABLE LookupValues (
Id CHAR(3) PRIMARY KEY,
Order INT
);
Query:
SELECT
u.*
FROM
users u
INNER JOIN
LookupTable l
ON
u.Id = l.Id
ORDER BY
l.Order
I have two tables, one for Customer and one for Item.
In Customer, I have a column called "preference", which stores a list of hard criteria expressed as a WHERE clause in SQL e.g. "item.price<20 and item.category='household'".
I'd like a query that works like this:
SELECT * FROM item WHERE interpret('SELECT preference FROM customer WHERE id = 1')
Which gets translated to this:
SELECT * FROM item WHERE item.price < 20 and item.category = 'household'
Example data model:
CREATE TABLE customer (
cust_id INT
preference VARCHAR
);
CREATE TABLE item (
item_id INT
price DECIMAL(19,4)
category VARCHAR
);
# Additional columns omitted for brevity
I've looked up casting and dynamic SQL but I haven't been able to figure out how I should do this.
I'm using PostgreSQL 9.5.1
I'm going to assume that preference is the same as my made up item_id column. You may need to tweak it to fit your case. For future questions like this it may pay to give us the table structures you are working with, it really helps us out!
What you are asking for is a subquery:
select *
from item
where item_id in (select
preference
from
customer
where id = 1)
What I would suggest though is a join:
select item.*
from item
join customer on item.item_id = customer.preference
where item.price<20 and
item.category='household'
customer.id = 1
I decided to change my schema instead, as it was getting pretty messy to store the criteria in preferences in that manner.
I restricted the kinds of preferences that could be specified, then stored them as columns in Customer.
After that, all the queries I wanted could be expressed as joins.
In MS Access, I am trying to implement a many-to-many table that will store 2-way relationships, similar to Association between two entries in SQL table. This table stores info such as "Person A and Person B are coworkers" "C and D are friends", etc. The table is like this:
ConstitRelationships
LeftId (number, primary key, foreign key to Constituents.ConstitId)
RightId (number, primary key, foreign key to Constituents.ConstitId)
Description (text)
Note that the primary key is a composite of the two Id fields.
Also the table has constraints:
[LeftId]<>[RightId] AND [LeftId]<[RightId]
The table is working ok in my Access project, except that I cannot figure out how to make an updateable query that I want to use as a datasheet subform so users can easily add/delete records and change the descriptions. I currently have a non-updatable query:
SELECT Constituents.ConstituentId, Constituents.FirstName,
Constituents.MiddleName, Constituents.LastName,
ConstitRelationships.Description, ConstitRelationships.LeftId,
ConstitRelationships.RightId
FROM ConstitRelationships INNER JOIN Constituents ON
(Constituents.ConstituentId =
ConstitRelationships.RightId) OR (Constituents.ConstituentId =
ConstitRelationships.LeftId);
If I ignore the possibility that the constituentId I want is in the leftId column, I can do this, which is updatable. So the OR condition in the inner join above is what's messing it up.
SELECT Constituents.ConstituentId, Constituents.FirstName,
Constituents.MiddleName, Constituents.LastName,
ConstitRelationships.Description, ConstitRelationships.LeftId,
ConstitRelationships.RightId
FROM ConstitRelationships INNER JOIN Constituents ON
(Constituents.ConstituentId =
ConstitRelationships.RightId) ;
I also tried this wacky iif thing to collapse the two LeftId and RightId fields into FriendId, but it was not updateable either.
SELECT Constituents.ConstituentId, Constituents.FirstName,
Constituents.MiddleName,
Constituents.LastName, subQ.Description
FROM Constituents
INNER JOIN (
SELECT Description, Iif([Forms]![Constituents Form]![ConstituentId] <>
ConstitRelationships.LeftId, ConstitRelationships.LeftId,
ConstitRelationships.RightId) AS FriendId
FROM ConstitRelationships
WHERE ([Forms]![Constituents Form]![ConstituentId] =
ConstitRelationships.RightId)
OR ([Forms]![Constituents Form]![ConstituentId] =
ConstitRelationships.LeftId)
) subQ
ON (subQ.FriendId = Constituents.ConstituentId)
;
How can I make an updatable query on ConstitRelationships, including a JOIN with the Constituent.FirstName MiddleName LastName fields?
I am afraid that is not possible. Because you use joins in your query over three tables it is not updatable. There is no way around this.
Here some detailed information about the topic: http://www.fmsinc.com/Microsoftaccess/query/non-updateable/index.html
As mentioned in the linked article one possible solution and in my opinion best solution for you would be the temporary table. It is a load of work compared to the easy "bind-form-to-a-query"-approach but it works best.
The alternative would be to alter your datascheme in that way that you do not need joins. But then denormalized data and duplicates would go rampage which makes the temporary table a favorable choice.
Scenario: A sampling survey needs to be performed on membership of 20,000 individuals. Survey sample size is 3500 of the total 20000 members. All membership individuals are in table tblMember. Same survey was performed the previous year and members whom were surveyed are in tblSurvey08. Membership data can change over the year (e.g. new email address, etc.) but the MemberID data stays the same.
How do I remove the MemberID/records contained tblSurvey08 from tblMember to create a new table of potential members to be surveyed (lets call it tblPotentialSurvey09). Again the record for a individual member may not match from the different tables but the MemberID field will remain constant.
I am fairly new at this stuff but I seem to be having a problem Googling a solution - I could use the EXCEPT function but the records for the individuals members are not necessarily the same from one table to next - just the MemberID may be the same.
Thanks
SELECT
* (replace with column list)
FROM
member m
LEFT JOIN
tblSurvey08 s08
ON m.member_id = s08.member_id
WHERE
s08.member_id IS NULL
will give you only members not in the 08 survey. This join is more efficient than a NOT IN construct.
A new table is not such a great idea, since you are duplicating data. A view with the above query would be a better choice.
I apologize in advance if I didn't understand your question but I think this is what you're asking for. You can use the insert into statement.
insert into tblPotentialSurvey09
select your_criteria from tblMember where tblMember.MemberId not in (
select MemberId from tblSurvey08
)
First of all, I wouldn't create a new table just for selecting potential members. Instead, I would create a new true/false (1/0) field telling if they are eligible.
However, if you'd still want to copy data to the new table, here's how you can do it:
INSERT INTO tblSurvey00 (MemberID)
SELECT MemberID
FROM tblMember m
WHERE NOT EXISTS (SELECT 1 FROM tblSurvey09 s WHERE s.MemberID = m.MemberID)
If you just want to create a new field as I suggested, a similar query would do the job.
An outer join should do:
select m_09.MemberID
from tblMembers m_09 left outer join
tblSurvey08 m_08 on m_09.MemberID = m_08.MemberID
where
m_08.MemberID is null