How to insert multiple instances based on an array - sql

We have two roles: Admin and Customer. There are a number of default users with email addresses following the pattern:
An Admin - admin1#.com, admin2#.com etc.
A Customer - user1#.com, user2#.com etc.
Then, we run the script for each combination (and in case with admins, it's done twice, because they're customers too).
insert into AspNetUserRoles values(
(select Id from AspNetUsers where Email = 'AAA'),
(select Id from AspNetRoles where Name = 'BBB'))
Now, based on my question, you can take a guess how it's resolved right now. For each new email, we add a statement or two. If we'd add a new role, we'd have to add a number of statement, possibly as many as the number of registered emails.
I sense there's away to declare a matrix on form:
a#.com, role1, role2
b#.com, role1,
c#.com
d#.com, role1, role3, role4
I've tried for a while but couldn't figure out the syntax, though. The actual DBA says it's not (easily) doable and that the script we have right now is as it's supposed to be done.
I suspect he's full of Christmas candy having been processed but, not being a DBA myself, I can't really argue, unless I have something that works. I also suspect that I didn't google the right way (i.e. I used wrong terms to describe what I want, due to my ignorance).
Edit
Realizing that the question might be misleading, I'll give an example in speudo-code to illustrate my intention.
List<Link> links = new List<Link> {
new {a1,b1}, new {a1,b2},
new {a2,b2},
new {a3,b1}, new {a3,b3}, new {a3,b4} }
foreach(Link in links)
ExecuteSql(
"insert into Links values(
(select Id from FirstTable where Name = link.A),
(select Id from SecondTable where Name = link.B))"
);
The part I can't figure out is how to declare such a list and how to loop through it.

1) Say we start by creating a temp table.
-- Create temp table for user and roles
CREATE TABLE #temp(
AspNetUser varchar(1000) ,
AspNetRoles varchar(1000));
2a) populate it from a File (eg userroles.csv)
a#.com,role1|b#.com,role1|c#.com,|d#.com,role1 role3 role4
Like this
-- Read from csv
BULK INSERT #temp FROM 'D:\userroles.csv'
WITH (
FIELDTERMINATOR =','
,ROWTERMINATOR ='|');
2b) OR do your own inserts in the script
INSERT INTO #temp
(AspNetUser, AspNetRoles)
VALUES
('a#.com','role1'),
('b#.com','role1'),
('c#.com',null),
('d#.com','role1 role3 role4')
3) Insert all combinations into the table by looking up the id's
-- Insert all found combinations
INSERT INTO AspNetUserRoles
SELECT users.Id, roles.Id
FROM
(
SELECT AspNetUser,
CAST ('<Role>' + REPLACE(AspNetRoles, ' ', '</Role><Role>') + '</Role>' AS XML) AS Data
FROM #temp
) AS A
CROSS APPLY Data.nodes ('/Role') AS Split(a)
INNER JOIN AspNetUsers users ON users.Email = AspNetUser
INNER JOIN AspNetRoles roles ON roles.Name = Split.a.value('.', 'VARCHAR(100)')
-- Clean up
drop table #textfile;
You can change delimiters SPACE, , and | to what you like.
You may want to do errorchecking for typos!

Related

Dynamically Insert into table while checking if the record already exists

I had some doubts on dynamic insertion of data while doing an insert statement so just wanted to get some assistance from you guys. I have to do multiple insert statements say around 1500 records based on 2 different criteria's below is just a sample of 1 insert statement.
Now while doing an insert statement I want to dynamically assign the USERID's and ROLEid's the 2 columns which you can see in the query below.
So for example where userid IN (500 different userid) and role id in (100 different) ones.
Insert into userrolelist (Userid, Roleid, IsDefault, EffectiveStart,
EffectiveEnd, Clientid, LastmodifiedUserId, LastmodifiedTimestamp)
Values (161514,1011,1,'2016-01-21 00:00:00.001',Null,16785,0,'2016-01-21
00:00:00.001')
I am sure there is a way to do dynamic insertion based on 2 different criteria's I am just confused as to how can I achieve that. Mainly also because for each criteria before insertion I need to check if that userid + roleid combination already exists in the table. Because if I dont check it and still do an insert it will throw an error because there is a constraint based on the 2 fields.
Any help on this matter would be appreciated. Please let me know if the question is not very clear and i can add a bit more explanation if required. Thank you.
You don't say where your lists of user IDs and role IDs are coming from, but because you specify different numbers for each of them, I assume that they are separate lists, rather than a single list of pairs. And I assume that they are stored in tables named userlist and rolelist, respectively. Then you can do the insert as follows:
insert into userrolelist
(Userid, Roleid, IsDefault, EffectiveStart, EffectiveEnd,
Clientid, LastmodifiedUserId, LastmodifiedTimestamp)
select
userid, roleid,
1,'2016-01-21 00:00:00.001',Null,16785,0,
'2016-01-21 00:00:00.001'
from
(select userid, roleid
from userlist
cross join rolelist
) as userrole
where
not exists (select 1 from userrolelist as ur where ur.userid=userrole.userid and ur.roleid=userrole.roleid);
The subquery constructs a list of all possible pairs of users and roles, so if you already have a list of pairs, you can simply use that in place of the subquery.

SQL query with pivot tables?

I'm trying to wrap by brain around how to use pivot tables for a query I need. I have 3 database tables. Showing relevant columns:
TableA: Columns = pName
TableB: Columns = GroupName, GroupID
TableC: Columns = pName, GroupID
TableA contains a list of names (John, Joe, Jack, Jane)
TableB contains a list of groups with an ID#. (Soccer|1, Hockey|2, Basketball|3)
TableC contains a list of the names and the group they belong to (John|1, John|3, Joe|2, Jack|1, Jack|2, Jack|3, Jane|3)
I need to create a matrix like grid view using a SQL query that would return a list of all the names from TableA (Y-axis) and a list of all the possible groups (X-axis). The cell values would be either true or false if they belong to the group.
Any help would be appreciated. I couldn't quite find an existing answer that helped.
You might try it like this
Here I set up a MCVE, please try to create this in your next question yourself...
DECLARE #Name TABLE (pName VARCHAR(100));
INSERT INTO #Name VALUES('John'),('Joe'),('Jack'),('Jane');
DECLARE #Group TABLE(gName VARCHAR(100),gID INT);
INSERT INTO #Group VALUES ('Soccer',1),('Hockey',2),('Basketball',3);
DECLARE #map TABLE(pName VARCHAR(100),gID INT);
INSERT INTO #map VALUES
('John',1),('John',3)
,('Joe',2)
,('Jack',1),('Jack',2),('Jack',3)
,('Jane',3);
This quer will collect the values and perform PIVOT
SELECT p.*
FROM
(
SELECT n.pName
,g.gName
,'x' AS IsInGroup
FROM #map AS m
INNER JOIN #Name AS n ON m.pName=n.pName
INNER JOIN #Group AS g ON m.gID=g.gID
) AS x
PIVOT
(
MAX(IsInGroup) FOR gName IN(Soccer,Hockey,Basketball)
) as p
This is the result.
pName Soccer Hockey Basketball
Jack x x x
Jane NULL NULL x
Joe NULL x NULL
John x NULL x
Some hints:
You might use 1 and 0 instead of x as SQL Server does not know a real boolean
You should add a pID to your names. Never join tables on real data (unless it is something unique and unchangeable [which means never acutally!!!])
UPDATE dynamic SQL (thx to #djlauk)
If you want a query which deals with any amount of groups you have to to this dynamically. But please be aware, that you loose the chance to use this in ad-hoc-SQL like in VIEW or inline TVF, which is quite a big backdraw...
CREATE TABLE #Name(pName VARCHAR(100));
INSERT INTO #Name VALUES('John'),('Joe'),('Jack'),('Jane');
CREATE TABLE #Group(gName VARCHAR(100),gID INT);
INSERT INTO #Group VALUES ('Soccer',1),('Hockey',2),('Basketball',3);
CREATE TABLE #map(pName VARCHAR(100),gID INT);
INSERT INTO #map VALUES
('John',1),('John',3)
,('Joe',2)
,('Jack',1),('Jack',2),('Jack',3)
,('Jane',3);
DECLARE #ListOfGroups VARCHAR(MAX)=
(
STUFF
(
(
SELECT DISTINCT ',' + QUOTENAME(gName)
FROM #Group
FOR XML PATH('')
),1,1,''
)
);
DECLARE #sql VARCHAR(MAX)=
(
'SELECT p.*
FROM
(
SELECT n.pName
,g.gName
,''x'' AS IsInGroup
FROM #map AS m
INNER JOIN #Name AS n ON m.pName=n.pName
INNER JOIN #Group AS g ON m.gID=g.gID
) AS x
PIVOT
(
MAX(IsInGroup) FOR gName IN(' + #ListOfGroups + ')
) as p');
EXEC(#sql);
GO
DROP TABLE #map;
DROP TABLE #Group;
DROP TABLE #Name;
I suspect it may be laborious to keep the pivot up to date if categories are added. Or maybe I just prefer Excel (if you ignore one major advantage). The following approach could be helpful too, assuming you do have Office 365.
I added the three tables using 3 CREATE TABLE statements and 3 INSERT statements based on the code I saw above. (The solutions make use of temporary tables to insert specific values, but I believe you already have the data in your three tables, called TableA, TableB, TableC).
CREATE TABLE TestName (pName VARCHAR(100));
INSERT INTO TestName VALUES('John'),('Joe'),('Jack'),('Jane');
CREATE TABLE TestGroup (gName VARCHAR(100),gID INT);
INSERT INTO TestGroup VALUES ('Soccer',1),('Hockey',2),('Basketball',3);
CREATE TABLE Testmap (pName VARCHAR(100),gID INT);
INSERT INTO Testmap VALUES
('John',1),('John',3)
,('Joe',2)
,('Jack',1),('Jack',2),('Jack',3)
,('Jane',3);
Then, in MS Excel, I added (there may be a shorter sequence but I'm still exploring) the three tables as queries from database > sql server database. After adding them, I added all three to the Data Model (I can elaborate if you ask).
I then inserted PivotTable from the ribbon, chose External data source, but opened the Tables tab (instead of Connections tab), to find my data model (mine was top of the list) and I clicked Open. At some point Excel prompted me to create relationships between tables and it did a good job of auto generating them for me.
After minor tweaks my PivotTable came out like this (I could also ask Excel to show the data as a PivotChart).
Pivot showing groups as columns and names as rows.
The advantage is that you don't have to revisit the PIVOT code in SQL if the list (of groups) changes. As I think someone else mentioned, consider using ids for pName as well, or another way to ensure that you are not stuck the next day if you have two persons named John or Jack.
In Excel you can choose when to refresh the data (or the pivot) and, after refresh, any additional categories will be added and counted.

Recommended way to deal with updating m2m table postgres

I have the below tables
A project table
project_id,project_name
A skill table
skill_id,skill_name
A project_skill table (many to many relationship)
project_skill_id,project_id,skill_id
The browser will have a form which asks the user to enter a project name and and SO style autocomplete for tags. I'm sending the below json format back to sql for insertion
{"project_name":"foo","skills":["bar","baz"]}
My question relates to a situation where the user gets to edit an existing project.Assuming the user removes "baz" from skills and includes "zed". How do i properly deal with updating the many to many table
{"project_name":"foo","skills":["bar","zed","biz"]}
Do i remove all records from the m2m table and do a fresh insert with the new skills?
remove all records based on project_id
insert new records of bar,zed,biz
Do i check in the server what was removed/added and remove only what was actually removed
remove baz from table
add biz
This also pertains to modifying project_name etc. Do i check what was modified and update the necessary or perform a complete delete and insert
I'd use a CTE with a MERGE (note this is SQL Server but Postgres should be similar):
;WITH src AS
(
SELECT p.project_id, s.skill_id
FROM
dbo.project AS p
INNER JOIN #input AS i ON p.project_name = i.project_name
INNER JOIN dbo.skill AS s ON i.skill_name = s.skill_name
)
MERGE INTO dbo.project_skill AS tgt
USING src
ON tgt.project_id = src.project_id AND tgt.skill_id = src.skill_id
WHEN NOT MATCHED BY TARGET THEN
INSERT (project_id, skill_id) VALUES (src.project_id, src.skill_id)
WHEN NOT MATCHED BY SOURCE THEN
DELETE;
where #input contains the new values:
DECLARE #input TABLE
(
project_name VARCHAR(100),
skill_name VARCHAR(100)
);

Database upgrade issues with SSDT

I've been using SSDT for my database design for the past 9 months on a project and I'm about to abandon it for DbUp. Hopefully, there's a simpler solution...
Here's the problem. I have a database with the following table:
Persons
-----------
Id (PK)
Name
Email
I would like to upgrade my database to allow a person to have multiple email addresses:
Persons EmailAddresses
----------- ----------------
ID (PK) ID (PK)
Name PersonId (FK)
Email
To do this all within SSDT without dataloss I would need to do some fancy Pre & Post deployment scripting:
-- PreDeployment.sql
IF EXISTS (SELECT 1 FROM sys.Tables where [name] = 'Persons')
BEGIN
CREATE Table TMP_Persons (ID, Name);
CREATE Table TMP_EmailAddresses (ID, PersonId, Email);
SELECT INTO TMP_Persons FROM Persons
SELECT INTO TMP_EmailAddresses FROM Persons
DELETE FROM Persons
END
-- PostDeployment.sql
IF EXISTS (SELECT 1 FROM sys.Tables where [name] = 'TMP_Persons')
BEGIN
SELECT INTO Persons FROM TMP_Persons
SELECT INTO EmailAddresses FROM TMP_EmailAddresses
DROP TABLE TMP_Persons;
DROP TABLE TMP_EmailAddresses;
END
This (although tricky) is do-able and I've been doing this for majority of my changes. However, the problem comes where you have multiple versions of your database. For example, I have the following scenarios:
New Developers - No prior database
Dev Machine - Database is very current
Production - Database is a week or more old
In the event that Production is more out-of-date than the dev machine (possibly from not deploying for a while or from needing to rollback) the above script may fail. This means that the Dev would need to know and take into account prior versions of the database.
For example, say that the Persons table was previously named Users. I would have to account for this possibility in my Pre-Deployment script.
-- PreDeployment.sql
IF EXISTS (SELECT 1 FROM sys.Tables where [name] = 'Users')
BEGIN
CREATE Table TMP_Persons (ID, Name);
CREATE Table TMP_EmailAddresses (ID, PersonId, Email);
SELECT INTO TMP_Persons FROM Users
SELECT INTO TMP_EmailAddresses FROM Users
DELETE FROM Persons
END
IF EXISTS (SELECT 1 FROM sys.Tables where [name] = 'Persons')
BEGIN
CREATE Table TMP_Persons (ID, Name);
CREATE Table TMP_EmailAddresses (ID, PersonId, Email);
SELECT INTO TMP_Persons FROM Persons
SELECT INTO TMP_EmailAddresses FROM Persons
DELETE FROM Persons
END
As time goes on and more variations occur the PreDeployment script is going to get very chaotic and error-prone. This just seems unmanageable to me. Aside from switching to DbUp or something else, is there a better way to do this within SSDT?

A Simple Sql Select Query

I know I am sounding dumb but I really need help on this.
I have a Table (let's say Meeting) which Contains a column Participants.
The Participants dataType is varchar(Max) and it stores Participant's Ids in comma separated form like 1,2.
Now my problem is I am passing a parameter called #ParticipantsID in my Stored Procedure and want to do something like this:
Select Participants from Meeting where Participants in (#ParticipantsID)
Unfortunately I am missing something crucial here.
Can some one point that out?
I've been there before... I changed the DB design to have one record contain a single reference to the other table. If you can't change your DB structures and you have to live with this, I found this solution on CodeProject.
New Function
IF EXISTS(SELECT * FROM sysobjects WHERE ID = OBJECT_ID(’UF_CSVToTable’))
DROP FUNCTION UF_CSVToTable
GO
CREATE FUNCTION UF_CSVToTable
(
#psCSString VARCHAR(8000)
)
RETURNS #otTemp TABLE(sID VARCHAR(20))
AS
BEGIN
DECLARE #sTemp VARCHAR(10)
WHILE LEN(#psCSString) > 0
BEGIN
SET #sTemp = LEFT(#psCSString, ISNULL(NULLIF(CHARINDEX(',', #psCSString) - 1, -1),
LEN(#psCSString)))
SET #psCSString = SUBSTRING(#psCSString,ISNULL(NULLIF(CHARINDEX(',', #psCSString), 0),
LEN(#psCSString)) + 1, LEN(#psCSString))
INSERT INTO #otTemp VALUES (#sTemp)
END
RETURN
END
Go
New Sproc
SELECT *
FROM
TblJobs
WHERE
iCategoryID IN (SELECT * FROM UF_CSVToTable(#sCategoryID))
You would not typically organise your SQL database in quite this way. What you are describing are two entities (Meeting & Participant) that have a one-to-many relationship. i.e. a meeting can have zero or more participants. To model this in SQL you would use three tables: a meeting table, a participant table and a MeetingParticipant table. The MeetingParticipant table holds the links between meetings & participants. So, you might have something like this (excuse any sql syntax errors)
create table Meeting
(
MeetingID int,
Name varchar(50),
Location varchar(100)
)
create table Participant
(
ParticipantID int,
FirstName varchar(50),
LastName varchar(50)
)
create table MeetingParticipant
(
MeetingID int,
ParticipantID int
)
To populate these tables you would first create some Participants:
insert into Participant(ParticipantID, FirstName, LastName) values(1, 'Tom', 'Jones')
insert into Participant(ParticipantID, FirstName, LastName) values(2, 'Dick', 'Smith')
insert into Participant(ParticipantID, FirstName, LastName) values(3, 'Harry', 'Windsor')
and create a Meeting or two
insert into Meeting(MeetingID, Name, Location) values(10, 'SQL Training', 'Room 1')
insert into Meeting(MeetingID, Name, Location) values(11, 'SQL Training', 'Room 2')
and now add some participants to the meetings
insert into MeetingParticipant(MeetingID, ParticipantID) values(10, 1)
insert into MeetingParticipant(MeetingID, ParticipantID) values(10, 2)
insert into MeetingParticipant(MeetingID, ParticipantID) values(11, 2)
insert into MeetingParticipant(MeetingID, ParticipantID) values(11, 3)
Now you can select all the meetings and the participants for each meeting with
select m.MeetingID, p.ParticipantID, m.Location, p.FirstName, p.LastName
from Meeting m
join MeetingParticipant mp on m.MeetingID=mp.MeetingID
join Participant p on mp.ParticipantID=p.ParticipantID
the above should produce
MeetingID ParticipantID Location FirstName LastName
10 1 Room 1 Tom Jones
10 2 Room 1 Dick Smith
11 2 Room 2 Dick Smith
11 3 Room 2 Harry Windsor
If you want to find out all the meetings that "Dick Smith" is in you would write something like this
select m.MeetingID, m.Location
from Meeting m join MeetingParticipant mp on m.MeetingID=mp.ParticipantID
where
mp.ParticipantID=2
and get
MeetingID Location
10 Room 1
11 Room 2
I have omitted important things like indexes, primary keys and missing attributes such as meeting dates, but it is clearer without all the goo.
Your table is not normalized. If you want to query for individual participants, they should be split into their own table, along the lines of:
Meeting
MeetingId primary key
Other stuff
Persons
PersonId primary key
Other stuff
Participants
MeetingId foreign key Meeting(MeetingId)
PersonId foreign key Persons(PersonId)
primary key MeetingId,PersonId
Otherwise, you have to resort to all sorts of trickery (what I call SQL gymnastics) to find out what you want. That trickery never scales well - your queries become slow very quickly as the table grows.
With a properly normalized database, the queries can remain fast well into the multi-millions of records (I work with DB2/z where we are used to truly huge tables).
There are valid reasons for sometimes reverting to second normal form (or even first) for performance but that should be a very hard thought out decision (and based on actual performance data). All databases should initially start of in 3NF.
SELECT * FROM Meeting WHERE Participants LIKE '%,12,%' OR Participants LIKE '12,%' OR Participants LIKE '%,12'
where 12 is the ID you are looking for....
Ugly, what a nasty model.
If I understand your question correctly, you are trying to pass in a comma separated list of participant ids and see if it is in your list. This link lists several ways to do such a thing"
[http://vyaskn.tripod.com/passing_arrays_to_stored_procedures.htm][1]
codezy.blogspot.com
If you store the participant ids in a comma-separated list (as text) in the database, you cannot easily query it (as a list) using SQL. You would have to resort to string-operations.
You should consider changing your schema to use another table to map meetings to participants:
create table meeting_participants (
meeting_id integer not null , -- foreign key
participant_id integer not null
);
That table would have multiple rows per meeting (one for each participant).
You can then query that table for individual participants, or number of participants, and such.
If participants is a separate data type you should be storing it as a child table of your meeting table. e.g.
MEETING
PARTICIPANT 1
PARTICIPANT 2
PARTICIPANT 3
Each participant would hold the meeting ID so you can do a query
SELECT * FROM participants WHERE meeting_id = 1
However, if you must store a comma separated list (for some external reason) then you can do a string search to find the appropriate record. This would be a very inefficient way to do a query though.
That is not the best way to store the information you have.
If it is all you have got then you need to be doing a contains (not an IN). The best answer is to have another table that links Participants to Meetings.
Try SELECT Meeting, Participants FROM Meeting CONTAINS(Participants, #ParticipantId)