mapping many to many items - sql

I'm using database that looks like this the following:
CREATE TABLE Students(
[StudentId] [int],
...,
[MajorId] [int]
);
CREATE TABLE Majors(
[MajorId] [int],
...,
[MajorDes] [varchar](20)
);
CREATE TABLE CourseRequiredForEachMajor(
[MajorId] [int],
...,
[CourseTitle] [varchar](20)
);
CREATE TABLE CoursesCompletedByStudents(
[StudentId] [int],
...,
[CourseTitle][varchar] (20)
);
INSERT INTO Majors ([MajorId] , .. , [MajorDes]) VALUES (1, ... . 'IT');
INSERT INTO Students ([StudentId], .. , [MajorId]) VALUES (1, ... . 1);
INSERT INTO CourseRequiredForEachMajor ([MajorId], .. , [CourseTitle]) VALUES (1, ... . 'IT101');
INSERT INTO CourseRequiredForEachMajor ([MajorId], .. , [CourseTitle]) VALUES (1,...,'IT302');
INSERT INTO CourseRequiredForEachMajor ([MajorId], .. , [CourseTitle]) VALUES(1,...,'IT321');
INSERT INTO CoursesCompletedByStudents ([StudentId], .. , [CourseTitle]) VALUES (1, ... . 'IT101');
INSERT INTO CoursesCompletedByStudents ([StudentId], .. , [CourseTitle]) VALUES (1,...,'IT302');
INSERT INTO CoursesCompletedByStudents ([StudentId], .. , [CourseTitle]) VALUES(2,...,'IT321');
I'm trying to list the missing courses for each student in order to complete the all the requirements for each major ( e.g. in the image above, I want to say that studentId# 1 need to take course IT321 in order to complete all requirement for majorId #1 ).
Also is there anyway that I can do to show percentage of completion ? ( e.g. studentId#1 completed two courses ( 66.6% ) out of 3 courses that are required by the major ). what is the way to do the calculations of this.
I really have no idea how to solve this, but this is my attempt:
Select CoursesCompletedByStudents.CourseTitle from Students,MajorsCourseRequiredForEachMajor,CoursesCompletedByStudents
where Students.MajorId= Major.MajorId and Studetns.StudentId = CoursesCompletedByStudents.[StudentId]
and CourseRequiredForEachMajor.[CourseTitle]=CoursesCompletedByStudents.[CourseTitle]

This query would give the required result.
with coursesCTE as
(
Select StudentId, CourseTitle, m.MajorId
from Students s
join Majors m
on s.MajorId = m.MajorID
join CourseRequiredForEachMajor cr
on cr.MajorId = m.MajorID
),
completedCTE
as
(
Select StudentId, cc.courseTitle, MajorId
from CoursesCompletedByStudents cc
join CourseRequiredForEachMajor cm
on cc.CourseTitle = cm.CourseTitle
)
Select * from coursesCTE
except
Select * from completedCTE
If you want comma separated course Ids, you can use the Stuff/XML path hack or STRING_AGG if you're using SQL Server 2017

Related

Is it possible to insert multiple values using insert in sql

Is it possible to insert in one single query multiple values into a table ? .
I have declared this table
declare global temporary table CFVariables
(
CF varchar(255)
)
with replace ;
then i inserted values into the table
INSERT INTO qtemp.CFVariables ( CF ) VALUES
('F01' ), ('T01' ), ('U01' ), ('CIP' ), ('L01' )
Is it possible to not insert the values in qtemp.CFVariables table this way ? but like In ('F01' , 'T01' , 'U01' , 'CIP' , 'L01' )
Then , i declared my second table :
declare global temporary table xVariables
(
CFC numeric(3),
CF varchar(255)
)
with replace ;
In this part i'm having a problem to insert into my table xVariables
I tried to use this to insert multiple values
INSERT INTO qtemp.xVariables ( CFC, CF ) VALUES
( 1, (select CF from qtemp.CFVariables ))
My query field because i'm inserting more then one row to the table .
How can i achieve this ?
Try
INSERT INTO qtemp.xVariables ( CFC, CF ) SELECT 1 AS CFC,CF from qtemp.CFVariables;
Try running an insert-select:
INSERT INTO qtemp.xVariables ( CFC, CF )
select 1, CF from qtemp.CFVariables
To restrict the records to be inserted, you will need to do something like this:
INSERT INTO qtemp.xVariables ( CFC, CF )
select 1, CF
from qtemp.CFVariables
where CF in ('F01' , 'T01' , 'U01' , 'CIP' , 'L01' )

'Merge Fields' - alike SQL Server function

I try to find a way to let the SGBD perform a population of merge fields within a long text.
Create the structure :
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
Now, create test values
INSERT INTO [store] (id, text)
VALUES (1, 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.')
INSERT INTO [statement] (id, store_id)
VALUES (1, 1)
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO [statement_merges] (statement_id, merge_field, user_data)
VALUES (1, '$$ASKED$$', '12')
At the time being my app is delivering the final statement, looping through merges, replacing in the stored text and output
Waw, stackoverflow is an amazing library of lost people in the IT
hell, and i have the feeling that 85% of the users found a solution,
personally I asked 12 questions.
I try to find a way to be code-independent and serve the output in a single query, as u understood, select a statement in which the stored text have been populated with user data. I hope I'm clear.
I looked on TRANSLATE function but it looks like a char replacement, so I have two choices :
I try a recursive function, replacing one by one until no merge_fields is found in the calculated text; but I have doubts about the performance of this approach;
There is a magic to do that but I need your knowledge...
Consider that I want this because the real texts are very long, and I don't want to store it more than once in my database. You can imagine a 3 pages contract with only 12 parameters, like start date, invoiced amount, etc... Everything else cant be changed for compliance.
Thank you for your time!
EDIT :
Thanks to Randy's help, this looks to do the trick :
WITH cte_replace_tokens AS (
SELECT replace(r.text, m.merge_field, m.user_data) as [final], m.merge_field, s.id, 1 AS i
FROM store r
INNER JOIN statement s ON s.store_id = r.id
INNER JOIN statement_merges m ON m.statement_id = s.id
WHERE m.statement_id = 1
UNION ALL
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field
)
select TOP 1 final from cte_replace_tokens ORDER BY i DESC
I will check with a bigger database if the performance is good...
At least, I can "populate" one statement, I need to figure out to be able to extract a list as well.
Thanks again !
If a record is updated more than once by the same update, the last wins. None of the updates are affected by the others - no cumulative effect. It is possible to trick SQL using a local variable to get cumulative effects in some cases, but it's tricky and not recommended. (Order becomes important and is not reliable in an update.)
One alternate is recursion in a CTE. Generate a new record from the prior as each token is replaced until there are no tokens. Here is a working example that replaces 1 with A, 2 with B, etc. (I wonder if there is some tricky xml that can do this as well.)
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
WITH cte_replace_tokens AS (
SELECT r.test, replace(r.test, l.token, l.value) as [final], l.id
FROM [Raw] r
CROSS JOIN #Token l
WHERE l.id = 1
UNION ALL
SELECT r.test, replace(r.final, l.token, l.value) as [final], l.id
FROM cte_replace_tokens r
CROSS JOIN #Token l
WHERE l.id = r.id + 1
)
select * from cte_replace_tokens where id = 6
It's not recommended to do such tasks inside sql engine but if you want to do that, you need to do it in a loop using cursor in a function or stored procedure like so :
DECLARE #merge_field nvarchar(30)
, #user_data nvarchar(MAX)
, #statementid INT = 1
, #text varchar(MAX) = 'Waw, stackoverflow is an amazing library of lost people in the IT hell, and i have the feeling that $$PERC_SAT$$ of the users found a solution, personally I asked $$ASKED$$ questions.'
DECLARE merge_statements CURSOR FAST_FORWARD
FOR SELECT
sm.merge_field
, sm.user_data
FROM dbo.statement_merges AS sm
WHERE sm.statement_id = #statementid
OPEN merge_statements
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
WHILE ##FETCH_STATUS = 0
BEGIN
set #text = REPLACE(#text , #merge_field, #user_data )
FETCH NEXT FROM merge_statements
INTO #merge_field , #user_data
END
CLOSE merge_statements
DEALLOCATE merge_statements
SELECT #text
Here is a recursive solution.
SQL Fiddle
MS SQL Server 2017 Schema Setup:
CREATE TABLE [dbo].[store]
(
[id] [int] NOT NULL,
[text] [nvarchar](MAX) NOT NULL
)
CREATE TABLE [dbo].[statement]
(
[id] [int] NOT NULL,
[store_id] [int] NOT NULL
)
CREATE TABLE [dbo].[statement_merges]
(
[statement_id] [int] NOT NULL,
[merge_field] [nvarchar](30) NOT NULL,
[user_data] [nvarchar](MAX) NOT NULL
)
INSERT INTO store (id, text)
VALUES (1, '$$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions.')
INSERT INTO store (id, text)
VALUES (2, 'Use The #_#')
INSERT INTO statement (id, store_id) VALUES (1, 1)
INSERT INTO statement (id, store_id) VALUES (2, 2)
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$PERC_SAT$$', '85%')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$ASKED$$', '12')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (1, '$$(*)$$', 'Wow')
INSERT INTO statement_merges (statement_id, merge_field, user_data) VALUES (2, ' #_#', 'Flux!')
Query 1:
;WITH Normalized AS
(
SELECT
store_id=store.id,
store.text,
sm.merge_field,
sm.user_data,
RowNumber = ROW_NUMBER() OVER(PARTITION BY store.id,sm.statement_id ORDER BY merge_field),
statement_id = st.id
FROM
store store
INNER JOIN statement st ON st.store_id = store.id
INNER JOIN statement_merges sm ON sm.statement_id = st.id
)
, Recurse AS
(
SELECT
store_id, statement_id, old_text = text, merge_field,user_data, RowNumber,
Iteration=1,
new_text = REPLACE(text, merge_field, user_data)
FROM
Normalized
WHERE
RowNumber=1
UNION ALL
SELECT
n.store_id, n.statement_id, r.old_text, n.merge_field, n.user_data,
RowNumber=r.RowNumber+1,
Iteration=Iteration+1,
new_text = REPLACE(r.new_text, n.merge_field, n.user_data)
FROM
Normalized n
INNER JOIN Recurse r ON r.RowNumber = n.RowNumber AND r.statement_id = n.statement_id
)
,ReverseOnIteration AS
(
SELECT *,
ReverseIteration = ROW_NUMBER() OVER(PARTITION BY statement_id ORDER BY Iteration DESC)
FROM
Recurse
)
SELECT
store_id, statement_id, new_text, old_text
FROM
ReverseOnIteration
WHERE
ReverseIteration=1
Results:
| store_id | statement_id | new_text | old_text |
|----------|--------------|------------------------------------------|--------------------------------------------------------------|
| 1 | 1 | Wow, stackoverflow...85%...12 questions. | $$(*)$$, stackoverflow...$$PERC_SAT$$...$$ASKED$$ questions. |
| 2 | 2 | Use TheFlux! | Use The #_# |
With the help of Randy, I think I've achieved what I wanted to do !
Known the fact that my real case is a contract, in which there are several statements that may be :
free text
stored text without any merges
stored text with one or
several merges
this CTE does the job !
WITH cte_replace_tokens AS (
-- The initial query dont join on merges neither on store because can be a free text
SELECT COALESCE(r.text, s.part_text) AS [final], CAST('' AS NVARCHAR) AS merge_field, s.id, 1 AS i, s.contract_id
FROM statement s
LEFT JOIN store r ON s.store_id = r.id
UNION ALL
-- We loop till the last merge field, output contains iteration to be able to keep the last record ( all fields updated )
SELECT replace(r.final, m.merge_field, m.user_data) as [final], m.merge_field, r.id, r.i + 1 AS i, r.contract_id
FROM cte_replace_tokens r
INNER JOIN statement_merges m ON m.statement_id = r.id
WHERE m.merge_field > r.merge_field AND r.final LIKE '%' + m.merge_field + '%'
-- spare lost replacements by forcing only one merge_field per loop
AND NOT EXISTS( SELECT mm.statement_id FROM statement_merges mm WHERE mm.statement_id = m.statement_id AND mm.merge_field > r.merge_field AND mm.merge_field < m.merge_field)
)
select s.id,
(select top 1 final from cte_replace_tokens t WHERE t.contract_id = s.contract_id AND t.id = s.id ORDER BY i DESC) as res
FROM statement s
where contract_id = 1
If the CTE solution with a cross join is too slow, an alternate solution would be to build a scalar fn dynamically that has every REPLACE required from the token table. One scalar fn call per record then is order(N). I get the same result as before.
The function is simple and likely not to be too long, depending upon how big the token table becomes...256 MB batch limit. I've seen attempts to dynamically create queries to improve performance backfire - moved the problem to compile time. Should not be a problem here.
if not object_id('tempdb..#Raw') is null drop table #Raw
CREATE TABLE #Raw(
[test] [varchar](100) NOT NULL PRIMARY KEY CLUSTERED,
)
if not object_id('tempdb..#Token') is null drop table #Token
CREATE TABLE #Token(
[id] [int] NOT NULL PRIMARY KEY CLUSTERED,
[token] [char](1) NOT NULL,
[value] [char](1) NOT NULL,
)
insert into #Raw values('123456'), ('1122334456')
insert into #Token values(1, '1', 'A'), (2, '2', 'B'), (3, '3', 'C'), (4, '4', 'D'), (5, '5', 'E'), (6, '6', 'F');
DECLARE #sql varchar(max) = 'CREATE FUNCTION dbo.fn_ReplaceTokens(#raw varchar(8000)) RETURNS varchar(8000) AS BEGIN RETURN ';
WITH cte_replace_statement AS (
SELECT a.id, CAST('replace(#raw,''' + a.token + ''',''' + a.value + ''')' as varchar(max)) as [statement]
FROM #Token a
WHERE a.id = 1
UNION ALL
SELECT n.id, CAST(replace(l.[statement], '#raw', 'replace(#raw,''' + n.token + ''',''' + n.value + ''')') as varchar(max)) as [statement]
FROM #Token n
INNER JOIN cte_replace_statement l
ON n.id = l.id + 1
)
select #sql += [statement] + ' END' from cte_replace_statement where id = 6
print #sql
if not object_id('dbo.fn_ReplaceTokens') is null drop function dbo.fn_ReplaceTokens
execute (#sql)
SELECT r.test, dbo.fn_ReplaceTokens(r.test) as [final] FROM [Raw] r

SQL Server: Insert batch with output clause

I'm trying the following
Insert number of records to table A with a table-valued parameter (tvp). This tvp has extra column(s) that are not in A
Get the inserted ids from A and the corresponding extra columns in the the tvp and add them to another table B
Here's what I tried
Type:
CREATE TYPE tvp AS TABLE
(
id int,
otherid int,
name nvarchar(50),
age int
);
Tables:
CREATE TABLE A (
[id_A] [int] IDENTITY(1,1) NOT NULL,
[name] [varchar](50),
[age] [int]
);
CREATE TABLE B (
[id_B] [int] IDENTITY(1,1) NOT NULL,
[id_A] [int],
[otherid] [int]
);
Insert:
DECLARE #a1 AS tvp;
DECLARE #a2 AS tvp
-- create a tvp (dummy data here - will be passed to as a param to an SP)
INSERT INTO #a1 (name, age, otherid) VALUES ('yy', 10, 99999), ('bb', 20, 88888);
INSERT INTO A (name, age)
OUTPUT
inserted.id_A,
inserted.name,
inserted.age,
a.otherid -- <== isn't accepted here
INTO #a2 (id, name, age, otherid)
SELECT name, age FROM #a1 a;
INSERT INTO B (id_A, otherid) SELECT id, otherid FROM #a2
However, this fails with The multi-part identifier "a.otherid" could not be bound., which I guess is expected because columns from other tables are not accepted for INSERT statement (https://msdn.microsoft.com/en-au/library/ms177564.aspx).
from_table_name
Is a column prefix that specifies a table included in the FROM clause of a DELETE, UPDATE, or MERGE statement that is used to specify the rows to update or delete.
So is there any other way to achieve this?
You cannot select value from a source table by using INTO operator.
Use OUTPUT clause in the MERGE command for such cases.
DECLARE #a1 AS tvp;
DECLARE #a2 AS tvp
INSERT INTO #a1 (name, age, otherid) VALUES ('yy', 10, 99999), ('bb', 20, 88888);
MERGE A a
USING #a1 a1
ON a1.id =a.[id_A]
WHEN NOT MATCHED THEN
INSERT (name, age)
VALUES (a1.name, a1.age)
OUTPUT inserted.id_A,
a1.otherId,
inserted.name,
inserted.age
INTO #a2;
INSERT INTO B (id_A, otherid) SELECT id, otherid FROM #a2

How to use merge statement to split one staging tables into two for loading from Staging to RealtionalDB?

I have following tables. I want to insert values into companyGroup and Comapany from test1 table. what would be better way CTE or Using Merge directly and how can i do that using tsql. Test1 is on database A and company and companygroup are on database B.
create table test1
(
companyID int identity
,CompanyName Varchar(50)
,[Group] Varchar(5)
)
INSERT INTO (CompanyName, [Group]) values ('Unknown', '0')
INSERT INTO (CompanyName, [Group]) values ('APPLE', 'IOS')
INSERT INTO (CompanyName, [Group]) values ('Google', 'Android')
INSERT INTO (CompanyName, [Group]) values ('Samsung', 'Android')
INSERT INTO (CompanyName, [Group]) values ('Lg', 'IOS')
create table CompanyGroup
(
Groupkey int identity (0,1) primary key
,GroupName varchar(5)
) ;
INSERT INTO (GroupName) VALUES ('Unknown')
create table Company
(
compnayKey int identity (0,1) primary key
,CompanyName Varchar(50)
,companyGroupKey int References CompanyGroup(GroupKey)
)
INSERT INTO (CompanyName,companyGroupKey) VALUES ('Unknown',0)
when I insert the values into company how can i convert group to int ? Any ideas? What would be Best TSQL for this load.
Try using this:
INSERT INTO CompanyGroup(GroupName)
SELECT [Group]
FROM test1
WHERE NOT EXISTS(SELECT 1
FROM CompanyGroup
WHERE GroupName = [Group]);
MERGE Company AS target
USING (SELECT t.CompanyName, g.Groupkey
FROM test1 AS t
LEFT JOIN CompanyGroup AS g ON t.[Group] = g.GroupName
) AS source
ON (target.CompanyName = source.CompanyName
AND target.companyGroupKey = source.Groupkey)
WHEN NOT MATCHED THEN
INSERT (CompanyName, companyGroupKey)
VALUES (source.CompanyName, source.Groupkey);

SQL Spatial JOIN By Nearest Neighbor

In the data below I am looking for a query so I can join the results of the 2 tables by nearest neighbor.
Some of the results in dbo.Interests Table will not be in the dbo.Details Table,
This Question finds the k nearest poins for a single point, I need this query to further reconcile data between the 2 tables
How can I extend this SQL query to find the k nearest neighbors?
IF OBJECT_ID('dbo.Interests', 'U') IS NOT NULL DROP TABLE dbo.Interests;
IF OBJECT_ID('dbo.Details', 'U') IS NOT NULL DROP TABLE dbo.Details;
CREATE TABLE [dbo].[Interests](
[ID] [int] IDENTITY(1,1) NOT NULL,
[NAME] [nvarchar](255) NULL,
[geo] [geography] NULL,
)
CREATE TABLE [dbo].[Details](
[ID] [int] IDENTITY(1,1) NOT NULL,
[NAME] [nvarchar](255) NULL,
[geo] [geography] NULL,
)
/*** Sample Data ***/
/* Interests */
INSERT INTO dbo.Interests (Name,geo) VALUES ('Balto the Sled Dog',geography::STGeomFromText('POINT(-73.97101284538104 40.769975451779729)', 4326));
INSERT INTO dbo.Interests (Name,geo) VALUES ('Albert Bertel Thorvaldsen',geography::STGeomFromText('POINT(-73.955996808113582 40.788611756916609)', 4326));
INSERT INTO dbo.Interests (Name,geo) VALUES ('Alice in Wonderland',geography::STGeomFromText('POINT(-73.966714294355356 40.7748020248959)', 4326));
INSERT INTO dbo.Interests (Name,geo) VALUES ('Hans Christian Andersen',geography::STGeomFromText('POINT(-73.96756141015176 40.774416211045626)', 4326));
/* Details */
INSERT INTO dbo.Details(Name,geo) VALUES ('Alexander Hamilton',geography::STGeomFromText('POINT(-73.9645616688172 40.7810234271951)', 4326));
INSERT INTO dbo.Details(Name,geo) VALUES ('Arthur Brisbane',geography::STGeomFromText('POINT(-73.953249720745731 40.791599412827864)', 4326));
INSERT INTO dbo.Details(Name,geo) VALUES ('Hans Christian Andersen',geography::STGeomFromText('POINT(-73.9675614098224 40.7744162102582)', 4326));
INSERT INTO dbo.Details(Name,geo) VALUES ('Balto',geography::STGeomFromText('POINT(-73.9710128455336 40.7699754516397)', 4326));
A brute force approach will be to compute the distance between all details X interests records:
SELECT *
FROM
(
SELECT *, rank=dense_rank() over (partition by IName, IGeo order by dist asc)
FROM
(
SELECT D.NAME as DName, D.geo as DGeo,
I.NAME as IName, I.geo as IGeo,
I.geo.STDistance(D.geo) AS dist
FROM dbo.Details D CROSS JOIN dbo.Interests I
) X
) Y
WHERE rank <= #k
NotE: #k is the target number of matches you are after