Crucial functionality of my app is to search bus stops by name which will return bus stop details as well as all services departing from it. I used php and MySQL and I was able to do so using GROUP_CONCAT. Now I'm planing to move to MS SQL and I did some initial work on this. I'm able to get all needed information using stored procedure like this:
SELECT Id, Name, Latitude, Longitude, Url,
(SELECT [dbo].[RoutesForStopId](Id)) 'Routes'
FROM Stops WHERE Stops.Name LIKE '%' + #Parameter + '%'
Where RoutesForStopId is scalar-valued function:
SELECT #Routes = STUFF((
SELECT DISTINCT ',' + r.ShortName FROM StopTimes st
INNER JOIN trips t ON t.TripId = st.TripId
INNER JOIN routes r ON r.Id = t.RouteId
WHERE st.StopId = #StopId
FOR XML PATH(''), TYPE).value('.', 'NVARCHAR(MAX)'), 1, 1, '')
FROM Stops WHERE Stops.Id = #StopId
It works as expected but response is very slow. I often get timeouts when calling this service.
Is there any way to speed up this query? I would appreciate a help on this as this is important piece of information for end user but being this slow I'm considering to get rid of this. It used to work much faster with MySQL db.
Thanks
Related
I am getting array from front end to perform filters according that inside the SQL query.
I want to apply a LIKE filter on the array. How to add an array inside LIKE function?
I am using Angular with Html as front end and Node as back end.
Array being passed in from the front end:
[ "Sports", "Life", "Relationship", ...]
SQL query is :
SELECT *
FROM Skills
WHERE Description LIKE ('%Sports%')
SELECT *
FROM Skills
WHERE Description LIKE ('%Life%')
SELECT *
FROM Skills
WHERE Description LIKE ('%Relationship%')
But I am getting an array from the front end - how to create a query for this?
In SQL Server 2017 you can use OPENJSON to consume the JSON string as-is:
SELECT *
FROM skills
WHERE EXISTS (
SELECT 1
FROM OPENJSON('["Sports", "Life", "Relationship"]', '$') AS j
WHERE skills.description LIKE '%' + j.value + '%'
)
Demo on db<>fiddle
As an example, for SQL Server 2016+ and STRING_SPLIT():
DECLARE #Str NVARCHAR(100) = N'mast;mode'
SELECT name FROM sys.databases sd
INNER JOIN STRING_SPLIT(#Str, N';') val ON sd.name LIKE N'%' + val.value + N'%'
-- returns:
name
------
master
model
Worth to mention that input data to be strictly controlled, since such way can lead to SQL Injection attack
As the alternative and more safer and simpler approach: SQL can be generated on an app side this way:
Select * from Skills
WHERE (
Description Like '%Sports%'
OR Description Like '%Life%'
OR Description Like '%Life%'
)
A simple map()-call on the words array will allow you to generate the corresponding queries, which you can then execute (with or without joining them first into a single string).
Demo:
var words = ["Sports", "Life", "Relationship"];
var template = "Select * From Skills Where Description Like ('%{0}%')";
var queries = words.map(word => template.replace('{0}', word));
var combinedQuery = queries.join("\r\n");
console.log(queries);
console.log(combinedQuery);
I'm not really good in SQL and I did some searching and couldn't really figured it out how to use STUFF. I mean it looks simple but when I use it, there's always an error.
Anyway, here is the portion of my code and I just want to combine 2 printer of the rows with the same computer name pulled from database, honestly I'm not even sure if I'm doing it right but please do correct me if I'm wrong. Really need the professionals to guide me and do let me know if the information is not sufficient. Thanks in advance.
Update:
Added more code, and I didn't mention clearly, my query is pulling different data from multiple tables. It may be a bit messy but it got what I wanted, my problem is combining the printer result.
SELECT DISTINCT sc.ComputerName AS 'Computer name',
*
*
*
ISNULL(c.PrinterName <-- error on this c.printername
(
STUFF
((
SELECT
'; ' + c.PrinterName
FROM PrinterList AS c
WHERE c.ComputerName = sc.ComputerName
FOR XML PATH(''), TYPE).value('.','nvarchar(max)'), 1, 2,'')), '') AS 'Printer'
FROM Computer sc
Full Join PCInfo AS a
ON a.ComputerName = sc.ComputerName
Full Join DriverList AS b
ON b.ComputerName = sc.ComputerName
Full Join PrinterList AS c
ON c.ComputerName = sc.ComputerName
WHERE RTRIM(LTRIM(REPLACE(lower(sc.ComputerName), ' ',''))) like '%example%'
The c alias is only available in the namespace of the subquery, but you do not reference it from there.
I would expect something like this:
SELECT c.PrinterName,
STUFF( (SELECT '; ' + c.PrinterName
FROM PrinterList AS c
WHERE c.ComputerName = sc.ComputerName
FOR XML PATH(''), TYPE
).value('.', 'nvarchar(max)'
), 1, 2, '')
) as printers
FROM computer c ;
There is no need for the join in the outer query. I also don't see why an outer join would be desired.
I've been working on outputting from SQL to XML and have finally come up with 2 queries that work fabulously for what I need with one problem. I did these independently as I am a novice with XML and now I need to combine them into one file for output. What I have is pretty complicated using dynamic SQL so I'm just going to post the dynamic part and exclude all of the variable and un-pivoting stuff I'm doing at the beginning. If I could just get result set 1 to be at the top and result set 2 just below it would suffice. I would like a wrapper tag (not sure if that's the term) as the very first and very last lines in this file. I'm not sure if anyone can tell me what I'm doing wrong just by looking at the dynamic code but I thought it would be worth a try. As I said, both of these work great but produce 2 separate files. Thanks in advance!
EXEC ('
SELECT
(SELECT GETDATE() AS STARTDATE,
GETDATE() AS ENDDATE
FOR XML PATH(''RECDATE''),ELEMENTS, TYPE),
(SELECT COUNT(*) AS SURVEY_COUNT
FROM PM_TEMP.dbo.tmpKansasCancerCenterExtract FOR XML PATH(''''), ELEMENTS, TYPE),
(SELECT ''PEPM'' as SERVICE,
DataCol as VARNAME,
Question as QUESTION_TEXT,
AnswerValue as ANSWER_TEXT
from PM_TEMP.dbo.tmpKansasCancerCenterCodeSheetExtract FOR XML PATH(''QUESTION''), ROOT(''QUESTION_MAP''), ELEMENTS, TYPE)
FOR XML PATH(''HEADER'')
SELECT PATID,CLIENTID,SERVICE,PATVISITDT,DATE,
(
SELECT VarName,Value
FROM PM_TEMP.dbo.tmpKansasCancerCenterExtract_AnalysisPivot P
INNER JOIN PM_TEMP.dbo.tmpKansasCancerCenterExtract E ON E.PatVisitID = P.PatVisitID
FOR XML PATH(''Response''), TYPE, ROOT(''Analysis'')
),
(
SELECT VarName,Value
FROM PM_TEMP.dbo.tmpKansasCancerCenterExtract_DemoPivot P
INNER JOIN PM_TEMP.dbo.tmpKansasCancerCenterExtract E ON E.PatVisitID = P.PatVisitID
FOR XML PATH(''Response''), TYPE, ROOT(''Demographics'')
)
FROM PM_TEMP.dbo.tmpKansasCancerCenterExtract
FOR XML PATH(''PatientLevelData''),TYPE, ROOT(''PatientLevelData'')
')
Given a table of "events" where each event may be associated with zero or more "speakers" and zero or more "terms", those records associated with the events through join tables, I need to produce a table of all events with a column in each row which represents the list of "speaker_names" and "term_names" associated with each event.
However, when I run my query, I have duplication in the speaker_names and term_names values, since the join tables produce a row per association for each of the speakers and terms of the events:
1|Soccer|Bobby|Ball
2|Baseball|Bobby - Bobby - Bobby|Ball - Bat - Helmets
3|Football|Bobby - Jane - Bobby - Jane|Ball - Ball - Helmets - Helmets
The group_concat aggregate function has the ability to use 'distinct', which removes the duplication, though sadly it does not support that alongside the custom separator, which I really need. I am left with these results:
1|Soccer|Bobby|Ball
2|Baseball|Bobby|Ball,Bat,Helmets
3|Football|Bobby,Jane|Ball,Helmets
My question is this: Is there a way I can form the query or change the data structures in order to get my desired results?
Keep in mind this is a sqlite3 query I need, and I cannot add custom C aggregate functions, as this is for an Android deployment.
I have created a gist which makes it easy for you to test a possible solution: https://gist.github.com/4072840
Look up the speaker/term names independently from each other:
SELECT _id,
name,
(SELECT GROUP_CONCAT(name, ';')
FROM events_speakers
JOIN speakers
ON events_speakers.speaker_id = speakers._id
WHERE events_speakers.event_id = events._id
) AS speaker_names,
(SELECT GROUP_CONCAT(name, ';')
FROM events_terms
JOIN terms
ON events_terms.term_id = terms._id
WHERE events_terms.event_id = events._id
) AS term_names
FROM events
I ran accross this problem as well, but came up with a method that I found a bit easier to comprehend. Since SQLite reports SQLite3::SQLException: DISTINCT aggregates must have exactly one argument, the problem seems not so much related to the GROUP_CONCAT method, but with using DISTINCT within GROUP_CONCAT...
When you encapsulate the DISTINCT 'subquery' within a REPLACE method that actually does nothing you can have the relative simplicity of nawfal's suggestion without the drawback of only being able to concat comma-less strings properly.
SELECT events._id, events.name,
(group_concat(replace(distinct speakers.name),'',''), ' - ') AS speaker_names,
(group_concat(replace(distinct speakers.name),'',''), ' - ') AS term_names
FROM events
LEFT JOIN
(SELECT et.event_id, ts.name
FROM terms ts
JOIN events_terms et ON ts._id = et.term_id
) terms ON events._id = terms.event_id
LEFT JOIN
(SELECT sp._id, es.event_id, sp.name
FROM speakers sp
JOIN events_speakers es ON sp._id = es.speaker_id
) speakers ON events._id = speakers.event_id
GROUP BY events._id;
But actually I would consider this a SQLite bug / inconsistency, or am I missing something?
That's strange that SQLite doesnt support that!.
At the risk of being down voted, only if it helps:
You can avail Replace(X, Y, Z). But you have to be sure you wont have valid , values in your columns..
SELECT events._id, events.name,
REPLACE(group_concat(distinct speakers.name), ',', ' - ') AS speaker_names,
REPLACE(group_concat(distinct terms.name), ',', ' - ') AS term_names
FROM events
LEFT JOIN
(SELECT et.event_id, ts.name
FROM terms ts
JOIN events_terms et ON ts._id = et.term_id
) terms ON events._id = terms.event_id
LEFT JOIN
(SELECT sp._id, es.event_id, sp.name
FROM speakers sp
JOIN events_speakers es ON sp._id = es.speaker_id
) speakers ON events._id = speakers.event_id
GROUP BY events._id;
The problem arises only with the group_concat(X,Y) expression, not with the group_concat(X) expression.
group_concat(distinct X) works well.
So, if the ',' is good for you, there is no problem, but if you want a ';' instead of ',' (and you are sure no ',' is in your original text) you can do:
replace(group_concat(distinct X), ',', ';')
Just to put a proper workaround (murb's answer is strangely parenthesized).
problem:
group_concat(distinct column_name, 'custom_separator') takes custom_separator as a part of distinct.
solution:
We need some no-op to let SQLite know that distinct finished (to wrap distinct and it's arguments).
No-op can be replace with empty string as a second parameter (documentation to replace).
group_concat(replace(distinct column_name, '', ''), 'custom_separator')
edit:
just found that it does not work :-( - can be called but distinct is not working anymore
There is a special case that does not work in sqlite : group_concat(DISTINCT X, Y)
Whereas in SQL you can use group_concat(DISTINCT X SEPARATOR Y) in sqlite you can't
This example : Select group_concat(DISTINCT column1, '|') from example_table group by column2;
gives the result : DISTINCT aggregates must have exactly one argument At line 1:
The solution :
select rtrim(replace(group_concat(DISTINCT column1||'#!'), '#!,', '|'),'#!') from example_table
We are having an issue where we are running the exact same query on two different servers, running the exact same service pack and update - 9.0.4211 - but receiving two differently formatted XML resultsets. Does anyone have any idea why/how this is happening?
Is there a setting/schema involved somehow in the way SQL Server generates the resulting XML format? I understand that we can transform the xml with the PATH option but at this point all of the client code is written to use the format from one of the result sets and using the path options still does not get us what the AUTO option does for the correct format.
Query -
SELECT distinct
operation.operationid,
#zoneId as ZoneID,
member.name, IsNull(member.firstname, '') as firstname, IsNull(member.lastname, '') as lastname, IsNull(member.email, '') as email, IsNull(member.memberid, '') as memberid, IsNull(member.type, '') as type, IsNull(member.AppMemberID, '') as AppMemberID
FROM IQSECURE_TaskOperations operation
join IQSECURE_RoleTasks r on operation.taskid = r.taskid
join IQOBASE_Rosters roster on r.roleid = roster.roleid
left outer join IQObase_Members member on roster.rosterid = member.Rosterid
WHERE operation.operationid = #operationid AND roster.zoneid = #zoneId and member.memberid is not null
union
SELECT distinct
operation.operationid,
#zoneID as ZoneID,
member.name, IsNull(member.firstname, '') as firstname, IsNull(member.lastname, '') as lastname, IsNull(member.email, '') as email, IsNull(member.memberid, '') as memberid, IsNull(member.type, '') as type, IsNull(member.AppMemberID, '') as AppMemberID
FROM IQSECURE_TaskOperations operation
join IQSECURE_RoleTasks r on operation.taskid = r.taskid
join IQOBASE_Rosters roster on r.roleid = roster.roleid
left outer join IQOBASE_Members_Exploded member on roster.rosterid = member.Rosterid
WHERE operation.operationid = #operationid AND roster.zoneid = #zoneid and member.memberid is not null
ORDER BY IsNull(member.type, ''), IsNull(member.lastname, ''), IsNull(member.firstname, ''), member.name
FOR XML AUTO, XMLDATA
Returned xml can be provided if need be.
Thanks in advance for any help provided.
Jon
You can check if there are any differences on the database:
SELECT *
FROM sys.configurations
ORDER BY name ;
And I would also check the client you are using. If you are using SQL Management Studio there is a menu option Tools / Options. This has setting that might affect what you are seeing.
This problem was a two part problem. The two databases were not exactly the same, one had SQL 2000 compatibility mode the other SQL 2005. The one with SQL 2000 was generating the correct XML format. So, one fix was to make the other SQL 2000 compatible. The other solution was to rewrite the select without the UNION. For some reason the UNION caused the incorrect XML format.