How do I remove duplicates in the following case in T-SQL?
I have a table with a column Code of type varchar(max).
It contains column value like truck/rail/truck/rail. I need the cell value to be truck/rail.
Other possibility is truck/rail/ship/truck need to be truck/rail/ship.
By using table valued function.
Thanks.
You can use String_Split along with String_agg to remove the duplicates.
DECLARE #t table(id int, val varchar(max))
insert into #t values(1,'truck/rail/truck/rail'), (2,'truck/rail/ship/truck')
SELECT t.id,STRING_AGG(splitval,'/') as newval FROM #t as t
cross apply (
SELECT distinct value from string_split(t.val,'/')) as ca(splitval)
group by t.id
id
newval
1
rail/truck
2
rail/ship/truck
Note1: String_Split, does not guarantee order. So, your concatenated results might be in different order from the original list, after duplicates removal. If you want to preserve the order, then we have to go for different solution using xml nodes or json array.
Note2: String_Split was introduced in SQL Server 2016. String_agg was introduced in SQL Server 2017. So, if you are using versions before that, you have to go for recursive CTE and CHARINDEX based solution.
If you know that the error exists, then just do an UPDATE where you replace the truck/rail/truck/rail with truck/rail using the REPLACE(Code,'truck/rail/truck/rail',truck/rail).
The same goes for your truck/rail/ship/truck issue.
If you need automatic detection and correction to be done, that's a whole 'nuther story but could still be done using nested REPLACES. Detection of the issue is the hard part. Personally, I'd be having a talk with the people that are providing the data.
Related
I have a column in a table that is json. It contains several columns within it.
Example:
Row1: "sTCounts":[{"dpsTypeTest":"TESTTRIAL","cnt":3033244.0}
Row2: "sTCounts":[{"dpsTypeTest":"TESTTRIAL","cnt":3.3}
I need to sum the cnt value for all rows in table. For instance, the above would produce a result of 3033247.3
I'm not familiar with stored procs enough to master. I thought the easiest route would be to create a temp table and extract the value into a column, and then write a query to sum the column values.
The problem is that it creates a column with datatype nvarchar(4000). It won't let me sum that column. I thought of changing the datatype but not sure how. I am trying CAST without luck.
select CAST(json AS varchar) AS JSON_VALUE(jsontext,
'$.sTCounts.cnt') AS PerfCount, TitleNumber
INTO dbo_Testing_Count0
from PerformanceTest
select sum(PerfCount)
from dbo_Testing_Count
Group by PerfCount
The error message is:
Incorrect syntax near 'jsontext'.
Any ideas? I am open to another method to sum the column or changing the datatype whichever the experts can aid on. I appreciate it.
The JSON you provide in your question is not valid... This seems to be just a fragment of a larger JSON. As your data starts with a [ you have to think of it as an array, so the simple json path '$.serviceTierCounts.cnt' won't work probably...
Try this, I've added the opening { and the closing brackets at the end:
DECLARE #mockupTable TABLE(ID INT IDENTITY, YourJson NVARCHAR(MAX));
INSERT INTO #mockupTable VALUES
(N'{"serviceTierCounts":[{"dpsType":"TRIAL","cnt":3033244.0}]}')
,(N'{"serviceTierCounts":[{"dpsType":"TRIAL","cnt":3.3}]}');
--You can read one scalar value using JSON_VALUE directly with a cast. But in this case I need to add [0]. This will tell the engine to read the first (zero-based index!) object's cnt property.
SELECT CAST(JSON_VALUE(YourJson,'$.serviceTierCounts[0].cnt') AS DECIMAL(14,4))
FROM #mockupTable
--But I think, that it's this what you are looking for:
SELECT *
FROM #mockupTable
CROSS APPLY OPENJSON(YourJson,'$.serviceTierCounts')
WITH(dpsType varchar(100)
,cnt decimal(14,4));
The WITH clause will return the object in typed columns side-by-side.
For easy proceeding, you can wrap this as a CTE and continue with the set in the following SELECT.
How to get Order Numbers(189,190) from following TrigParams field in SQL server 2014 Database.
TrigParams
{"OWLSObjKey":{"key":"OWLSObjKey","value":"189","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"189","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}
{"OWLSObjKey":{"key":"OWLSObjKey","value":"190","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"190","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}
You see from the other answers, that SQL-Server 2016+ with broad JSON support would be of great help. But without you are not lost. You can use string methods:
credits to Panagiotis Kanavos for the MCVE
declare #table table (trigparams nvarchar(2000))
insert into #table
values
('{"OWLSObjKey":{"key":"OWLSObjKey","value":"189","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"189","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}'),
('{"OWLSObjKey":{"key":"OWLSObjKey","value":"190","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"190","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}')
--This is the query
select LEFT(CutOff,CHARINDEX('"',CutOff)-1)
from #table t
CROSS APPLY(SELECT STUFF(t.trigparams,1,CHARINDEX('"value":"',t.trigparams)+8,'')) A(CutOff);
The idea in short:
Within the APPLY we will use STUFF() to write nothing over the first characters up to the number you are looking for (after the first occurance of "value":". This is returned as Column CutOff. We can now use LEFT() to pick the number only.
The question doesn't specify the server version, the table schema or whether the string represents a single value or the contents of two separate rows.
I'll just assume it's SQL Server 2016 which supports JSON and the text comes from two separate rows. I'll also assume the query only needs to return the data as single value.
In this case a simple call to JSON_VALUE('$.OWLSObjKey.value') will return the data. JSON_VALUE returns a single value from a well-formed JSON string :
declare #table table (trigparams nvarchar(2000))
insert into #table
values
('{"OWLSObjKey":{"key":"OWLSObjKey","value":"189","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"189","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}'),
('{"OWLSObjKey":{"key":"OWLSObjKey","value":"190","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"190","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}')
select JSON_VALUE(trigparams,'$.OWLSObjKey.value') As SomeKey
from #table
This returns :
SomeKey
189
190
Assuming you're using SQL Server 2016+, use OPENJSON:
SELECT O.OrderNumber
FROM (VALUES('{"OWLSObjKey":{"key":"OWLSObjKey","value":"189","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"189","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}'),
('{"OWLSObjKey":{"key":"OWLSObjKey","value":"190","type":null},"OWLSObjType":{"key":"OWLSObjType","value":"17","type":null},"ObjKey":{"key":"ObjKey","value":"190","type":null},"ObjType":{"key":"ObjType","value":"17","type":null}}')) V(TrigParams)
CROSS APPLY OPENJSON(V.TrigParams) WITH (OWLSObjKey nvarchar(MAX) AS JSON) OK
CROSS APPLY OPENJSON(OK.OWLSObjKey) WITH (OrderNumber int '$.value') O;
If not, then SQL Server is not your friend for this without using a CLR Function. If you can't use CLR, then I would suggest using a different tool to read the JSON value.
I need to re-assign all SortID's, starting from 1 until MAX (SortID) from a subset of records of table Beleg, using SQL-92, after one of the SortID's has changed (for example from 444 to 444.1). I have tried several ways (for example SET #a:=0; UPDATE table SET field=#a:=#a+1 WHERE whatever='whatever' ORDER BY field2), but it didn't work, as these solutions all need a special kind of SQL, like SQLServer or Oracle, etc.
The SQL that I use is SQL-92, implemented in FileMaker (INSERT and UPDATE are available, though, but nothing fancy).
Thanks for any hint!
Gary
From what I know, SQL-92 is a standard and not a language. So you can say you are using T-SQL, which is mostly SQL-92 compliant, but you can't say I program SQL Server in SQL-92. The same applies to FileMaker.
I suppose you are trying to update your table through ODBC? The Update statement looks OK, but there are no variables if FileMaker SQL (and I am not sure using a variable inside query will give you result you expect, I think you will set SortId in every row to 1). You are thinking about doing something like Window functions with row() in TSQL, but I do not think this functionality is available.
The easiest solution is to use FileMaker, resetting the numbering for a column is really a trivial task which takes seconds. Do you need help with this?
Edit:
I was referring to TSQL functions rank() and row_number(), there is no row() function in TSQL
I finally got the answer from Ziggy Crueltyfree Zeitgeister on the Database Administrators copy of my question.
He suggested to break this down into multiple steps using a temporary table to store the results:
CREATE TABLE sorting (sid numeric(10,10), rn int);
INSERT INTO sorting (sid, rn)
SELECT SortID, RecordNumber FROM Beleg
WHERE Year ( Valuta ) = 2016
AND Ursprungskonto = 1210
ORDER BY SortID;
UPDATE Beleg SET SortID = (SELECT rn FROM sorting WHERE sid=Beleg.SortID)
WHERE Year ( Valuta ) = 2016
AND Ursprungskonto = 1210;
DROP TABLE sorting;
Of course! I just keep the table definition in Filemaker (let the type coercion be done by Filemaker this way), and filling and deleting from it with my function: RenumberSortID ().
I have a database column containing a string that might look something like this u/1u/3u/19/g1/g4 for a particular row.
Is there a performant way to get all rows that have at least one of the following elements ['u/3', 'g4'] in that column?
I know I can use AND clauses, but the number of elements to verify against varies and could become large..
I am using RoR/ActiveRecord in my project.
in sql server, you can use XML to convert your list of search params into a record set, then cross join that with the base table, and do charIndex() to see if the column contains the substring.
Since i don't know your table or column names, i used a table (persons) that i already had data in, which has a column 'phone_home'. To search for any phone number that contains '202' or '785' i would use this query:
select person_id,phone_home,Split.data.value('.', 'VARCHAR(10)')
from (select *, cast('<n>202</n><n>785</n>' as XML) as myXML
from persons) as data cross apply myXML.nodes('/n') as Split(data)
where charindex(Split.data.value('.', 'VARCHAR(10)'),data.phone_Home) > 0
you will get duplicate records if it matches more than one value, so throw a distinct in there and remove the Split from the select statement if that is not desired.
Using xml in sql is voodoo magic to me...i got the idea from this post http://www.sqljason.com/2010/05/converting-single-comma-separated-row.html
no idea what performance is like...but at least there aren't any cursors or dynamic sql.
EDIT: Casting the XML is pretty slow, so i made it a variable so it only gets cast once.
declare #xml XML
set #xml = cast('<n>202</n><n>785</n>' as XML)
select person_id,phone_home,Split.persons.value('.', 'VARCHAR(10)')
from persons cross apply #xml.nodes('/n') as Split(persons)
where charindex(Split.persons.value('.', 'VARCHAR(10)'),phone_Home) > 0
I have a column that stores data like (42,12). Now I want to fetch 42 or 12 (two different select queries). I have searched and found some similar but much more complex scenarios. Is there any easy way of doing it? I am using MSSQL Server 2005.
Given there will always be only two values and they will be integer
The reason you have this problem is because the database (which you may not have any control over), violates first normal form. Among other things, first normal form says that each column should hold a single value, not multiple values. This is bad design.
Now, having said this, the first solution that pops into my head is to write a UDF that parses the value in this column, based on the delimiter and returns either the first or second value.
You can try something like this
DECLARE #Table TABLE(
Val VARCHAR(50)
)
INSERT INTO #Table (Val) SELECT '42,12'
SELECT *,
CAST(LEFT(Val,CHARINDEX(',',Val)-1) AS INT) FirstValue,
CAST(RIGHT(Val,LEN(Val) - CHARINDEX(',',Val)) AS INT) SecondValue
FROM #Table
You can use something like this:
SELECT SUBSTRING_INDEX(field, ',', 1)
Note: Its not the efficient way of doing things in rdbms. Consider normalizing your Database.