The file is valid from what I can tell, it doesn't have "blanks" or other common issues that create this error. Using SQL Server Tools 2018, get the same error using Visual Studio 2019 running SSIS. What am I doing wrong here?
EDIT POST SOLUTION - I would recommend using OUTER APPLY instead of the CROSS APPLY here. Null values will cause rows to be lost otherwise. Hopefully save someone else my headaches!
JSON Example:
{"organizationAffiliations": [{"organizationId": 2001, "locationId": 3029960, "planIds": [5, 13, 19, 20, 24, 40]}]}
SQL Code in Question
-- organization affiliations query -- This should work but doesn't --
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON=BulkColumn
FROM OPENROWSET (BULK 'C:\FileDirectory\filename.json', SINGLE_CLOB) IMPORT
SELECT
JSON_Value (c.value, '$.organizationId') as organizationId,
JSON_Value (c.value, '$.locationId') as locationId,
JSON_Value (p.value, '$.planIds') as planIds
FROM OPENJSON (#JSON, '$.organizationAffiliations') as c
CROSS APPLY OPENJSON (c.value, '$.planIds') as p
PlanIDs is an array. Based on the CROSS APPLY, I'm assuming this is the structure you are looking for
SELECT
JSON_Value (c.value, '$.organizationId') as organizationId,
JSON_Value (c.value, '$.locationId') as locationId,
P.value as planID
FROM OPENJSON (#JSON, '$.organizationAffiliations') as c
CROSS APPLY OPENJSON (c.value, '$.planIds') as p
Results
Related
I need to "translate" this statement to SQL server
regexp_replace(Main.LOCK, '\/\/.*', '') TARGET
I need to get rid of this signs because before (or after, depends how you look) I use this one
LISTAGG(stock.LOCATION_NO, '//') WITHIN GROUP (ORDER BY isnull(QTY_OH,0)+isnull(QTY_TR,0) - isnull(QTY_RS, 0) desc) LOCK
Neither Regex and Listagg can be used within SQL server
What you see, what I'm trying to do (and it worked very well in Oracle) is to get the TARGET value that contains Main.LOCK with MAXIMUM value of
isnull(QTY_OH,0)+isnull(QTY_TR,0) - isnull(QTY_RS, 0)
Now I can't translate it properly to SQL server
Also, the error I've get are:
Msg 195, Level 15, State 10, Line 12
'regexp_replace' is not a recognized built-in function name.
Msg 10757, Level 15, State 1, Line 49
The function 'LISTAGG' may not have a WITHIN GROUP clause.
Can anyone help here?
SQL Server ver 18.8
Warehouse Ver 13.0
Microsoft SQL Server 2016 (SP2-GDR) (KB4583460) - 13.0.5103.6 (X64)
Nov 1 2020 00:13:28
Copyright (c) Microsoft Corporation
Standard Edition (64-bit) on Windows Server 2016 Standard 10.0 (Build 14393: ) (Hypervisor)
The regexp_replace() is doing something pretty simple. It is taking the portion of the string before '//', if that is there.
In the more recent versions of SQL Server, you can use string_agg() and left():
string_agg(left(main.lock,
charindex('//', main.lock + '//')- 1
), '//'
) within group (order by coalesce(qty_oh, 0) + coalesce(qty_tr, 0) - coalesce(qty_rs, 0) desc)
SQL Server has no built-in Regex functions but they are available via CLR. The good news is you don't need Regex in SQL Server. Everything I used to do with RegEx I now handle using NGrams8k. It's easy and performs much better. I've built a few functions using NGrams8K that would be helpful for this problem and many others. First we have PatReplace8K, second is Translate8K (updated code for both below.) A third option is PatExtract8K (follow the link for the code).
Examples of each performing the text transformation. With each function I'm just removing the Alpha characters and the numbers from 0-5 from "SomeString":
--==== Sample Data
DECLARE #table TABLE (SomeId INT IDENTITY, SomeString VARCHAR(40));
INSERT #table(SomeString) VALUES(NEWID()),(NEWID()),(NEWID()),(NEWID()),(NEWID());
--==== Using Patreplace8k
SELECT t.SomeString, f.NewString
FROM #table AS t
CROSS APPLY samd.patReplace8K(t.SomeString,'[0-5A-F-]','') AS f
--==== Using Translate8K
SELECT t.SomeString, f.NewString
FROM #table AS t
CROSS APPLY samd.Translate8K(t.SomeString,'[012345ABCDEF-]','') AS f
--==== samd.patExtract8K
SELECT t.SomeString,
NewString = STRING_AGG(f.item,'') WITHIN GROUP (ORDER BY f.ItemNumber)
FROM #table AS t
CROSS APPLY samd.patExtract8K(t.SomeString,'[0-5A-F-]') AS f
GROUP BY t.SomeString;
Each Return:
SomeString NewString
---------------------------------------- -----------------
0818BEF3-E0B3-4B3B-AA97-649E43EB16AF 8897696
3077EE8B-9E92-4337-9E2F-97DABE2E4623 7789979976
6BCD8194-F993-42DB-AF4A-D8289F8F8DA3 6899988988
C1F152DF-8B6F-4C14-AF6F-AC8869099FDB 866886999
F877D888-245E-4CEB-84B7-1CFF6E03B974 87788887697
To perform you string aggregation you can use XML PATH(), or STRING_AGG. Here's an example using both techniques and PatReplace8k:
SELECT NewString = STUFF((
SELECT '//'+NewString
FROM #table AS t
CROSS APPLY samd.patReplace8K(t.SomeString,'[0-5A-F-]','') AS f
ORDER BY f.NewString
FOR XML PATH('')),1,2,'');
SELECT STRING_AGG(f.NewString,'//') WITHIN GROUP (ORDER BY f.NewString)
FROM #table AS t
CROSS APPLY samd.patReplace8K(t.SomeString,'[0-5A-F-]','') AS f;
In each case I get what I want:
NewString
----------------------------------------------------
6967899797//777689886//868796//8887789//88989
Translate Function:
CREATE OR ALTER FUNCTION samd.Translate8K
(
#string VARCHAR(8000), -- Input
#pattern VARCHAR(100), -- characters to replace
#key VARCHAR(100) -- replacement characters
)
/*
Purpose:
Standard Translate function - the fastest UDF version in the game. Enjoy.
For more about TRANSLATE see: https://www.w3schools.com/sql/func_sqlserver_translate.asp
Requires:
NGrams8K; get you some here:
https://www.sqlservercentral.com/articles/nasty-fast-n-grams-part-1-character-level-unigrams
Designed By Alan Burstein; May, 2021
*/
RETURNS TABLE WITH SCHEMABINDING AS RETURN
SELECT NewString = ISNULL(REPLACE(tx.NewString,CHAR(0),''),#string)
FROM
(
SELECT STRING_AGG(CAST(t.tKey+ng.Token AS CHAR(1)),'')
WITHIN GROUP (ORDER BY ng.Position)
FROM samd.ngrams8k(#string,1) AS ng
CROSS APPLY (VALUES(#key+REPLICATE(CHAR(0),100))) AS tx(NewKey)
CROSS APPLY (VALUES(CHARINDEX(ng.Token,#pattern))) AS pos(N)
CROSS APPLY (VALUES(SUBSTRING(tx.NewKey,pos.N,1))) AS t(tKey)
) AS tx(NewString);
Patreplace8k:
CREATE OR ALTER FUNCTION [samd].[patReplace8K]
(
#string VARCHAR(8000),
#pattern VARCHAR(50),
#replace VARCHAR(20)
)
/*****************************************************************************************
[Purpose]:
Given a string (#string), a pattern (#pattern), and a replacement character (#replace)
patReplace8K will replace any character in #string that matches the #Pattern parameter
with the character, #replace.
[Author]:
Alan Burstein
[Compatibility]:
SQL Server 2008+
[Syntax]:
--===== Basic Syntax Example
SELECT pr.NewString
FROM samd.patReplace8K(#String,#Pattern,#Replace) AS pr;
[Developer Notes]:
1. #Pattern IS case sensitive but can be easily modified to make it case insensitive
2. There is no need to include the "%" before and/or after your pattern since since we
are evaluating each character individually
3. Certain special characters, such as "$" and "%" need to be escaped with a "/"
like so: [/$/%]
4. Functions that use samd.ngrams8k will see huge performance gains when the optimizer
generates a parallel execution plan. One way to get a parallel query plan (if the
optimizer does not choose one) is to use make_parallel by Adam Machanic found here:
sqlblog.com/blogs/adam_machanic/archive/2013/07/11/next-level-parallel-plan-porcing.aspx
As is the case with functions which leverage samd.ngrams or samd.ngrams8k,
samd.patReplace8K is almost always dramatically faster with a parallel execution
plan. On my PC (8 logical CPU, 64GB RAM, SQL 2019) samd.patReplace8K is about 4X
faster when executed using all 8 of my logical CPUs.
5. samd.patReplace8K is deterministic. For more about deterministic functions see:
https://msdn.microsoft.com/en-us/library/ms178091.aspx
[Examples]:
--===== 1. Replace numeric characters with a "*"
SELECT pr.NewString
FROM samd.patReplace8K('My phone number is 555-2211','[0-9]','*') AS pr;
--==== 2. Using againsts a table
DECLARE #table TABLE(OldString varchar(60));
INSERT #table VALUES ('Call me at 555-222-6666'), ('phone number: (312)555-2323'),
('He can be reached at 444.665.4466 on Monday.');
SELECT t.OldString, pr.NewString
FROM #table AS t
CROSS APPLY samd.patReplace8K(t.oldstring,'[0-9]','*') AS pr;
[Revision History]:
-----------------------------------------------------------------------------------------
Rev 00 - 20141027 Initial Development - Alan Burstein
Rev 01 - 20141029 - Redesigned based on the dbo.STRIP_NUM_EE by Eirikur Eiriksson
(see: http://www.sqlservercentral.com/Forums/Topic1585850-391-2.aspx)
- change how the cte tally table is created
- put the include/exclude logic in a CASE statement instead of a WHERE clause
- Added Latin1_General_BIN Colation
- Add code to use the pattern as a parameter. - Alan Burstein
Rev 02 - 20141106 - Added final performance enhancement (more cudos to Eirikur Eiriksson)
- Put 0 = PATINDEX filter logic into the WHERE clause
Rev 03 - 20150516 - Updated to deal with special XML characters - Alan Burstein
Rev 04 - 20170320 - changed #replace from char(1) to varchar(1) for whitespace handling
- Alan Burstein
Rev 05 - 20200515 - Complete rewrite using samd.NGrams
- changed PATINDEX(...)=0 to: PATINDEX()&0x01=0;
- Changed CASE statement to IIF; Dropped collation - Alan Burstein
*****************************************************************************************/
RETURNS TABLE WITH SCHEMABINDING AS RETURN
SELECT NewString = STRING_AGG(IIF(PATINDEX(#pattern,col.Token)&0x01=0,col.Token,
#replace),'') WITHIN GROUP (ORDER BY ng.position)
FROM samd.NGrams8K(#string,1) AS ng
CROSS APPLY (VALUES(ng.token)) AS col(Token);
I need to update a string to amend any aliases - which can be 'H1.', 'H2.', 'H3.'... etc - to all be 'S.' and am struggling to work out the logic.
For example I have this:
'H1.HUB_CUST_ID, H2.HUB_SALE_ID, H3.HUB_LOC_ID'
But I want this:
'S.HUB_CUST_ID, S.HUB_SALE_ID, S.HUB_LOC_ID'
If you could use wildcards in REPLACE, I'd do something like this REPLACE(#string, 'H%.H', 'S.H').
Theoretically, there is no limit to how many H# aliases there could be. In practice there will almost definitely be less than 10.
Is there a better way than a nested replace of H1 - H10 separately, which both looks messy in a script and carries a small risk if more tables are joined in future?
SQL Server doesn't support pattern replacement. You are better off using a different language, that does support pattern/REGEX replacement or implementing a CLR function.
That said, however, considering you said that the value would always be below 10 you could brute force it, but it's not "pretty".
SELECT REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(YourString,'H1.','S.'),'H2.','S.'),'H3.','S.'),'H4.','S.'),'H5.','S.'),'H6.','S.'),'H7.','S.'),'H8.','S.'),'H9.','S.')
FROM YourTable ...
You can convert your string to XML and then convert it into simple table:
DECLARE #txt nvarchar(max) = N'H1.HUB_CUST_ID, H2.HUB_SALE_ID, H3.HUB_LOC_ID',
#x xml
SELECT #x = '<a al="' + REPLACE(REPLACE(#txt,', ','</a><a al="'),'.','">')+ '</a>'
SELECT t.c.value('#al', 'nvarchar(max)') as alias_name,
t.c.value('.','nvarchar(max)') as col_name
FROM #x.nodes('/a') t(c)
Output:
alias_name col_name
H1 HUB_CUST_ID
H2 HUB_SALE_ID
H3 HUB_LOC_ID
You can put results into temp table, amend them using LIKE 'some basic pattern' and then build new string.
If you don't care about the result order, you can unaggregate and reaggregate:
select t.*, v.new_val
from t cross apply
(select string_agg(concat('S1', stuff(s.value, 1, charindex('.'), '') - 1, ',') within group (order by (select null) as newval
from string_split(t.col, ',') s
) s;
Note: This assumes that all values start with the prefix you want to replace -- as your sample data suggests. A case expression can be used if there are exceptions.
You can actually get the original ordering -- assuming no duplicates -- using charindex():
select t.*, v.new_val
from t cross apply
(select string_agg(concat('S1', stuff(s.value, 1, charindex('.'), '') - 1, ',')
within group (order by charindex(s.value, t.col)
) as newval
from string_split(t.col, ',') s
) s;
How to make the query below work?
I used the concept from another question SQL Server 2016 JSON: Select array of strings instead of array of objects
But when I tried the select below it doesn't work
SELECT
(SELECT line AS "line"
FROM
(SELECT
CONCAT('5th', ' ', '566') AS "line"
UNION ALL
SELECT 'Complement' AS LINE
)
FOR JSON PATH) AS "address.lines"
The version of the SQL Server SELECT ##VERSION = Microsoft SQL Server 2016
The result from the query is
Msg 156, Level 15, State 1, Line 11
Incorrect syntax near the keyword 'FOR'.
The excepted result is [{"line":"5th 566"},{"line":"Complement"}], with this result I will use the function from SQL Server 2016 JSON: Select array of strings instead of array of objects to remove "line" and get the final result ["5th 566","Complement"]
I need to use the UNION ALL because I have two different values from the same table to result in an array called address.lines at the JSON result
If I understand your question correctly, you just need an alias (t) to solve this error:
Statement:
SELECT (
SELECT [Line] FROM (
SELECT CONCAT('5th', ' ', '566') AS [Line]
UNION ALL
SELECT 'Complement' AS [Line]
) t
FOR JSON PATH
)
AS "address.lines"
Output:
address.lines
[{"Line":"5th 566"},{"Line":"Complement"}]
Of course, you can use only JSON functions to generate the final JSON output:
Statement:
DECLARE #json nvarchar(max) = N'[]'
SELECT #json = JSON_MODIFY(#json, 'append $', [Line])
FROM (
SELECT CONCAT('5th', ' ', '566') AS [Line]
UNION ALL
SELECT 'Complement' AS [Line]
) t
SELECT #json AS [address.lines]
Output:
address.lines
["5th 566","Complement"]
I have a JSON column [gca] that looks like this
[{
"i":"https://some.image.URL 1",
"u":"https://some.product.url",
"n":"Product 1 Name",
"q":"1",
"sk":"sku number 1",
"st":"$499.99"
},
{
"i":"https://some.image.URL 2",
"u":"https://some.product.url",
"n":"Product 2 Name",
"q":"1",
"sk":"sku number 2",
"st":"$499.99"
}]
I want to extract values specific to position. For example:
JSON_VALUE ([gca], '$[0].i') + ', ' + JSON_VALUE ([gca], '$[1].i')
So the result would be a string
image url 1, image url 2
I tried the cross apply solution from this answer, but I get this error:
JSON text is not properly formatted. Unexpected character 'h' is found at position 0
Expected results
-- https://i.stack.imgur.com/UaRjQ.png
Your statement is correct and based on the JSON data in the question, the reason for this "... JSON text is not properly formatted. Unexpected character 'h' is found at position 0' error ... " error is somewhere else.
Your JSON text is a valid JSON array, so you have two possible approaches to get your expected results:
if this JSON array has always two items, you should use JSON_VALUE() to access each item
if the count of the items is not known, you should use OPENJSON() with additional CROSS APPLY operator and string aggregation function.
Table:
CREATE TABLE #Data (
[gca] nvarchar(max)
)
INSERT INTO #Data
([gca])
VALUES
(N'[{"i":"https://some.image.URL 1","u":"https://some.product.url","n":"Product 1 Name","q":"1","sk":"sku number 1","st":"$499.99"},{"i":"https://some.image.URL 2","u":"https://some.product.url","n":"Product 2 Name","q":"1","sk":"sku number 2","st":"$499.99"}]'),
(N'[{"i":"https://some.image.URL 1","u":"https://some.product.url","n":"Product 1 Name","q":"1","sk":"sku number 1","st":"$499.99"},{"i":"https://some.image.URL 2","u":"https://some.product.url","n":"Product 2 Name","q":"1","sk":"sku number 2","st":"$499.99"}]')
Statements:
-- For fixed structure with two items
SELECT JSON_VALUE ([gca], '$[0].i') + ', ' + JSON_VALUE ([gca], '$[1].i') AS [http]
FROM #Data
-- For JSON array with multiple items and SQL Server 2017+
SELECT j.*
FROM #Data d
CROSS APPLY (
SELECT STRING_AGG([http], ',') AS [http]
FROM OPENJSON(d.[gca]) WITH ([http] varchar(max) '$.i')
) j
-- For JSON array with multiple items
SELECT STUFF(j.[http], 1, 1, N'') AS [http]
FROM #Data d
CROSS APPLY (
SELECT CONCAT(',', [http])
FROM OPENJSON(d.[gca]) WITH ([http] varchar(max) '$.i')
FOR XML PATH('')
) j([http])
Output:
-------------------------------------------------
http
-------------------------------------------------
https://some.image.URL 1,https://some.image.URL 2
https://some.image.URL 1,https://some.image.URL 2
Note, that JSON support was intoduced in SQL Server 2016 and STRING_AGG() was introduced in SQL Server 2017. For string aggregation in earlier versions use FOR XML PATH.
I have a database with 2 columns
A B
-- --
X 1995
Y 2005
C 1962
D 2003
I'm trying to create a SQL statement that will take a string of comma delimited values and return a json list of values in B where any value in the string is in A
so if the comma delimited string was 'X,C' the json list would be [1995,1962]
I've been using json path to try this, but I can't get it exactly like I want it and I've been spinning my wheels for too long
This is what I've tried:
Select mt.B as json_list_b_values
From [dbo].[myTable] mt
Where mt.A in (Select value From String_Split('X,C', ',')) for json path
This is the ouput:
[ {"json_list_b_values":"1995"}, {"json_list_b_values":"1962"} ]
As you're on 2016 you can't use STRING_AGG, but you can use the old tried and tested FOR XML PATH and STUFF method:
DECLARE #list varchar(8000) = 'X,C';
WITH VTE AS(
SELECT *
FROM(VALUES('X',1995),
('Y',2005),
('C',1962),
('D',2003)) V(A,B))
SELECT '[' + STUFF((SELECT CONCAT(',',V.B)
FROM VTE V
CROSS APPLY STRING_SPLIT(#list,',') SS
WHERE SS.[value] = V.A
FOR XML PATH('')),1,1,'') + ']';
I am no SQL expert. So please bear with me.
Step 1 - Handle the CSV input
You are already doing this by using the IN clause. I would store these results in a table variable
Step 2 - Convert the query results to a simple JSON array
I can think of the function STRING_AGG(). This will concatenate the rows into a flat string.
E.g. Join the FirstName column into a comma delimited string
SELECT STRING_AGG ( ISNULL(FirstName,'N/A'), ',') AS csv
FROM Person.Person;
Will produce the following
John,N/A,Mike,Peter,N/A,N/A,Alice,Bob
Code snippet based on your example
I am using a table variable #result1 to hold the result from Step 1.
SELECT '[ "' + string_agg( json_list_b_values, '", "') + '" ]' FROM #result1
MSDN reference for STRING_AGG()
https://learn.microsoft.com/en-us/sql/t-sql/functions/string-agg-transact-sql?view=sql-server-2017
CORRECTION
The function STRING_AGG() is available in SQL 2017.