I have column A with value hello.
I need to migrate it into new column AJson with value ["hello"].
I have to do this with Sql Server command.
There are different commands FOR JSON etc. but they serialize value with column name.
This is the same value that C# method JsonConvert.SerializeObject(new List<string>(){"hello"} serialization result would be.
I can't simply attach [" in the beginning and end because the string value may contain characters which without proper serialization will break the json string.
My advice is you just make a lot of nested replaces and then do it yourself.
FOR JSON is intended for entire JSON, and therefore not valid without keys.
Here is a simple example that replaces the endline with \n
print replace('ab
c','
','\n')
Backspace to be replaced with \b.
Form feed to be replaced with \f.
Newline to be replaced with \n.
Carriage return to be replaced with \r.
Tab to be replaced with \t.
Double quote to be replaced with "
Backslash to be replaced with \
My approach was to use these 3 commands:
UPDATE Offers
SET [DetailsJson] =
(SELECT TOP 1 [Details] AS A
FROM Offers AS B
WHERE B.Id = Offers.Id
FOR JSON PATH, WITHOUT_ARRAY_WRAPPER)
UPDATE Offers
SET [DetailsJson] = Substring([DetailsJson], 6, LEN([DetailsJson]) - 6)
UPDATE Offers
SET [DetailsJson] = '[' + [DetailsJson] + ']'
..for op's answer/table..
UPDATE Offers
SET [DetailsJson] = concat(N'["', string_escape([Details], 'json'), N'"]');
declare #col nvarchar(100) = N'
a b c " : [ ] ]
x
y
z'
select concat(N'["', string_escape(#col, 'json'), N'"]'), isjson(concat(N'["', string_escape(#col, 'json'), N'"]'));
Related
In SQL Server 2017, I'd like to "SELECT" a JSON object embedded within another as a string so we can store/process them later.
eg JSON:
[
{"key1":"value1",
"level2_Obj":{"key2":"value12"}
},
{"key1":"value2",
"level2_Obj":{"key22":"value22"}
},
]
From above JSON, I'd like to SELECT whole of the level2Obj JSON object, see below for what I'd like to see the "selection" result.
value1 |{"key2" :"value12"}
value2 |{"key22":"value22"}
I tried below with no luck:
SELECT * FROM
OPENJSON(#json,'$."data1"')
WITH(
[key1] nvarchar(50),
[embedded_json] nvarchar(max) '$."level2Obj"'
) AS DAP
Can some one please help how I select the contents of the 2nd level JSON object as a string?
The idea is to Write 1st level JSON properties into individual cells and rest of JSON levels into a single column of type nvarchar(max) (i.e whole of sub-level JSON object into a single column as a string for further processing in later stages).
Good day,
Firstly, Your JSON text is not properly formatted. There is extra comma after the last object in the array. I will remove this extra comma for the sake of the answer, but if this is the format you have then first step will be to clear the text and make sure that is is well formatted.
Please check if this solve your needs:
declare #json nvarchar(MAX) = '
[
{
"key1":"value1",
"level2_Obj":{"key2":"value12"}
}
,
{
"key1":"value2",
"level2_Obj":{"key22":"value22"}
}
]
'
SELECT JSON_VALUE (t1.[value], '$."key1"'), JSON_QUERY (t1.[value], '$."level2_Obj"')
FROM OPENJSON(#json,'$') t1
I recently found a bug with some Access SQL queries that I can't seem to track down. I have a fairly straightforward SQL query that I use to retrieve data from an access database that's "managed" in an older application (ie the data is already in the database and I have no real control over what's in there).
import pyodbc
MDB = '******.MDB'
DRV = '{Microsoft Access Driver (*.mdb)}'
PWD = ''
con = pyodbc.connect('DRIVER={};DBQ={};PWD={}'.format(DRV, MDB, PWD))
sql = ('SELECT Estim.PartNo, Estim.Descrip, Estim.CustCode, Estim.User_Text1, Estim.Revision, ' +
'Estim.Comments, Routing.PartNo AS RPartNo, Routing.StepNo, Routing.WorkCntr, Routing.VendCode, ' +
'Routing.Descrip AS StepDescrip, Routing.SetupTime, Routing.CycleTime, ' +
'Routing.WorkOrVend, ' +
'Materials.PartNo as MatPartNo, Materials.SubPartNo, Materials.Qty, ' +
'Materials.Unit, Materials.TotalQty, Materials.ItemNo, Materials.Vendor ' +
'FROM (( Estim ' +
'INNER JOIN Routing ON Estim.PartNo = Routing.PartNo ) ' +
'INNER JOIN Materials ON Estim.PartNo = Materials.PartNo )')
if 'PartNo' in kwargs:
key = kwargs['PartNo']
sql = sql + 'WHERE Estim.PartNo=?'
cursor = con.cursor().execute(sql, key)
# use this for debuging only
num = 0
for row in cursor.fetchall():
num += 1
return num
This works fine for all PartNo except when PartNo contains a decimal point. Curiously, when PartNo contains a decimal point AND a hyphen, I get the appropriate record(s).
kwargs['PartNo'] = "100.100-2" # returns 1 record
kwargs['PartNo'] = "200.100" # returns 0 records
Both PartNos exist when viewed in the other application, so I know there should be records returned for both queries.
My first thought was to ensure kwargs['PartNo'] is a string key = str(kwargs['PartNo']) with no change.
I also tried to places quotes around the 'PartNo' value with no success. key = '\'' + kwargs['PartNo'] + '\''
Finally, I tried to escape the . with no success (I realize this would break most queries, but I'm just trying to track down the issue with a single period) key = str(kwargs['partNo']).replace('.', '"."')
I know using query parameters should handle all the escaping for me, but at this point, I'm just trying to figure out what's going on. Any thoughts on this?
So the issue isn't with the query parameters - everything works as it should. The problem is with the SQL statement. I incorrectly assumed - and never checked - that there was a record in the Materials table that matched PartNo.
INNER JOIN Materials ON Estim.PartNo = Materials.PartNo
will only return a record if PartNo is found in both tables, which in this particular case it is not.
Changing it to
LEFT OUTER JOIN Materials ON Estim.PartNo = Materials.PartNo
produces the expected results. See this for info on JOINS. https://msdn.microsoft.com/en-us/library/bb243855(v=office.12).aspx
As for print (repr(key)) - flask handles the kwarg type upstream properly
api.add_resource(PartAPI, '/api/v1.0/part/<string:PartNo>'
so when I ran this in the browser, I got the "full length" strings. When run in the cmd line using python -c ....... I was not handling the argument type properly as Gord pointed out, so it was truncating the trailing zeros. I didn't think the flask portion was relevant, so I never added that in the original question.
After I did some internet research I found out that nodes function for xml processing of Sql Server does not accept a variable as parameter. If I try to use a variable a get the following error:
The argument 1 of the XML data type method "nodes" must be a string literal.
This is what I am trying to do:
SET #xPath = REPLACE(#xPath,'|','/')
SET #xPath = '/' + #xPath + '/text()'
SELECT #ParsedHtml = #ParsedHtml + CONVERT(nvarchar(max),Col.query('.')) + ';' FROM #Html.nodes(#xPath)AS T(Col)
Is there any hack i can do to make nodes function accept a variable as parameter???
You can use a variable in the path like this:
SET #xPath = REPLACE(#xPath,'|','/')
set #ParsedHtml = ''
SELECT #ParsedHtml = #ParsedHtml + CONVERT(nvarchar(max),Col.query('.')) + ';'
FROM #Html.nodes('/*[local-name()=sql:variable("#xPath")]/text()')AS T(Col)
I don't think you can do multiple levels in one variable in the path, which it looks like you are trying to do. You'd have to specify each level with a parameter of it's own. If you need to do a dynamic number of path levels, you might look into using dynamic sql.
EDIT: My only pending issue is c) (True and False on file, bit on database, I can't change neither the file nor the database scheme, there's hundreds of terabytes I can't touch).
The system receives a file (hundreds of thousands of them, actually) with a certain format. Things are:
a) First type is a uniqidentifier (more on this later)
b) On the database, the table's first 4 values are generated by the database (they are related to dates), meaning that those 4 values are not found on the files (all the rest are -and are in order-, even if it's always their representation as text or they are empty)
c) Bit values are represented with a False/True on the file.
So, the issue for 1 is that in the text file I receive as input, the uniqidentifier is using brackets. When I tried to generate the file with the format nul options using the bcp command tool, it would make it a sqlchar with 37 characters (which makes no sense to me, since it would either be 36 or 38).
Row separator is "+++\r\n", column separator is "©®©".
How would I go about generating the format files? I've been stuck with this for some time, I never used bcp before and errors I've got don't really tell much ("Unexpected EOF encountered in BCP data-file")
Am I supposed to specify all the columns in the format file or just the ones I desire to read from the files I get?
Thanks!
NOTE: I can't provide the SQL schema since it's for the company I work for. But it's pretty much: smalldate, tinyint tinyint tinyint (this four are generated by the db), uniqidentifier, chars, chars, more varchars, some bits, more varchars, some nvarchar. ALL values, except for those generated by the db, accept null.
My current problem is with the skipping the first 4 columns.
http://msdn.microsoft.com/en-us/library/ms179250(v=SQL.105).aspx
I followed that guide but somehow it's not working. Here's the changes (I'm just hard-changing column names to keep privacy of the project, even if it sounds stupid)
This is the one generated with bcp (with format nul -c) -note I put it as link 'cause it's not that short-
http://pastebin.com/4UkpPp1n
The second one, which is supposed to do the same but ignoring the first 4 columns is in the next pastebin:
http://pastebin.com/Lqj6XSbW
Yet it is not working. The error is "Error = [Microsoft][SQL Native Client]The number of fields provided for bcp operation is less than the number of columns on the server.", which was supposed to be the purpose of all that.
Any help will be greatly appreciated.
I'd create a new table with a CHAR(38) for the GUID. Import your data into this staging table, then translate it with CAST(SUBSTRING(GUID, 2, 36) AS UNIQUEIDENTIFIER) to import the staging data into your permanent table. This approach also works well for dates in odd formats, numbers with currency symbols, or generally any kind of poorly-formatted input.
BCP format files are a little touchy, but fundamentally aren't too complicated. If that part continues to give you trouble, one option is to import the whole row as a single VARCHAR(1000) field, then split it up within SQL - if you're comfortable with SQL text processing that is.
Alternately, if you are familiar with some other programming language, like Perl or C#, you can create a script to pre-process your inputs into a more friendly form, like tab-delimited. If you're not familiar with some other programming language, I suggest you pick one and get started! SQL is a great language, but sometimes you need a different tool; it's not great for text processing.
If you're familiar with C#, here's my code to generate a format file. No one gets to make fun of my Whitestone indentation :P
private static string CreateFormatFile(string filePath, SqlConnection connection, string tableName, string[] sourceFields, string[] destFields, string fieldDelimiter, string fieldQuote)
{
string formatFilePath = filePath + ".fmt";
StreamWriter formatFile = null;
SqlDataReader data = null;
try
{
// Load the metadata for the destination table, so we can look up fields' ordinal positions
SqlCommand command = new SqlCommand("SELECT TOP 0 * FROM " + tableName, connection);
data = command.ExecuteReader(CommandBehavior.SchemaOnly);
DataTable schema = data.GetSchemaTable();
Dictionary<string, Tuple<int, int>> metadataByField = new Dictionary<string, Tuple<int, int>>();
foreach (DataRow row in schema.Rows)
{
string fieldName = (string)row["ColumnName"];
int ordinal = (int)row["ColumnOrdinal"] + 1;
int maxLength = (int)row["ColumnSize"];
metadataByField.Add(fieldName, new Tuple<int, int>(ordinal, maxLength));
}
// Begin the file, including its header rows
formatFile = File.CreateText(formatFilePath);
formatFile.WriteLine("10.0");
formatFile.WriteLine(sourceFields.Length);
// Certain strings need to be escaped to use them in a format file
string fieldQuoteEscaped = fieldQuote == "\"" ? "\\\"" : fieldQuote;
string fieldDelimiterEscaped = fieldDelimiter == "\t" ? "\\t" : fieldDelimiter;
// Write a row for each source field, defining its metadata and destination field
for (int i = 1; i <= sourceFields.Length; i++)
{
// Each line contains (separated by tabs): the line number, the source type, the prefix length, the field length, the delimiter, the destination field number, the destination field name, and the collation set
string prefixLen = i != 1 || fieldQuote == null ? "0" : fieldQuote.Length.ToString();
string fieldLen;
string delimiter = i < sourceFields.Length ? fieldQuoteEscaped + fieldDelimiterEscaped + fieldQuoteEscaped : fieldQuoteEscaped + #"\r\n";
string destOrdinal;
string destField = destFields[i - 1];
string collation;
if (destField == null)
{
// If a field is not being imported, use ordinal position zero and a placeholder name
destOrdinal = "0";
fieldLen = "32000";
destField = "DUMMY";
collation = "\"\"";
}
else
{
Tuple<int, int> metadata;
if (metadataByField.TryGetValue(destField, out metadata) == false) throw new ApplicationException("Could not find field \"" + destField + "\" in table \"" + tableName + "\".");
destOrdinal = metadata.Item1.ToString();
fieldLen = metadata.Item2.ToString();
collation = "SQL_Latin1_General_CP1_CI_AS";
}
string line = String.Join("\t", i, "SQLCHAR", prefixLen, fieldLen, '"' + delimiter + '"', destOrdinal, destField, collation);
formatFile.WriteLine(line);
}
return formatFilePath;
}
finally
{
if (data != null) data.Close();
if (formatFile != null) formatFile.Close();
}
}
There was some reason I didn't use a using block for the data reader at the time.
It seems as if it is not possible for BCP to understand True and False as bit values. It's better to either go with SSIS or first replace the contents of the text (not a good idea to create views or anything like that, it is more overhead).
I want to update a row which has some html tags inside. For instance:
src='/imagem.png'></ p></ body>
> UPDATE ISTANBUL_TABLE SET TEXT = '<
> body>< p>< img src='/imagem.png '></
> p></ body>' WHERE 1=1
You see after src=' means the query ends, but it does not end. How can i solve it without using " (double comma)? Any solution please?
best regards bk
You need to escape the single-quotes, by typing them twice:
UPDATE ISTANBUL_TABLE SET TEXT = '< body>< p>< img src=''/imagem.png ''>' WHERE 1=1
Also, your WHERE clause is nonsensical and can be dropped entirely
UPDATE ISTANBUL_TABLE SET TEXT = '<body><p><img src=''/imagem.png''>'
Use parameterised SQL:
UPDATE ISTANBUL_TABLE SET TEXT = #HTML WHERE...
Then from your calling code, you just pass in the #HTML parameter and don't need to double up the single quotes.