I've found the solution for more complex than this but I'm not able to make this work...
I've a column called data of type json. The JSON structure is as follows:
{"actions": ["action1","action2","action3", ..., "actionN"]}
So, let's say I've 3 rows with the following data:
{"actions": ["work","run"]}
{"actions": ["run","eat","sleep", 'walk']}
{"actions": ["eat","run","work"]}
I want to retrieve the rows where work is included in actions array.
I've tried something similar to what is posted here:
Query for element of array in JSON column, but since each element inside the array is just a json string, I got stuck there.
Then, I tried something like:
SELECT * from table t WHERE 'work' in ...
but this also failed to get the values as an string array to put it there.
Using PostgreSql 9.3.
Since you are using the json type, which is just a string internally, the simplest solution is:
SELECT * FROM table WHERE strpos(data::text, 'work') > 0;
It defeats the purpose of using json in the first place, but it is (very likely) faster than the json parser in this simple case.
A json solution would be:
SELECT * FROM (
SELECT *, unnest(data->"actions")::text AS action FROM table) AS foo
WHERE action = 'work';
Related
I have some records in my database in a column called Swatch, that are JSON to describe a colour swatch. Each record looks something like this:
{"colors":{"color-1":"transparent","color-2":"transparent"}}
Basically it's one big colour object with lots of key value pairs. I would like to manipulate this JSON, using SQL, to achieve something like this instead to allow customisable colour names.
{"colors":[{"name": "Red", "color":"#00000"},{"name": "Green", "color":"#00000"}]
(Hex codes are just for demo purposes, I know they're not legit).
How can I achieve this? It is possible using SQL Server?
It may be possible, but a lot depends on the data and what you are trying to achieve.
There is a set of JSON functions available, most of them added in SQL 2016, and a pair more should be available in SQL Server 2022.
If you want to erase the current structure and data in favor of a new one that's easy.
If you want to convert your current structure to a different one, that might be pretty hard, but it's highly data related.
Let's consider your samples as the actual data you want to convert
{"colors":{"color-1":"transparent","color-2":"transparent"}}
{"colors":[{"name": "Red", "color":"#00000"},{"name": "Green", "color":"#00000"}]}
Here are the issues and complications I see
how can you say what color-1 is? what name does it have? that's human operation unless you have some ColorCode-ColorName table
Do you have a fixed/known amount of color-N keys? to get the property value cleanly you will need to reference them explicitly. (but I'm sure you can find other ways... without using JSON functions tho)
to change the structure you need to extract the data and build the new JSON string, basically a lot of string concatenation that will require quite some time to build and test
JSON function samples
I've played around with the JSON data and created some sample queries, I hope trying them will help you understand what it means to manipulate JSON data, and choose if SQL Server is a fitting tool for your task
-- key-values property
DECLARE #jdata2 as varchar(200) = '{"colors":{"color-1":"transparent","color-2":"transparent"}}'
SELECT
JSON_VALUE(#jdata2,'$."colors"."color-1"') AS Color1
,JSON_VALUE(#jdata2,'$."colors"."color-2"') AS Color2
,JSON_VALUE(#jdata2,'$."colors"."color-3"') AS Color3
GO
-- objects and arrays
DECLARE #jdata as varchar(200) = '{"company":"Contoso","colors":[{"name": "Red", "color":"#00000"},{"name": "Green", "color":"#00000"}]}'
SELECT JSON_QUERY(#jdata,'$.colors[0]')
SELECT JSON_VALUE(#jdata,'$.colors[0].name')
SELECT JSON_VALUE(#jdata,'$.company')
--Update property
SELECT JSON_MODIFY(#jdata,'$.company', 'Northwind')
--Add property
SELECT JSON_MODIFY(#jdata,'$.country', 'Italy')
--Add new Object
SELECT JSON_MODIFY(#jdata,'$.salesman', JSON_QUERY('{"name":"Mario","surname":"Rossi"}'))
--Append new Object in an Object array
SELECT JSON_MODIFY(#jdata,'append $.colors', JSON_QUERY('{"name":"Yellow", "color":"#00000"}','$'))
------ About DELETING
--Delete (whole) Property works fine
SELECT JSON_MODIFY(#jdata,'$.colors', NULL)
-- deleting sometihng inside an array is not fine at all
-- Should delete 1 value/object from the array... but no, 'null,' is left instead
SELECT JSON_MODIFY(#jdata,'$.colors[1]', NULL)
-- To "delete" properly pass the whole array or object array omitting the deleted value...
SELECT JSON_MODIFY(#jdata,'$.colors', JSON_QUERY('[{"name": "Green", "color":"#00000"}]'))
I am trying to create an index that returns entire data object of the documents in a collection;
here is the code:
CreateIndex({
name: "users_by_data",
source: Collection("users"),
values: { field: ['data'] }
})
but after creation it says: Values Not set (using ref by default)
If I specifically define fields (separately by their name), it will behave as expected, but data isn't working. the question is:
Is it impossible (e.g. for performance reasons) or am I doing it wrong?
side note: I am aware that I can do Lambda function on Paginate and achieve similar result, but this question specifically is about the Index level;
You can currently index regular values (strings, numbers, dates, etc) and you can index an array which will more or less 'unroll' the array into separate index entries. However, what you are trying, indexing an object is not possible at this point. An object (like data) will be ignored if you try to index it.
Currently, you have two options:
as you mentioned, using Map/Get at query time.
listing all values of the data object in the index since you can select specific values of an object in the index (which is however less flexible if new attributes arrive later on in the object)
We intend to support indexing of objects in the future, I can't provide an ETA yet though. There is a feature request on our forums as well that you can vote up: https://forums.fauna.com/t/object-as-terms-instead-of-scalar-s/628
You're going to want to use the Select function on the Ref you get back from the Index if you only want the data field back.
For an individual document, you can do something like this
Select( "data",
Get(
Match(
Index("yourIndexName"),
**yourIndexTerm // Could point to String/Number/FQL Ref
)
)
)
For a list of documents, you can use Paginate as you said but you can still pull the data property out of each document
Map(
Paginate(
Match(
Index("yourIndexName"),
**yourIndexTerm // Could point to String/Number/FQL Ref
)
),
Lambda("doc", Select("data", Get(Var("doc"))))
)
I need to execute a SQL query, which converts a String column to a Array and then validate the size of that array
I was able to do it easily with postgresql:
e.g.
select
cardinality(string_to_array('a$b','$')),
cardinality(string_to_array('a$b$','$')),
cardinality(string_to_array('a$b$$$$$','$')),
But for some reason trying to convert String on vertica to array is not that simple, Saw this links:
https://www.vertica.com/blog/vertica-quick-tip-dynamically-split-string/
https://forum.vertica.com/discussion/239031/how-to-create-an-array-in-vertica
And much more that non of them helped.
I also tried using:
select REGEXP_COUNT('a$b$$$$$','$')
But i get an incorrect value - 1.
How can i Convert String to array on Vertica and gets his Length ?
$ has a special meaning in a regular expression. It represents the end of the string.
Try escaping it:
select REGEXP_COUNT('a$b$$$$$', '[$]')
You could create a UDx scalar function (UDSF) in Java, C++, R or Python. The input would be a string and the output would be an integer. https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ExtendingVertica/UDx/ScalarFunctions/ScalarFunctions.htm
This will allow you to use language specific array logic on the strings passed in. For example in python, you could include this logic:
input_list = input.split("$")
filtered_input_list = list(filter(None, input_list))
list_count = len(filtered_input_list)
These examples are a good starting point for writing UDx's for Vertica. https://github.com/vertica/UDx-Examples
I wasn't able to convert to an array - but Im able to get the length of the values
What i do is convert to Rows an use count - its not best performance wise
But with this way Im able to do also manipulation like filtering of each value between delimiter - and i dont need to use [] for characters like $
select (select count(1)
from (select StringTokenizerDelim('a$b$c','$') over ()) t)
Return 3
In my Room model, I have an attribute named available_days, which is being stored as an array.
For example:
Room.first.available_days
=> ["wed", "thurs", "fri"]
What is the best way to find all Rooms where the size of the array is equal to 3?
I've tried something like
Room.where('LENGTH(available_days) = ?', 3)
with no success.
Update: the data type for available_days is a string, but in order to store an array, I am serializing the attribute from my model:
app/models/room.rb
serialize :available_days
Can't think of a purely sql way of doing it for sqlite since available_days is a string.
But here's one way of doing it without loading all records at once.
rooms = []
Room.in_batches(of: 10).each_record do |r|
rooms << r if r.available_days.length == 3
end
p rooms
If you're using postgres you can parse the serialized string to an array type, then query on the length of the array. I expect other databases may have similar approaches. How to do this depends on how the text is being serialized, but by default for Rails 4 should be YAML, so I expect you data is encoded like this:
---
- first
- second
The following SQL will remove the leading ---\n- as well as the final newline, then split the remaining string on - into an array. It's not strictly necessary to cleanup the extra characters to find the length, but if you want to do other operations you may find it useful to have a cleaned up array (no leading characters or trailing newline). This will only work for simple YAML arrays and simple strings.
Room.where("ARRAY_LENGTH(STRING_TO_ARRAY(RTRIM(REPLACE(available_days,'---\n- ',''),'\n'), '\n- '), 1) = ?", 3)
As you can see, this approach is rather complex. If possible you may want to add a new structured column (array or jsonb) and migrate the serialized string into the a typed column to make this easier and more performant. Rails supports jsonb serialization for postgres.
I have seen this:
$makeModels = Make::model()->findAll(array("select"=>"fieldMake", "order"=>"fieldMake DESC"));
is it possible to do so with findAllByAttributes too? I need to only select one column values. I mean I don't want findAllByAttributes return array of object I need only an array of values.
If you only want an array of values use a custom SQL command with CDbCommand
Take a look at the following url for more information:
http://www.yiiframework.com/doc/api/1.1/CDbCommand
It will probably look something like this:
Yii::app()->db->createCommand('your sql query')->queryColumn();