I want to know how to parse JSON if the key is dynamic, and how to define the path in such cases in bigquery?
PFA image with JSON format.
I am trying to get the endTime values in JSON.
I am stuck at response.data.menus. and not able to define path post menus as the key is dynamic.
Try below
create temp function get_keys(input string) returns array<string> language js as """
return Object.keys(JSON.parse(input));
""";
create temp function get_values(input string) returns array<string> language js as """
return Object.values(JSON.parse(input));
""";
create temp function get_leaves(input string) returns string language js as '''
function flattenObj(obj, parent = '', res = {}){
for(let key in obj){
let propName = parent ? parent + '.' + key : key;
if(typeof obj[key] == 'object'){
flattenObj(obj[key], propName, res);
} else {
res[propName] = obj[key];
}
}
return JSON.stringify(res);
}
return flattenObj(JSON.parse(input));
''';
select *
from (
select
keys[safe_offset(2)] as manu_id,
keys[safe_offset(6)] as regularHours,
keys[safe_offset(7)] as key,
val
from your_table, unnest([struct(get_leaves(response) as leaves)]),
unnest(get_keys(leaves)) key with offset
join unnest(get_values(leaves)) val with offset using(offset),
unnest([struct(split(key, '.') as keys)])
where ends_with(key, 'startTime')
or ends_with(key, 'endTime')
)
pivot (any_value(val) for key in ('startTime', 'endTime'))
if applied to sample response you provided in your question
output is
Note: I would not expected above to 100% fit your requirements/expectations - but at least it should give you good direction!
Related
Let's say I have:
type User struct {
ID int64 `json:"id"
Posts []Post `json:"posts"
}
type Post struct {
ID int64 `json:"id"
Text string `json:"text"
}
The SQL query:
WITH temp AS (SELECT u.id AS user_id, p.id AS post_id, p.text AS post_text FROM users u JOIN posts p ON u.id=p.user_id)
SELECT user_id, ARRAY_AGG(ARRAY[post_id::text, post_text])
FROM temp
GROUP BY user_id
)
What I want is to scan rows from the query above into a slice of User objects:
import (
"context"
"fmt"
"github.com/jackc/pgx/v4/pgxpool"
"github.com/lib/pq"
)
var out []User
rows, _ := client.Query(context.Background(), query) // No error handling for brevity
for rows.Next() {
var u User
if err := rows.Scan(&u.ID, pq.Array(&u.Posts)); err != nil {
return
}
out = append(out, u)
}
Pretty much expectedly, the code above fails with:
pq: cannot convert ARRAY[4][2] to StringArray
This makes sense, but is there a way to read the SQL output into my slice of users?
Scanning of multi-dimensional arrays of arbitrary types, like structs, is not supported by lib/pq. If you want to scan such an array you'll have to parse and decode it yourself in a custom sql.Scanner implementation.
For example:
type PostList []Post
func (ls *PostList) Scan(src any) error {
var data []byte
switch v := src.(type) {
case string:
data = []byte(v)
case []byte:
data = v
}
// The data var holds the multi-dimensional array value,
// something like: {{"1","foo"}, {"2","bar"}, ...}
// The above example is easy to parse but too simplistic,
// the array is likely to be more complex and therefore
// harder to parse, but not at all impossible if that's
// what you want.
return nil
}
If you want to learn more about the PostgreSQL array representation syntax, see:
Array Input and Output Syntax
An approach that does not require you to implement a parser for PostgreSQL arrays would be to build and pass JSON objects, instead of PostgreSQL arrays, to array_agg. The result of that would be a one-dimensional array with jsonb as the element type.
SELECT user_id, array_agg(jsonb_build_object('id', post_id, 'text', post_text))
FROM temp
GROUP BY user_id
Then the implementation of the custom sql.Scanner just needs to delegate to lib/pq.GenericArray and another, element-specific sql.Scanner, would delegate to encoding/json.
type PostList []Post
func (ls *PostList) Scan(src any) error {
return pq.GenericArray{ls}.Scan(src)
}
func (p *Post) Scan(src any) error {
var data []byte
switch v := src.(type) {
case string:
data = []byte(v)
case []byte:
data = v
}
return json.Unmarshal(data, p)
}
type User struct {
ID int64 `json:"id"`
Posts PostList `json:"posts"`
}
If I have a json object and I wanted to access to most nested elements, but I don't know the name of a preceding object, is there a function that can do this?
Consider below approach
create temp function extract_keys(input string) returns array<string> language js as """
return Object.keys(JSON.parse(input));
""";
create temp function extract_values(input string) returns array<string> language js as """
return Object.values(JSON.parse(input));
""";
create temp function extract_all_leaves(input string) returns string language js as '''
function flattenObj(obj, parent = '', res = {}){
for(let key in obj){
let propName = parent ? parent + '.' + key : key;
if(typeof obj[key] == 'object'){
flattenObj(obj[key], propName, res);
} else {
res[propName] = obj[key];
}
}
return JSON.stringify(res);
}
return flattenObj(JSON.parse(input));
''';
select json,
array(
select as struct key, value
from unnest(extract_keys(leaves)) key with offset
join unnest(extract_values(leaves)) value with offset
using(offset)
) most_nested_elements
from your_table, unnest([struct(extract_all_leaves(json) as leaves)])
if applied to sample data in your question
output is
Not general solution like #Mikhail's answer but with some assumption you can also use a regular expression to extract most nested fields.
WITH sample AS (
SELECT '{"field": {"properties": {"custom_name": {"type":"string", "label":"custom_name"}}}}' AS json
)
SELECT TRIM(SPLIT(kv, ':')[OFFSET(0)], '"') AS key,
TRIM(SPLIT(kv, ':')[OFFSET(1)], '"') AS value
FROM sample, UNNEST(REGEXP_EXTRACT_ALL(json, r'(\"\w+\":\s*[^{][\w\"]+)')) kv
;
Anyone have any insight into why the following results in false instead of true?
CREATE TEMPORARY FUNCTION test()
RETURNS BOOL
LANGUAGE js
AS
"""
const d = new Date();
return d instanceof Date;
""";
SELECT test();
Returns false (unexpected)
WORKAROUND:
CREATE TEMPORARY FUNCTION test()
RETURNS BOOL
LANGUAGE js
AS
"""
const d = new Date();
return Object.prototype.toString.call(d) === '[object Date]';
""";
SELECT test();
Returns true (as expected)
The instanceof Date seams not to work.
For other objects, like String it works fine.
There is a JavaScript workaround possible:
CREATE TEMPORARY FUNCTION test()
RETURNS BOOL
LANGUAGE js
AS
"""
var d = new Date();
var s= new String('String created with constructor');
//return s instanceof String;
return typeof d.getMonth === 'function';
""";
SELECT test();
How to substring method in React native? I tried all methods which are given below but none of the method worked.
substring, slice, substr
The substring method is applied to a string object.
The substring() method extracts the characters from a string, between two specified indices, and returns the new substring.
This method extracts the characters in a string between "start" and "end", not including "end" itself.
If "start" is greater than "end", this method will swap the two arguments, meaning str.substring(1, 4) == str.substring(4, 1).
If either "start" or "end" is less than 0, it is treated as if it were 0.
Note: The substring() method does not change the original string.
The way to use it is this:
var str = "Hello world!";
var res = str.substring(1, 4);
// res value is "ell"
https://www.w3schools.com/jsref/jsref_substring.asp
You can use it :
var str = "Hello world!";
var res = str.substring(0, 4); // output is Hello
if you get from JSON
{item.name.substring(0, 4)}
from text
this is text.substring(0, 5) // output is: this i
Method 1: With Variables
var str = "Demo String";
var res = str.substring(2, 5); //starts from 0
Method 2: With States Directly
<Text>{this.state.str.substring(0, 7)}</Text>
try
(obj.str).substring(1, 4);
Another alternative, you can make simple function. and then print in your jsx.
var number = "62857123456"
const slice = {
phone: (input: string = '') => {
let output = '';
// you can use substring, slice, substr
output = input.slice(2,14);
return output;
},
};
Finally, print on your jsx by calling the function that was created.
{slice.phone(number)}
I have faced the same situation For this the solution is place all js code in a function and call it externally
class AboutMe extends Component {
displayAboutMe(){
var data = this.props.getAboutMeQuery;
if(data.loading){
return(<div>Loading Books</div>)
}else{
var aboutMe = JSON.stringify(this.props.getAboutMeQuery.aboutme);
console.log(aboutMe);
var res = aboutMe.substring(12,453);
console.log(res);
}
}
render(){
this.displayAboutMe()
return (
<div id="profile-section"></div>
)}}
I need to make a request with a multi-value param,its value must be a very big array to test an api param validation. One of my implementation:
Scenario: filter_too_long_multivalue
* def localPath = endPointBase + 'filter'
* def generateLongParam =
"""
function(){
var paramValue = [];
for(var idx = 0; idx<1002; idx++){
paramValue.push('r')
}
var params = {};
params.pl = paramValue;
return params;
}
"""
* def tooLongParam = generateLongParam()
* print tooLongParam
Given path localPath
And params tooLongParam
When method get
Then match response == authorizationSchema.empty
Then the request is:
GET http://XXXXXXXXXXXXX?pl=%5Bobject+Array%5D
I have tried differents ways, but always the same result...
How can I obtain a weel formed request?
Thank you.
Yes, you have hit one of those edge cases, can you let me know if this works, I'll also see if this can be improved.
Sometimes what is returned by a JS function is not exactly the format that Karate likes. The workaround is to use json instead of def to type-convert - refer doc: https://github.com/intuit/karate#type-conversion
* def fun =
"""
function(){
var temp = [];
for(var i = 0; i < 5; i++) {
temp.push('r');
}
return temp;
}
"""
* json array = fun()
Given url demoBaseUrl
And path 'echo'
And params { pl: '#(array)' }
When method get
Then status 200
Which resulted in:
1 > GET http://127.0.0.1:60146/echo?pl=r&pl=r&pl=r&pl=r&pl=r