This is my first query expecting some answers. I'm using the below report request and I'm getting error for pageToken .
enter code here
return analytics.reports().batchGet(
body={
'reportRequests': [
{
'viewId': VIEW_ID,
'pageToken': '100001',
'pageSize': 100000,
'dateRanges': [{'startDate': 'yesterday', 'endDate': 'yesterday'}],
'metrics': [{'expression': 'ga:sessions'},{'expression':'ga:users'},
{'expression': 'ga:bounces'},{'expression':'ga:pageviews'}],
'dimensions': [{'name': 'ga:dimension1'},{'name': 'ga:date'},
{'name':'ga:pageTitle'}]
}]
}).execute()
My query is for my particular query I have data more than 100000 rows and I want to load all the rows into particular table. Please can someone help me how to get all the data from the request.
Related
I am passing sales orders grouped by item into a function. I need to load the sales orders to modify the sales order record that it has already been processed. How would I go about getting all sales orders grouped under the item?
var itemId = result.getValue({
name: 'item',
summary: 'GROUP'
});
The above code gets me the item id for what the orders in the given result are being grouped by. There could be any number of sales within a single group.
Running a grouped saved search in SuiteScript doesn't give you the ability to "drill down" like you can when running a similar search in the user interface. As soon as you have a summary criteria for any column in a saved search, Netsuite requires that all your columns be summarized. If you want to get the internal IDs for the Sales Orders, then they will have to be returned as a summarized column in your saved search code.
Something like this:
const searchObj = search.create({
type: search.Type.SALES_ORDER,
filters: [
['mainline', 'is', 'F']
//,'AND', any other filters you need...
],
columns: [
search.createColumn({
name: "item",
summary: "GROUP"
}),
search.createColumn({
name: "formula(text)",
summary: "MAX",
formula: "ns_concat({internalid})"
})
]
}).run().each(function(result) {
const itemId = result.getValue({name: "item", summary: "GROUP"})
const salesOrderIds = result.getValue({
name: "formula(text)",
summary: "MAX",
formula: "ns_concat({internalid})"
}).split(',')
return true
})
I am running two database queries to retrieve data that I will outputting in a message embed. The queries are returning the proper rows when I just dump the entire result into the console. However, whenever I try to output the actual value for one of the rows, it displays as undefined in the message embed.
From what I've found based on examples, rows[0].somevalue should be outputting the correct results.
let mentionedUser = message.mentions.members.first();
let captainUser = client.users.find(user => user.id == `${mentionedUser.id}`);
con.query(`SELECT * FROM captains WHERE id = '${mentionedUser.id}';SELECT * FROM results WHERE captain = '${captainUser.username}'`, [2, 1], (err, rows) => {
if(err) throw err;
console.log(rows);
const infoEmbed = new Discord.RichEmbed()
.setColor("#1b56af")
.setAuthor('Captain Information', client.user.displayAvatarURL)
.setThumbnail('https://i.imgur.com/t3WuKqf.jpg')
.addField('Captain Name', `${mentionedUser}`, true)
.addField('Cap Space', `${rows[0].credits}`, true) // Returns undefined
message.channel.send(infoEmbed);
});
This is the console result
[ [ RowDataPacket {
id: '91580646270439424',
team_name: 'Resistance',
credits: 85,
roster_size: 2 } ],
[ RowDataPacket { id: 'Sniper0270', captain: 'BTW8892', credits: 10 },
RowDataPacket { id: 'Annex Chrispy', captain: 'BTW8892', credits: 5 } ] ]
In the code posted above, the expected output of rows[0].credits should output 85. No error codes are present, it just displayed as "undefined" in the message embed.
You are executing two queries inside a single query call. It looks like the mysql library returns an array of arrays in this scenario where the first value is the result of the first query and the second is the result of the second query. This is non standard. Normally you would either execute each query in its own query call or you would use a union to join the two queries into a single resultset.
this is not the practical way to send query request , as query is a single statement excluding the bulk update , you cannot execute two different query using a single con.query , it is not a proper way. execute them separately
I have a table with the following structure:
and the following data in it:
[
{
"addresses": [
{
"city": "New York"
},
{
"city": "San Francisco"
}
],
"age": "26.0",
"name": "Foo Bar",
"createdAt": "2016-02-01 15:54:25 UTC"
},
{
"addresses": [
{
"city": "New York"
},
{
"city": "San Francisco"
}
],
"age": "26.0",
"name": "Foo Bar",
"createdAt": "2016-02-01 15:54:16 UTC"
}
]
What I'd like to do is recreate the same table (same structure) but with only the latest version of a row. In this example let's say that I'd like to group by everything by name and take the row with the most recent createdAt.
I tried to do something like this: Google Big Query SQL - Get Most Recent Column Value but I couldn't get it to work with record and repeated fields.
I really hoped someone from Google Team will provide answer on this question as it is very frequent topic/problem asked here on SO. BigQuery definitelly not friendly enough with writing Nested / Repeated stuff back to BQ off of BQ query.
So, I will provide the workaround I found relatively long time ago. I DO NOT like it, but (and that is why I hoped for the answer from Google Team) it works. I hope you will be able to adopt it for you particular scenario
So, based on your example, assume you have table as below
and you expect to get most recent records based on createdAt column, so result will look like:
Below code does this:
SELECT name, age, createdAt, addresses.city
FROM JS(
( // input table
SELECT name, age, createdAt, NEST(city) AS addresses
FROM (
SELECT name, age, createdAt, addresses.city
FROM (
SELECT
name, age, createdAt, addresses.city,
MAX(createdAt) OVER(PARTITION BY name, age) AS lastAt
FROM yourTable
)
WHERE createdAt = lastAt
)
GROUP BY name, age, createdAt
),
name, age, createdAt, addresses, // input columns
"[ // output schema
{'name': 'name', 'type': 'STRING'},
{'name': 'age', 'type': 'INTEGER'},
{'name': 'createdAt', 'type': 'INTEGER'},
{'name': 'addresses', 'type': 'RECORD',
'mode': 'REPEATED',
'fields': [
{'name': 'city', 'type': 'STRING'}
]
}
]",
"function(row, emit) { // function
var c = [];
for (var i = 0; i < row.addresses.length; i++) {
c.push({city:row.addresses[i]});
};
emit({name: row.name, age: row.age, createdAt: row.createdAt, addresses: c});
}"
)
the way above code works is: it implicitely flattens original records; find rows that belong to most recent records (partitioned by name and age); assembles those rows back into respective records. final step is processing with JS UDF to build proper schema that can be actually written back to BigQuery Table as nested/repeated vs flatten
The last step is the most annoying part of this workaround as it needs to be customized each time for specific schema(s)
Please note, in this example - it is only one nested field inside addresses record, so NEST() fuction worked. In scenarious when you have more than just one
field inside - above approach still works, but you need to involve concatenation of those fields to put them inside nest() and than inside js function to do extra splitting those fields, etc.
You can see examples in below answers:
Create a table with Record type column
create a table with a column type RECORD
How to store the result of query on the current table without changing the table schema?
I hope this is good foundation for you to experiment with and make your case work!
When I pass an input field of repeated record type into Bigquery UDF, it keeps saying that the input field is not found.
This is my 2 rows of data:
{"name":"cynthia", "Persons":[ { "name":"john","age":1},{"name":"jane","age":2} ]}
{"name":"jim","Persons":[ { "name":"mary","age":1},{"name":"joe","age":2} ]}
This is the schema of the data:
[
{"name":"name","type":"string"},
{"name":"Persons","mode":"repeated","type":"RECORD",
"fields":
[
{"name": "name","type": "STRING"},
{"name": "age","type": "INTEGER"}
]
}
]
And this is the query:
SELECT
name,maxts
FROM
js
(
//input table
[dw_test.clokTest_bag],
//input columns
name, Persons,
//output schema
"[
{name: 'name', type:'string'},
{name: 'maxts', type:'string'}
]",
//function
"function(r, emit)
{
emit({name: r.name, maxts: '2'});
}"
)
LIMIT 10
Error I got when trying to run the query:
Error: 5.3 - 15.6: Undefined input field Persons
Job ID: ord2-us-dc:job_IPGQQEOo6NHGUsoVvhqLZ8pVLMQ
Would someone please help?
Thank you.
In your list of input columns, list the leaf fields directly:
//input columns
name, Persons.name, Persons.age,
They'll still appear in their proper structure when you get the records in your UDF.
I'm performing the following lookback snapshot. I got the data back but i also need the ValidFrom Date of the Completed record. It's not a part of the _PreviousValues record. How can i get that to come back with my query?
Thanks!
find: {
'_TypeHierarchy': 'HierarchicalRequirement',
'Children':null,'ScheduleState':'Accepted',
'_PreviousValues.ScheduleState':'Completed',
'_ValidFrom': { '$gte':startDate},
'_ValidTo': { '$lte': endDate},
},
fetch: ['FormattedID','Name','_ValidFrom','_ValidTo','BlockedReason','_User','WorkProduct','ScheduleState','_PreviousValues.ScheduleState','AcceptedDate'],
// order: 'OpenedDate DESC',
hydrate: ['FormattedID','Name','_ValidFrom','_ValidTo','BlockedReason','_User','WorkProduct','ScheduleState','_PreviousValues.ScheduleState','AcceptedDate'],
compress: true,
It looks like what we need is something like _PreviousValues.ScheduleState._ValidFrom, but it does not exist.
I think it is not possible to get _ValidFrom value of the _PreviousValues.ScheduleState from the same query, and a separate query is needed.
For example, this query:
https://rally1.rallydev.com/analytics/v2.0/service/rally/workspace/1234/artifact/snapshot/query.js?find={"Project":5678,"_TypeHierarchy":"HierarchicalRequirement","ScheduleState":"Accepted", "_PreviousValues.ScheduleState": "Completed"}&fields=["ObjectID","_ValidFrom","_ValidTo","ScheduleState","_PreviousValues.ScheduleState"]&hydrate=["ScheduleState","_PreviousValues.ScheduleState"]&compress=true
will return _PreviousValues object which only includes state value:
_PreviousValues: {
ScheduleState: "Completed"
}
Let's say one of the results has ObjectID 777.
The second query will use ObjectID(s) of the results of the first query to get the time interval when the story was in the "Completed" state:
https://rally1.rallydev.com/analytics/v2.0/service/rally/workspace/1234/artifact/snapshot/query.js?find={"ObjectID":777,"ScheduleState": "Completed","_PreviousValues.ScheduleState": "In-Progress"}&fields=["ObjectID","_ValidFrom","_ValidTo","ScheduleState"]&hydrate=["ScheduleState"]
It may return more than one snapshot, and depending on what fields are fetched there may not be an indication what changed between those snapshots (e.g. in this case TaskStatus and TaskRemainingTotal) but in any case the earliest snapshot's _ValidFrom value, _ValidFrom: "2013-06-17T18:51:36.931Z" is the date you are looking for
Results:
[
{
_ValidFrom: "2013-06-17T18:51:36.931Z",
_ValidTo: "2013-06-17T18:51:44.382Z",
ObjectID: 12353154323,
ScheduleState: "Completed"
},
{
_ValidFrom: "2013-06-17T18:55:50.897Z",
_ValidTo: "2013-06-18T20:53:01.755Z",
ObjectID: 12353154323,
ScheduleState: "Completed"
}
]
If you are writing a code, you will get the _ValidFrom of the first element of the array of objects.