Mongoose saves in UTC but needs to retrieve in different timezones depends on the user timezone - express

Mongoose saves all dates in UTC format that's OK.
Each user has a specific timezone. It stores the in user's document.
I would like to send any documents to the client with converted dates in the user's timezone.
I know how to format one value, but I'm looking for a solution to avoid transformation one-by-one.
The workaround solution will be a pass-through transform function on the client for each value or recursive modification response.
Any thoughts?

You can do this in aggregation.
Lets say you have this data:
[
{
"_id": {
"$oid": "5f18b5c87f9f9c0fd8322b60"
},
"createdAt": {
"$date": "2020-07-22T21:55:20.575Z"
},
},
{
"_id": {
"$oid": "5f19efac5cfa75483865eaa2"
},
"createdAt": {
"$date": "2020-07-23T20:14:36.108Z"
},
}
]
you can do this:
const timezone = "America/Chicago"
Model.aggregate([
{
$set: {
localTime: {
$dateToString: {
date: "$createdAt",
timezone
}
}
},
}
]);
The result of the aggregation will be:
[
{
"_id": ObjectId("5f18b5c87f9f9c0fd8322b60"),
"createdAt": ISODate("2020-07-22T21:55:20.575Z"),
"localTime": "2020-07-22T16:55:20.575Z"
},
{
"_id": ObjectId("5f19efac5cfa75483865eaa2"),
"createdAt": ISODate("2020-07-23T20:14:36.108Z"),
"localTime": "2020-07-23T15:14:36.108Z"
}
]
Demo example: https://mongoplayground.net/p/7IOGMrC2sf5

Related

Searching in Mongo by a date within at least one period

I have a sample document in Mongo with sample fields:
{
"_id": 1,
"feeCode": "SAMPLE_FEE_CODE",
"feeSuspensionPeriods": [
{
"startDate": "2021-01-01",
"endDate": "2021-06-30"
},
{
"startDate": "2022-01-01",
"endDate": "2022-06-30"
},
{
"startDate": "2023-01-01"
}
]
}
I need to write a query which will extract this document for me by feeCode and date. The date must contain at least one of the feeSuspensionPeriods.
For example:
When I ask for "SAMPLE_FEE_CODE" and "2021-01-01" parameters, it will return this document to me.
When I ask for "SAMPLE_FEE_CODE" and "2021-07-01" parameters, the document will not return it to me.
Use $elemMatch for this:
db.collection.find({
feeCode: "SAMPLE_FEE_CODE",
feeSuspensionPeriods: {
$elemMatch: {
startDate: {$lte: "2021-05-03"},
endDate: {$gte: "2021-05-03"}
}
}
See how it works on the playground example
you should try "elemMatch"
eg:
_db._collectionName.find({
feeCode: "SAMPLE_FEE_CODE",
feeSuspensionPeriods: {
$elemMatch: {
startDate: {$lte: "yyyy-mm-dd"},
endDate: {$gte: "yyyy-mm-dd"}
}
}

Extract info from JSON string

I am querying CloudTrail logs from S3 bucket via Amazon Athena.
My goal is to extract start/stop time for given instance id.
I am using query as:
SELECT
eventName,
eventTime,
if(responseelements like '%i-0000000000000%' ,'i-0000000000000') as instanceId
FROM cloudtrail_logs_pp
WHERE (responseelements like '%i-0000000000000%' )
AND (eventName = 'StopInstances' OR eventName = 'StartInstances')
AND "timestamp" BETWEEN '2022/01/01'
AND '2022/08/01'
ORDER BY eventTime
The issue I am facing is of duplicated entries via API call.
The structure of json file is:
{
"eventVersion": "1.08",
"eventTime": "2022-06-22T05:15:33Z",
"eventName": "StartInstances",
"requestParameters": {
"instancesSet": {
"items": [{
"instanceId": "i-00000"
}]
}
},
"responseElements": {
"requestId": "e95d270a",
"instancesSet": {
"items": [{
"instanceId": "i-00000",
"currentState": {
"code": 0,
"name": "pending"
},
"previousState": {
"code": 80,
"name": "stopped"
}
}]
}
},
"sessionCredentialFromConsole": "true"
},
However there are few entries where current and previous state are same.
How can I enhance my query to remove those entries?
Also there are cases when multiple instances were stopped / started - so can't use index in the query.

express-graphql: How to remove external "data" object layer.

I am replacing an existing REST endpoint with GraphQL.
In our existing REST endpoint, we return a JSON array.
[{
"id": "ABC"
},
{
"id": "123"
},
{
"id": "xyz"
},
{
"id": "789"
}
]
GraphQL seems to be wrapping the array in two additional object layers. Is there any way to remove the "data" and "Client" layers?
Response data:
{
"data": {
"Client": [
{
"id": "ABC"
},
{
"id": "123"
},
{
"id": "xyz"
},
{
"id": "789"
}
]
}
}
My query:
{
Client(accountId: "5417727750494381532d735a") {
id
}
}
No. That was the whole purpose of GraphQL. To have a single endoint and allow users to fetch different type/granularity of data by specifying the input in a query format as opposed to REST APIs and then map them onto the returned JSON output.
'data' acts as a parent/root level container for different entities that you have queried. Without these keys in the returned JSON data, there won't be any way to segregate the corresponding data. e.g.
Your above query can be modified to include another entity like Owner,
{
Client(accountId: "5417727750494381532d735a") {
id
}
Owner {
id
}
}
In which case, the output will be something like
{
"data": {
"Client": [
...
],
"Owner": [
...
]
}
}
Without the 'Client' and 'Owner' keys in the JSON outout, there is no way to separate the corresponding array values.
In your case, you can get only the array by doing data.Client on the returned output.

How to prepare Google Natural Language Proscessing output (json) for Big Query

I'm trying to query the output of a Natural Language Processing (NLP) call in Big Query (BQ) but I'm struggling to get the output in the right format for BQ.
I understand that BQ takes json files (as newline delimited) - but just not sure that (a) the output of NLP is json newline delimited and (b) if my schema is correct.
Here's the json output I'm working with:
{
"entities": [
{
"name": "Rowling",
"type": "PERSON",
"metadata": {
"wikipedia_url": "http://en.wikipedia.org/wiki/J._K._Rowling"
},
"salience": 0.65751493,
"mentions": [
{
"text": {
"content": " J.",
"beginOffset": -1
}
},
{
"text": {
"content": "K. Rowl",
"beginOffset": -1
}
}
]
},
{
"name": "LONDON",
"type": "LOCATION",
"metadata": {
"wikipedia_url": "http://en.wikipedia.org/wiki/London"
},
"salience": 0.14284456,
"mentions": [
{
"text": {
"content": "\ufeffLON",
"beginOffset": -1
}
}
]
},
{
"name": "Harry Potter",
"type": "WORK_OF_ART",
"metadata": {
"wikipedia_url": "http://en.wikipedia.org/wiki/Harry_Potter"
},
"salience": 0.0726779,
"mentions": [
{
"text": {
"content": "th Harry Pot",
"beginOffset": -1
}
},
{
"text": {
"content": "‘Harry Pot",
"beginOffset": -1
}
}
]
},
{
"name": "Deathly Hallows",
"type": "WORK_OF_ART",
"metadata": {
"wikipedia_url": "http://en.wikipedia.org/wiki/Harry_Potter_and_the_Deathly_Hallows"
},
"salience": 0.022565609,
"mentions": [
{
"text": {
"content": "he Deathly Hall",
"beginOffset": -1
}
}
]
}
],
"language": "en"
}
Is there a way to send the output directly to big query via the command line in Google Cloud shell?
Any information would be greatly appreciated!
Thanks
Glad you found my Harry Potter blog post! I'd recommend storing the NL API's JSON response as a string in BigQuery and then using a user-defined function to query it. You should be able to run the following (the table is publicly viewable) to get a count of how often each entity appears in the JSON you posted:
SELECT
COUNT(*) as entity_count, entity
FROM
JS(
(SELECT entities FROM [sara-bigquery:samples.hp_udf]),
entities,
"[{ name: 'entity', type: 'string'}]",
"function(row, emit) {
try {
x = JSON.parse(row.entities);
entities = x['entities'];
entities.forEach(function(data) {
emit({ entity: data.name });
});
} catch (e) {}
}"
)
GROUP BY entity
ORDER BY entity_count DESC
send the output directly to big query via the command line in Google Cloud shell
Look at this page, and search for "bq load"
https://cloud.google.com/bigquery/bq-command-line-tool
Here they have some example about json schema.
Schema to load json data to google big query

Query index in Cloudant doesn't return expected data

I have a Cloudant DB on Bluemix with an index defined as:
{
"index": {
"fields": [
{ "typ": "asc" },
{ "sen": "asc" },
{ "tim": "asc" }
]
},
"type": "json"
}
WHen I have a query of the form
{
"selector": {
"tim": {"$gt": millisecs},
"typ": "H"
},
"fields": ["sen","val","tim"],
"sort": [
{ "typ": "asc" },
{ "sen": "asc" },
{ "tim": "asc" }
],
"limit": readCount
}
it works perfectly. If I want to get everything, i.e. remove the condition typ="H", I get the error
"error":"no_usable_index","reason":"There is no index available for this selector."
I get the same response if I have "typ" : { "$in": ["H", "T"] }. I would have expected that the more generic query would work better than the one with extra selectors.
I just don't understand how this could be!
"typ" is the first field of your index, so is the basis of the ordering.
"tim", if it's the only element of the query, doesn't take advantage of the index, so it would trigger a full table scan if that query was allowed.
However you can ask explicitely for a full table scan if you add:
"_id": { "$gt": null }
See the doc, your case is not really described, but I think it's implied.
Did you try to create separate indexes for these fields and run the same query?