how can i update data on pymongo well? - pymongo

i inserted data into mongodb to use pymongo like this.
data = {
'page_url': product_link,
'static': {
'created_at': datetime.datetime.now(),
'created_at_gmt': datetime.datetime.now(),
},
'created_at': datetime.datetime.now(),
'created_at_gmt': datetime.datetime.now(),
'updated_at': datetime.datetime.now(),
'updated_at_gmt': datetime.datetime.now()
db.product.insert_one(data)
then when i updated data, i removed created_at and created_at_gmt.
data = {
'page_url': product_link,
'static': {
},
'updated_at': datetime.datetime.now(),
'updated_at_gmt': datetime.datetime.now()
db.product.update_one({'page_url': product_link}, {"$set": data}, upsert=True)
but after i updated it, when i checked db data on compass, there was no created_at and created_at_gmt in static. on in static. there was created_at and created_at_gmt on same depth with static.
i don't know why it heppend...
can you help me ?

You should only update the keys you want to in the $set stage.
current_datetime = datetime.datetime.now()
db.product.update_one({
'page_url': product_link
}, {
"$set": {
'updated_at': current_datetime ,
'updated_at_gmt': current_datetime
}
}, upsert=True)
Like the above example, you should only reference the keys you want to update.
Have a look at $unset update command if you want to remove a key from a MongoDB Document.
Also, store the datetime value in a variable, so that it is uniform.

Related

Pymongo update a document with a condition

I'm trying to update a document under a condition with pymongo, but failed to figure out a way. Use this document as an example:
{"_id":"mol001",
"deid":["a001", "a003", "a005"],
"count":3}
For _id of mol001, I'm trying to append a tag to deid and update count to 4:
{"_id":"mol001",
"deid":["a001", "a003", "a005", "b001"],
"count":4}
One thing needs to be aware of is the count value. if It's larger than 10, the document will not be updated. Below is what I came up with:
mol = "mol001"
b001 = "b001"
try:
## in case mol001 doesn't exist, use upset = True
count = coll.find_one({"_id": mol}, {"_id": False, "count": 1})['count']
except:
count = 0
if count <= 10:
coll.update_one({"_id": mol}, {'$push': {'deid': b001}}, upsert=True)
coll.update_one({"_id": mol}, {"$inc": {"count": 1}}, upsert=True)
This was very inefficient since it needs to do one query and update twice. Is there a way to use $cond to do the update in one sentence?
Here's one way to do it.
db.collection.update({
"_id": "mol001",
"count": {
"$lte": 10
}
},
{
"$push": {
"deid": "b001"
},
"$inc": {
"count": 1
}
},
{
"upsert": true
})
Try it on mongoplayground.net.
You can combine the two operations into a single update operation:
coll.update_one({'_id': mol}, {'$push': {'deid': b001}, '$inc': {'count': 1}}, upsert=True)

Delete and add attributes with array.map and the spread operator

I'm trying to mangle data returned from an api. I've got an array of objects returned. I want to delete the password field and then add a couple of additional fields. I'd like to use the spread operator but my process feels a bit clunky.
myArray.map( item => {
const newItem = { ...item };
delete newItem.password;
newItem.saved = true;
return newItem;
});
Is there a nicer way to do this?
Given an array of objects -
const myArrayOfObjects = [
{id: 1, keyToDelete: 'nonsense'},
{id: 2, keyToDelete: 'rubbish'}
];
Delete the attribute keyToDelete, and add a new key newKey with the value "someVar".
myArrayOfObjects.map(({ keyToDelete, ...item}) => { ...item, newKey:'someVar'});
Updating the array to
[
{id: 1, newKey:'someVar'},
{id: 2, newKey:'someVar'}
]
See this great post for more information on the deletion method.

ChartJS Unix Time Values changed

I'm importing data from a Sybase database into ChartJS in VueJs2. I'm using the vue-chart module
I push the timestamps into an array as Unix times using
this.firstIn(new Date(tnaDetails[0].Sunday_FirstIn).getTime())
So:
[Sunday_FirstIn:2010-01-17 08:00:00.0]
Would convert to
1263708000000
Which I then add to the dataset:
datasets: [{
type: 'line',
label: "First In",
backgroundColor: "green",
data: this.firstIn,
fill: false
}
]
However, when the data is plotted on the graph, the values are changed. The above Unit Timestamp becomes
1263700000000
Which obviously returns the wrong time. I'm not doing anything to the ticks in the options.
Below is a result of the numbers being changed. The console has the original data:
Is there a setting that alters the precision/values of numbers in ChartJS that I'm not aware of?
Thanks.
Seth
For anyone who has any similar problem in future, I patched together a few solutions I found.
Firstly, from here Unix Timestamp in JavaScript, I wrote the method:
getTimeString: function(dateString) {
var hours = new Date(dateString).getHours();
var mins = new Date(dateString).getMinutes();
return Math.round((new Date("1970-02-01 " + hours + ":" + mins)).getTime());
}
The important part here is to make sure you have the same day. Not doing this will cause the ChartJS graph to plot the times in different places on the y-axis, even if the hours are the same.
Then from this StackOverFlow question and the related plunker, in the chart options, I have:
{
responsive: true,
maintainAspectRatio: false,
scales: {
yAxes: [{
position: 'left',
ticks: {
callback: value => {
let date = moment(value);
if (date.diff(moment('1970-02-01 23:59:59'), 'minutes') === 0) {
return null;
}
return date.format('H:mm');
},
stepSize: 3.6e+6
}
}]
}, //end scales
tooltips: {
callbacks: {
label: function(toolTipItem, data) {
let date = moment(toolTipItem.yLabel);
if (date.diff(moment('1970-02-01 23:59:59'), 'minutes') === 0) {
return null;
}
return date.format('H:mm');
}
}
}
}
Pay attention to the callbacks. They will format the time, calculating the difference from a set time to the time you need plotted. In the first function, you could really use any day, it wouldn't matter, as long as it's the same day. The stepSize will display hourly intervals on the yAxis.

RavenDB facet takes to long query time

I am new to ravendb and trying it out to see if it can do the job for the company i work for .
i updated a data of 10K records to the server .
each data looks like this :
{
"ModelID": 371300,
"Name": "6310I",
"Image": "0/7/4/9/28599470c",
"MinPrice": 200.0,
"MaxPrice": 400.0,
"StoreAmounts": 4,
"AuctionAmounts": 0,
"Popolarity": 16,
"ViewScore": 0.0,
"ReviewAmount": 4,
"ReviewScore": 40,
"Cat": "E-Cellphone",
"CatID": 1100,
"IsModel": true,
"ParamsList": [
{
"ModelID": 371300,
"Pid": 188396,
"IntVal": 188402,
"Param": "Nokia",
"Name": "Manufacturer",
"Unit": "",
"UnitDir": "left",
"PrOrder": 0,
"IsModelPage": true
},
{
"ModelID": 371305,
"Pid": 398331,
"IntVal": 1559552,
"Param": "1.6",
"Name": "Screen size",
"Unit": "inch",
"UnitDir": "left",
"PrOrder": 1,
"IsModelPage": false
},.....
where ParamsList is an array of all the attributes of a single product.
after building an index of :
from doc in docs.FastModels
from docParamsListItem in ((IEnumerable<dynamic>)doc.ParamsList).DefaultIfEmpty()
select new { Param = docParamsListItem.IntVal, Cat = doc.Cat }
and a facet of
var facetSetupDoc = new FacetSetup
{
Id = "facets/Params2Facets",
Facets = new List<Facet> { new Facet { Name = "Param" } }
};
and search like this
var facetResults = session.Query<FastModel>("ParamNewIndex")
.Where(x => x.Cat == "e-cellphone")
.ToFacets("facets/Params2Facets");
it takes more than a second to query and that is on only 10K of data . where our company has more than 1M products in DB.
am i doing something wrong ?
In order to generate facets, you have to check for each & every individual value of docParamsListItem.IntVal. If you have a lot of them, that can take some time.
In general, you shouldn't have a lot of facets, since that make no sense, it doesn't help the user.
For integers, you usually use ranges, instead of the actual values.
For example, price within a certain range.
You use just the field for things like manufacturer, or the MegaPixels count, where you have lot number or items (about a dozen or two)
You didn't mention which build you are using, but we made some major improvements there recently.

Is it possible to turn an array returned by the Mongo GeoNear command (using Ruby/Rails) into a Plucky object?

As a total newbie I have been trying to get the geoNear command working in my rails application and it appear to be working fine. The major annoyance for me is that it is returning an array with strings rather than keys which I can call on to pull out data.
Having dug around, I understand that MongoMapper uses Plucky to turn the the query resultant into a friendly object which can be handled easily but I haven't been able to find out how to transform the result of my geoNear query into a plucky object.
My questions are:
(a) Is it possible to turn this into a plucky object and how do i do that?
(b) If it is not possible how can I most simply and systematically extract each record and each field?
here is the query in my controller
#mult = 3963 * (3.14159265 / 180 ) # Scale to miles on earth
#results = #db.command( {'geoNear' => "places", 'near'=> #search.coordinates , 'distanceMultiplier' => #mult, 'spherical' => true})
Here is the object i'm getting back (with document content removed for simplicity)
{"ns"=>"myapp-development.places", "near"=>"1001110101110101100100110001100010100010000010111010", "results"=>[{"dis"=>0.04356444023196527, "obj"=>{"_id"=>BSON::ObjectId('4ee6a7d210a81f05fe000001'),...}}], "stats"=>{"time"=>0, "btreelocs"=>0, "nscanned"=>1, "objectsLoaded"=>1, "avgDistance"=>0.04356444023196527, "maxDistance"=>0.0006301239824196907}, "ok"=>1.0}
Help is much appreciated!!
Ok so lets say you store the results into a variable called places_near:
places_near = t.command( {'geoNear' => "places", 'near'=> [50,50] , 'distanceMultiplier' => 1, 'spherical' => true})
This command returns an hash that has a key (results) which maps to a list of results for the query. The returned document looks like this:
{
"ns": "test.places",
"near": "1100110000001111110000001111110000001111110000001111",
"results": [
{
"dis": 69.29646421910687,
"obj": {
"_id": ObjectId("4b8bd6b93b83c574d8760280"),
"y": [
1,
1
],
"category": "Coffee"
}
},
{
"dis": 69.29646421910687,
"obj": {
"_id": ObjectId("4b8bd6b03b83c574d876027f"),
"y": [
1,
1
]
}
}
],
"stats": {
"time": 0,
"btreelocs": 1,
"btreelocs": 1,
"nscanned": 2,
"nscanned": 2,
"objectsLoaded": 2,
"objectsLoaded": 2,
"avgDistance": 69.29646421910687
},
"ok": 1
}
To iterate over the responses just iterate as you would over any list in ruby:
places_near['results'].each do |result|
# do stuff with result object
end