Meteor: meteor-collectionapi PUT overwrites the entire object instead of just updating the field I want - api

I'm trying to use the meteor-collectionapi package to update my database. I've set up a basic collection to test out the functionality.
I'm starting with this data:
{ "name" : "Darrell David", "age" : "18", "gender" : "Male", "_id" : "8BW9Yg2oKByBGdnSa" }
{ "name" : "Julie Smith", "age" : "21", "gender" : "Female", "_id" : "fAaFwCEXLzrmejnJK" }
{ "name" : "Todd Davis", "age" : "32", "gender" : "Male", "_id" : "ixKjhkTmjrNte2DjP" }
Now, I want to update the gender of the first player to "Female" so I call this using CURL:
curl -H "X-Auth-Token: 97f0ad9e24ca5e0408a269748d7fe0a0" -X PUT -d "{\"$set\":{\"gender\":\"Female\"}}" http://localhost:3000/collectionapi/players/8BW9Yg2oKByBGdnSa
And what I wind up with is this:
{ "_id" : "8BW9Yg2oKByBGdnSa", "" : { "gender" : "Female" } }
{ "name" : "Julie Smith", "age" : "21", "gender" : "Female", "_id" : "fAaFwCEXLzrmejnJK" }
{ "name" : "Todd Davis", "age" : "32", "gender" : "Male", "_id" : "ixKjhkTmjrNte2DjP" }
The first player has been completely overwritten and the name and age fields have been lost.
What am I missing here? When I execute this command in the MongoDB console it works perfectly:
db.players.update(
{ _id: "8BW9Yg2oKByBGdnSa" },
{ $set: { gender: "Female" } }
)

I'm guessing that bash is replacing "$set" with an empty environment variable
eg. echo "$set" vs echo "\$set"
so update your PUT command to:
curl -H "X-Auth-Token: 97f0ad9e24ca5e0408a269748d7fe0a0" -X PUT -d "{\"\$set\":{\"gender\":\"Female\"}}" http://localhost:3000/collectionapi/players/8BW9Yg2oKByBGdnSa
By default Collection.update() will replace a document if no modifiers are present ($set, $unset, $push, $pull etc). So the command being sent to the server is to replace the document with {"":{"gender":"Female"}}

Related

BQ load job failing when trying to create table from AVRO file

I am trying to create a BQ Table from AVRO file. I am getting this error when i run the BQ load job:
"Error while reading data, error message: The Apache Avro library
failed to parse the header with the following error: Unexpected type
for default value. Expected long, but found null: null"
The Schema of the AVRO file is:
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
I am not sure what is wrong with the schema or anything else, because of which I am unable to load the data.
The problem is most likely the fields that have type int but you have null as the default value. For example:
"name" : "type_id",
"type" : "int",
"default" : null
The default should either be changed to be an integer or the type should be changed to be a union that includes null (like many of the other fields).

Remove Subdocument items with $pull

I'm trying to remove items from subdocuments using ExpressJS and Mongoose but it is only removing the first items, not the sub items.
So I want to remove "subitem 2" in the messages Array
This is the structure:
{
"_id" : ObjectId("5c4ee94b30ebd71cbed89a35"),
"title" : "Test",
"subitem" : [
{
"_id" : ObjectId("5c4ee95630ebd71cbed89a36"),
"title" : "Item 1",
"messages" : [
{
"_id" : ObjectId("5c4ee95f30ebd71cbed89a37"),
"type" : "single_article",
"date" : "Jan 28, 2019",
"title" : "subitem 1",
"text" : ""
}
]
},
{
"_id" : ObjectId("5c4ee96830ebd71cbed89a38"),
"title" : "item 2",
"messages" : [
{
"_id" : ObjectId("5c4ee96e30ebd71cbed89a39"),
"type" : "single_article",
"date" : "Jan 28, 2019",
"title" : "subitem 2",
"text" : ""
}
]
}
],
"__v" : 0
}
And this is the $pull method:
getController.deleteRec = function(req,res,collection){
var id = req.params.id;
console.log(id);
collection.updateOne({'subitem.messages._id': id}, {$pull: {'subitem.0.messages': {"_id": id}}}).
then(function(result){
console.log(result);
});
};
Now I know why it is only deleting the first item because I have "subitem.0.messages". How can I loop over this, so it can delete all items?
You can use $ as a wildcard index, removing all elements in the array matching your query like this:
{$pull: {'subitem.$.messages': {"_id": id}}}
if you want to remove multiple documents:
{$pull: {'subitem.$.messages': {"_id": {$in : [id, id2, id3...]}}}}

How to use i18next-node-mongodb-backend?

Did anyone knows how to store data in mongoDB to be used with https://github.com/gian788/i18next-node-mongodb-backend. Tried couple of things and its not translating keys.
Here is how I am storing my data in mongo
{"en" : { "translation" : { "FirstName" : "First Name", "LastName" : "Lastwq Name" } } }
While printing it always prints key with error
i18next::translator: missingKey en translation FirstName FirstName
Any help?
The format is the following :
{
"_id" : ObjectId("581f3645caa6b31c074dd91c"),
"language" : "en",
"namespace" : "translation",
"data" : {
"FirstName" : "First Name",
"LastName" : "Lastwq Name"
}
}
Obviously you can change the namespace in the init options

Elasticsearch - Extracting PDF content and encoding with base64

I want to be able to extract content from a PDF file and to be able to search within that content using ElasticSearch.
I did install elasticsearch/elasticsearch-mapper-attachments/2.6.0
I have created a new index named "docs".
I did create a file named "tmp.json" with that content :
{"title": "file.pdf", "file": "IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg=="}
I did execute the following :
curl -X PUT "http://localhost:9200/docs/attachment/_mapping" -d '{
"attachment": {
"properties" : {
'file" : {
"type" : "attachment",
"fields" : {
"title" : {"store":"yes"},
"file":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":"yes"}
}
}
}
}
}'
and the following :
curl -X POST "http://localhost:9200/docs/attachment" -d #tmp.json
The problem is that the content is stored as it is in the file.
I was expecting the content to be decoded, like so :
base64.b64decode("IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==")
That gives :
b'"God Save the Queen" (alternatively "God Save the King"'
To encode in base64, here what I do :
import json, base64
file64 = base64.b64encode(open('file.pdf', "rb").read()).decode('ascii')
f = open('tmp.json', 'w')
data = {"file":file64, "title":fname}
json.dump(data,f)
f.close()
I would like to be able to see the content using kibana (but for now I see only the base64 data ...)
This didn't work :
curl -X PUT "http://localhost:9200/docs/attachment/_mapping" -d '{
"attachment": {
"properties" : {
"content" : {
"type" : "attachment",
"fields" : {
"title" : {"store":"yes"},
"content":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":"yes"}
}
}
}
}
}'
This worked, and I can see the content of the PDF through Kibana :
curl -X PUT "http://localhost:9200/docs" -d '{
"mappings" : {
"attachment" : {
"properties" : {
"content" : {
"type" : "attachment",
"fields" : {
"content" : { "store" : "yes" },
"author" : { "store" : "yes" },
"title" : { "store" : "yes"},
"date" : { "store" : "yes" },
"keywords" : { "store" : "yes", "analyzer" : "keyword" },
"name" : { "store" : "yes" },
"content_length" : { "store" : "yes" },
"content_type" : { "store" : "yes" }
}
}
}
}
}
}'

restart jobtracker through cloudera manager API

I am trying to restart Mapreduce Jobtracker through Cloudera Manager API. Stats for Jobtracker is as follows :
local-iMac-399:$ curl -u 'admin:admin' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/mapreduce/roles/mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86'
{
"name" : "mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86",
"type" : "JOBTRACKER",
"serviceRef" : {
"clusterName" : "cluster",
"serviceName" : "mapreduce"
},
"hostRef" : {
"hostId" : "24259373-7e71-4089-8251-faf055e42ad7"
},
"roleUrl" : "http://hadoop-namenode.dev.com:7180/cmf/roleRedirect/mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86",
"roleState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "JOB_TRACKER_FILE_DESCRIPTOR",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_GC_DURATION",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_HOST_HEALTH",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_LOG_DIRECTORY_FREE_SPACE",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_SCM_HEALTH",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_UNEXPECTED_EXITS",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_WEB_METRIC_COLLECTION",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "STALE",
"haStatus" : "ACTIVE",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"commissionState" : "COMMISSIONED",
"roleConfigGroupRef" : {
"roleConfigGroupName" : "mapreduce-JOBTRACKER-BASE"
}
}
local-iMac-399:$
Dont know How do I use API to restart just Jobtracker ?
I tried to restart Hive service using following command but got some error
local-iMac-399:$curl -X POST -u 'admin:admin' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/hive/roleCommands/restart'
{
"message" : "No content to map due to end-of-input\n at [Source: org.apache.cxf.transport.http.AbstractHTTPDestination$1#4169c499; line: 1, column: 1]"
}
I would appreciate if someone help in understanding how to use Cloudera Manager API
Based on the information provided, this is how you'd invoke the CM API JobTracker restart
curl -u 'admin:admin' -X POST -H "Content-Type:application/json" -d '{"items":["mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86"]}' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/mapreduce/roleCommands/restart'