I'm trying to use the meteor-collectionapi package to update my database. I've set up a basic collection to test out the functionality.
I'm starting with this data:
{ "name" : "Darrell David", "age" : "18", "gender" : "Male", "_id" : "8BW9Yg2oKByBGdnSa" }
{ "name" : "Julie Smith", "age" : "21", "gender" : "Female", "_id" : "fAaFwCEXLzrmejnJK" }
{ "name" : "Todd Davis", "age" : "32", "gender" : "Male", "_id" : "ixKjhkTmjrNte2DjP" }
Now, I want to update the gender of the first player to "Female" so I call this using CURL:
curl -H "X-Auth-Token: 97f0ad9e24ca5e0408a269748d7fe0a0" -X PUT -d "{\"$set\":{\"gender\":\"Female\"}}" http://localhost:3000/collectionapi/players/8BW9Yg2oKByBGdnSa
And what I wind up with is this:
{ "_id" : "8BW9Yg2oKByBGdnSa", "" : { "gender" : "Female" } }
{ "name" : "Julie Smith", "age" : "21", "gender" : "Female", "_id" : "fAaFwCEXLzrmejnJK" }
{ "name" : "Todd Davis", "age" : "32", "gender" : "Male", "_id" : "ixKjhkTmjrNte2DjP" }
The first player has been completely overwritten and the name and age fields have been lost.
What am I missing here? When I execute this command in the MongoDB console it works perfectly:
db.players.update(
{ _id: "8BW9Yg2oKByBGdnSa" },
{ $set: { gender: "Female" } }
)
I'm guessing that bash is replacing "$set" with an empty environment variable
eg. echo "$set" vs echo "\$set"
so update your PUT command to:
curl -H "X-Auth-Token: 97f0ad9e24ca5e0408a269748d7fe0a0" -X PUT -d "{\"\$set\":{\"gender\":\"Female\"}}" http://localhost:3000/collectionapi/players/8BW9Yg2oKByBGdnSa
By default Collection.update() will replace a document if no modifiers are present ($set, $unset, $push, $pull etc). So the command being sent to the server is to replace the document with {"":{"gender":"Female"}}
Related
I am trying to create a BQ Table from AVRO file. I am getting this error when i run the BQ load job:
"Error while reading data, error message: The Apache Avro library
failed to parse the header with the following error: Unexpected type
for default value. Expected long, but found null: null"
The Schema of the AVRO file is:
{
"type" : "record",
"name" : "Pair",
"namespace" : "org.apache.avro.mapred",
"fields" : [ {
"name" : "key",
"type" : "int",
"doc" : ""
}, {
"name" : "value",
"type" : {
"type" : "record",
"name" : "CustomerInventoryOrderItems",
"namespace" : "com.test.customer.order",
"fields" : [ {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "inventory_order_items",
"type" : {
"type" : "map",
"values" : {
"type" : "array",
"items" : {
"type" : "record",
"name" : "CustomerInventoryOrderItem",
"fields" : [ {
"name" : "order_item_id",
"type" : "int",
"default" : null
}, {
"name" : "updated_at",
"type" : "long"
}, {
"name" : "created_at",
"type" : "long"
}, {
"name" : "product_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "type_id",
"type" : "int",
"default" : null
}, {
"name" : "event_id",
"type" : [ "null", "int" ],
"default" : null
}, {
"name" : "price",
"type" : [ "null", "double" ],
"default" : null
}, {
"name" : "tags",
"type" : [ "null", "string" ],
"default" : null
}, {
"name" : "estimated_ship_date",
"type" : [ "null", "long" ],
"default" : null
} ]
}
}
}
} ]
},
"doc" : "",
"order" : "ignore"
} ]
}
I am not sure what is wrong with the schema or anything else, because of which I am unable to load the data.
The problem is most likely the fields that have type int but you have null as the default value. For example:
"name" : "type_id",
"type" : "int",
"default" : null
The default should either be changed to be an integer or the type should be changed to be a union that includes null (like many of the other fields).
I'm trying to remove items from subdocuments using ExpressJS and Mongoose but it is only removing the first items, not the sub items.
So I want to remove "subitem 2" in the messages Array
This is the structure:
{
"_id" : ObjectId("5c4ee94b30ebd71cbed89a35"),
"title" : "Test",
"subitem" : [
{
"_id" : ObjectId("5c4ee95630ebd71cbed89a36"),
"title" : "Item 1",
"messages" : [
{
"_id" : ObjectId("5c4ee95f30ebd71cbed89a37"),
"type" : "single_article",
"date" : "Jan 28, 2019",
"title" : "subitem 1",
"text" : ""
}
]
},
{
"_id" : ObjectId("5c4ee96830ebd71cbed89a38"),
"title" : "item 2",
"messages" : [
{
"_id" : ObjectId("5c4ee96e30ebd71cbed89a39"),
"type" : "single_article",
"date" : "Jan 28, 2019",
"title" : "subitem 2",
"text" : ""
}
]
}
],
"__v" : 0
}
And this is the $pull method:
getController.deleteRec = function(req,res,collection){
var id = req.params.id;
console.log(id);
collection.updateOne({'subitem.messages._id': id}, {$pull: {'subitem.0.messages': {"_id": id}}}).
then(function(result){
console.log(result);
});
};
Now I know why it is only deleting the first item because I have "subitem.0.messages". How can I loop over this, so it can delete all items?
You can use $ as a wildcard index, removing all elements in the array matching your query like this:
{$pull: {'subitem.$.messages': {"_id": id}}}
if you want to remove multiple documents:
{$pull: {'subitem.$.messages': {"_id": {$in : [id, id2, id3...]}}}}
Did anyone knows how to store data in mongoDB to be used with https://github.com/gian788/i18next-node-mongodb-backend. Tried couple of things and its not translating keys.
Here is how I am storing my data in mongo
{"en" : { "translation" : { "FirstName" : "First Name", "LastName" : "Lastwq Name" } } }
While printing it always prints key with error
i18next::translator: missingKey en translation FirstName FirstName
Any help?
The format is the following :
{
"_id" : ObjectId("581f3645caa6b31c074dd91c"),
"language" : "en",
"namespace" : "translation",
"data" : {
"FirstName" : "First Name",
"LastName" : "Lastwq Name"
}
}
Obviously you can change the namespace in the init options
I want to be able to extract content from a PDF file and to be able to search within that content using ElasticSearch.
I did install elasticsearch/elasticsearch-mapper-attachments/2.6.0
I have created a new index named "docs".
I did create a file named "tmp.json" with that content :
{"title": "file.pdf", "file": "IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg=="}
I did execute the following :
curl -X PUT "http://localhost:9200/docs/attachment/_mapping" -d '{
"attachment": {
"properties" : {
'file" : {
"type" : "attachment",
"fields" : {
"title" : {"store":"yes"},
"file":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":"yes"}
}
}
}
}
}'
and the following :
curl -X POST "http://localhost:9200/docs/attachment" -d #tmp.json
The problem is that the content is stored as it is in the file.
I was expecting the content to be decoded, like so :
base64.b64decode("IkdvZCBTYXZlIHRoZSBRdWVlbiIgKGFsdGVybmF0aXZlbHkgIkdvZCBTYXZlIHRoZSBLaW5nIg==")
That gives :
b'"God Save the Queen" (alternatively "God Save the King"'
To encode in base64, here what I do :
import json, base64
file64 = base64.b64encode(open('file.pdf', "rb").read()).decode('ascii')
f = open('tmp.json', 'w')
data = {"file":file64, "title":fname}
json.dump(data,f)
f.close()
I would like to be able to see the content using kibana (but for now I see only the base64 data ...)
This didn't work :
curl -X PUT "http://localhost:9200/docs/attachment/_mapping" -d '{
"attachment": {
"properties" : {
"content" : {
"type" : "attachment",
"fields" : {
"title" : {"store":"yes"},
"content":{
"type":"string",
"term_vector":"with_positions_offsets",
"store":"yes"}
}
}
}
}
}'
This worked, and I can see the content of the PDF through Kibana :
curl -X PUT "http://localhost:9200/docs" -d '{
"mappings" : {
"attachment" : {
"properties" : {
"content" : {
"type" : "attachment",
"fields" : {
"content" : { "store" : "yes" },
"author" : { "store" : "yes" },
"title" : { "store" : "yes"},
"date" : { "store" : "yes" },
"keywords" : { "store" : "yes", "analyzer" : "keyword" },
"name" : { "store" : "yes" },
"content_length" : { "store" : "yes" },
"content_type" : { "store" : "yes" }
}
}
}
}
}
}'
I am trying to restart Mapreduce Jobtracker through Cloudera Manager API. Stats for Jobtracker is as follows :
local-iMac-399:$ curl -u 'admin:admin' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/mapreduce/roles/mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86'
{
"name" : "mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86",
"type" : "JOBTRACKER",
"serviceRef" : {
"clusterName" : "cluster",
"serviceName" : "mapreduce"
},
"hostRef" : {
"hostId" : "24259373-7e71-4089-8251-faf055e42ad7"
},
"roleUrl" : "http://hadoop-namenode.dev.com:7180/cmf/roleRedirect/mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86",
"roleState" : "STARTED",
"healthSummary" : "GOOD",
"healthChecks" : [ {
"name" : "JOB_TRACKER_FILE_DESCRIPTOR",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_GC_DURATION",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_HOST_HEALTH",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_LOG_DIRECTORY_FREE_SPACE",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_SCM_HEALTH",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_UNEXPECTED_EXITS",
"summary" : "GOOD"
}, {
"name" : "JOB_TRACKER_WEB_METRIC_COLLECTION",
"summary" : "GOOD"
} ],
"configStalenessStatus" : "STALE",
"haStatus" : "ACTIVE",
"maintenanceMode" : false,
"maintenanceOwners" : [ ],
"commissionState" : "COMMISSIONED",
"roleConfigGroupRef" : {
"roleConfigGroupName" : "mapreduce-JOBTRACKER-BASE"
}
}
local-iMac-399:$
Dont know How do I use API to restart just Jobtracker ?
I tried to restart Hive service using following command but got some error
local-iMac-399:$curl -X POST -u 'admin:admin' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/hive/roleCommands/restart'
{
"message" : "No content to map due to end-of-input\n at [Source: org.apache.cxf.transport.http.AbstractHTTPDestination$1#4169c499; line: 1, column: 1]"
}
I would appreciate if someone help in understanding how to use Cloudera Manager API
Based on the information provided, this is how you'd invoke the CM API JobTracker restart
curl -u 'admin:admin' -X POST -H "Content-Type:application/json" -d '{"items":["mapreduce-JOBTRACKER-0675ebab2b87e3869e0d90167cf4bf86"]}' 'http://hadoop-namenode.dev.com:7180/api/v6/clusters/Cluster%201/services/mapreduce/roleCommands/restart'