I am a complete beginner in PostgreSQL. And I was really amazed by the hstore datatype provided by Postgres. Well, I am using the Rails 3 framework and developing a simple app that uses PostgreSQL. I want to store an array of hashes in a field.
For Eg.:
authors: [
{
name: "abc",
email: "abc#example.com"
},
{
name: "xyz",
email: "xyz#example.com"
}
]
Is this possible in PostgreSQL using Rails 3? If so, can somebody give insights on how?
Thanks
It's certainly possible to create an array-of-hstore column in Rails 4 with e.g. a column spec like this in the table creation:
t.hstore :properties, :array => true
However, there's an encoding bug in Rails 4.0 that unfortunately renders them unusable; essentially you can read from them and they present correctly as arrays of hashes, but not write.
I've opened an issue (with fix patch) at https://github.com/rails/rails/issues/11135 which hopefully will be incorporated soon.
I'm not sure if they'll allow you to have an hstore array, but there are a few active record extensions that add hstore and array types. e.g.:
https://github.com/funny-falcon/activerecord-postgresql-arrays
https://github.com/engageis/activerecord-postgres-hstore
https://github.com/tlconnor/activerecord-postgres-array
Don't miss the Rails 4-related improvements, too:
http://blog.remarkablelabs.com/2012/12/a-love-affair-with-postgresql-rails-4-countdown-to-2013
Related
I am using Audit.Net library to log EntityFramework actions into a database (currently everything into one AuditEventLogs table, where the JsonData column stores the data in the following Json format:
{
"EventType":"MyDbContext:test_database",
"StartDate":"2021-06-24T12:11:59.4578873Z",
"EndDate":"2021-06-24T12:11:59.4862278Z",
"Duration":28,
"EntityFrameworkEvent":{
"Database":"test_database",
"Entries":[
{
"Table":"Offices",
"Name":"Office",
"Action":"Update",
"PrimaryKey":{
"Id":"40b5egc7-46ca-429b-86cb-3b0781d360c8"
},
"Changes":[
{
"ColumnName":"Address",
"OriginalValue":"test_address",
"NewValue":"test_address"
},
{
"ColumnName":"Contact",
"OriginalValue":"test_contact",
"NewValue":"test_contact"
},
{
"ColumnName":"Email",
"OriginalValue":"test_email",
"NewValue":"test_email2"
},
{
"ColumnName":"Name",
"OriginalValue":"test_name",
"NewValue":"test_name"
},
{
"ColumnName":"OfficeSector",
"OriginalValue":1,
"NewValue":1
},
{
"ColumnName":"PhoneNumber",
"OriginalValue":"test_phoneNumber",
"NewValue":"test_phoneNumber"
}
],
"ColumnValues":{
"Id":"40b5egc7-46ca-429b-86cb-3b0781d360c8",
"Address":"test_address",
"Contact":"test_contact",
"Email":"test_email2",
"Name":"test_name",
"OfficeSector":1,
"PhoneNumber":"test_phoneNumber"
},
"Valid":true
}
],
"Result":1,
"Success":true
}
}
Me and my team has a main aspect to achieve:
Being able to create a search page where administrators are able to tell
who changed
what did they change
when did the change happen
They can give a time period, to reduce the number of audit records, and the interesting part comes here:
There should be an input text field which should let them search in the values of the "ColumnValues" section.
The problems I encountered:
Even if I map the Json structure into relational rows, I am unable to search in every column, with keeping the genericity.
If I don't map, I could search in the Json string with LIKE mssql function but on the order of a few 100,000 records it takes an eternity for the query to finish so it is probably not the way.
Keeping the genericity would be important, so we don't need to modify the audit search page every time when we create or modify a new entity.
I only know MSSQL, but is it possible that storing the audit logs in a document oriented database like cosmosDB (or anything else, it was just an example) would solve my problem? Or can I reach the desired behaviour using relational database like MSSQL?
Looks like you're asking for an opinion, in that case I would strongly recommend a document oriented DB.
CosmosDB could be a great option since it supports SQL queries.
There is an extension to log to CosmosDB from Audit.NET: Audit.AzureCosmos
A sample query:
SELECT c.EventType, e.Table, e.Action, ch.ColumnName, ch.OriginalValue, ch.NewValue
FROM c
JOIN e IN c.EntityFrameworkEvent.Entries
JOIN ch IN e.Changes
WHERE ch.ColumnName = "Address" AND ch.OriginalValue = "test_address"
Here is a nice post with lot of examples of complex SQL queries on CosmosDB
Backstory:
I'm building an e-commerce web app (online store)
Now I got to the point of choosing a database system and an appropriate design.
I got stuck with developing a design for product attributes
I've been considering of choosing NoSQL (MongoDB) or SQL database systems
I need you advice and help
The problem:
When you choose a product type (e.g. table) it should show you the corresponding filters for such a type (e.g. height, material etc.). When you choose another type, say "car", it provides you with the car specific filter attributes (e.g. fuel, engine volume)
For example, here on one popular online store if you choose a data storage type you get a filter fo this type attributes, such as hard drive size or connection type
Question
What approach is the best for such a problem? I described some below, but maybe you have your own thoughts in regard to it
MongoDB
Possible solution:
You can implement such product attrs structure pretty easy.
You can create one collection with a field attrs for each product and put there whatever you want, like they suggest here (field "details"):
https://docs.mongodb.com/ecosystem/use-cases/product-catalog/#non-relational-data-model
The structure will be
Problem:
With such a solution you don't have product types at all so you can't filter the products out by their types. Each product contains it's own arbitrary structure in attrs field and don't follow any pattern
Ir maybe I can somehow go with this approach?
SQL
There are solutions like single table where all the products store in one table and you end up with as many fields as an attribute number of all the products taken together.
Or for every product type you create a new table
But I won't consider these ones. One is very bulky and another one isn't much flexible and requires a dynamic scheme design
Possible solution
There is one pretty flexible solution called EAV https://en.wikipedia.org/wiki/Entity%E2%80%93attribute%E2%80%93value_model
Our schema would be:
EAV
Such a design may be done on MongoDB system, but I'm not sure it's been made for such a normalised structure
Problem
The schema is going to get really huge and really hard to query and grasp
If you choose SQL database, take a look PostgreSQL which supports JSON features. Not necessarily you need to follow Database normalization.
If you choose MongoDB, you need to store attrs array with generic {key:"field", value:"value"} pairs.
{id:1, attrs:[{key: "prime", value: true}, {key:"height", value:2}, {key:"material", value:"wood"},{key:"color", "value":"brown"}]}
{id:2, attrs:[{key: "prime", value: true}, {key:"fuel", value:"gas"}, {key:"volume", "value":3}]}
{id:3, attrs:[{key: "prime", value: true}, {key:"fuel", value:"diesel"}, {key:"volume", "value":1.5}]}
Then you define Multi-key index like this:
db.collection.createIndex({"attrs.key":1, "attrs.value":1})
If you want apply step-by-step filters, use MongoDB aggregation with $elemMatch operator
☑ Prime
☑ Fuel
☐ Other
...
☑ Volume 3
☐ Volume 1.5
Query's representation
db.collection.aggregate([
{
$match: {
$and: [
{
attrs: {
$elemMatch: {
key: "prime",
value: true
}
}
},
{
attrs: {
$elemMatch: {
key: "fuel"
}
}
},
{
attrs: {
$elemMatch: {
key: "volume",
"value": 3
}
}
}
]
}
}
])
MongoPlayground
When I query the API api/issues/ for issues with fields="id", I get back an array of issues similiar to this:
[
{ "id": "2-120" }
]
This works for further calls because 2-120 can be used in calls to /api/issues/{id}. However, I also need to display those IDs to users, which are more comfortable with project-based IDs, like EX-10. (Also, the whole browser user-interface is structured around those project issues ids)
What I tried:
Had a look at the Issue JSON Schema docs, which do not seem to contain an additional ID
Tried to find out if they can be converted manually, which does not seem to be the case.
So, how can I convert global issue IDs, like 2-120, to project issue IDs, like EX-10?
After looking at the schema again, I simply overlooked idReadable. So, a request to api/issues/PA-102?fields=id,idReadable will give you both types of IDs.
{ "id": "2-120", "idReadable": "PA-20" }
I'm building a Zapier app for a platform that have dynamic fields. I have an API that returns the list of fields for one of my resource (for example) :
[
{ name: "First Name", key: "first_name", type: "String" },
{ name: "Civility", key: "civility", type: "Multiple" }
]
I build my action's inputFields based on this API :
create: {
[...],
operation: {
inputFields: [
fetchFields()
],
[...]
},
}
The API returns type that are list of values (i.e : Civility), but to get these values I have to make another API call.
For now, what I have done is in my fetchFields function, each time I encounter a type: "Multiple", I do another API call to get the possible values and set it as choices in my input field. However this is expensive and the page on Zapier takes too much time to display the fields.
I tried to use the z.dehydrate feature provided by Zapier but it doesn't work for input choices.
I can't use a dynamic dropdown here as I can't pass the key of the field possible value I'm looking for. For example, to get back the possible values for Civility, I'll need to pass the civility key to my API.
What are the options in this case?
David here, from the Zapier Platform team.
Thanks for writing in! I think what you're doing is possible, but I'm also not 100% that I understand what you're asking.
You can have multiple API calls in the function (which it sounds like you are). In the end, the function should return an array of Field objects (as descried here).
The key thing you might not be aware of is that subsequent steps have access to a partially-filled bundle.inputData, so you can have a first function that gets field options and allows a user to select something, then a second function that runs and pulls in fields based on that choice.
Otherwise, I think a function that does 2 api calls (one to fetch the field types and one to turn them into Zapier field objects) is the best bet.
If this didn't answer your question, feel free to email partners#zapier.com or join the slack org (linked at the bottom of the readme) and we'll try to solve it there.
Ok, this is probably a stupid question but I have been reading and trying different queries and for some reason I cannot get this to work without throwing an error. This is my first time working with MongoDB and it is in an RoR project. We set up charities to have a twitter handle field, but it was not put into the model originally. So we populated the DB with charities, but now none of them have the twitter handle field. I added it to the model so now all others created will have it.
My issue is when I try to update the charities already in my DB I keep getting an error pointing at $set:
namespace :add_tw_handles_fields_2013_6_13 do
desc "add_tw_handle"
task :add_tw_handle => :environment do |t, args|
# db.charity.update( { featured: false }, { $set: { tw_handle : "test"}}, false, true)
# got your 6
Charity.update({ },
{
$set: { "tw_handle": "test"}
},
{ multi: true }
})
end
end
I tried the 2 synax calls above, I was reading in these 2 docs http://docs.mongodb.org/manual/reference/method/db.collection.update/ http://docs.mongodb.org/manual/core/update/#Updating-The%24positionaloperator.
I always get this error tho:
add_tw_handles_fields_2013_6_13.rake:16: syntax error, unexpected ':', expecting tASSOC
$set: {
As far as I can tell that is the correct syntax. I am running this in the script so I don't think I need the db. before my Model name (as shown in the uncommented update) right? I am new to this, but I literally copied and pasted the example and filled out my info, and nothing. I then tried adding a query, but there is never an error until it gets to $set: and I have no idea why. It is exactly as is shown in the Mongo docs linked above.
Any insight into what my issue would be greatly appreciated.
Thanks,
Alan
The error you're getting is from Ruby, not MongoDB, because you're trying to use MongoDB's JSON syntax inside of Ruby, which Ruby does not like. :) Your update query looks fine but you need to translate it to Ruby syntax which is a bit different.
coll.update( { }, { "$set" => { "tw_handle" => "test" } } );
will work assuming coll is your Collection object.
See here for a good tutorial (written by the MongoDB Ruby team) on using the Ruby driver.
Ok so after looking around and talking to the other dev on this project this is what I have found:
Inside a rails script you want to access the objects through rails not the mongoDB directly:
Following example is for running a script from within lib/tasks
Uses Rails activerecord to update the entry through the framework
m = ModelName.find("51b610972f52760fcc003331")
m.update_attributes( :attribute_name => "what you want to assign" )
m.save
Finds the object that has the id given from your model in rails (accesses mongo db directly)
object = ModelName.find({
"$in" => {
"_id" => "51b610972f52760fcc003331"
}
})
object.first.update_attributes(:attribute_name => "what you want to assign")
From within the console
Within the rails console you can use the first segments syntax using the activerecord models to access the objects you are querying for. But if you want to go directly into the mongo console the syntax is slightly different then above.
after going to the root directory of your project launch mongo in the console and find an object from one of your collections based on its id:
>mongo
>show dbs
>use dbsname
>show collections
>db.collection_name.find({ _id: { $in: [ ObjectId("51b610972f52760fcc003331") ] }})
Hopefully this is helpful to others just learning, syntaxs are different in each. I was told that mongo console runs with javascript, where the rails console (and within the project) is using the activerecord to run the call then manipulate the MongoDB. The commenter above was only addressing accessing the mongoDB directly from within a script. So hopefully this is a little more complete in the different ways you could run into this issue.