How to get Mongoid working with existing MongoDB collcetions? - ruby-on-rails-3

I'm working on a quite simple ruby on rails website using MongoDB as the database. My idea was to get this website working with Mongoid such that I can display certain contents from the already existing mongodb collection. I've checked the Internet for tutorials about how to use Mongoid, the problem is all of them are about how to create your Mongodb with rails rather than using an existing one. Could anyone tell how to do what I want? Thanks a lot.

Until you define models for your collections,
you will have to drop down to the Moped driver level to examine existing collections.
Here's a link to the Moped documentation - http://mongoid.org/en/moped/docs/driver.html
And here are some hints to get you started.
console-input.rb
session = Mongoid.default_session
collection = session['test_collection']
collection.insert({name: 'George Washington'})
session.collection_names
collection.find.first
$ rails c < console-input.rb
Loading development environment (Rails 3.2.14)
Switch to inspect mode.
session = Mongoid.default_session
<Moped::Session seeds=["localhost:27017"] database=sandbox_mongoid3_development>
collection = session['test_collection']
#<Moped::Collection:0x007ff2ceb73c20 #database=#<Moped::Database:0x007ff2ceb68b40 #session=<Moped::Session seeds=["localhost:27017"] database=sandbox_mongoid3_development>, #name="sandbox_mongoid3_development">, #name="test_collection">
collection.insert({name: 'George Washington'})
nil
session.collection_names
["test_collection"]
collection.find.first
{"_id"=>"528bdc60e277b0dd1681771a", "name"=>"George Washington"}

Related

How do I launch a SmartSim orchestrator without models?

I'm trying to prototype using the SmartRedis Python client to interact with the SmartSim Orchestrator. Is it possible to launch the orchestrator without any other models in the experiment? If so, what would be the best way to do so?
It is entirely possible to do that. A SmartSim Experiment can contain different types of 'entities' including Models, Ensembles (i.e. groups of Models), and Orchestrator (i.e. the Redis-backed database). None of these entities, however, are 'required' to be in the Experiment.
Here's a short script that creates an experiment which includes only a database.
from SmartSim import Experiment
NUM_DB_NODES = 3
exp = Experiment("Database Only")
db = exp.create_database(db_nodes=NUM_DB_NODES)
exp.generate(db)
exp.start(db)
After this, the Orchestrator (with the number of shards specified by NUM_DB_NODES) will have been spunup. You can then connect the Python client using the following line:
client = smartredis.Client(db.get_address()[0],NUM_DB_NODES>1)

Flask-migrate change db before upgrade

I have a multi-tenancy structure set up where each client has a schema set up for them. The structure mirrors the "parent" schema, so any migration that happens needs to happen for each schema identically.
I am using Flask-Script with Flask-Migrate to handle migrations.
What I tried so far is iterating over my schema names, building a URI for them, scoping a new db.session with the engine generated from the URI, and finally running the upgrade function from flask_migrate.
#manager.command
def upgrade_all_clients():
clients = clients_model.query.all()
for c in clients:
application.extensions["migrate"].migrate.db.session.close_all()
application.extensions["migrate"].migrate.db.session = db.create_scoped_session(
options={
"bind": create_engine(generateURIForSchema(c.subdomain)),
"binds": {},
}
)
upgrade()
return
I am not entirely sure why this doesn't work, but the result is that it only runs the migration for the db that was set up when the application starts.
My theory is that I am not changing the session that was originally set up when the manager script runs.
Is there a better way to migrate each of these schemas without setting multiple binds and using the --multidb parameter? I don't think I can use SQLALCHEMY_BINDS in the config since these schemas need to be able to be dynamically created/destroyed.
For those who are encountering the same issue, the answer to my specific situation was incredibly simple.
#manager.command
def upgrade_all_clients():
clients = clients_model.query.all()
for c in clients:
print("Upgrading client '{}'...".format(c.subdomain))
db.engine.url.database = c.subdomain
_upgrade()
return
The database attribute of the db.engine.url is what targets the schema. I don't know if this is the best way to solve this, but it does work and I can migrate each schema individually.

ClientCacheConfiguration is not saved to table

Was using CacheConfiguration in Ignite until I stuck with issue on how to authenticate.
Because of that I was starting to change the CacheConfiguration to clientCacheConfiguration. However after converting it to CacheConfiguration I started to notice that it
does not able to save into table because it lack of method setIndexedTypes eg.
Before
CacheConfiguration<String, IgniteParRate> cacheCfg = new CacheConfiguration<>();
cacheCfg.setName(APIConstants.CACHE_PARRATES);
cacheCfg.setIndexedTypes(String.class, IgniteParRate.class);
New
ClientCacheConfiguration cacheCfg = new ClientCacheConfiguration();
cacheCfg.setName(APIConstants.CACHE_PARRATES);
//cacheCfg.setIndexedTypes(String.class, IgniteParRate.class); --> this is not provided
I still need the table to be populated so it easier for us to verify ( using Client IDE like DBeaver)
Any way to solve this issue?
If you need to create tables/cache dynamically using the thin-client, you'll need to use the setQueryEntities() method to define the columns available to SQL "manually". (Passing in the classes with annotations is basically a shortcut for defining the query entities.) I'm not sure why setIndexedTypes() isn't available in the thin-client; maybe a question for the developer mailing list.
Alternatively, you can define your caches/tables in advance using a thick client. They'll still be available when using the thin-client.
To add to existing answer, you can also try to use cache templates for that.
https://apacheignite.readme.io/docs/cache-template
Pre-configure templates, use them when creating caches from thin client.

Ruby mongodb: Three newly created objects doesn't appear to exist during testing

I'm using mongodb to store some data. Then I have a function that gets the object with the latest timestamp and one with the oldest. I haven't experienced any issues during development or production with this method but when I try to implement a test for it the test fails approx 20% of the times. I'm using rspec to test this method and I'm not using mongoid or mongomapper. I create three objects with different timestamps but get a nil response since my dataset contains 0 objects. I have read a lot of articles about write_concern and that it might be the problem with "unsafe writes" but I have tried almost all the different combinations with these parameters (w, fsync, j, wtimeout) without any success. Does anyone have any idea how to solve this issue? Perhaps I have focused too much with the write_concern track and that the problems lies somewhere else.
This is the method that fetches the latest and oldest timestamp.
def first_and_last_timestamp(customer_id, system_id)
last = collection(customer_id).
find({sid:system_id}).
sort(["t",Mongo::DESCENDING]).
limit(1).next()
first = collection(customer_id).
find({sid:system_id}).
sort(["t",Mongo::ASCENDING]).
limit(1).next()
{ min: first["t"], max: last["t"] }
end
Im inserting data using this method where data is a json object.
def insert(customer_id, data)
collection(customer_id).insert(data)
end
I have reverted back to use the default for setting up my connection
Mongo::MongoClient.new(mongo_host, mongo_port)
I'm using the gem mongo (1.10.2). I'm not using any fancy setup for my mongo database. I've just installed mongo using brew on my mac and started it. The version of my mongo database is v2.6.1.

Can't find mapping model for migration - UIManagedDocument Core Data Migration

I have two versions of my model Model001.xcdatamodel and Model002.xcdatamodel. These two are in the Model.xcdatamodeld bundle.
I also have a Model001to002.xcmappingmodel which is not part of the Model.xcdatamodeld. I checked: both the xcmappingmodel and the xcdatamodeld get copied into the .app bundle.
My managed object context is initialized like this:
NSURL *documentModel = [bundle URLForResource:#"Model"
withExtension:#"momd"]; managedObjectModel = [[NSManagedObjectModel alloc]
initWithContentsOfURL:documentModel]; return managedObjectModel;
I also set these properties on my overridden initWithFileURL: in my UIManagedObject subclass.
NSMutableDictionary *options = [NSMutableDictionary dictionaryWithDictionary:self.persistentStoreOptions];
[options setObject:#YES forKey:NSMigratePersistentStoresAutomaticallyOption];
[options setObject:#YES forKey:NSInferMappingModelAutomaticallyOption];
self.persistentStoreOptions = [options copy];
But when I try to open a documet, I get the following error:
Can't find mapping model for migration
-- UPDATE --
Even if I do a manual migration
[NSMappingModel mappingModelFromBundles:#[[NSBundle mainBundle]]
forSourceModel:sourceObjectModel
destinationModel:self.managedObjectModel];
this returns nil. Although I double checked that the Model001to002.cdm is in the app bundle. It has to be in the app bundle right?
A "gotcha" with mapping models is that you are not allowed to make any changes to the models after you created the mapping. If you do, you will also get this error.
OK, solved the problem by removing all core data files from Xcode, reading them and setting the source and destination of the mapping model again.
Damn you Xcode!
You are not allowed to make any changes to the source/destination model after you have created the mapping models.
If you do make some changes,
mappingModelFromBundles:forSourceModel:destinationModel: will not be able to find the mapping model file
addPersistentStoreWithType:configuration:URL:options:error: with {NSInferMappingModelAutomaticallyOption: #NO} will report an error "Can't find mapping model for migration"
migrateStoreFromURL:type:options:withMappingModel:toDestinationURL:destinationType:destinationOptions:error: will report an error "Mismatch between mapping and source/destination models"
So, just recreate the mapping model and copy every change you made in the old one.
TL;DR
At least as of Xcode 8/9, open the mapping model then from the Editor menu select Refresh data models. Usually it seems you need to restart Xcode. If that doesn't do it you might try re-selecting the destination at the bottom of the model editor.
More Tips
Definitely NEVER change a model after it has been distributed in an app build.
For this example, let's say you have published Data Model 1 (DM1) and are making a migration to DM2. If you set DM2 as the active version then run your app, a migration will run on your persistent store. If you then make another change to DM2, run your app... Boom!
The issue is that your store has already been migrated to "DM2" but the data in the store doesn't fit into the model anymore. And, we can't migrate from DM2 to DM2 again.
It may seem like an obvious solution to go ahead and create DM3. It is
usually a good idea though to minimize the number of models and
migrations while you are developing.
So... now you have a persistent store that has been migrated to a defunct DM2. How do you test the migration again? You could revert your app and generate some data with DM1 but I prefer to use backups
Creating a backup
Before you run your app with DM2 you can copy the existing store (with DM1) to use for later test migrations. On macOS you can easily do this manually. The code below should do the trick as well. Typically you wouldn't want to ship this, rather you could just put it somewhere before your normal CD stack opens, run the app, then stop the app (maybe place a breakpoint just after then end the run via Xcode).
let fm = FileManager.default
let url = // The store URL you would use in ↓
// try coordinator.addPersistentStore(ofType: NSSQLiteStoreType, configurationName: nil, at: url, options: nil)
let dir = url.deleteLastPathComponent().appendingPathComponent("Backup", isDirectory: true).appendingPathComponent("DM1", isDirectory: true)
print("Saving DB backup for DM1")
if !fm.fileExists(atPath: dir.path) {
do {
// Create a directory
try fm.createDirectory(at: dir, withIntermediateDirectories: true, attributes: nil)
let backupURL = dir.appendingPathComponent(url.lastPathComponent)
try fm.copyItem(at: url, to: backupURL)
}
catch {
print("Failed to save DB backup")
}
}
Oops, I need to make another change...
If you run your migration to DM2 then realize you need to make another change, you'll want to re-test your migration from DM1 -> DM2. This is where the backup comes in.
Same way you made the backup, run this code.
let fm = FileManager.default
let url = // The store URL you would use to add the store
let dir = url.deleteLastPathComponent().appendingPathComponent("Backup", isDirectory: true).appendingPathComponent("DM1", isDirectory: true)
let backupURL = dir.appendingPathComponent(url.lastPathComponent)
if fm.fileExists(atPath: backupURL.path) {
do {
fm.removeItem(at: url.path)
try fm.copyItem(at: backupURL, to: url)
}
catch {
print("Failed to restore DB backup")
}
}
You now have a restored DM1 store and have made changes to DM2. If you run the app the migration might succeed but it won't use your custom mapping model.
Remember if you are using a custom mapping, you will still need to use the Refresh Data Models technique before the mapping model will work.
This can happen if your test device's store is from a version of the data model that no longer exists.
For example I had Data Model Version 7, then I made Data Model Version 8. I made a mapping model to go from 7 to 8. Then I ran it on my test device and everything was happy.
Then I made some more changes to 8.
The thing to realize is that in Core Data, every model has a hash identifier that the system creates by taking a checksum of the xcdatamodel file. So if you make even a slight change, even if you didn't create a new version, it sees it as a different version. These versions' identifiers are NSStoreModelVersionHashes (see documentation here).
So in other words, I ended up with:
Data Model 7 (release) - 0plcXXRN7XHKl5CcF+fwriFmUpON3ZtcI/AfK748aWc=
Data Model 8 (beta) - qeN1Ym3TkWN1G6dU9RfX6Kd2ccEvcDVWHpd3LpLgboI=
Data Model 8 (release) - EqtMzvRnVZWkXwBHu4VeVGy8UyoOe+bi67KC79kphlQ=
Instead of making a version 9, and saving the original version 8 in the data model history, I just updated 8, figuring automatic migration could take care of me. Well, it couldn't, and I couldn't make a mapping between the two, because the old (beta) version of 8 was gone.
I did it that way because it was an intermediary internal build (not a release) so it wasn't a big deal, but it did throw me for a loop!
If it wasn't an internal build and I needed to make this work, I could go back to the (beta) commit and pull out that xcdatamodel file for 8 (beta), rename the (release) version to 9, then stick it into the release build and make a mapping model between 8 and 9.
However since it was just an internal beta build, we just erased and reinstalled the app on test devices. We did verify that, when going from 7 (release) to 8 (release), the migration went smoothly.
Removing Coredata files from its path an re - run project is worked for me