I need to replicate from one read/write Pimcore instance to another Pimcore instance read-only.
Is there a smarter way to achieve that than
- replicating the database (master slave model)
- "rsync"ing the media and class file system folders?
There are APIs to export and import contents and structure, but is this reliable with high volumes ?
Thanks
Possibility 1:
Never tried this, but you could post the object / asset / ... to an controller on the other instance. To do that, you need to extend the save-function of all objects or hook into the save event.
On the "slave" instance, you can then check if an object / asset... with this path and key exists, then update, otherwise create a new.
Possibility 2:
Do it with polling: Instance A tells B about it's objects, ... via a controller. B then iterates and clones it.
Related
Prerequisites:
I have 2 methods in the network API:
return list of entities (just basic info: name, id, etc.)
return detailed info about entity
The requirement is to save only objects created by processing the second request (save to sqlite) and leave non-full objects without saving.
Also, the 'main' context should contain only full objects from 1st request, and any other 'temporary' context should contain all the others objects.
I've tried to create two instances of NSPersistentStoreCoordinator and use them for different types of contexts, but it seems that for one NSManagedObjectModel can exist only one coordinator (the pointer points to the same adress).
If I understand you correctly, then I think your best option is to only create a managed object once you're sure you want it to persist in Core Data storage. You may need another layer of non-managed objects to contain data for the "non-full" entities
(This would be something like Data Transfer Objects from Java EE programming.)
You can not save indiscriminately from within the same MOC. Saving the MOC always saves everything in it.
So, you can use a separate MOC that is never saved, and then just "move" those objects to the main MOC when they are ready to be saved.
The solution that exactly solves my problem is to create two 'forks' of core data stack:
one with default config and mainContext
the second (new instance of the same NSManagedObjectModel, new
NSPersistentStore (inMemory) and new NSPersistenStoreCoordinator )
Is it possible to find out which properties were saved on a managed object after the save occurs? For example, I have someone listening for managed object context saves, (NSManagedObjectContextDidSaveNotification) and I want to know which properties on the objects were saved.
The NSManagedObjectContextDidSaveNotification does contain all three bits of information you would need to sync with a server. Check the [notification userInfo] and you will find three sets inside: NSInsertedObjectsKey, NSUpdatedObjectsKey, and NSDeletedObjectsKey
If you want to know what properties on an entity have changed that would require that you track them yourself using KVO. I would recommend against this as the odds of that level of effort being worth it over just pushing the entire object up to a server are slim.
Update #2
On further poking around:
From the NSManagedObjectContextWillSaveNotification you could loop through each set and reference the changedValues method. You could keep a reference to that dictionary until after you receive the NSManagedObjectContextDidSaveNotification and then process the changes. Still sounds very heavy to me.
Update
What is your end goal?!?!
If you are trying to figure out what to push to a server then being at the attribute level is too low. You should be syncing at the entity level.
If you are just trying to keep some internal consistency inside of your application then you are thinking way, way too low level. This is a solved problem. Core Data solved it.
Why don't you get them when they are about to be saved. Subscribe to NSManagedObjectContextWillSaveNotification and check insertedObjects, updatedObjects and deletedObjects of the NSManagedObjectContext.
Update:
Even easier, get the user info of the NSManagedObjectContextDidSaveNotification
From the documentation:
Typically, on thread A you register for the managed object context
save notification, NSManagedObjectContextDidSaveNotification. When you
receive the notification, its user info dictionary contains arrays
with the managed objects that were inserted, deleted, and updated on
thread B.
http://developer.apple.com/library/ios/#documentation/cocoa/conceptual/coredata/Articles/cdConcurrency.html#//apple_ref/doc/uid/TP40003385-SW1
Here's the solution I settled with. I have one singleton class that is notified when a context saves. The NSManagedObjectContextWillSave notification tells me which things have changed so I store them in a dictionary with the key being the context that saved. Then when I get the NSManagedObjectContextDidSave notification, I check the dictionary for the associated context. Finally, I remove that entry from the dictionary. Does that seem reasonable?
Two views in my application need to load same information when restoring state. My idea was, to avoid saving it twice, to have one view create another in init orcreatePartControl if it wasn't created yet. However,
PlatformUI.getWorkbench().getActiveWorkbenchWindow().getActivePage().showView(...)
doesn't work there, as getActivePage() returns null. Is it possible to work around this?
Delegate to a manager or service to load/maintain/save the shared state. That will ensure the first access initializes your information. When the view is instantiated just go to the manager and retrieve the information. If the user never instantiates your view, then you never had to do the extra work.
In the general case, you can't create/instantiate one view while creating/activating another view. Eclipse won't allow it, and will generate ERRORs in the error log.
EDIT:
3 standard persistence patterns I've seen used (and/or misused :-) are:
1) Have your plugin get its state location and simply serialize you state out there. (location provided for free if you subclass org.eclipse.core.runtime.Plugin) You can do it in your activator stop(BundleContext) method. You can uses classes like org.eclipse.ui.XMLMemento to serialize to/from XML if you don't already have a solution.
2) if you subclass org.eclipse.ui.plugin.AbstractUIPlugin you can use org.eclipse.ui.plugin.AbstractUIPlugin.getDialogSettings() to store your state. Potentially a little bulky as you would have to keep it up to date.
3) have your common manager update a preference, potentially using another serialization technique.
I was wondering if there is a way to share an NSManagedObject between two or more NSManagedObjectContext objects running in the same thread.
I have the following problem: I have one main context shared through all my code in the application and several different contexts that are created for each remote fetch request that I issue. (I created a custom class that fetches remotely and inserts all the objects found in the server in his own NSManagedObjectContext). Those fetch requests may run simultaneously since they use NSURLConnection objects that may end at different times. If the same remote object gets fetched by different connections, I will end up with duplicates at the moment of saving and merging the context with the main one. (That is, objects that have the same remote ID but a different objectID).
One possible solution would be to save (and so persist) every object as soon as it is created but I can't do that because it may have some relationships that may still have not been filled and won't validate during the save operation.
I'm really looking forward to a method that allows you to share the same non-persistent instance of an object between context. If anybody has encountered this issue and came up with a solution, I would be pleased to know!
Context cannot communicate between each other save through their stores. However, you can insert a managed object with a nil managed object context and it will be independent (albeit without relationships) of any context. You could pass that independent managed object around however you wished and insert it into a context when you needed to persist it. This is dangerous but possible.
However, if you're not running each connection on a separate thread then you don't gain anything by having multiple context. Each connection object will activate its delegate in sequence on the main thread. In this case, your easiest solution would be to use the same delegate for all the connections and let the delegate handle the insertions into a single context. To prevent duplication, just do a fetch on the remoteID and see if you get back an extant object before inserting a new object for that remoteID.
I don't think what you want to do is possible. I mean if you want to share changes between different contexts, you got to use notifications and merge it whenever did save or did change occur. But in your case, I'd say just use 1 context and save in the end. Or a less elegant way: save all the remote ids temporary in your app and check before inserting new ones. In this case, you can continue use multiple contexts and save after each didfinishloading.
Let me begin with an illustrative example (assume the implementation is in a statically typed language such as Java or C#).
Assume that you are building a content management system (CMS) or something similar. The data is hierarchically organised into Folders. Each folder has a collection of children; a child may be a Page or a Folder. All items are stored within a root folder. No cycles are allowed. We have an acyclic graph.
The system will have a remote API and instances of Folder and Page must be serialized / de-serialized across the network. With a typical implementation of folder, in which a folder's children are a List, serialization of the root node would send the entire graph. This is unacceptable for obvious reasons.
I am interested to hear people have solved this problem in the past.
I have two potential suggestions:
Navigation by query: Change the domain model so that the folder class contains only a list of IDs for each child. To access a child we must query for it. Serialisation is now trivial since the graph ends at a well defined point. The major downside is that we lose type safety - the ID could be for something other than a folder/child.
Stop and re-attach: During serialization stop whenever we detect a reference to a folder or page, send the ID instead. When de-serializing we must then look up the corresponding object for each ID and re-attach it at the relevant position in the nascent object.
I don't know what kind of API you are trying to build, but your suggestion #1 sounds like it is close to what is recommended for REST style services and APIs. Basically, a Folder object would contain a list of URLs to its children.
The Navigation by query solution was used for NFS. By reading through your question, it looks to me, as if you're trying to implements kind of a file system yourself.
If you're looking specifically into sending objects over the network there is always CORBA. Aside from that there is DCOM and the newer WCF. But wait there is more like RMI. Furthermore there are Web Services. I'll stop here now.
Suppose You model the whole tree with every element being a Node, specialisations of Node being Folder and, umm, Leaf. You have a "root" Node. Nodes have a methods
canHaveChildren()
getChildren()
Leaf nodes have the obvious behaviours (never even need to hit the network)
Folders getChildren() get the next set of nodes.
I did devise a system with Restful services along these lines. Seemed to be reasonably easy to program to.
I would not do it by the Navigation by query method. Simply because I would like to stick with the domain model where folders contains folders or pages.
Customizing the serialization might also be tricky, bug prone and difficult to change\understand.
I would suggest that you introduce and object like FolderBowser in your model which takes an id and gives you a list of contents of the folder. That will make your service operations simpler.
Cheers,
Unmesh
The classical solution is probably to use a proxy pattern, where some of the graph is sent over the network and some of the folders are replaced by proxies that will not have their lists of children populated until they are queried. A round trip to the server takes a significant amount of time and it will probably result in too many requests if all folders are proxies (this would yield a new request each time the contents of a folder is inspected), so you want to go for some trade off between the size of each chunk of data and the number of server requests needed in a typical scenario. This is of course application specific, but sending the contents of all child folders in for instance depth 2 might be a useful strategy...
Long story short: What will probably work best is your solution #1 with the exception that you want to send more than one folder at a time because of the overhead of a round trip to the server...