Is dulwich thread safe? - dulwich

In other words, if I have a MemoryRepo instance, can I be performing a:
"fetch(), updating references and performing a send_pack()" on one thread and
traversing the commit graph or even introducing a change into the object_store and local references in another thread (using the same MemoryRepo instance)?

Yes, MemoryRepo is meant to be thread-safe.

Related

Core Data insert objects in NSOperation and sync

I'm saving objects from network response in NSOperation. As I understand for merge MOC's changes from background threads to main I can use mergeChangesFromContextDidSaveNotification or performBlock with parent context.
What should I prefer to use mergeChangesFromContextDidSaveNotification or performBlock?
What are the pros and cons of each merge method?
NSOperation executed in background context. So when I call performBlock will be created new thread or not?
The answer depends very much on whether your background MOC has the main MOC as a parent.
You only need to use performBlock if you are using MOCs with private/main queue concurrency.
And if you are, to get commands executed in the corresponding correct queue, you'd use performBlock.
So even if you have to use mergeChanges-- and you don't have to with child contexts-- you would nest mergeChanges in a performBlock! But the reason you don't have to with a child context is because your "background thread" MOC should could be a child of the main thread MOC. So all you have to do is save the child to merge changes into the parent. (Note that the parent also needs to be saved)
So to get back to the question:
mergeChanges if you're not pushing changes from child to parent MOC with a save in the child
use performBlock everywhere but on operations on the mainQueueMOC on the main thread if you have using multiple MOCs with different concurrency types
I'm not sure if two methods you mentioned provide equivalent functionality: you use performBlock to perform random code on the tread of the receiver managed object context. And you use mergeChangesFromContextDidSaveNotification to process NSNotification from another context save. If I got it right up to here, your only chose is the latter.
Potentially you can create a child context within NSOperation, saved data from child context propagates into parent context automatically.
To elaborate further, you would use performBlock to modify parent's managed objects from the background - it basically dispatches you to the thread of the managed object context to allow perform operations in a safe manner
Searching answer on my question I found Concurrent Core Data Stacks Performance article. This article answer that if you user old core data stack with sync data using mergeChangesFromContextDidSaveNotification method you get better performance.
Also I found apple example ThreadedCoreData. In this example they still using sync with mergeChangesFromContextDidSaveNotification.
So answer on my question is if you carry on performance you should use mergeChangesFromContextDidSaveNotification if you what simpler realization you should use nested contexts.

Should my block based methods return on the main thread or not when creating an iOS cloud integration framework?

I am in the middle of creating a cloud integration framework for iOS. We allow you to save, query, count and remove with synchronous and asynchronous with selector/callback and block implementations. What is the correct practice? Running the completion blocks on the main thread or a background thread?
For simple cases, I just parameterize it and do all the work i can on secondary threads:
By default, callbacks will be made on any thread (where it is most efficient and direct - typically once the operation has completed). This is the default because messaging via main can be quite costly.
The client may optionally specify that the message must be made on the main thread. This way, it requires one line or argument. If safety is more important than efficiency, then you may want to invert the default value.
You could also attempt to batch and coalesce some messages, or simply use a timer on the main run loop to vend.
Consider both joined and detached models for some of your work.
If you can reduce the task to a result (remove the capability for incremental updates, if not needed), then you can simply run the task, do the work, and provide the result (or error) when complete.
Apple's NSURLConnection class calls back to its delegate methods on the thread from which it was initiated, while doing its work on a background thread. That seems like a sensible procedure. It's likely that a user of your framework will not enjoy having to worry about thread safety when writing a simple callback block, as they would if you created a new thread to run it on.
The two sides of the coin: If the callback touches the GUI, it has to be run on the main thread. On the other hand, if it doesn't, and is going to do a lot of work, running it on the main thread will block the GUI, causing frustration for the end user.
It's probably best to put the callback on a known, documented thread, and let the app programmer make the determination of the effect on the GUI.

Making pthread_rwlock_wrlock recursive

I have a problem regarding the behaviour of the pthread function pthread_rwlock_wrlock. The specification linked above states that when one thread has locked the lock for writing and the same thread locks it again, it results in undefined behaviour (I could actually observe this in that on x86 Linux calling this function is a noop and on PowerPC Linux it stalls the thread).
The behaviour I need would be a read write lock that has the following characteristics:
read-locking by a thread succeeds if:
the lock is not held by any thread
the lock is only read-locked by zero or more threads (including the calling thread) and possibly read- or write locked by the calling thread
write-locking succeeds when:
the lock is not held by any other thread
only the current thread is holding the lock (for reading or writing)
With a pthread_mutex_t, the recursiveness of the lock can be controlled via an initialization flag, but this is not possible for pthread_rwlock_t.
What are my options? I've never actually had to implement this kind of concurrency primitive in C, and I think I'm missing some obvious solution here.
To be honest, recursive locking does have some uses but generally it's a hack. I can't seem to find the article right now, but Butenhof has a nice rant on this.
Back to the question. You could keep a thread-specific flag that signals: "I have the lock". Set it right after locking and unset it before unlocking. Since this is the only thread accessing it, you should be safe. So when trying to lock you simply need to check: "Hey, is this thing locked already?".
As a side note: are you sure the design is okay if a thread tries to lock twice ?
EDIT
Found the article.
But if that's all that's necessary, why does POSIX have recursive
mutexes?
Because of a dare.

Identify a GCD thread

I have written a Core Data abstraction class which holds the persistent store, object model and object context. To make the multithreading easier, I have written the accessor for the object context so that it returns a instance that is only available for the current thread by using [NSThread currentThread] to identify the threads.
This works perfectly as long as I don't use GCD, which I want to use as replacement for the old NSThread's. So my question is, how do I identify a GCD thread? The question applies for both iOS and Mac OS X but I guess that its the same for both platforms.
You could check whether dispatch_get_current_queue() returns anything. I like Jeremy's idea of transitioning to a CD-context-per-queue instead of CD-context-per-thread model using the queue's context storage though.
Perhaps you can store the CD context for each thread in the GCD context using dispatch_set_context()
The contextForCurrentThread helper method in Magical Record is very similar to what to said (i.e. keep one context per thread). The GCD execution block, while running on a single queue, can potentially run on any thread managed by GCD, which will cause some random crashes. Check this article: http://saulmora.com/2013/09/15/why-contextforcurrentthread-doesn-t-work-in-magicalrecord/

Undoing Core Data insertions that are performed off the main thread

I'm working on some code that uses an NSOperation to import data. I'd like for the user to be able to undo the NSManagedObject instances that are created during the import operation.
From what I can tell, it's impossible to use the NSManagedObjectContext -undoManager for any operations that are performed off of the main thread. From the Core Data Programming Guide section on Use Thread Confinement to Support Concurrency, we have these two conditions:
Only objectID should be passed
between managed object contexts (on
separate threads)
Managed objects
must be saved in a context before
the objectID can be used.
This makes sense since the managed objects need to be moved from private storage (NSManagedObjectContext) to public storage (NSPersistentStore) before they can be shared.
Unfortunately, the -save: message also causes any managed objects in the undo stack to be removed. From the Memory Management Using Core Data section of the same guide:
Managed objects that have pending
changes (insertions, deletions, or
updates) are retained by their context
until their context is sent a save:,
reset , rollback, or dealloc message,
or the appropriate number of undos to
undo the change.
I've tried several things to work around this limitation, and everything eventually leads back to bulk of the work happening on the main thread (and spinning beach balls.) Any clues to getting undo working with objects created off the main thread would be very much appreciated.
--
An enhancement Radar has been submitted: rdar://problem/8977725
This answer will probably be a bit of a back and forth. If I understand the issue correctly, you are doing an import but when the import is done you want the user to be able to select what gets saved from the import?
If that is not correct, please fix my assumptions and I will update this answer.
If it is correct then what you can do is:
Change your background object creation to
NSEntityDescription *myEntity = ... //Entity from your context
[[NSManagedObject alloc] initWithEntity:myEntity
insertIntoManagedObjectContext:nil];
Store these entities in an array.
Pass the entities back to your main thread as needed.
Release on any objects you don't want to keep
Call [myMainContext insertObject:managedObject] on any you want to keep.
Perform a save on the NSManagedObjectContext.
Since these entities are not part of a NSManagedObjectContext yet they only exist in memory and should be thread safe since they are not yet tied down to a NSManagedObjectContext.
This is of course theoretical and will require testing. However it should accomplish your goal.
Not an expert, but I think what you're going to need to do is create a second context to perform the operations, then merge the two contexts together. You should be able to manage the merge as an undo step. Note, this only works if you're treating the entire set of operations as one undo step, as far as the user is concerned.
Suppose that you use a separate context for the background thread, and once it's done, push a [[backgroundContext undoManager] undo] onto the foreground thread's undo stack? I've never tried anything like that, but off the top of my head I can't think of a reason it shouldn't work.
One option may be to make your import thread persistent. Even when the thread is finished importing, it goes into an idle loop state. This way your threaded ManagedObjectContext is persisted in the proper thread. Then when the user wishes to undo a change, send a message to the thread to use the undomanager.
It's incredibly likely you've considered this and you're likely only looking for a solution using the existing undoManager, but just in case:
Since you're inserting objects and not updating existing ones, you do have the power to tag them with a transaction id as each batch is imported, deleting them in a background thread in the case of an undo. A simple incremented NSNumber is sufficient for the tag.
Inelegant, but workable.