I use it like this in the code.
Project.refreshLocal(IResource.DEPTH_ONE, null);
Project.delete(false, true, new NullProgressMonitor());
Then I got the thread information and found the following problems.
Many threads are in the WAITING(on object monitor) state.
"pool-2-thread-2" #69 prio=5 os_prio=0 tid=0x000000001fa76000 nid=0x1b54 in Object.wait() [0x0000000032f0e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000cfac5630> (a java.lang.Object)
at java.lang.Object.wait(Unknown Source)
at org.eclipse.core.internal.jobs.ThreadJob.waitForRun(ThreadJob.java:316)
- locked <0x00000000cfac5630> (a java.lang.Object)
at org.eclipse.core.internal.jobs.ThreadJob.joinRun(ThreadJob.java:205)
at org.eclipse.core.internal.jobs.ImplicitJobs.begin(ImplicitJobs.java:95)
at org.eclipse.core.internal.jobs.JobManager.beginRule(JobManager.java:297)
at org.eclipse.core.internal.resources.WorkManager.checkIn(WorkManager.java:124)
at org.eclipse.core.internal.resources.Workspace.prepareOperation(Workspace.java:2243)
at org.eclipse.core.internal.resources.Resource.refreshLocal(Resource.java:1542)
at cn.com.agree.studio.server.utils.ResourceRefreshLocal.refreshLocal(ResourceRefreshLocal.java:25)
- locked <0x00000000c00c5e50> (a org.eclipse.core.internal.resources.Project)
at cn.com.agree.studio.server.service.search.GetJavaMetadataService.transferClassMetaInfo(GetJavaMetadataService.java:175)
at cn.com.agree.studio.server.service.search.GetJavaMetadataService.lambda$0(GetJavaMetadataService.java:153)
- locked <0x00000000c31d7958> (a java.lang.Class for cn.com.agree.studio.server.service.search.GetJavaMetadataService)
at cn.com.agree.studio.server.service.search.GetJavaMetadataService$$Lambda$326/115772063.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Then below is the code near the Monitor.
The class name of this class is 'org.eclipse.core.internal.jobs.ThreadJob'.
while (true) {
// monitor is foreign code so do not hold locks while calling into monitor
if (interrupted || isCanceled(monitor))
// Condition #4.
throw new OperationCanceledException();
// Try to run the job. If result is null, this job was allowed to run.
// If the result is successful, atomically release thread from waiting
// status.
blockingJob = manager.runNow(threadJob, true);
if (blockingJob == null) {
// Condition #1.
waiting = false;
return threadJob;
}
Thread blocker = blockingJob.getThread();
// the rule could have been transferred to this thread while we were waiting
if (blocker == currentThread && blockingJob instanceof ThreadJob) {
// now we are just the nested acquire case
result = (ThreadJob) blockingJob;
// push expects a compatible rule, otherwise an exception will be thrown
result.push(threadJob.getRule());
// rule was either accepted or both jobs have null rules
ruleCompatibleAndTransferred = true;
result.isBlocked = threadJob.isBlocked;
// Condition #3.
return result;
}
// just return if lock listener decided to grant immediate access
if (manager.getLockManager().aboutToWait(blocker))
// Condition #2.
return threadJob;
// Notify the lock manager that we're about to block waiting for the scheduling rule
manager.getLockManager().addLockWaitThread(currentThread, threadJob.getRule());
synchronized (blockingJob.jobStateLock) {
try {
// Wait until we are no longer definitely blocked (not running).
// The actual exit conditions are listed above at the beginning of
// this while loop
int state = blockingJob.getState();
//ensure we don't wait forever if the blocker is waiting, because it might have yielded
//to me
if (state == Job.RUNNING && canBlock)
blockingJob.jobStateLock.wait();
else if (state != Job.NONE)
blockingJob.jobStateLock.wait(250);
} catch (InterruptedException e) {
// This thread may be interrupted via two common scenarios. 1) If
// the UISynchronizer is in use and this thread is a UI thread
// and a syncExec() is performed, this thread will be interrupted
// every 1000ms. 2) If this thread is allowed to be blocked and
// the progress monitor was canceled, the internal JobManager
// worker thread will interrupt this thread so cancellation can
// be carried out.
interrupted = true;
}
}
// Going around the loop again. Ensure we're not marked as waiting for the thread
// as external code is run via the monitor (Bug 262032).
manager.getLockManager().removeLockWaitThread(currentThread, threadJob.getRule());
}
So I don’t know why the thread is waiting and how to solve the problem.
Many thanks in advance!
Related
Past few days I am learning coroutines, most of thee concepts are clear but I don't understand the implementation of the delay function.
How delay function is resuming the coroutine after the delayed time? For a simple program, there is only one main thread, and to resume the coroutine after the delayed time I assume there should be another timer thread that handles all the delayed invocations and invokes them later. Is it true? Can someone explain the implementation detail of the delay function?
TL; DR;
When using runBlocking, delay is internally wrapped and runs on same thread and when using any other dispatcher it suspends and is resumed by resuming the continuation by event-loop thread. Check the long answer below to understand the internals.
Long answer:
#Francesc answer is pointing correctly but is somewhat abstract, and still does not explains how actually delay works internally.
So, as he pointed to the delay function:
public suspend fun delay(timeMillis: Long) {
if (timeMillis <= 0) return // don't delay
return suspendCancellableCoroutine sc# { cont: CancellableContinuation<Unit> ->
cont.context.delay.scheduleResumeAfterDelay(timeMillis, cont)
}
}
What it does is "Obtains the current continuation instance inside suspend functions and suspends the currently running coroutine after running the block inside the lambda"
So this line cont.context.delay.scheduleResumeAfterDelay(timeMillis, cont) is going to be executed and then the current coroutine gets suspended i.e. frees the current thread it was stick on.
cont.context.delay points to
internal val CoroutineContext.delay: Delay get() = get(ContinuationInterceptor) as? Delay ?: DefaultDelay
that says if ContinuationInterceptor is implementation of Delay then return that otherwise use DefaultDelay which is internal actual val DefaultDelay: Delay = DefaultExecutor a DefaultExecutor which is internal actual object DefaultExecutor : EventLoopImplBase(), Runnable {...} an implementation of EventLoop and has a thread of its own to run on.
Note: ContinuationInterceptor is an implementation of Delay when coroutine is in the runBlocking block in order to make sure the delay run on same thread otherwise it is not. Check this snippet to see the results.
Now I couldn't find implemenation of Delay created by runBlocking since internal expect fun createEventLoop(): EventLoop is an expect function which is implemented from outside, not by the source. But the DefaultDelay is implemented as follows
public override fun scheduleResumeAfterDelay(timeMillis: Long, continuation: CancellableContinuation<Unit>) {
val timeNanos = delayToNanos(timeMillis)
if (timeNanos < MAX_DELAY_NS) {
val now = nanoTime()
DelayedResumeTask(now + timeNanos, continuation).also { task ->
continuation.disposeOnCancellation(task)
schedule(now, task)
}
}
}
This is how scheduleResumeAfterDelay is implemented it creates a DelayedResumeTask with the continuation passed by delay, and then calls schedule(now, task) which calls scheduleImpl(now, delayedTask) which finally calls delayedTask.scheduleTask(now, delayedQueue, this) passing the delayedQueue in the object
#Synchronized
fun scheduleTask(now: Long, delayed: DelayedTaskQueue, eventLoop: EventLoopImplBase): Int {
if (_heap === kotlinx.coroutines.DISPOSED_TASK) return SCHEDULE_DISPOSED // don't add -- was already disposed
delayed.addLastIf(this) { firstTask ->
if (eventLoop.isCompleted) return SCHEDULE_COMPLETED // non-local return from scheduleTask
/**
* We are about to add new task and we have to make sure that [DelayedTaskQueue]
* invariant is maintained. The code in this lambda is additionally executed under
* the lock of [DelayedTaskQueue] and working with [DelayedTaskQueue.timeNow] here is thread-safe.
*/
if (firstTask == null) {
/**
* When adding the first delayed task we simply update queue's [DelayedTaskQueue.timeNow] to
* the current now time even if that means "going backwards in time". This makes the structure
* self-correcting in spite of wild jumps in `nanoTime()` measurements once all delayed tasks
* are removed from the delayed queue for execution.
*/
delayed.timeNow = now
} else {
/**
* Carefully update [DelayedTaskQueue.timeNow] so that it does not sweep past first's tasks time
* and only goes forward in time. We cannot let it go backwards in time or invariant can be
* violated for tasks that were already scheduled.
*/
val firstTime = firstTask.nanoTime
// compute min(now, firstTime) using a wrap-safe check
val minTime = if (firstTime - now >= 0) now else firstTime
// update timeNow only when going forward in time
if (minTime - delayed.timeNow > 0) delayed.timeNow = minTime
}
/**
* Here [DelayedTaskQueue.timeNow] was already modified and we have to double-check that newly added
* task does not violate [DelayedTaskQueue] invariant because of that. Note also that this scheduleTask
* function can be called to reschedule from one queue to another and this might be another reason
* where new task's time might now violate invariant.
* We correct invariant violation (if any) by simply changing this task's time to now.
*/
if (nanoTime - delayed.timeNow < 0) nanoTime = delayed.timeNow
true
}
return SCHEDULE_OK
}
It finally sets the task into the DelayedTaskQueue with the current time.
// Inside DefaultExecutor
override fun run() {
ThreadLocalEventLoop.setEventLoop(this)
registerTimeLoopThread()
try {
var shutdownNanos = Long.MAX_VALUE
if (!DefaultExecutor.notifyStartup()) return
while (true) {
Thread.interrupted() // just reset interruption flag
var parkNanos = DefaultExecutor.processNextEvent() /* Notice here, it calls the processNextEvent */
if (parkNanos == Long.MAX_VALUE) {
// nothing to do, initialize shutdown timeout
if (shutdownNanos == Long.MAX_VALUE) {
val now = nanoTime()
if (shutdownNanos == Long.MAX_VALUE) shutdownNanos = now + DefaultExecutor.KEEP_ALIVE_NANOS
val tillShutdown = shutdownNanos - now
if (tillShutdown <= 0) return // shut thread down
parkNanos = parkNanos.coerceAtMost(tillShutdown)
} else
parkNanos = parkNanos.coerceAtMost(DefaultExecutor.KEEP_ALIVE_NANOS) // limit wait time anyway
}
if (parkNanos > 0) {
// check if shutdown was requested and bail out in this case
if (DefaultExecutor.isShutdownRequested) return
parkNanos(this, parkNanos)
}
}
} finally {
DefaultExecutor._thread = null // this thread is dead
DefaultExecutor.acknowledgeShutdownIfNeeded()
unregisterTimeLoopThread()
// recheck if queues are empty after _thread reference was set to null (!!!)
if (!DefaultExecutor.isEmpty) DefaultExecutor.thread // recreate thread if it is needed
}
}
// Called by run inside the run of DefaultExecutor
override fun processNextEvent(): Long {
// unconfined events take priority
if (processUnconfinedEvent()) return nextTime
// queue all delayed tasks that are due to be executed
val delayed = _delayed.value
if (delayed != null && !delayed.isEmpty) {
val now = nanoTime()
while (true) {
// make sure that moving from delayed to queue removes from delayed only after it is added to queue
// to make sure that 'isEmpty' and `nextTime` that check both of them
// do not transiently report that both delayed and queue are empty during move
delayed.removeFirstIf {
if (it.timeToExecute(now)) {
enqueueImpl(it)
} else
false
} ?: break // quit loop when nothing more to remove or enqueueImpl returns false on "isComplete"
}
}
// then process one event from queue
dequeue()?.run()
return nextTime
}
And then the event loop (run function) of internal actual object DefaultExecutor : EventLoopImplBase(), Runnable {...} finally handles the tasks by dequeuing the tasks and resuming the actual Continuation which was suspended the function by calling delay if the delay time has reached.
All suspending functions work the same way, when compiled it gets converted into a state machine with callbacks.
When you call delay what happens is that a message is posted on a queue with a certain delay, similar to Handler().postDelayed(delay) and, when the delay has lapsed, it calls back to the suspension point and resumes execution.
You can check the source code for the delay function to see how it works:
public suspend fun delay(timeMillis: Long) {
if (timeMillis <= 0) return // don't delay
return suspendCancellableCoroutine sc# { cont: CancellableContinuation<Unit> ->
cont.context.delay.scheduleResumeAfterDelay(timeMillis, cont)
}
}
So if the delay is positive, it schedules the callback in the delay time.
If each UIApplication initializes a RunLoop like this on the main thread
void CFRunLoopRun(void) { /* DOES CALLOUT */
int32_t result;
do {
result = CFRunLoopRunSpecific(CFRunLoopGetCurrent(), kCFRunLoopDefaultMode, 1.0e10, false);
CHECK_FOR_FORK();
} while (kCFRunLoopRunStopped != result && kCFRunLoopRunFinished != result);
}
Why it doesn't block the whole execution since it is an infinite loop? How could an infinite loop and my code on the main thread work together on a single thread without context switching?
It does block the current thread. However, part of what CFRunLoopRunSpecific does is call your code on that thread. When you return, it goes back to CFRunLoopRunSpecific, which then calls other code.
I have a piece of code, which is processing a queue synchronously and asynchronously. I'm using OCMock to test the thing and individually I can test both cases (synchronous and asynchronous) but when I test for both at the same time I get trouble.
To verify that the queue is processed correctly I'm passing it a mocked listener and from this listener I'm then asking if it got all the notifications propagated by the queue processor. I have two tests and in the first test (asynchronous) these expectations are met but with the second test (synchronous) I get this error:
OCMockObject[JHQueueListener] : 4 expected methods were not invoked:
startedProcessingQueue
startedToProcessQueueItem:OCMockObject[JHQueueItem]
finishedProcessingQueueItem:OCMockObject[JHQueueItem]
finishedProcessingQueue
Here's a link to the project:
https://github.com/jphollanti/queue-processor
And here's a link to the test:
https://github.com/jphollanti/queue-processor/blob/master/QueueProcessorTests/JHQueueProcessorTests.m
Issue #1: The references are fine but when it comes to the tests, threading is expected to work incorrectly. The problem here is that a new thread is started and in that new thread the status of the queue is set as in progress. But it takes longer to start up a new thread than it does for the main thread to ask for the status and this results in the fact that the queue is not (yet) in progress. Adding a delay of some 10000ms should help things a lot. Like so:
...
[queue processQueueAsynchronously:queueItems];
usleep(10000);
BOOL wentToThread = NO;
while ([queue isInProgress]) {
wentToThread = YES;
...
}
...
Also, calling dispatch_async(dispatch_queue_t queue, ^(void)block) takes a lot of time and this adds up to the "random" nature of the issues.
Issue #2: Calling dispatch_async(dispatch_get_main_queue(), ^{ ... } from the main thread causes the block to be sent to some queue which is executed at some time (don't know how it works). This is why the second test (synchronous) failed. Using something like this helps:
if ([NSThread isMainThread]) {
for (id <JHQueueListener> listener in listeners) {
[listener startedToProcessQueueItem:queueItem];
}
} else {
dispatch_async(dispatch_get_main_queue(), ^{
for (id <JHQueueListener> listener in listeners) {
[listener startedToProcessQueueItem:queueItem];
}
});
}
I need to block and wake a process using SIGUSR2 and SIGUSR1 respectively. Below here's my signal handler sub routine. How do I wake a process blocked by pause?
void sig_handler(int sig) {
static int i = 1;
if(sig == SIGUSR2) {
pause();
}
else if(sig == SIGUSR1) {
/* I don't what to write here */
}
}
Also, I read somewhere pause() is not a good programming practice, is there any other means to suspend a process for some time?
See this page
In general, doing a lot of works in signals is ... tricky. Some things are not async-signal-safe, and therefore it makes robust programming there a bit difficult. In your case, pause() waits for a signal to arrive, but since you are calling it from the signal handler, it is not going to work there (I think).
As to making the process sleep and resume on signals. Look at the page I linked above. The best way is to have the signal handlers simply set flags and have the main thread (i.e. in main() or in an event loop) react to these flags. As recommended by the page, use sigsuspend when SIGUSR2 is received to pause the process until SIGURS1 is received.
It's simple. Use the 'kill' system call-
void sig_handler(int sig) {
static int i = 1;
if(sig == SIGUSR2) {
pause();
}
else if(sig == SIGUSR1) {
kill(<pid of process to wake up>, sig);
// make sure that process with pid has registered for sig
}
}
I would like to implement a multi-threaded, non-blocking file open. Ideally, the desired solution would be to make a call to open() & have it return immediately, and do something like register a callback to be called (or handle a signal, or conditional variable) when the open() operation is actually complete. To that end, I wrote a little test driver that creates multiple simultaneous threads and tries to open the same file. I would have hoped the return from openat() to be an invalid file descriptor, with an errno == EAGAIN, but the open call seems to always block until the open completes successfully.
Is there an implementation of this approach for a non-blocking open()?
Thanks in advance.
Reference Thread Code:
void* OpenHandler(void* args)
{
// Declarations removed
Dir = "/SomeDir";
if ((DirFd = open(Dir, O_RDONLY )) < 0) {
printf("********Error opening Directory*******\n");
return NULL;
}
do {
FileFd = openat(DirFd, &FileName[DirLen], O_RDONLY | O_NONBLOCK);
/* If open failed */
if (FileFd == -1) {
if (errno == EAGAIN)
printf("Open would block\n");
else {
printf("Open failed\n");
pthread_exit(NULL);
}
}
else
Opened = 1;
} while (!Opened);
pthread_exit(NULL);
}
open() and openat() always fully resolve the open request in one shot (normally, this doesn't need to sleep, but it can if directory entries need to be brought in from disk or over the network).
To do what you want, you'll have to build a thread pool of file-opening threads, that perform the open() on behalf of the thread you want to continue working and notify it when the open is complete. Unless you're opening a lot of files on very slow network filesystems, I doubt the juice will be worth the squeeze.