I found that asio's doc about synchronization by strand is obscure. It just says that in a thread pool design asio app, the handler can be run in any thread which had call io_service::run(). Using a strand to warp these handler can make their execution concurrent correctly. In its example3, all handle_read are wrap by strand, and I think the variables in Connection class such as buffer has been synchronized by strand, different thread calls handle_read will gets up-to-date data, that is OK. But what about there is a data member defined in Connection class which also accessed by a handler was not wrap by strand? I think this is a problem, isn't it?
In its doc example3, why handle_accept was not wrap by a strand? The new_connection_ is accessed by multi threads: new_connection_.reset called by thread A and server::handle_accept called by thread B. I think it needs data synchronization here or else thread B might use a out-of-date new_connection_ that its reset have not been called yet.
HTTP Server 3 is designed in such a way that it does not actually need a strand.
A fundamental trait of Boost.Asio is that a handler will be called at most once for a given operation. This behavior allows for the call path of asynchronous programming to be envisioned more as a call chain.
For example, examine the illustrated call chain for the server accepting connections:
server::server(...)
{
start_accept(); --.
} |
.----------------'
| .----------------------------------------.
V V |
void server::start_accept() |
{ |
new_connection_.reset(new connection(...)); |
acceptor_.async_accept(..., handle_accept); --. |
} | |
.--------------------------------------------' |
| |
V |
void server::handle_accept(...) |
{ |
if (!error) |
{ |
new_connection_->start(); |
} |
start_accept(); ---------------------------------'
}
As shown in the illustration, only a single asynchronous event chain is present. With no possibility of concurrent execution of the handlers or operations on new_connection, it is said to be running in an implicit strand. The thread in which the handler, server::handle_accept, runs is inconsequential.
The connection::handle_read call chains and more details about strands are answered in this question.
I think you are somehow missing the meaning of strand. It does not synchronize data access. It syncronize handler calls. This can be understood as "all handlers wrapped by given strand will NOT be called concurrently".
So, your first questsion: if some handler is not wrapped by strand - it can be called concurrently now. So its subject to sync issues and/or RC. Note if you wrap in one place does not mean you are protected from RC; this should be done in every call. Since strand does not know what you calling from other thread unless you use .wrap
Second question: In given example start_accept setting up accept handler handle_accept, and handle_accept is setting new accept handler (via calling start_accept). So, they will not be called concurrently since you cannot create 2 or more async_accept events. Sure, if other thread call start_accept for same "Server" instance - this example can/will fail, but doing so is a clear mistake.
Related
SharedFlow has just been introduced in coroutines 1.4.0-M1, and it is meant to replace all BroadcastChannel implementations (as stated in the design issue decription).
I have a use case where I use a BroadcastChannel to represent incoming web socket frames, so that multiple listeners can "subscribe" to the frames.
The problem I have when I move to a SharedFlow is that I can't "end" the flow when I receive a close frame, or an upstream error (which I would like to do to inform all subscribers that the flow is over).
How can I make all subscriptions terminate when I want to effectively "close" the SharedFlow?
Is there a way to tell the difference between normal closure and closure with exception? (like channels)
If MutableSharedFlow doesn't allow to convey the end of the flow to subscribers, what is the alternative if BroadcastChannel gets deprecated/removed?
The SharedFlow documentation describes what you need:
Note that most terminal operators like Flow.toList would also not complete, when applied to a shared flow, but flow-truncating operators like Flow.take and Flow.takeWhile can be used on a shared flow to turn it into a completing one.
SharedFlow cannot be closed like BroadcastChannel and can never represent a failure. All errors and completion signals should be explicitly materialized if needed.
Basically you will need to introduce a special object that you can emit from the shared flow to indicate that the flow has ended, using takeWhile at the consumer end can make them emit until that special object is received.
I think a possible solution is creating a boolean flag isValid and publicly expose only flows with .takeWhile { isValid }. Then just call isValid = false and sFlow.emit() when you want to close all subscribers.
Possible implementation:
private var isValid = true // In real scenario use atomic boolean
private val _sharedFlow = MutableSharedFlow<Unit>()
val sharedFlow: Flow<Unit> get() = _sharedFlow.takeWhile { isValid }
suspend fun cancelSharedFlow() {
isValid = false
_sharedFlow.emit(Unit)
}
EDIT: In my case .emit() was always suspending so I had to use BufferOverflow.DROP_LATEST (which is not suitable for many usecases). Not sure if the problem is in this example or elsewhere in my app. If you see a problem, please comment :)
I'd like to organize a thread barrier: given a single lock object, any thread can obtain it and continue thread's chain further, but any other thread will stay dormant on the same lock object until the first thread finishes and releases the lock.
Let's express my intention in code (log() simply prints string in a log):
val mutex = Semaphore(1) // number of permits is 1
source
.subscribeOn(Schedulers.newThread()) // any unbound scheduler (io, newThread)
.flatMap {
log("#1")
mutex.acquireUninterruptibly()
log("#2")
innerSource
.doOnSubscribe(log("#3"))
.doFinally {
mutex.release()
log("#4")
}
}
.subscribe()
It actually works well, i can see how multiple threads show log "#1" and only one of them propagates further, obtaining lock object mutex, then it releases it and i can see other logs, and next threads comes into play. OK
But sometimes, when pressure is quite high and number of threads is greater, say 4-5, i experience DEADLOCK:
Actually, the thread that has acquired the lock, prints "#1" and "#2" but it then never print "#3" (so doOnSubscribe() not called), so it actually stops and does nothing, not subscribing to innerSource in flatMap. So all threads are blocked and app is not responsive at all.
My question - is it safe to have blocking operation inside flatMap? I dig into flatMap source code and i see the place where it internally subscribes:
if (!isDisposed()) {
o.subscribe(new FlatMapSingleObserver<R>(this, downstream));
}
Is it possible that thread's subscription, that has acquired lock, was disposed somehow?
You can use flatMap second parameter maxConcurrency and set it to 1, so it does what you want without manually locking
The code:
In thread 1:
boost::async_read(socket, buffer, strand.wrap(read_handler));
In thread 2:
strand.post([](){socket.async_write_some(buffer, strand.wrap(write_handler))});
It is clear that read_handler, async_write_some, write_handler protected by strand, they will not concurrent. However, async_read is an composed operation, it will call zero or more times to async_read_some, those async_read_some also need protect by strand or else they might concurrent with async_write_some in thread 2.
But from the code, strand only wrap read_handler, how asio make all intermediate operations(async_read_some) also wrapped by the strand?
In short, asio_handler_invoke enables one to customize the invocation of handlers in the context of a different handler. In this case, the object returned from strand.wrap() has a custom asio_handler_invoke strategy associated with it that will dispatch handlers into the strand that wrapped the initial handler. Conceptually, it is as follows:
template <typename Handler>
struct strand_handler
{
void operator()();
Handler handler_;
boost::asio::strand dispatcher_;
};
// Customize invocation of Function within context of custom_handler.
template <typename Function>
void asio_handler_invoke(Function function, strand_handler* context)
{
context->dispatcher_.dispatch(function);
}
strand_handler wrapped_completion_handler = strand.wrap(completion_handler);
using boost::asio::asio_handler_invoke;
asio_handler_invoke(intermediate_handler, &wrapped_completion_handler);
The custom asio_handler_invoke hook is located via argument-dependent lookup. This detail is documented in the Handler requirement:
Causes the function object f to be executed as if by calling f().
The asio_handler_invoke() function is located using argument-dependent lookup. The function boost::asio::asio_handler_invoke() serves as a default if no user-supplied function is available.
For more details on asio_handler_invoke, consider reading this answer.
Be aware that an operation may be attempted within the initiating function. The documentation is specific that intermediate handlers will be invoked within the same context as the final completion handler. Therefore, given:
assert(strand.running_in_this_thread());
boost::async_read(socket, buffer, strand.wrap(read_handler));
the boost::async_read itself must be invoked within the context of strand to be thread-safe. See this answer for more details on thread-safety and strands.
I'm trying to force CAN signals to given values using COM interface of CANalyzer. Since there is no COM method to send CAN messages, I'm implementing a workaround using CAPL:
void SendMySignal(int value) {
message MyMessage msg;
msg.MySignal = value;
output(msg);
}
This works fine, however since MyMessage and MySignal are referenced statically (by name) here, I'll have to implement N functions to be able to send N signals (or an N-way switch statement, etc). Is there a way to avoid the hassle and access signals inside a message by string? Something like this:
void SendSignal(int MessageID, char SignalName, int value)
I'm also open to alternative solutions in case I have missed something in the COM interface. If there is a solution which only works for CANoe, I can ask my boss for a license, but of course I'd prefer to do without.
there is such function, but it is restricted to be used only in test nodes
long setSignal(char signalName[], double aValue);
you can find details in:
CAPL Function Overview » Test Feature Set / Signal Access » SetSignal
Special Use Case: Signal is not known before Measurement Start
and take care about not to send for each signal a new message to avoid bus over-flooding. In my opinion it is a better style to set all signals for whole message and to send it on change only when it is not cyclic. Signal updates in cyclic messages mostly have to be sent in next cycle.
In the book 'Operating System Concepts' - 9th Edition - Chapter 3 - Page 117 - Page 120 it says:
Both processes (the parent and the child) continue execution at the instruction after the fork(), with one difference: the return code for the fork() is zero for the new (child) process, whereas the (nonzero) process identifier of the child is returned to the parent.
The only difference is that the value of pid (the process identifier) for the child process is zero, while that for the parent is an integer value greater than zero (in fact, it is the actual pid of the child process).
Please can someone explain this concept to me.
Whenever a processor(CPU) runs a program , it stores the line number of the code it is executing (more formally address pointing to that instruction).The computer stores it in a register (kind of variable) called as stack pointer. So the stack pointer(SP) will store the current instruction processor has to execute (or run). This way computer track which instruction should be executed .
When a program runs , it is allocated some small memory in the computer's main memory.Here's where all our code along with important registers(including SP) which help processor to keep track of program when it's running.Too a process is uniquely identified by Process ID(PID).
Now let me come to your question. Whenever we call fork , a copy of the program from which you called fork is created. This copy is called as the "child process" and our original process is known as parent process.
When the copy created , all the memory that your program has been allocated is copied to some other place in memory (which is now child process's memory).So an identical running program(process) is created.
Now this copied memory contains the SP of the parent process , so whenever processor runs the program it directly runs the program from the same fork call line (since SP will store this line as current instruction when the process is created).Since our fork call was successful , it has to return a non-negative value (denoting success of fork system call)So it return 0 to the child process and child process ID to the parent(since the current Process ID was pointing here too).
Returning Child Process ID to parent makes a good deal since it's better that parent can keep a track of child process created from it.Too returning child a 0 make the deal even better as we have to return a non-negative number and other positive number may be some process's PID.
Run cd /proc in your linux system . All the directories having some numeral name are pid of some process(which may be active/inactive).Read more about it to clear the concept.
Hope that clears your doubt :).
when you call fork in your code it returns two values in case call is success one for parent (the code run by parent) and zero for child (the code run by child)
int childparent;
childparent = fork();
if (childparent >=0)
{
if (childparent==0)
//child code
if (childparent>0)
//parent code
}
the pid 0 mention in your sentence is not the process id shown by shell command ps
Edit: yes the code running in the parent (if childparent>0) case is running in context of that specific child which is just created. so return value to parent is the child actual Process ID (PID). if you fork in your simple code and sleep for long time in code enough to run ps you can match the PIDs shown in PS and printf in parent the return value of the fork() (printf("%d",childparent))
fork() makes a complete copy of the current process by creating a new process and then filling it with the current process. It chooses one to be the parent and the other to be the child. It indicates the difference only by adjusting the return value of fork(). If the process(es) ignore the return value, they behave identically. (There are some rarely significant other differences between the processes related to signal handling and delivery, file descriptors opened with special attributes, memory maps, etc.)
When you think you begin to understand it, look at this to see if you do.
I had this doubt when I was reading the book too. The answer is as follows:
When the main program (parent) executes fork(), a copy of its address space, including the program and all data, is created. System call fork() returns the child process ID to the parent and returns 0 to the child process. Both the parent and the child process can now start their execution from the immediate next line after the fork system call.
Let me illustrate this with a simple example.
Consider this code:
main()
{
pid=fork();
if(pid == 0) // Condition to determine Parent/Child Process
ChildProcess();
else
ParentProcess();
}
void ChildProcess()
{
//Some Arbitrary Code
}
void ParentProcess()
{
//Some Arbitrary Code
}
This snippet explains that based on a condition, both the processes (parent and child) can now execute in their own pre-defined way.
In the above example, say the process id of the child is 3456, then the parent would get that id as the return value from fork. However, the child will always get the process-id as 0 and then the execution continues.
The design of the fork() call is such because the complete administration of the child process(es) can now be handled by the parent and the parent can always keep a note of which of the child processes terminates, normally or abnormally, by implicitly/explicitly invoking exit() system call. A parent may also simply wait for the child by making a wait() system call which then return the pid of the child and in this way, the parent can keep a note of which child has terminated.
This is how child process creation and termination is handled.
There is one more thing I would like to add here which is not completely relevant to the question but I think would be helpful.
You would also have noticed the description of the exec() system call immediately after this discussion in the book. In short, both the discussions explain this:
Forking provides a way for an existing process to start a new one, but
what about the case where the new process is not part of the same
program as parent process? This is the case in the shell; when a user
starts a command it needs to run in a new process, but it is unrelated
to the shell.
This is where the exec system call comes into play. exec will replace
the contents of the currently running process with the information
from a program binary.
Thus the process the shell follows when launching a new program is to
firstly fork, creating a new process, and then exec (i.e. load into
memory and execute) the program binary it is supposed to run.
If you would like to know more about the fork() system call then you should also know about its internal implementation, especially how the clone() system call works.
Reference Sites:
The fork() system call article by MTU
How Fork & Exec work together