I am creating several ByteBuddy classes (using DynamicTypeBuilder) and loading them. The creation of these classes and the loading of them happens on a single thread (the main thread; I do not spawn any threads myself nor do I submit anything to an ExecutorService) in a relatively simple sequence.
I have noticed that running this in a unit test several times in a row yields different results. Sometimes the classes are created and loaded fine. Other times I get errors from the generated bytecode when it is subsequently used (often in the general area of where I am using withArgumentArrayElements, if it matters; ArrayIndexOutOfBoundsErrors and the like; again other times this all works fine (with the same inputs)).
This feels like a race condition, but as I said I'm not spawning any threads. Since I am not using threads, only ByteBuddy (or the JDK) could be. I am not sure where that would be. Is there a ByteBuddy synchronization mechanism I should be using when creating and loading classes with DynamicTypeBuilder.make() and getLoaded()? Maybe some kind of class resolution is happening (or not happening!) on a background thread or something at make() time, and I am accidentally somehow preventing it from completing? Maybe if I'm going to use these classes immediately (I am) I need to supply a different TypeResolutionStrategy? I am baffled, as should be clear, and cannot figure out why a single-threaded program with the same inputs should produce generated classes that behave differently from run to run.
My pattern for loading these classes is:
Try to load the (normally non-existent) class using Class#forName(name, true, Thread.currentThread().getContextClassLoader()).
If (when) that fails, create the ByteBuddy-generated class and load it using the usual ByteBuddy recipes.
If that fails, it would be only because some other thread might have created the class already. In this unit test, there is no other thread. In any case, if a failure were to occur here, I repeat step 1 and then throw an exception if the load fails.
Are there any ByteBuddy-specific steps I should be taking in addition or instead of these?
Phew! I think I can chalk this up to a bug in my code (thank goodness). Briefly, what looked like concurrency issues was (most likely) an issue with accidentally shared classnames and HashMap iteration order: when one particular subclass was created-and-then-loaded, the other would simply be loaded (not created) and vice versa. The net effect was effects that looked like those of a race condition.
Byte Buddy is fully thread-safe. But it does attempt to create a class every time you invoke load what is a fairly expensive operation. To avoid this, Byte Buddy offers the TypeCache mechanism that allows you to implement an efficient cache.
Note that libraries like cglib offer automatic caching. Byte Buddy does not do this since the cache uses all inputs as keys and references them statically what can easily create memory leaks. Also, the keys are rather inefficient which is why Byte Buddy chose this approach.
Related
var list = listOf("one", "two", "three")
fun One() {
list.forEach { result ->
/// Does something here
}
}
fun Two() {
list = listOf("four", "five", "six")
}
Can function One() and Two() run simultaneously? Do they need to be protected by locks?
No, you dont need to lock the variable. Even if the function One() still runs while you change the variable, the forEach function is running for the first list. What could happen is that the assignment in Two() happens before the forEach function is called, but the forEach would either loop over one or the other list and not switch due to the assignment
if you had a println(result) in your forEach, your program would output either
one
two
three
or
four
five
six
dependent on if the assignment happens first or the forEach method is started.
what will NOT happen is something like
one
two
five
six
Can function One() and Two() run simultaneously?
There are two ways that that could happen:
One of those functions could call the other. This could happen directly (where the code represented by // Does something here in One()⁽¹⁾ explicitly calls Two()), or indirectly (it could call something else which ends up calling Two() — or maybe the list property has a custom setter which does something that calls One()).
One thread could be running One() while a different thread is running Two(). This could happen if your program launches a new thread directly, or a library or framework could do so. For example, GUI frameworks tend to have one thread for dispatching events, and others for doing work that could take time; and web server frameworks tend to use different threads for servicing different requests.
If neither of those could apply, then there would be no opportunity for the functions to run simultaneously.
Do they need to be protected by locks?
If there's any possibility of them being run on multiple threads, then yes, they need to be protected somehow.
99.999% of the time, the code would do exactly what you'd expect; you'd either see the old list or the new one. However, there's a tiny but non-zero chance that it would behave strangely — anything from giving slightly wrong results to crashing. (The risk depends on things like the OS, CPU/cache topology, and how heavily loaded the system is.)
Explaining exactly why is hard, though, because at a low level the Java Virtual Machine⁽²⁾ does an awful lot of stuff that you don't see. In particular, to improve performance it can re-order operations within certain limits, as long as the end result is the same — as seen from that thread. Things may look very different from other threads — which can make it really hard to reason about multi-threaded code!
Let me try to describe one possible scenario…
Suppose Thread A is running One() on one CPU core, and Thread B is running Two() on another core, and that each core has its own cache memory.⁽³⁾
Thread B will create a List instance (holding references to strings from the constant pool), and assign it to the list property; both the object and the property are likely to be written to its cache first. Those cache lines will then get flushed back to main memory — but there's no guarantee about when, nor about the order in which that happens. Suppose the list reference gets flushed first; at that point, main memory will have the new list reference pointing to a fresh area of memory where the new object will go — but since the new object itself hasn't been flushed yet, who knows what's there now?
So if Thread A starts running One() at that precise moment, it will get the new list reference⁽⁴⁾, but when it tries to iterate through the list, it won't see the new strings. It might see the initial (empty) state of the list object before it was constructed, or part-way through construction⁽⁵⁾. (I don't know whether it's possible for it to see any of the values that were in those memory locations before the list was created; if so, those might represent an entirely different type of object, or even not a valid object at all, which would be likely to cause an exception or error of some kind.)
In any case, if multiple threads are involved, it's possible for one to see list holding neither the original list nor the new one.
So, if you want your code to be robust and not fail occasionally⁽⁶⁾, then you have to protect against such concurrency issues.
Using #Synchronized and #Volatile is traditional, as is using explicit locks. (In this particular case, I think that making list volatile would fix the problem.)
But those low-level constructs are fiddly and hard to use well; luckily, in many situations there are better options. The example in this question has been simplified too much to judge what might work well (that's the down-side of minimal examples!), but work queues, actors, executors, latches, semaphores, and of course Kotlin's coroutines are all useful abstractions for handling concurrency more safely.
Ultimately, concurrency is a hard topic, with a lot of gotchas and things that don't behave as you'd expect.
There are many source of further information, such as:
These other questions cover some of the issues.
Chapter 17: Threads And Locks from the Java Language Specification is the ultimate reference on how the JVM behaves. In particular, it describes what's needed to ensure a happens-before relationship that will ensure full visibility.
Oracle has a tutorial on concurrency in Java; much of this applies to Kotlin too.
The java.util.concurrent package has many useful classes, and its summary discusses some of these issues.
Concurrent Programming In Java: Design Principles And Patterns by Doug Lea was at one time the best guide to handling concurrency, and these excerpts discuss the Java memory model.
Wikipedia also covers the Java memory model
(1) According to Kotlin coding conventions, function names should start with a lower-case letter; that makes them easier to distinguish from class/object names.
(2) In this answer I'm assuming Kotlin/JVM. Similar risks are likely apply to other platforms too, though the details differ.
(3) This is of course a simplification; there may be multiple levels of caching, some of which may be shared between cores/processors; and some systems have hardware which tries to ensure that the caches are consistent…
(4) References themselves are atomic, so a thread will either see the old reference or the new one — it can't see a bit-pattern comprising parts of the old and new ones, pointing somewhere completely random. So that's one problem we don't have!
(5) Although the reference is immutable, the object gets mutated during construction, so it might be in an inconsistent state.
(6) And the more heavily loaded your system is, the more likely it is for concurrency issues to occur, which means that things will probably fail at the worst possible time!
Some Vulkan objects (eg vkPipelines, vkCommandBuffers) are able to be created/allocated in arrays (using size + pointer parameters). At a glance, this appears to be done to make it easier to code using common usage patterns. But in some cases (eg: when creating a C++ RAII wrapper), it's nicer to create them one at a time. It is, of course, simple to achieve this.
However, I'm wondering whether there are any significant downsides to doing this?
(I guess this may vary depending on the actual object type being created - but I didn't think it'd be a good idea to ask the same question for each object)
Assume that, in both cases, objects are likely to be created in a first-created-last-destroyed manner, and that - while the objects are individually created and destroyed - this will likely happen in a loop.
Also note:
vkCommandBuffers are also deallocated in arrays.
vkPipelines are destroyed individually.
Are there any reasons I should modify my RAII wrapper to allow for array-based creation/destruction? For example, will it save memory (significantly)? Will single-creation reduce performance?
Remember that vkPipeline creation does not require external synchronization. That means that the process is going to handle its own mutexes and so forth. As such, it makes sense to avoid locking those internal mutexes whenever possible.
Also, the process is slow. So being able to batch it up and execute it into another thread is very useful.
Command buffer creation doesn't have either of these concerns. So there, you should feel free to allocate whatever CBs you need. However, multiple creation will never harm performance, and it may help it. So there's no reason to avoid it.
Vulkan is an API designed around modern graphics hardware. If you know you want to create a certain number of objects up front, you should use the batch functions if they exist, as the driver may be able to optimize creation/allocation, resulting in potentially better performance.
There may (or may not) be better performance (depending on driver and the type of your workload). But there is obviously potential for better performance.
If you create one or ten command buffers in you application then it does not matter.
For most cases it will be like less than 5 %. So if you do not care about that (e.g. your application already runs 500 FPS), then it does not matter.
Then again, C++ is a versatile language. I think this is a non-problem. You would simply have a static member function or a class that would construct/initialize N objects (there's probably a pattern name for that).
The destruction may be trickier. You can again have static member function that would destroy N objects. But it would not be called automatically and it is annoying to have null/husk objects around. And the destructor would still be called on VK_NULL_HANDLE. There is also a problem, that a pool reset or destruction would invalidate all the command buffer C++ objects, so there's probably no way to do it cleanly/simply.
Is the JVM (any of them) ever re-compiling code that has already been compiled at runtime?
It depends on what you mean by re-compiling, but the HotSpot VM will discard code that relies on optimistic assumptions when they are proven to be wrong or no longer relevant. See deoptimization:
Deoptimization is the process of changing an optimized stack frame to an unoptimized one. With respect to compiled methods, it is also the process of throwing away code with invalid optimistic optimizations, and replacing it by less-optimized, more robust code.
The fourth point is particularly interesting:
If a class is loaded that invalidates an earlier class hierarchy analysis, any affected method activations, in any thread, are forced to a safepoint and deoptimized.
This applies to optimistic method inlining as described in this paper:
A class hierarchy analysis (CHA) is
used to detect virtual call sites where currently only one suitable method exists.
This method is then optimistically inlined. If a class is loaded later that adds
another suitable method and, therefore, the optimistic assumption no longer
holds, the method is deoptimized.
Good day all,
I'm having a hell of a time figuring out which multithreading approach to utilize in my current work project. Since I've never written a multithreaded app in my life, this is all confusing and very overwhelming. Without further ado, here's my background story:
I've been assigned to take over work on a control application for a piece of test equipment in my companies R&D lab. The program has to be able to send and receive serial communications with three different devices semi-concurrently. The original program was written in VB 6 (no multithreading) and I did plan on just modding it to work with the newer products that need to be tested until it posed a safety hazard when the UI locked up due to excessive serial communications during a test. This resulted in part of the tester hardware blowing up, so I decided to try rewriting the app in VB.Net as I'm more comfortable with it to begin with and because I thought multithreading might help solve this problem.
My plan was to send commands to the other pieces of equipment from the main app thread and spin the receiving ends off into their own threads so that the main thread wouldn't lock up when timing is critical. However, I've yet to come to terms with my options. To add to my problems, I need to display the received communications in separate rich text boxes as they're received while the data from one particular device needs to be parsed by the main program, but only the text that results from the most current test (I need the text box to contain all received data though).
So far, I've investigated delegates, handling the threads myself, and just began looking into BackgroundWorkers. I tried to use delegates earlier today, but couldn't figure out a way to update the text boxes. Would I need to use a call back function to do this since I can't do it in the body of the delegate function itself? The problem I see with handling threads myself is figuring out how to pass data back and forth between the thread and the rest of the program. BackgroundWorkers, as I said, I just started investigating so I'm not sure what to think about them yet.
I should also note that the plan was for the spawned threads to run continuously until somehow triggered to stop. Is this possible with any of the above options? Are there other options I haven't discovered yet?
Sorry for the length and the fact that I seem to ramble disjointed bits of info, but I'm on a tight deadline and stressed out to the point I can't think straight! Any advice/info/links is more than appreciated. I just need help weighing the options so I can pick a direction and move forward. Thanks to everybody who took the time to read this mess!
OK, serial ports, inter-thread comms, display stuff in GUI components like RichTextBox, need to parse incoming data quickly to decode the protocol and fire into a state-machine.
Are all three serial ports going to fire into the same 'processControl' state-machine?
If so, then you should probably do this by assembling event/data objects and queueing them to the state-machine run by one thread,(see BlockingCollection). This is like hugely safer and easier to understand/debug than locking up the state-engine with a mutex.
Define a 'comms' class to hold data and carry it around the system. It should have a 'command' enum so that threads that get one can do the right thing by switching on the enum. An 'Event' member that can be set to whatever is used by the state-engine. A 'bool loadChar(char inChar)' that can have char-by-char data thrown into it and will return 'true' only if a complete, validated protocol-unit has been assembled, checked and parsed into data mambers. A 'string textify()' method that dumps info about the contained data in text form. A general 'status' string to hold text stuff. An 'errorMess' string and Exception member.
You probably get the idea - this comms class can transport anything around the system. It's encapsulated so that a thread can use it's data and methods without reference to any other instance of comms - it does not need any locking. It can be queued to work threads on a Blocking Collection and BeginInvoked to the GUI thread for displaying stuff.
In the serialPort objects, create a comms at startup and load a member with the serialPort instance. and, when the DataReceived event fires, get the data from the args a char at a time and fire into the comms.loadChar(). If the loadChar call returns true, queue the comms instance to the state-machine input BlockingCollection and then immediately create another comms and start loading up the new one with data. Just keep doing that forever - loading up comms instances with chars until they have a validated protocol unit and queueing them to the state-machine. It may be that each serial port has its own protocol - OK, so you may need three comms descendants that override the loadChar to correctly decode their own protocol.
In the state-machine thread, just take() comms objects from the input and do the state-engine thing, using the current state and the Event from the comms object. If the SM action routine decides to display something, BeginInvoke the comms to the GUI thread with the command set to 'displaySomeStuff'. When the GUI thread gets the comms, it can case-switch on the command to decide what to display/whatever.
Anyway, that's how I build all my process-control type apps. Data flows around the system in 'comms' object instances, no comms object is ever operated on by more than one thead at a time. It's all done by message-passing on either BlockingCollection, (or similar), queues or BeginInvoke() if going to the GUI thread.
The only locks are in the queues and so are encapsulated. There are no explicit locks at all. This means there can be no explicit deadlocks at all. I do get headaches, but I don't get lockups.
Oh - don't go near 'Thread.Join()'.
My app contains several singletons (following from this tutorial). I've noticed however, when the app crashes because of a singleton, it becomes nearly impossible to figure out where it came from. The app breakpoints at the main function giving an EXEC_BAD_ACCESS even though the problem lies in one of the Singleton objects. Is there a guide to how would I debug my singleton objects if they were problematic?
if you don't want to change your design (as recommended in my other post), then consider the usual debugging facilities: assertions, unit tests, zombie tests, memory tests (GuardMalloc, scribbling), etc. this should identify the vast majority of issues one would encounter.
of course, you will have some restrictions regarding what you can and cannot do - notably regarding what cannot be tested independently using unit tests.
as well, reproducibility may be more difficult in some contexts when/if you are dealing with a complex global state because you have created several enforced singletons. when the global state is quite large and complex - testing these types independently may not be fruitful in all cases since the bug may appear only in a complex global state found in your app (when 4 singletons interact in a specific manner). if you have isolated the issue to interactions of multiple singleton instances (e.g. MONAudioFileCache and MONVideoCache), placing these objects in a container class will allow you to introduce coupling, which will help diagnose this. although increasing coupling is normally considered a bad thing; this does't really increase coupling (it already exists as components of the global state) but simply concentrates existing global state dependencies -- you're really not increasing it as much as you are concentrating it when the state of these singletons affect other components of the mutable global state.
if you still insist on using singletons, these may help:
either make them thread safe or add some assertions to verify mutations happen only on the main thread (for example). too many people assume an object with atomic properties implies the object is thread safe. that is false.
encapsulate your data better, particularly that which mutates. for example: rather than passing out an array your class holds for the client to mutate, have the singleton class add the object to the array it holds. if you truly must expose the array to the client, then return a copy. ths is just basic ood, but many objc devs expose the majority of their ivars disregarding the importance of encapsualtion.
if it's not thread safe and the class is used in a mutithreaded context, make the class (not the client) implement proper thread safety.
design singletons' error checking to be particularly robust. if the programmer passes an invalid argument or misuses the interface - just assert (with a nice message about the problem/resolution).
do write unit tests.
detach state (e.g. if you can remove an ivar easily, do it)
reduce complexity of state.
if something is still impossible to debug after writing/testing with thorough assertions, unit tests, zombie tests, memory tests (GuardMalloc, scribbling), etc,, you are writing programs which are too complex (e.g. divide the complexity among multiple classes), or the requirements do not match the actual usage. if you're at that point, you should definitely refer to my other post. the more complex the global variable state, the more time it will take to debug, and the less you can reuse and test your programs when things do go wrong.
good luck
I scanned the article, and while it had some good ideas it also had some bad advice, and it should not be taken as gospel.
And, as others have suggested, if you have a lot of singleton objects it may mean that you're simply keeping too much state global/persistent. Normally only one or two of your own should be needed (in addition to those that other "packages" of one sort or another may implement).
As to debugging singletons, I don't understand why you say it's hard -- no worse than anything else, for the most part. If you're getting EXEC_BAD_ACCESS it's because you've got some sort of addressing bug, and that's nothing specific to singleton schemes (unless you're using a very bad one).
Macros make debugging difficult because the lines of code they incorporate can't have breakpoints put in them. Deep six macros, if nothing else. In particular, the SYNTHESIZE_SINGLETON_FOR_CLASS macro from the article is interfering with debugging. Replace the call to this macro function with the code it generates for your singleton class.
ugh - don't enforce singletons. just create normal classes. if your app needs just one instance, add them to something which is created once, such as your app delegate.
most cocoa singleton implementations i've seen should not have been singletons.
then you will be able to debug, test, create, mutate and destroy these objects as usual.
the good part is course that the majority of your global variable pains will disappear when you implement these classes as normal objects.