This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
Are there any Java VMs which can save their state to a file and then reload that state?
Is there a way to stop the JVM, save its states in the hard disk and then resume the execution from the same point it was stopped? Something like the hibernation of windows, linux, etc?
When it's running on a Unix, you can certainly suspend it by e.g. hitting "^Z" on the command line you started it from, or sending it a SIGSTOP signal. This doesn't quite do what you want. It's not written directly to disk (though it may be swapped out). It won't survive a system restart. Unlike an image file, you can't copy or restore it.
There are also a variety of hacks that let some image based systems (smalltalk, emacs, etc.) "unexec()" themselves and save a copy on a disk. These will break any network connections or open files. Most approaches also require the cooperation of the program being saved, especially to gracefully handle the connections to the outside world being severed.
Finally, you could run the JVM in a VM, and suspend the VM. There, at least, connections to files inside the VM will be saved.
Related
I'm using a Pharo 6.1 image to do some CSV file processing. Sometimes my image just becomes unresponsive — I have to terminate the Pharo VM from Task Manager.
Is it necessary to use somethings like:
ensure:[toStream close]
ensure:[fromStream close]
What's a simple but reliable approach to reading and especially writing files with Pharo?
The image is not really unresponsive but the UI thread for sure is when you do a long operation like reading a big CSV file.
The best would be to for a process for doing it.
Like in:
[ target doSomeLongThing ] fork.
You can view the process in the process browser from the world menu and terminate it there.
Now that is not really something one will think about when trying things out interactively in a playground.
What I do to alleviate this situation is twofold:
Put these things in a test. Why? Because tests have a maximum duration and if things are stuck, they will come back with a time exceeded notification from the test.
Start the Zinc REPL on a port, so that I can access the system from a web browser if the UI thread is kind of stuck and restart the UI thread.
I wish the interrupt key would work in all cases but well, it doesn't and is a solid annoyance.
I am of the opinion that images should be considered as discardable artifacts and CI job should be building them regularly (like daily) so that we have some recovery path.
This is not really cool as yes, using an image and being able to come back to it without having to rebuild stuff all the time when doing explorations shouldn't frustrate us with UI thread blockages. I hate that. I have an image that is stuck on restart from some Glamour problem, it is infuriating as it cannot be debugged or anything.
You can also use:
(FileLocation imageDirectory / 'somefile.csv') asFileReference readStreamDo: [ :stream |
"do something with the stream" ].
This will do the ensure: bit for you. Cleaner. Easier.
For CSV, also give a shot to NeoCSV as it will do the file handling for you as well.
HTH
I have a project which scans a large file (2.5GB) picking out strings which will then be written to some subset of several hundred files.
It would be fastest just to use normal buffered writes but
I'm worried about running out of filehandles.
I want to be able to watch the progress of the files while they're being written.
I would prefer as little loss as possible if the process is interrupted. Incomplete files are still partially useful.
So instead I open in read/write mode, append the new line, and close again.
This was fast enough much of the time but I have found that on certain OSes this behaviour is a severe pessimization. Last time I ran it on my Windows 7 netbook I interrupted it after several days!
I can implement some kind of MRU filehandle manager which keeps so many files open and flushes after so many write operations each. But is this overkill?
This must be a common situation, is there a "best practice", a "pattern"?
Current implementation is in Perl and has run on Linux, Solaris, and Windows, netbooks to phat servers. But I'm interested in the general problem: language-independent and cross-platform. I've thought of writing the next version in C or node.js.
On Linux, you can open a lot of files (thousands). You can limit the number of opened handles in a single process with the setrlimit syscall and the ulimit shell builtin. You can query them with the getrlimit syscall and also using /proc/self/limits (or /proc/1234/limits for process of pid 1234). The maximum number of system-wide opened files is thru /proc/sys/fs/file-max (on my system, I have 1623114).
So on Linux you could not bother, and open many files at once.
And I would suggest to maintain a memoized cache of opened files, and use them if possible (in a MRU policy). Don't open and close each file too often, only when some limit has been reached... (e.g. when an open did fail).
In other words, you could have your own file abstraction (or just a struct) which knows the file name, may have an opened FILE* (or a null pointer) and keep the current offset, maybe also the last time of opening or writing, then manage a collection of such things in a FIFO discipline (for those having an opened FILE*). You certainly want to avoid close-ing (and later re-open-ing) a file descriptor too often.
You might occasionally (i.e. once a few minutes) call sync(2), but don't call it too often (certainly not more than once per 10 seconds). If using buffered FILE-s don't forget to sometimes fflush them. Again, don't do that very often.
From what experience I have programming whenever a program has a problem it crashes, whether it is from an unhanded exception or a piece of code that should have been checked for errors, but was not and threw one. What would cause a program to completely freeze a system to the point of requiring a restart.
Edit: Thanks for the answers. As for the language and OS this question was inspired by me playing Fallout and the game freezing twice in an hour causing me to have to restart the xbox, so I am guessing c++.
A million different things. The most common that come to mind are:
Spawning too many threads or processes, which drowns the OS scheduler.
Gobbling too much RAM, which puts the memory manager into page-fault hell.
In a Dotnet/Java type environment its quite difficult to seize a system up, because the Runtime keeps you code at a distance from the OS.
Closer to the metal say C or C++, Assembly etc you have to play fair with the rest of the system - If you dont have it already grab a copy of Petzold and observe/experiment yourself with the amount of 'boilerplate' code to get a single Window running...
Even closer, down at the driver level all sorts of things can happen...
There are number of reasons, being internal or external that leads to deadlocked application, more general case is when something is being asked for by a program but is not given that leads to infinite waiting, the practical example to this is, a program writes some text to a file, but when it is about to open a file for writing, same file is opened by any other application, so the requesting app will wait (freeze in some cases if not coded properly) until it gets exclusive control of the file.
And a critical freeze that leads to restarting the system is when the file which is asked for is something which very important for the OS. However, you may not need to restart the system in order to get it back to normal, unless the program which was frozen is written in a language that produces native binary, i.e. C/C++ to be precise. So if application is written in a language which works with the concept of managed code, like any .NET language, it will not need a system restart to get things back to normal.
page faults, trying to access inaccessible data or memory(acces violation), incompatible data types etc.
I'd like to have a small (not doing too damn much) daemon running on a little server, watching a directory for new files being added to it (and any directories in the main one), and calling another Clojure program to deal with that new file.
Ideally, each file would be added to a queue (a list represented by a ref in Clojure?) and the main process would take care of those files in the queue on a FIFO basis.
My question is: is having a JVM up running this little program all the time too much a resource hog? And do you have any suggestions as to how go about doing this?
Thank you very much!
EDIT: Another question I should ask: should I run this as its own instance (using less memory) and have it launch a new JVM when a file is seen, or have it on the same JVM the Clojure code that will process the file?
As long as it is running fine now and it has no memory leaks it should be fine.
From the daemon terminology I gather it is on a unix clone, and in this case best is to start it from an init script, or from the rc.local script. Unfortunately details differ from OS to OS to be more specific.
Limit the memry using -Xmx=64m or something to make sure it fails before taking down the rest of the services. Play a bit with the number to find the lowest reliable size.
Also, since clojures claim to fame is its ability to deal with concurrency it make a lot of sense to only run one JVM with all functionality running on it in multiple threads. The overhead of spawning new processes is already very big and if it is a JVM which needs to JIT and warm up its memory management, doubly so. On a resource constrained machine could pose a problem. and on a resource rich machine this is a waste.
I always found that the JVM is not made to quickly run something script like and exit again. It is really not made for that use case in my opinion
.
Whether this is possible I don't know, but it would mighty useful!
I have a process that fails periodically (running in Windows 2000). I then have just one chance to react to it before having to restart it and painfully wait for it to fail again. I didn't write the process so don't have the source to debug. The failure is seemingly random.
With a snapshot of the process I could repeatedly and quickly test reactions to the failure.
I had thought of running inside a VM but this isn't possible in this instance.
EDIT:
#Jon Cage asked:
When you say a snapshot, you mean capturing a process when it's about to fail (including memory, program state etc. etc.) ...and then replaying it's final few seconds repeatedly to see what effect it has on some other component?
This is exactly what I mean!
I think minidump is what you are looking for.
You can also used Userdump:
The User Mode Process Dumper
(userdump) dumps any running Win32
processes memory image (including
system processes such as csrss.exe,
winlogon.exe, services.exe, etc) on
the fly, without attaching a debugger,
or terminating target processes.
Generated dump file can be analyzed or
debugged by using the standard
debugging tools.
This article shows you how to use it.
My best bet is to start the process in a debugger (OllyDbg being my preferred tool).
The process will pause on an exception, and you can try to figure out what happened shortly before that.
This needs some understanding of assembler and does not allow to create a snapshot of the process for later analysis. You would need to write your own debugger for that - it should be theoretically possible.