Is time stamp used in process to free its resources if it holds resources for long time ??
if yes then in process state diagram there is no connection between block(wait) state and terminate state but both are connected via running state.so here a anomaly in concept arises that if a process has to quit from wait state then it has to go via running state.
The OS can't indiscriminately kill a process if it holds a resource for a long time.
Would you kill a 24/7 server who waits on a socket?
Would you kill a process who keeps a file opened? After how long? What if the process actually needs to keep the file opened for days? If you think it can't possibly need that much, let me give you this scenario: a computationally intensive process who takes days to compute all it's data. Big data. It uses a file as a "cache/buffer". It writes to it and reads from it continuously. As such it holds the file open all this time.
Would you kill a process who waits on a lock? After how much time? 10 minutes? 1 hour? 10 hours? 5 days? What if the purpose of that process is to do a cleanup or smth after some other process has released the lock, be it if that other process held it for 1 second or for 2 weeks?
Related
Is there a way to listen or wait for a specific time (e.g. 11:30 am) every day. The only way I know how is to set a timer that checks for the current time every 60 seconds which I have actually implemented using a backgroundworker. But is there a way to just wait and listen for the specified time (similar to monitoring for directory changes) and then take some action?
Thanks in advance.
Typically, rather than having a program resident in memory waiting, you would setup a Scheduled Task for this (or a cron job on linux). The scheduled task will run the program at the appropriate time. The program can still check (validate) the expected time if needed, but it shouldn't just always sit in the background using up resources if it's only going to run once per day.
The scheduled task is also better because it will recover automatically from computer reboots, crashes, etc. If something happens that interrupts your program's normal running, the scheduled task will still be able to run.
This is especially important in the .Net world, because .Net requires you to be very careful writing long-lived programs to avoid address space fragmentation. The .Net garbage collector is good at freeing up and returning old memory to the operating system, but over time your program's virtual address space can become fragmented and eventually you will not be able to allocate new memory any longer.
Even if this is part of a larger program, where there are also other things happening based on user interactions, it's still a good idea to split this off into a separate process.
I'm reading Google's Site Reliability Engineering book and ran across the word crashloop which I've never heard before and have not been able to locate a definition
"If a task tries to use more resources than it requested, Borg kills the task and restarts it (as a slowly crashlooping task is usually preferable to a task that hasn’t been restar‐ ted at all)."
What is a crashloop and how does it compare to an infinite loop if at all?
A crashloop is when a process crashes and is restarted by a watchdog daemon, indefinitely.
That is, the history is:
Process starts at time T.
Process crashes at time T+1.
Watchdog daemon restarts process.
Process started at time T+2.
Process crashes at time T+3.
Watchdog daemon restarts process.
Process starts...etc.
Here, the watchdog deamon is Borg, and the process is encapsulated into a task.
In general, in distributed computing if you want something to eventually succeed, you have to write down your intent for it to be completed and you need a worker to loop continually to act on this intent. This is "at least once delivery" of a work item.
Here, the intent is that the task runs (written down into Borg), and Borg itself is running the loop that is constantly trying to make sure the task runs. This is why when a task crashes, it is restarted. When a task crashes repeatedly, together you end up with a crashloop.
I setup a trigger that starts recording of CPU/thread/telemetry, sleep for a while, and write out a snapshot file, then stop recording of CPU/thread/telemetry with reset data.
It seems a snapshot file grows over time. Is this because of accumulated thread/telemetry data? If so, it will eventually eat up all the heap if it runs for very long like a year? Anyway to reset its data?
Thanks,
Takenori
I am a newbie to Cassandra - I have been searching for information related to commits and crash recovery in Cassandra on a single node. And, hoping someone can clarify the details.
I am testing Cassandra - so, set it up on a single node. I am using stresstool on datastax to insert millions of rows. What happens if there is an electrical failure or system shutdown? Will all the data that was in Cassandra's memory get written to disk upon Cassandra restart (I guess commitlog acts as intermediary)? How long is this process?
Thanks!
Cassandra's commit log gives Cassandra durable writes. When you write to Cassandra, the write is appended to the commit log before the write is acknowledged to the client. This means every write that the client receives a successful response for is guaranteed to be written to the commit log. The write is also made to the current memtable, which will eventually be written to disk as an SSTable when large enough. This could be a long time after the write is made.
However, the commit log is not immediately synced to disk for performance reasons. The default is periodic mode (set by the commitlog_sync param in cassandra.yaml) with a period of 10 seconds (set by commitlog_sync_period_in_ms in cassandra.yaml). This means the commit log is synced to disk every 10 seconds. With this behaviour you could lose up to 10 seconds of writes if the server loses power. If you had multiple nodes in your cluster and used a replication factor of greater than one you would need to lose power to multiple nodes within 10 seconds to lose any data.
If this risk window isn't acceptable, you can use batch mode for the commit log. This mode won't acknowledge writes to the client until the commit log has been synced to disk. The time window is set by commitlog_sync_batch_window_in_ms, default is 50 ms. This will significantly increase your write latency and probably decrease the throughput as well so only use this if the cost of losing a few acknowledged writes is high. It is especially important to store your commit log on a separate drive when using this mode.
In the event that your server loses power, on startup Cassandra replays the commit log to rebuild its memtable. This process will take seconds (possibly minutes) on very write heavy servers.
If you want to ensure that the data in the memtables is written to disk you can run 'nodetool flush' (this operates per node). This will create a new SSTable and delete the commit logs referring to data in the memtables flushed.
You are asking something like
What happen if there is a network failure at the time data is being loaded in Oracle using SQL*Loader ?
Or what happens Sqoop stops processing due to some condition while transferring data?
Simply whatever data is being transferred before electrical failure or system shutdown, it will remain the same.
Coming to second question, when ever the memtable runs out of space, i.e when the number of keys exceed certain limit (128 is default) or when it reaches the time duration (cluster clock), it is being stored into sstable, immutable space.
Microsoft's documentation states:
Background tasks are meant to be short-lived tasks that do not consume a lot of resources.
It also says:
Each app on the lock screen receives 2 seconds of CPU time every 15 minutes, which can be used by all of the background tasks of the app. At the end of 15 minutes, each app on the lock screen receives another 2 seconds of CPU time for use by its background tasks.
I need to run a background task every two minutes to update my live-tile.
My app is a lock-screen-app.
Computation is within the CPU and network usage constraints
Can I create a permanent background task (e.g. something which polls a web service and pulls information, waits and loops) to create a OneShot TimeTrigger every two minutes or is there a better way of doing this?
My concern with the background task option is whether the runtime would deem the task inactive while it was sleeping and close it or something else like there's a limit on the number of times a live tile can be updated within 15 minutes...
Yes, if by long lived you mean under 25 minutes.
Time triggers cannot execute more frequent than 15 minutes. Creating a OneShot trigger that executes in 2 minutes is, that's an interesting idea and should work. Yes, background tasks can register other background tasks to keep this chain going. Should the user's machine be off when it execs it will queue later.
Having said that, updating your tile that frequently & using a background task is not a wise solution. Because, it is unreliable. Background tasks can be disabled, for one. But every 15 minutes, you are going to exceed your quota. Try using a Scheduled tile instead.