I've started using a database at work that is based off SQL and Unix.
I am surprised to learn, that if someone requests for a change to be made to their details at around 5PM or a certain date, then the person who is allocated the incident then has to WAIT until 5pm and make the changes manually.
I'm surprised a button that says 'Apply changes later' does not exist, there is only a 'Save' button.
I have seen complicated solutions using Java on stackoverflow, but I am not familiar with UNIX or SQL, and googling brings no results.
Would it be a simple fix?
It wouldn't have to account for any time differences, and I'm assuming would just work off System clock; and I know Java has a calendar function that I assume works off the PC clock.
Java
Java does indeed have a sophisticated facility for scheduling a future task to be executed. See the ScheduledExecutorService class.
Rather than specify a date-time, you pass the schedule method a number of nanoseconds, or milliseconds, or seconds, or minutes, or hours, or days. You also pass a TimeUnit enum instance to indicate which granularity.
And, yes, Java depends on the host operating system for its clock to track the date-time.
Task Master
I suggest using your database to track the jobs to be run, in conjunction with Java. If using only Java, the scheduled jobs would exist only in memory and would disappear if the Java app exits or crashes.
Instead, the Java app on launch should check the database for any pending jobs, and schedule them with an executor. Each job on completion should mark the database "task master" table row as finished.
Related
I'm in a situation where I have a server running sql 2012 with roughly two hundred scheduled jobs (all are SSIS package executions). I'm facing a directive from management where I need to run some custom software to create a bug report ticket whenever a job fails. Right now I'm relying on half the jobs jobs notifying an operator on failure, while the other half do like a "go to step X- send failure email" for each step on failure, where "step X" is some sql that queries the DB and sends out an email saying which job failed at which step.
So what I'm looking for is some universal solution where I can have every job do the same thing when it fails (in this case, run some program that creates a bug tracking ticket). I am trying to avoid the situation where I manually go into every single job and add a new step at the end, with all previous steps changing to "go to step Y on failure" where step Y is this thing that creates the bug report.
My first thought was to create a new job that queries the execution history tables and looks for unhandled failures and then does the bug report creation itself. However, I already made the mistake of presenting this idea to the manager and was told it's not a viable solution because it's "reactive and not proactive" and also not creating tickets in real-time. I should know better than to brainstorm with non-programming management but it's too late, so that option is off the table and I haven't been able to uncover any other methods.
Any suggestions?
I'm proposing this as an answer, though it's not a technical solution. Present the possible solutions and let the manager decide:
Update all the Agent Jobs - This will take a lot of time and every job will need to be tested, which will also take a lot of time. I'd guess 2-8 weeks depending on how it's done.
Create an error handler job that monitors the logs and creates tickets based on those errors. This has two drawbacks - it is not "real-time" (as desired by the manager) and something will need to be put into place to insure errors are only reported once. This has the upside of being one change to manage. Also it can be made near real time if it were run on the minute.
A third option, which would be more a preliminary step, is to create an error report based off of the logs. This will help to understand the quantity and types of failures. This may help to shape the ultimate solution - do we want all these tickets, can they be broken up into different categories, do we want tickets for errors that are self-healing (i.e. connection errors which have built-in retries)?
Is there a way to listen or wait for a specific time (e.g. 11:30 am) every day. The only way I know how is to set a timer that checks for the current time every 60 seconds which I have actually implemented using a backgroundworker. But is there a way to just wait and listen for the specified time (similar to monitoring for directory changes) and then take some action?
Thanks in advance.
Typically, rather than having a program resident in memory waiting, you would setup a Scheduled Task for this (or a cron job on linux). The scheduled task will run the program at the appropriate time. The program can still check (validate) the expected time if needed, but it shouldn't just always sit in the background using up resources if it's only going to run once per day.
The scheduled task is also better because it will recover automatically from computer reboots, crashes, etc. If something happens that interrupts your program's normal running, the scheduled task will still be able to run.
This is especially important in the .Net world, because .Net requires you to be very careful writing long-lived programs to avoid address space fragmentation. The .Net garbage collector is good at freeing up and returning old memory to the operating system, but over time your program's virtual address space can become fragmented and eventually you will not be able to allocate new memory any longer.
Even if this is part of a larger program, where there are also other things happening based on user interactions, it's still a good idea to split this off into a separate process.
Time shift software like Time Machine enables you to test your application based on past or future time. Time Machine® guarantees no modification are needed in the database and the application.
I was just curious about how this software works internally. Does anyone have any thoughts about this?
These applications intercept the date and time calls of the application and return a date and time from a virtual clock.
Time Machine intercepts your file system’s date and time calls. If the
caller is configured with a virtual clock and the program is not on
the exclusion list, virtual time is returned; otherwise system time is
returned.
Source (for the mentioned Time Machine software): http://www.solution-soft.com/sites/default/files/wysiwyg/TMDataSheet.pdf
I actually wrote one from scratch for windows. I have to say it is an interesting endeavor, and the information is available on the web to write one from scratch.
The code I designed the intercept off of is out on codeproject.
The Three Functions I hooked were:
GetSystemTime
GetLocalTime
GetSystemTimeAsFileTime
These were the only ones I needed to capture to replace all time calls within .net.
The process isn't so much hard as it is difficult to maintain, especially if you are operating on a virtual system like Citrix.
I'm working on a distributed producer/consumer system using a messaging queue. The part that I'm interested in parallelising is the consumer side of it, and I'm happy with what I have for that.
However, I'm not sure what to do about the producer. I only need one producer running at a time since the load of the producing part of my system is not too high, but I want a reliable way of managing it, as in starting, stopping, restarting, and mainly, monitor it so that if the producer host fails another one can pick up.
If it helps, I'm happy with my consumer algorithm, the one that queues jobs, since it's fault tolerant to be down for a period of time and pick up the stuff that happened during the time it was down.
I'm sure there are tools or at least known patterns to do this and not reinvent the wheel.
I'm using rabbitmq but can use activemq, or even refactor into storm or something like that if needed, my code is not complex so far.
After a couple of weeks thinking about it, the simplest solution came to mind, and I'm actually very pleased with it, so I'll share it in case you find it useful, or point out if you think of any downsides, it seems to be working fine so far.
I have created the simplest table in my DB, called heartbeat, with a single timestamp field called ts, and is meant to have a single row all the time.
I start all of my potential producers every 5 minutes (quartz), and they do an update of the table if the ts fields is older than now() - 5 minutes. Because the update call is blocking, I'll have no db threading issues. Now, if the update returns > 0 it means that it actually modified the value of ts, and then I execute the actual producing code (queue jobs). If the update returns 0, it did not modify the table, because someone else did less than 5 minutes ago, and therefore this producer won't do anything until it checks again in 5 minutes.
Obviously the 5 minutes value is configurable, and this allows for a very neat upgrade with small changes to be able to execute several producers at the same time, if I ever had that need.
I wish to develop an application in VB.NET to provide to following functionality and hope you can give me some pointers on which direction to take.
I need some kind of “server” type component which sits in the background monitoring request from users and performing various task. (this component can be install locally or centrally)
The users submit an instruction to the “server” to perform a certain task at a designated date and time. (or perform the task straight away)
The “server” would perform the task at the desired date and time and inform the user the result of the task.
I have thought of using a central database to which the user writes the instructions. The “server” could read from the database to obtain the instructions, and write the result back to the database.
I want a fast reaction to the instructions, so the “server” must poll the database every few seconds; I fear this may be detrimental to performance. Also how do I get the server to perform the task at the desired time?
Again checking all outstanding tasks against the current time is not very efficient, so I thought about utilising the Windows Scheduler, but I am not sure of the best way of integrating this functionality.
I would be grateful for any ideas, pointers or suggestions.
Have you looked at quartz.net? It's a scheduling framework which might be useful to you.
We have a similar system where we work, utilising a webservice to accept requests, run them when required, and notify callers with the results if necessary.
In our case the callers were other applications and not people.
The web service consisted of the following methods: (rough version, not exact)
int AddJob(string jobType, string input, datetime startTime) // schedules job and sets timer to call StartJobs when needed, and then returns job id
void GetResults(int jobId, out string status, out string output) // gets results (status="queued / running / completed / failed")
void StartJobs() //called via a timer as needed to kick off scheduled jobs
We also built in checks to limit how many jobs of could run simultaneously, and whether they could retry if they failed, and emails admins if any jobs fail the last attempt.
Our version is much more comprehensive than this, with the jobs actually being webservices themselves, supporting simultaneous running, built-in workflow so jobs can wait on others, but maybe it will give you some ideas. It's not a trivial project, but was fun to implement!