I am working on an application, where the requirements is, I have to store EmailSchedules in a Database Table which contains some columns along with the EmailingDate. I have used Quartz.NET Scheduler to Schedule the Email saved in table. I created a service where there is a function name StartAsync() to Manage scheduling. I am calling the function from Startup.cs class. That means, when the application is run, then it call the function. Now the problem is, once the application is run, and then I insert a new schedule in table, the Quartz Schedular do trigger the changes as it do not know about the changes. That's why I am looking for something which will continuously monitor the table and if any changes occurred (Insert/Update/Delete) then it'll inform the Scheduler.
I tried with Worker Service but did not work.
Is there any that can help in this situation? Please help if you have any solution.
Related
I have a pretty complicated sql script which cleans up my database and will be run manually from time to time. Immediatelly after the script has run I have to start a recurring hangfire job which is usually executed only once per day.
In order not to forget to run the script I would like to trigger the execution directly from the sql script. Is there a simple way to modify one of the hangfire tables to achieve this task?
Hangfire core version is 1.7.5
UPDATE:
Unfortunately I have not found any hangfire table containing the recurring jobs. Therefore I guess they are not persisted at all.
Would it be possible to run a job by just inserting a new line in the HangFire.Job table? Or is this a bad idea?
Sorry for not following up this question here on stackoverflow. Due to the lack of a pure hangfire solution I have finally implemented some custom logic with the help of a separate "DatabaseState" table.
My DatabaseState table has the following fields: DateOfDbUpdate(datetime) and HangfireExecuted(bool). Whenever I run my complicated SQL-script I just insert a new entry in the table with the current time and HangfireExecuted = false.
In the code I have created a recurring hangfire job which checks every few seconds if there is a new entry in the DatabaseState table with HangfireExecuted == false. If it finds such an entry it will start the cleanup job.
Unfortunately this is not a very clean and direct solution. I would be happy if someone else comes up with a real hangfire solution.
How do I know when to invalidate the cache, if a table change is made from an outside source?
I have an api call that returns an employee table. The first time this call is made, I will cache the results so that on subsequent calls it will pull the data from the cache instead of the database. This makes sense, however, what happens if someone adds a new record to the employee table from outside of the api, how does the cache know that it is now invalid?
If the user made the change to the employee table through the API I can capture that, but we have a separate desktop app that doesn't use the API, and that app can directly make changes to the employee table. Is there any accepted standards for handling this?
The only possible solution I can think of is to add a trigger to the employee table, and somehow use that to know when a table has changed. But, we have over a thousand tables, and we are making an api call for each table - So, I do not think that adding a thousand triggers to our database is an acceptable solution.
Yes you could add a trigger as suggested. Or you could use a caching system that support expiry time/sliding expiry. So you would be serving up stale data some of the time but not always.
As the other answer a suggests your trigger idea is ok, however as you've stated that would be a lot of triggers.
If your cache is not local to the API, which i assume it isn't if triggers would be able to access. Could you not access it from your desktop application? You could invalidate your cache by removing the employee record from the cache with the desktop application when it makes a successful change to the employee table.
It boils down to..
You have a cache (which is essentially a read store).
You have two options to update it
- Either it times out and fetches (which is ok, if you dont need up to the minute real time data)
- Or is has to be told its data is no longer valid.
Two ways to solve this
Push model
Pull model
Push Model: Using a database trigger for SQL server table to populate an intermediate audit table and polling that using a background task.
Pull Model: Using CLR Trigger and pushing the updates to an API. Whenever DML happens the CLR trigger will call the Api, qhich in-turn can update the cache!
Hope this helps!
In a media management system my task is to create a workflow automation. Currently, i have created it using SQL Server triggers and the UI using ASP.NET with JQuery.
For Ex:
When a new file enters the system the trigger works and it will update the database metadata table with some data for that file.
Millions of assets get through the system. Is it ideal to have triggers to do this process.
Is there a better way to create this automation?
Is there a "best practice" to do this kind of works?
I'm having the same issue and data enters my central asset database on several ways (may differ from client to client).
So I also want to create an easily customizable workflow in the data layer (no other dependencies)
As the other people mention, triggers may affect the parent activity.
That is overcome by writing your action that should be performed away to a queue table.
Example Trigger. Hardware.Status = "Issue Work Order"
INSERT INTO Queue (Created, Task, Completed) VALUES (GETUTCDATE(),"EXEC dbo.IssueWorkOrder(123)",0);
The insert of a record into your queue table will reduce the problems as highlighted by other user comments.
The you build a scheduling tool (hangfire, sql tasks, or whatever), that execute tasks in the queue in the data order it wAS added.
Now, of course in practice it's not as simple as that. You will have to address the following:
What if the step fails2
Dependencies of previous steps to first have been completed
Multiple operators changing a record. (the deploy time between the job step being executed, and another person updating the same record.
I guess #2 and #3 is an issue with any workflow engine / pipleline. To address this a locking mechanism must be put in place.
I have two applications(web and a desktop app) that uses entity framework which use a common sql server database. They have unit of work pattern implemented and it keeps the context in the session or in the relevant thread. My question is how to update context of another application when one application updates something on the database ?
As an example let say the windows service has added some row to a table. How can the web application context get that one at the same time it is inserted.
Context in scenario of a web application should only last per the request. From what I see, you have to implement something as an event from database level as that seems to be the common place. This can be done using Triggers
In your scenario, you should perform following steps (just doing a drawing board scenario)
Add triggers at database level for each table, which will basically throw an event to the application layer.
Somehow extract those triggers into stored procedures, so that you can use with EF
Thereafter, implement a layer that sits on both the application whose primary responsibility is to notify the user of a change in the database by other application and then update the request by clicking a button(which in turn update the context). Basically the database level trigger, triggers something on the respective UI.
The meat of the work lies in the third point. You can achieve it in many ways. Alternatives are writing a service that polls another service (which accepts alerts from db trigger) for checking the modifications. so the logical separation could be like db --> service that accepts the change notification --> service that polls the notification service --> application
Above works logically and theoretically but hope it helps you out and I would be keen to know how you go about doing this.
I am currently working on a project with specific requirements. A brief overview of these are as follows:
Data is retrieved from external webservices
Data is stored in SQL 2005
Data is manipulated via a web GUI
The windows service that communicates with the web services has no coupling with our internal web UI, except via the database.
Communication with the web services needs to be both time-based, and triggered via user intervention on the web UI.
The current (pre-pre-production) model for web service communication triggering is via a database table that stores trigger requests generated from the manual intervention. I do not really want to have multiple trigger mechanisms, but would like to be able to populate the database table with triggers based upon the time of the call. As I see it there are two ways to accomplish this.
1) Adapt the trigger table to store two extra parameters. One being "Is this time-based or manually added?" and a nullable field to store the timing details (exact format to be determined). If it is a manaully created trigger, mark it as processed when the trigger has been fired, but not if it is a timed trigger.
or
2) Create a second windows service that creates the triggers on-the-fly at timed intervals.
The second option seems like a fudge to me, but the management of option 1 could easily turn into a programming nightmare (how do you know if the last poll of the table returned the event that needs to fire, and how do you then stop it re-triggering on the next poll)
I'd appreciate it if anyone could spare a few minutes to help me decide which route (one of these two, or possibly a third, unlisted one) to take.
Why not use a SQL Job instead of the Windows Service? You can encapsulate all of you db "trigger" code in Stored Procedures. Then your UI and SQL Job can call the same Stored Procedures and create the triggers the same way whether it's manually or at a time interval.
The way I see it is this.
You have a Windows Service, which is playing the role of a scheduler and in it there are some classes which simply call the webservices and put the data in your databases.
So, you can use these classes directly from the WebUI as well and import the data based on the WebUI trigger.
I don't like the idea of storing a user generated action as a flag (trigger) in the database where some service will poll it (at an interval which is not under the user's control) to execute that action.
You could even convert the whole code into an exe which you can then schedule using the Windows Scheduler. And call the same exe whenever the user triggers the action from the Web UI.
#Vaibhav
Unfortunately, the physical architecture of the solution will not allow any direct communication between the components, other than Web UI to Database, and database to service (which can then call out to the web services). I do, however, agree that re-use of the communication classes would be the ideal here - I just can't do it within the confines of our business*
*Isn't it always the way that a technically "better" solution is stymied by external factors?