I have splunk alerts set up. however when an application server is restarted, many log entries are created which trigger these alerts. I would like to either ignore these log entries or ignore the alerts when an application server is restarted.
Short of being able to do that, is there a way to annotate the splunk timeline? that way I could annotate the timeline and when people get alerted they can open the report and see that a server reboot occurred. Other tools with timelines allow this sort of annotation.
The best way to implement 'safe work times' is by using a lookup file.
use the date_day date_hour type of fields to set the safe time, and then use the servername as the lookup field to get the data in, then use a where clause to filter out safe times.
lookup file
host safe_begin safe_end
myHost 1900 2200
Query:
.... | where date_hour!>=safe_begin AND date_hour!<=safe_end
after that, set your alert accordingly.
Related
I want to create an automate alert for our data such as null, unequal amount, etc. It'll send to our email. The data stored in bigquery. How do i create this?
it automatically send to our email if the database have null value, empty value, unequal.
e.g
Look!
Table invoice have 0 rows detected. You can check from XX, and XY table.
or for the advance
Look!
Table invoice have 0 rows detected. You can check from XX, and XY table. This causes because XYZ.
The easiest way to resolve this request is to develop a custom solution on top of app engine or cloud functions, so that you can run your own queries and trigger email notifications.
This approach will not alert you in real-time but based on an schedule.
The other option if your use case is related to INSERT statements and real-time analysis is to take advantage of the logging alerts.
This will trigger an alert based on INSERT events, so you need to build your own query and setup the alert accordingly.
You can start with something like this:
resource.type="bigquery_resource"
protoPayload.methodName="jobservice.insert"
If you don't want to create a custom solution, it sounds like the Great Expectations framework could be a good fit for you.
This is a system for defining properties you expect of your data. It can be connected to BigQuery. It can be configured to send you emails if your tests ('expectations') fail.
You would still need to host the system yourself (they have a cloud solution coming in 2023).
If you want to get started fast and don't have easy access to hosting, you could also consider DBT (data build tool). DBT can connect to BigQuery and allows you to run tests (though this is not its main purpose). These tests can include checks for null data. You can write custom tests for more complex checks.
I mention it because they already have a paid for cloud solution, DBT Cloud, and yes, you can send email notifications if your tests fail. Depending on circumstances, this could be the fastest way to get what you want.
I have created a View in BigQuery and want to set it to update. When trying to save a query in a Schedule Query, an error occurs: Scheduled Query Error
This error is about permissions. You need these permissions to create a query scheduler with BigQuery:
bigquery.transfers.update or (bigquery.jobs.create and
bigquery.transfers.get )
bigquery.jobs.create
bigquery.datasets.update
Another option is adding this role roles/bigquery.admin , including all the permissions you need to schedule or modify a query.
You can read more information about permissions.
When I run a query, unless I manually specify the data location to run the query, BQ defaults to the US multi-region and uses resources in the US.
But then, The BQ Data Transfer Service throws an error since I are now trying to export the data from a query that was executed in US to a table that sits in EU.
So, to fix that, before you schedule the query, I have to go under the query detailed settings, and change the Data Location to EU
Then save these settings, and finish scheduling the query.
Before that, make sure you have enable the Billing
enter image description here
And then maybe because of BigQuery Data Transfer Service (trusted through the pop-up windows……)
bother me a lot……
If the pop-up window does not appear, you should check the address bar to make sure the window is not blocked, if it's not the cause,you can try to change the location of data(related to where you store your data),and submit.
enter image description here
I had the same problem and it turned out to be the pop-up blocker on my Firefox browser. The pop-up blocker on Chrome also stopped the scheduled query working but Chrome was a bit more visible that pop-ups were blocked.
On Firefox, click on the circled icon in the address bar to see blocked pop-ups
Change the pop-up permission to 'Allow'.
I had exactly this generic error message 'Scheduled query error', and it was driving me mad. I checked all permissions and locations etc.
I turns out that the 'Time Travel Window' was set to less than 48 hours for the dataset I was trying to write into. Which apparently BigQuery doesn't like.
To fix this I executed this query
ALTER SCHEMA `PROJECTNAME.DATASETNAME`
SET OPTIONS(
max_time_travel_hours = 72);
Problem: I created 10 AlwaysOn Availability Groups with SQL Server without a problem. Suddenly, it stopped worked and I kept getting this ONLY on the "create the listener" part:
Msg 19471, Level 16, State 0, Line 9
The WSFC cluster could not bring the Network Name resource with DNS name 'L_MyListener' online. The DNS name may have been taken or have a conflict with existing name services, or the WSFC cluster service may not be running or may be inaccessible. Use a different DNS name to resolve name conflicts, or check the WSFC cluster log for more information.
Sometimes I also got Msg 19476. This was all maddening because one moment I was creating listeners and availability groups, feeling like a guru, and then everything stopped and I lost hours of time.
So how do you solve this? Of course, Microsoft's own suggested text in the above error description was NOT helpful.
Apparently, each listener is really creating a mini "computer" in Active Directory if you look. And... here's the kicker, a domain user can only join a computer to a domain a limited number of times and that default is 10. Who would have thought that adding listeners equates with joining users to a domain!???! Microsoft really should have made this listener thing more intuitive, at least in their description text of possible problems.
Well, on your Domain Controller, open ADSI Edit, configure it the first time to look at your default naming context of your domain, like "DC=yourname..." with the CN= rows below that. Now, right click on the "DC=" line, choose Properties and navigate down to ms-DS-MachineAccountQuota and increase the limit from 10 to something else like 100.
You may need to run "GPUPDATE /FORCE" on the SQL Servers where you want to try again to add the listener. You may also have to clean up the mess it left (i.e. delete and restore the bad attempt at setting up your group and listener) before you try again.
With SQL Server 2016 supposedly going to require each database be in its own group, with its own listener, people will hit this limit of 10 quite easily!
I hope this helps you. If so, please mark this as the answer on the left. Of course, there are other reasons why people may get this error, as in the Microsoft error now but this whole post is for people who had it working just fine and then suddenly it stopped.
I have an requirement to execute macro-script kept on remote machine. DCOM Configuration setting is not set to allow communication between two computers. Even it's not possible to change the DCOM setting in my scenario.
Can I achieve the same requirement using Remote Desktop connection and then invoking macro script (through VBA code)?
I don't know exactly how to invoke script kept on remote machine using VBA?
Any idea/suggestions welcomed!!
This may not exactly answer your question, but may give you an idea.
I faced a the same problem with an application built in Access and the only way I could achieve this is by programming the macro I wanted to run into the application and then let the application periodically check a table on a share drive for "Go" command every few seconds.
On the share drive there was a table like this:
command_id | user | command
======================================
8 | "john.doe" | "update_me"
And all the applications were checking this table. If the app identified itself as user "john.doe", it decided to run the pre-programmed macro "update_me" and delete the row afterwards to prevent it from running again.
You can easily program more advanced stuff, like parameters, periodical execution etc.
It's not perfect, but it works quite nicely. Currently, I have the table on the SQL server and the command request takes cca 0.01s and I use it very often.
i have one windows form application in v s 2010.
i want to maintain a FLAG in Which there is two values of FLAG
1) yes and 2) no
i want that when a form is load a Flag value is yes and The form is close the flag value is no in my back end Database
It is same like sing in person on any web site.....
But my Problem is that how can i maintain it any how the my computer is turn off .......
i store a database on single computer that is server.but i also want a code for on same computer,also
I would do it the opposite way: when your application exits gracefully, write a certain value to your database. When you start your application and do not find this value, you know that your application exited unexpected, e.g. by an apprupt power loss of your computer.
I think what you want is keep a record of whether your application are online.So when your app start,it switch the flag in database to yes,when app close,it switch the flag to no,right?
But if the application close abnormally,it may not have the change to switch the flag,this is what we need to handle.
I suggest you attach a timestamp field with the flag in database,and if your app is running,update the timestamp in a fix period,like send a heart beat to database.
Actually we can remove the flag,just use the time stamp field.It store the last heart beat the app sent to the database.
Then we check this last heart beat time,if it's longer than the heart beat interval,we know the client application is closed,disconnected,or down for some reason.
It's not 100% accurate in all time,but it works.