I am working on splunk dashboard. Below is the sample table and query
index="myindex" message="ApiImpl" "succeed=true"
| rex field=message "execution_time=(?.*)" | table method response_time | stats avg(response_time) as "avg", min(response_time) AS "min", max(response_time) As "max" by method
**method avg max min**
create 34 99 22
update 31 189 21
delete 30 69 29
Now on the result table if I click on 189 in update row. in the new or same window it should open me the same search along with method=update and response_time=189. Since the table is simple we can make out. But my table is very big when I click on particular cell it should open with the selected filter.
The new search result should open like below. Or should open log event directly with the update method which is taking max responce time 189
**method avg max min**
update 31 189 21
Could you please help me to provide a way like a query or table options to get the new search?
Related
I'm a newbie as far as Splunk is concerned with modest regex skills.
We have events with the following patterns:
fallbackAPIStatus={api1=133:..., api2=472:...,api3=498:...}
fallbackAPIStatus={api1=3535:...}
fallbackAPIStatus={api2=252:...,api3=655:...}
The numeric value indicates the response times and the ellipsis inidcates fields that I'm not interested in.
The number of apis within the braces is dynamic (between 1 and 4)
I want to be able to create a table as follows:
apiName TotalRequests Max-Response-Time Min-Response-Time
api1 2 3535 133
api2 2 472 252
api2 2 655 498
Here's my search:
index=my_logs sourcetype=my_sourcetype | rex field=_raw "fallbackAPIStatus=\{(?P<fallBackApis>[^\}]+)\}" | eval temp=split(fallBackApis,",") | rex field=temp "(?P<apiName>[a-zA-Z-]+)=(?P<responseTime>[0-9]+):"|stats count as TotalRequests max(responseTime) as Max-Response-Time min(responseTime) as Min-Response-Time by apiName
I'm able to get the TotalRequests right but I'm not able to get the correct max and min response times
Can someone advise what I'm doing wrong here?
I think there is an issue with your field extraction, the following works fine
| eval temp=split(fallBackApis,",") | rex field=temp "(?<apiName>\S+)=(?<responseTime>\d+):"
I have seen a similar question asked How to get second highest value among multiple columns in SQL ... however the solution won't work for Microsoft Access (Row_Number/Over Partition isn't valid in Access).
My Access query includes dozens of fields. I would like to create a new field/column that would return the second highest value of 10 specific columns that are included in the query, I will call this field "Cover". Something like this:
Product Bid1 Bid2 Bid3 Bid4 Cover
Watch 104 120 115 108 115
Shoe 65 78 79 76 18
Hat 20 22 19 20 20
I can do a really long SWITCH formula such as the following equivalent Excel formula:
IF( AND(Bid1> Bid2, Bid1 > Bid3, Bid1 > Bid4), Bid1,
AND(Bid2> Bid1, Bid2 > Bid3, Bid2 > Bid4), Bid2,
.....
But there must be a more efficient solution. A MAXIF equivalent would work perfectly if MS-Access Query had such a function.
Any ideas? Thank you in advance.
This would be easier if the data were laid out in a more normalized way. The clue is the numbered field names.
Your data is currently organized as a Pivot (known in Access as crosstab), but can easily be Unpivoted.
This data is much easier to work with if laid in a more normalized fashion which is this case would be:
Product Bid Amount
--------- ----- --------
Watch 1 104
Watch 2 120
Watch 3 115
Watch 4 108
Shoe 1 65
Shoe 2 78
Shoe 3 79
Shoe 4 76
Hat 1 20
Hat 2 22
Hat 3 19
Hat 4 20
This way querying becomes simpler.
It looks like you want the maximum of the bids, grouped by Product, so:
select Product, max(amount) as maxAmount
from myTable
group by product
Really, we shouldn't be storing text fields at all, so Product should be an ID number, with associated Product Names stored once in a separate table, instead of several times in the this one, like:
ProdID ProdName
-------- ----------
1 Watch
2 Shoe
3 Hat
... but that's another lesson.
Generally speaking repeating of anything should be avoided... that's pretty much the purpose of a database... but the links below will explain than I. :)
Quackit : Microsoft Access Tutorial
YouTube : DB Planning
Microsoft : Database Design Basics
Microsoft : Database Normalization Basics
Wikipedia : Database Normalization
I have a django web app with postgres db; the general operation is that every day I have an array of values that need to be stored in one of the tables.
There is no foreseeable need to query the values of the array but need to be able to plot the values for a specific day.
The problem is that this array is pretty big and if I were to store it in the db, I'd have 60 million rows per year but if I store each row as a blob object, I'd have 60 thousand rows per year.
Is is a good decision to use a blob object to reduce table size when you do not want to query with the row of values?
Here are the two options:
option1: keeping all
group(foreignkey)| parent(foreignkey) | pos(int) | length(int)
A | B | 232 | 45
A | B | 233 | 45
A | B | 234 | 45
A | B | 233 | 46
...
option2: collapsing the array into a blob:
group(fk)| parent(fk) | mean_len(float)| values(blob)
A | B | 45 |[(pos=232, len=45),...]
...
so I do NOT want to query pos or length but I want to query group or parent.
An example of read query that I'm talking about is:
SELECT * FROM "mytable"
LEFT OUTER JOIN "group"
ON ( "group"."id" = "grouptable"."id" )
ORDER BY "pos" DESC LIMIT 100
which is a typical django admin list_view page main query.
I tried loading the data and tried displaying the table in the django admin page without doing any complex query (just a read query).
When I get pass 1.5 millions rows, the admin page freezes. All it takes is a some count query on that table to cause the app to crash so I should definitely either keep the data as a blob or not keep it in the db at all and use the filesystem instead.
I want to emphasize that I've used django 1.8 as my test bench so this is not a postgres evaluation but rather a system evaluation with django admin and postgres.
Ruby on Rails' ActiveRecord's destroy_all method generates one SQL DELETE query per record. We have a page in a web application that allows a user to delete associated records. Since the deleted_at timestamps are off by a few microseconds, it becomes challenging to group all the records that were deleted at the same time. Here is some example data:
id | deleted_at
----+----------------------------
71 |
45 | 2014-04-29 18:35:00.676153
46 | 2014-04-29 18:35:00.685657
47 | 2014-04-30 21:11:00.73096
48 | 2014-04-30 21:11:00.738533
49 | 2014-04-30 21:11:00.745232
50 |
51 |
(8 rows)
So you can see there were two events here, one affecting 2 rows (with ids 45 and 46) and one affecting 3 rows (with ids 47, 48, and 49). My question is how can I query this table so that each event is grouped into a single row? I've considered using extract(microseconds from r.deleted_at) but that would fail if the seconds were wrapped. I want a query, or even a function that compares each record and groups those with deleted_at within a certain threshold of each other.
Edit: I should mention that I can't use delete_all because I want callbacks to be run. We are using the paranoia gem to soft-delete records.
I found a workaround. Instead of depending on ActiveRecord callbacks, I can simply set the timestamp manually using update_all. So instead of Model.where(...).delete_all I do Model.where(...).update_all(deleted_at: Time.current). Now I can easily group by timestamp.
If anyone wants to answer the original PostgreSQL query question, I'll be glad to choose the correct answer for points.
I am designing a system containing logical steps with some actions associated (but the actions are not part of the question, but they are crucial for each step in the list)!
The ting is that I need to create a way to define all the logical steps in an ordered way, so that I can get the list by query, and also make modifications later on!
Anyone with some experience in this kind of database design?
I have been thinking of having a column named wizard_steps (or something similar), and then use priority to make the order, but for some reason i feel that this design at some point will fail (due to items with same priority, adding new items would then have to rearrange the rest of the items, and so forth)!
Another design I have been thinking about is the use of "next item" as a column in the wizard_step column, but I don't feel this is the correct step eighter!
So to summarize; I am trying to make a list (and the design should be open enought to support multiple lists) of elements where the order is crucial!
Any ideas on how the database should look like?
Thanks!
EDIT: I found this yii component I will check out: http://www.yiiframework.com/extension/simpleworkflow/
Might be a good solution!
If I get you well, your main concern is to create a schema that supports ordered lists and can provide easy insert/reordering of items.
The following table design:
id_list item_priority foreign_itemdef_id
1 1 245
1 2 32
1 3 45
2 1 156
2 2 248
2 3 127
coupled to a table with item definition will be easily queried but will be difficult to maintain, especially for insertions
That one:
id_list first_item_id
1 45
2 38
coupled to the linked list:
item_id next_item foreign_itemdef_id
45 381 56
381 NULL 59
38 39 89
39 42 78
42 NULL 45
Will be both difficult to query and update (you should update the linked list inside a transaction, otherwise your linked list can get corrupted).
I would prefer the first solution for simplicity.
Depending on your update frequency, you may consider using large increments between item_priority to help insertion:
id_list item_priority foreign_itemdef_id
1 1000 245
1 2000 32
1 3000 45
2 1000 156
2 2000 248
2 3000 127
1 2500 46 -- late insertion
1 2750 47 -- late insertion
EDIT:
Here's a query that will hopefully make room for an insertion: it increments priority of all rows above the argument
$query_make_room_for_new_item = "UPDATE item_priority_table SET item_priority = item_priority + 1 WHERE item_priority > ". $new_item_position_priority ." AND id_list = ".$id_list;
Then insert your item with priority $new_item_position_priority