I have a report where I am working with event logs. I have created a table with fields that are extracted from the event logs.
This is my splunk query:
| stats count as Total_by_Requester values(*) as * by Requester_Id
| table Type_of_Call LOB DateTime_Stamp Policy_Number Requester_Id Last_Name State City Zip Total_by_Requester
| addcoltotals labelfield=Type_of_Call label="Grand Total" Total_by_Requester
Here I am taking the count of Requester_Id and then displaying it in Total_by_Requester field/column in the table and then doing the addcoltotals command to the get the total of Tota_by_Requester field
The issue that this query has is that it is displaying the Grand Total right underneath the Type_of_Call column, I want to display the Grand Total in its own column after the Total_by_Requester column
Picture of the issue
I have tried doing this query which brings the Grand Total to it's own column and has the right value but gets rid of all the other columns:
| stats count as Total_by_Requester values(*) as * by Requester_Id
| stats sum(Total_by_Requester) as Grand_Total
| table Type_of_Call LOB DateTime_Stamp Policy_Number Requester_Id Last_Name State City Zip Total_by_Requester Grand_Total
Issue in picture:
The labelfield option to addcoltotals tells the command where to put the added label. If the specified field name already exists then the label will go in that field, but if the value of the labelfield option is new then a new column will be created.
However, to create an entirely separate Grand_Total field, use the appendpipe command. The command applies a set of commands to the existing result set without triggering a new search.
| table Type_of_Call LOB DateTime_Stamp Policy_Number Requester_Id Last_Name State City Zip count
| appendpipe
[ stats sum(count) as Grand_Total ]
Related
I am working with event logs which contain many fields. I am trying to isolate 1 field and get a count of the value of that field and display the count in an existing table as a new field
This is my log:
LOG_LEVEL="INFO" MESSAGE="Type_of_Call = Sample Call LOB = F DateTime_Stamp = 2022-10-10T21:10:53.900129 Policy_Number = 12-AB-1234-5 Requester_Id = A1231301 Last_Name = SAMPLE State = IL City = Chicago Zip 12345" APPLICATION_VERSION="appVersion_IS_UNDEFINED"
This is my splunk query:
| stats count, values(*) as * by Requester_Id
| table Type_of_Call LOB DateTime_Stamp Policy_Number Requester_Id Last_Name State City Zip
The issue that this query has is that it is grouping the Requester Id field into 1 row and not displaying the count at all.
This is what the table and the issue look like :
What I want is that I need to make the rows unique and display the count of the Requester Id in a new field.
For example: if there are 2 logs with the same Requester_Id with value "abc", I would still display those two logs separately in a table because it would have other fields different such as the date and time but I would like to display the count of the Requester_Id as 2 in a new field in the same table.
updated picture of the total:
Add the count field to the table command.
To get the total count at the end, use the addcoltotals command.
| table Type_of_Call LOB DateTime_Stamp Policy_Number Requester_Id Last_Name State City Zip count
| addcoltotals labelfield=Type_of_Call label="Total Events" count
We are having to search through hundreds of alerts on a daily basis to test a new fraud system. The problem is we have to review every alert which is very time consuming.
The 1st query gets the list of alerts and the details for each.
The 2nd query takes the used ID, and search for 3 specific events which can be 0 to many.
If the are no records, then add "NULL" to user ID, "N" to the device ID, and $0.00 to the amount
The results of both queries need to be put into a table for extract.
index=mbank_p_database sourcetype=mbank_event EventTypeID=1095
| dedup OLBUserID
| table _time, SessionID, EventTypeID, OLBUserID, score, risk_rating, reason_code
| sort _time
index=mbank_p_database sourcetype=mbank_event EventTypeID=1000 OR EventTypeID=1011 OR EventTypeID=1012 OLBUserID=<Results from 1st query>
| table UDID, Amount
If there are no results in the second query for the User ID, then make "UDID"="N", and "Amount"=$0.00
``
Table layout of results of combined query:
--_time = Date\Timestamp (1st Query)
--SessionID = "SessionID" (1st Query)
--OLBUserID = "UserID" (1st Query)
--Deposit? = ("Y" or "N") (2nd Query)
--score = "Score" (1st Query)
--risk_rating = "Rating" (1st Query)
--reason_code = "Reason Code" (1st Query)
"" immediately brings to mind a subsearch, but that won't work in this case because the subsearch returns too many fields.
Try combining the two searches using stats. Something like this:
index=mbank_p_database sourcetype=mbank_event (EventTypeID=1095 OR EventTypeID=1000 OR EventTypeID=1011 OR EventTypeID=1012)
| stats values(*) as * by OLBUserID
| table _time, SessionID, EventTypeID, OLBUserID, score, risk_rating, reason_code
I eventually had to change the way the search works. First you have to search for all items that have "EventTypeID=1095" only. Then take those results, and add them to the sub-search. I added blank fields so the users could export it directly to a spreadsheet to do there research.
Final SPL:
index=mbank_p_database sourcetype=mbank_event EventTypeID=1095
| dedup OLBUserID
| join type=left SessionID
[ search index=mbank_p_database sourcetype=mbank_event EventTypeID=1000 OR EventTypeID=1011 OR EventTypeID=1012]
| eval "Deposit?"=case(Amount<=0.0000, "N", Amount>0.0000, "Y")
| table _time, SessionID, UDID, OLBUserID, "Deposit?", Amount, "Bank#", "Acct Type", "Acct#", "Fraud?", "Comments", score, risk_rating, reason_code
| sort -_time
Im very new to MS Access and I'm struggling a bit.
I have a table of postcodes that look like this:
+-----------+
| Postcode |
+-----------+
| Wa13 657 |
| eC2B 984 |
| eq8 987 |
+-----------+
And another table with 10 fields.
One of the fields is pickup address which has entries that look like this:
+------------------------------------------+
| pickup address |
+------------------------------------------+
| 69, example entry road, London, wa13 657 |
| 87, example entry road, London, eC2B 984 |
+------------------------------------------+
I'm looking to count the number of times that each postcode is spotted within the pickup address field.
The postcode has to be full, i.e. partial entries such as eC2B should not be counted.
So firstly I'd like to get a total count. If possible, secondly I'd like to vary this by date ranges, which are stored in separate fields in the second table.
Any help appreciated!
You could use a join with calculated join criteria:
select p.postcode, count(*)
from Postcodes p inner join Addresses a on instr(a.[pickup address],p.postcode) > 0
group by p.postcode
Change Postcodes to the name of the table containing your postcodes, and Addresses to the name of the table containing your pickup addresses (and change the postcode and pickup address field names as appropriate if necessary).
If you wish to further filter the results by date, simply add a WHERE clause:
select p.postcode, count(*)
from Postcodes p inner join Addresses a on instr(a.[pickup address],p.postcode) > 0
where a.date >= #2018-12-06#
group by p.postcode
(Change date to the name of your date field; you'll need to provide more information about the structure of your data if this date should reference another field).
Note that this type of query cannot be displayed by the MS Access Query Designer and can only be viewed as SQL.
Hello i am new to teradata. I am loading flat file into my TD DB using fast load.
My data set(CSV FILE) contains some issues like some of the rows in city column contains proper data but some of the rows contains NULL. The values of the city columns which contains NULL are stored into the next column which is zip code and so on. At the end some of the rows contains extra columns due to the extra NULL in rows. Examples is given below. How to resolve these kind of issues in fastload? Can someone answer this with SQL example?
City Zipcode country
xyz 12 Esp
abc 11 Ger
Null def(city's data) 12(zipcode's data) Por(country's data)
What about different approach. Instead of solving this in fast load, load your data to temporary table like DATABASENAME.CITIES_TMP with structure like below
City | zip_code | country | column4
xyz | 12 | Esp |
NULL | abc | 12 | Por
In next step create target table DATABASENAME.CITY with the structure
City | zip_code | country |
As a final step you need to run 2 INSERT queries:
INSERT INTO DATABASENAME.CITY (City, zip_code, country)
SELECT City, zip_code, country FROM DATABASENAME.CITIES_TMP
WHERE CITY not like 'NULL'/* WHERE CITY is not null - depends if null is a text value or just empty cell*/;
INSERT INTO DATABASENAME.CITY (City, zip_code, country)
SELECT Zip_code, country, column4 FROM DATABASENAME.CITIES_TMP
WHERE CITY like 'NULL' /* WHERE CITY is null - depends if null is a text value or just empty cell*/
Of course this will work if all your data looks exacly like in sample you provide.
This also will work only when you need to do this once in a while. If you need to load data few times a day it will be a litte cumbersome (not sure if I used proper word in this context) and then you should build some kind of ETL process with for example Talend tool.
I have a relatively large (as in >10^6 entries) table called "things" which represent locateable objects, e.g. countries, areas, cities, streets, etc. They are used as a tree of objects with a fixed depth, so the table structure looks like this:
id
name
type
continent_id
country_id
city_id
area_id
street_id
etc.
The association inside "things" is 1:n, i.e. a street or area always belongs to a defined city and country (not two or none); the column city_id for example contains the id of the "city" thing for all the objects which are inside that city. The "type" column contains the type of thing (Street, City, etc) as a string.
This table is referenced in another table "actions" as "thing_id". I am trying to generate a table of action location statistics showing the number of active and inactive actions a given location has. A simple JOIN like
SELECT count(nullif(actions.active, 1)) AS icount,
count(nullif(actions.active, 0)) AS acount,
things.name AS name, things.id AS thing_id, things.city_id AS city_id
FROM "actions"
LEFT JOIN things ON actions.thing_id = things.id
WHERE UPPER(substring(things.name, 1, 1)) = UPPER('A')
AND actions.datetime_at BETWEEN '2012-09-26 19:52:14' AND '2012-10-26 22:00:00'
GROUP BY things.name, things.id ORDER BY things.name
will give me a list of "things" (starting with 'A') which have actions associated with them and their active and inactive count like this:
icount | acount | name | thing_id | city_id
------------------------------------------------------------------
0 5 Brooklyn, New York City | 25 | 23
1 0 Manhattan, New York City | 24 | 23
3 2 New York City | 23 | 23
Now I would like to
only consider "city" things (that's easy: filter by type in "things"), and
in the active/inactive counts, use the sum of all actions happening in this city - regardless of whether the action is associated with the city itself or something inside the city (= having the same city_id). With the same dataset as above, the new query should result in
icount | acount | name | thing_id | city_id
------------------------------------------------------------------
4 7 New York City | 23 | 23
I do not need the thing_id in this table (since it would not be unique anyway), but since I do need the city's name (for display), it is probably just as easy to also output the ID, then I don't have to change as much in my code.
How would I have to modify the above query to achieve this? I'd like to avoid additional trips to the database, and advanced SQL features such as procedures, triggers, views and temporary tables, if possible.
I'm using Postgres 8.3 with Ruby 1.9.3 on Rails 3.0.14 (on Mac OS X 10.7.4).
Thank you! :)
You need to count actions for all things in the city in an independent subquery and then join to a limited set of things:
SELECT c.icount
,c.acount
,t.name
,t.id AS thing_id
,t.city_id
FROM (
SELECT t.city_id
,count(nullif(a.active, 1)) AS icount
,sum(a.active) AS acount
FROM things t
LEFT JOIN actions a ON a.thing_id = t.id
WHERE t.city_id = 23 -- to restrict results to one city
GROUP BY t.city_id
) c -- counts per city
JOIN things t USING (city_id)
WHERE t.name ILIKE 'A%'
AND t.datetime_at BETWEEN '2012-09-26 19:52:14'
AND '2012-10-26 22:00:00'
ORDER BY t.name, t.id;
I also simplified a number of other things in your query and used table aliases to make it easier to read.