How to use a function in select along with all the records in Sequalize? - sql

Here is a Sequalize query below which retrieves a transformed value based on the table column value.
courses.findAll({
attributes: [ [sequelize.fn('to_char', sequelize.col('session_date'), 'Day'), 'days']]
});
The above sequlaize query will return result equal to as followed SQL query.
select to_char(bs.session_date, 'Day') as days from courses bs;
Expected output:
I want the transformed value which is in attributes along with all records like below. I know we can mention all the column names in attributes array but it is a tedious job. Any shortcut similar to asterisk in SQL query.
select to_char(bs.session_date, 'Day') as days,* from courses bs;
I tried the below sequalize query but no luck.
courses.findAll({
attributes: [ [sequelize.fn('to_char', sequelize.col('session_date'), 'Day'), 'days'],'*']
});

The attributes option can be passed an object as well as an array of fields for finer tuning in situations like this. It's briefly addressed in the documentation.
courses.findAll({
attributes: {
include: [
[ sequelize.fn('to_char', sequelize.col('session_date'), 'Day'), 'days' ]
]
}
});
By using include we're adding fields to the courses.* selection. Likewise we can also include an exclude parameter in the attributes object which will remove fields from the courses.* selection.

There is one shortcut to achieve the asterisk kind of selection in Sequalize. Which can be done as follows...
// To get all the column names in an array
let attributes = Object.keys(yourModel.rawAttributes);
courses.findAll({
attributes: [...attributes ,
[sequelize.fn('to_char', sequelize.col('session_date'), 'Day'), 'days']]
});
This is a work around there may be a different option.

Related

Using Athena to get terminatingrule from rulegrouplist in AWS WAF logs

I followed these instructions to get my AWS WAF data into an Athena table.
I would like to query the data to find the latest requests with an action of BLOCK. This query works:
SELECT
from_unixtime(timestamp / 1000e0) AS date,
action,
httprequest.clientip AS ip,
httprequest.uri AS request,
httprequest.country as country,
terminatingruleid,
rulegrouplist
FROM waf_logs
WHERE action='BLOCK'
ORDER BY date DESC
LIMIT 100;
My issue is cleanly identifying the "terminatingrule" - the reason the request was blocked. As an example, a result has
terminatingrule = AWS-AWSManagedRulesCommonRuleSet
And
rulegrouplist = [
{
"nonterminatingmatchingrules": [],
"rulegroupid": "AWS#AWSManagedRulesAmazonIpReputationList",
"terminatingrule": "null",
"excludedrules": "null"
},
{
"nonterminatingmatchingrules": [],
"rulegroupid": "AWS#AWSManagedRulesKnownBadInputsRuleSet",
"terminatingrule": "null",
"excludedrules": "null"
},
{
"nonterminatingmatchingrules": [],
"rulegroupid": "AWS#AWSManagedRulesLinuxRuleSet",
"terminatingrule": "null",
"excludedrules": "null"
},
{
"nonterminatingmatchingrules": [],
"rulegroupid": "AWS#AWSManagedRulesCommonRuleSet",
"terminatingrule": {
"rulematchdetails": "null",
"action": "BLOCK",
"ruleid": "NoUserAgent_HEADER"
},
"excludedrules":"null"
}
]
The piece of data I would like separated into a column is rulegrouplist[terminatingrule].ruleid which has a value of NoUserAgent_HEADER
AWS provide useful information on querying nested Athena arrays, but I have been unable to get the result I want.
I have framed this as an AWS question but since Athena uses SQL queries, it's likely that anyone with good SQL skills could work this out.
It's not entirely clear to me exactly what you want, but I'm going to assume you are after the array element where terminatingrule is not "null" (I will also assume that if there are multiple you want the first).
The documentation you link to say that the type of the rulegrouplist column is array<string>. The reason why it is string and not a complex type is because there seems to be multiple different schemas for this column, one example being that the terminatingrule property is either the string "null", or a struct/object – something that can't be described using Athena's type system.
This is not a problem, however. When dealing with JSON there's a whole set of JSON functions that can be used. Here's one way to use json_extract combined with filter and element_at to remove array elements where the terminatingrule property is the string "null" and then pick the first of the remaining elements:
SELECT
element_at(
filter(
rulegrouplist,
rulegroup -> json_extract(rulegroup, '$.terminatingrule') <> CAST('null' AS JSON)
),
1
) AS first_non_null_terminatingrule
FROM waf_logs
WHERE action = 'BLOCK'
ORDER BY date DESC
You say you want the "latest", which to me is ambiguous and could mean both first non-null and last non-null element. The query above will return the first non-null element, and if you want the last you can change the second argument to element_at to -1 (Athena's array indexing starts from 1, and -1 is counting from the end).
To return the individual ruleid element of the json:
SELECT from_unixtime(timestamp / 1000e0) AS date, action, httprequest.clientip AS ip, httprequest.uri AS request, httprequest.country as country, terminatingruleid, json_extract(element_at(filter(rulegrouplist,rulegroup -> json_extract(rulegroup, '$.terminatingrule') <> CAST('null' AS JSON) ),1), '$.terminatingrule.ruleid') AS ruleid
FROM waf_logs
WHERE action='BLOCK'
ORDER BY date DESC
I had the same issue but the solution posted by Theo didn't work for me, even though the table was created according to the instructions linked to in the original post.
Here is what worked for me, which is basically the same as Theo's solution, but without the json conversion:
SELECT
from_unixtime(timestamp / 1000e0) AS date,
action,
httprequest.clientip AS ip,
httprequest.uri AS request,
httprequest.country as country,
terminatingruleid,
rulegrouplist,
element_at(filter(ruleGroupList, ruleGroup -> ruleGroup.terminatingRule IS NOT NULL),1).terminatingRule.ruleId AS ruleId
FROM waf_logs
WHERE action='BLOCK'
ORDER BY date DESC
LIMIT 100;

Select all Valid starting letters with Sequelize

I have a list of countries which will be separated by starting letter so for example when you click on 'A' it will make an API call to return all the countries beginning with 'A'.
However there are some letters that don't have any countries in our system, and these may change as we update out data.
I want to have a query that will let me know which letters do not have any countries that begin with them, so that I can disable them.
I can do this be running a findOne query for every single letter in the alphabet... but that is not neat or performant. Is there a way to get the data from a single query?
I am able to get the desired result by using a substring function within a distinct function.
const result = await Countries.findAll({
attributes: [
[
sequelize.fn(
'DISTINCT',
sequelize.fn('substring', sequelize.col('countryName'), 1, 1),
),
'letter',
],
],
group: [sequelize.fn('substring', sequelize.col('countryName'), 1, 1)],
raw: true,
})

How to add checkbox column in vue-tables-2?

I have list of users table created in vue-tables-2. I want to add checkbox column before each row to select multiple row and before header to select all row like gmail.
can anyone help me out ?
it is very simple.
you should add a column to your vuetables column, just this.
imagine you are defining your tables column in fileds variable.ok ?
i mean something like below,just put this code inside your table coulmn:
fields: [
{
name: '__checkbox',
titleClass: 'center aligned',
dataClass: 'center aligned'
},
//...
]
if you wana get more information take look at this link:
https://github.com/ratiw/vuetable-2-tutorial/wiki/lesson-11

SQL Query to JSONiq Query

I want to convert an SQL query into a JSONiq Query, is there already an implementation for this, if not, what do I need to know to be able to create a program that can do this ?
I am not aware of an implementation, however, it is technically feasible and straightforward. JSONiq has 90% of its DNA coming from XQuery, which itself was partly designed by people involved in SQL as well.
From a data model perspective, a table is mapped to a collection and each row of the table is mapped to a flat JSON object, i.e., all fields are atomic values, like so:
{
"Name" : "Turing",
"First" : "Alan",
"Job" : "Inventor"
}
Then, the mapping is done by converting SELECT-FROM-WHERE queries to FLWOR expressions, which provide a superset of SQL's functionality.
For example:
SELECT Name, First
FROM people
WHERE Job = "Inventor"
Can be mapped to:
for $person in collection("people")
where $person.job eq "Inventor"
return project($person, ("Name", "First"))
More complicated queries can also be mapped quite straight-forwardly:
SELECT Name, COUNT(*)
FROM people
WHERE Job = "Inventor"
GROUP BY Name
HAVING COUNT(*) >= 2
to:
for $person in collection("people")
where $person.job eq "Inventor"
group by $name := $person.name
where count($person) ge 2
return {
name: $name,
count: count($person)
}
Actually, if for had been called from and return had been called select, and if these keywords were written uppercase, the syntax of JSONiq would be very similar to that of SQL: it's only cosmetics.

Summation of sub document field based on a group unexpected behaviour

*I have a document structured in which we embedded the salary Document into the Employee_Detail Document.As per the MongoDb documantation ,we can use $Unwind to deconstruct the Document and use aggregated pipeline...But its not working. i am using the below script...*
{
"_id" : ObjectId("5763d4a54da83b98f269878a"),
"First_Name" : "fgfg",
"Department" : "QA",
"Salary" : {
"HRA" : "1200",
"Basic" : "2000",
}
})
And i want to get sum of basic salary based on department Like
then Expected output is
Department Total_Basic**
QA 2000
I have used the following code to get the output. I have used the $unwind to deconstruct the document.and use aggregated pipeline to group the department(Sum of basic Salary).
db.Employee_Detail.aggregate([
{$unwind:"$Salary"}, {$group: {"_id": "$Department", total_Basic: {$sum: "$Salary.Basic" }
}}
])
But i get the below Result.
Department Total_Basic
QA 0
I think $unwind is not Working. Please suggest
Your main problem is the type of the field Basic is a string. Second, you do not need to use unwind unless the field Salary contains an array.
So perform an update to convert the types of Basic and HRA to floats, (see this stackoverflow question)
And then an aggregate operation like this will give you the desired result:
db.Employee_Detail.aggregate([
{$group: {"_id": "$Department", total_Basic: {$sum: "$Salary.Basic" }}
])