I need to parametrize a SQL Query to use in JMETER such that every time it triggers, a random value is picked from the list of values to be used in IN clause.
Parent Query - Select * from Employee where Emp_Id in ( 3,9,11,12,13) and Dept_Name in('HR',IT','Admin','Audit')
Post the Parameterization when i trigger the Query through JDBC Request the request run for different user needs to have random selection made.
Ex:
Query 1 should be like - Select * from Employee where Emp_Id in ( 3,9) and Dept_Name in('HR',IT')
Query 2 should be like - Select * from Employee where Emp_Id in ( 11,12,13) and Dept_Name in('HR',IT','Admin')
I am trying to use CSV Data Set Config but not able to achieve the above output.
First of all I would recommend reconsidering your whole approach because tests needs to be repeatable, if you need to check all the possible combinations of the emp and dept IDs - go for pairwise testing, store the generated queries in the CSV file and parameterize the queries using CSV Data Set Config.
If you still want to make the number of arguments absolutely random you can go for the following approach:
Add User Defined Variables to your Test Plan and define the following variables there:
Emp_Id=3,9,11,12,13
Dept_Name=HR,IT,Admin
Amend your query to include JMeter's __groovy() function like:
Select * from Employee where Emp_Id in (${__groovy(def values = vars.get('Emp_Id').split('\,').collect(); values.shuffle(); values.take(org.apache.commons.lang3.RandomUtils.nextInt(1\,values.size())).join('\,'),)}) and Dept_Name in(${__groovy(def values = vars.get('Dept_Name').split('\,').collect{value -> "'" + value + "'"}; values.shuffle(); values.take(org.apache.commons.lang3.RandomUtils.nextInt(1\,values.size())).join('\,'),)})
Demo:
Related
I have a long query and I'm looking for a way to simplify it for the executer.
For example, I have this query:
select function_1(r_set) from (select collect_set(records) as r_set from (select function_2(<column_name>) as records from <table_name>) as record_t) as record_set_t;
function_1 and function_2 are custom UDFs.
Since everything besides the table name and column name are constant, Is it possible to define some kind of alias or procedure to a query, with column name and table name as a parameter?
or even wrap it somehow with shorter execution command?
I look for something like:
# set alias for long query somehow
set MyQueryAlias = select function_1(r_set) from (select collect_set(records) as r_set from (select function_2(<column_name>) as records from <table_name>) as record_t) as record_set_t;
# execute the query with table name and column name as a parameter
exec MyQueryAlias <table_name> <column_name>
My purpose is to make it easy for other users to use the saved query on different tables and columns.
I have some useful queries, I'd like to build a few more complex ones that needs them as sub queries. Can I call them by name ?
I'v seen the 'save view' option and was able to build new queries that used saved views.
Does this method refreshes the saved view each time a top query uses it, by re-executing the relevant queries ? or is it just a named query result, that I have to rerun each time to refresh ?
other suggestions to build queries in modular fashion ? For example when I change the days range I select from I want all subqueries to use the range.
In programming it's either using promoters or globals, how to do this in BigQuery ?
Whilst it is very diffcult to address your questions due to its broadness. I will answer them with general guidelines and examples for each doubt.
Regarding your first question, about subqueries and calling queries by an alias. I have 2 considerations about these:
1) You can use subqueries with WITH. So, you perform your transformations in the data, save it in a temporary table and reference it in the following (sub)query. Moreover, every time you run the code, all the queries will be executed. Below is an example,
WITH data as (
SELECT "Alice" AS name, 39 AS age, "San Francisco" AS city UNION ALL
SELECT "Marry" AS name, 35 AS age, "San Francisco" AS city UNION ALL
SELECT "Phill" AS name, 18 AS age, "Boston" AS city UNION ALL
SELECT "Robert" AS name, 10 AS age, "Tampa" AS city
),
greater_30 AS (
SELECT * FROM data
WHERE age > 30
),
SF_30 AS (
SELECT * FROM greater_30
WHERE city = "San Francisco"
)
SELECT * FROM SF_30
and the output,
Row name age city
1 Alice 39 San Francisco
2 Marry 35 San Francisco
2) Create a Stored Procedure: procedures are blocks of statements which can be called from other queries and also executed recursively ( call one procedure inside other). In order to create and store a procedure you have to specify the project and dataset where it will be saved. As well as, its name. Below is an example (using a BigQuery public dataset),
#creating the procedure
CREATE or replace PROCEDURE project_id.ataset.chicago_taxi(IN trip_sec INT64, IN price INT64)
BEGIN
CREATE TEMP TABLE taxi_rides AS
SELECT * FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips`
WHERE trip_seconds > trip_sec and fare >price
LIMIT 10000
;
END;
Now, you can call the procedure using CALL. As follows:
DECLARE trip_sec INT64 DEFAULT 30;
DECLARE price INT64 DEFAULT 30;
CALL `project_id.ataset.chicago_taxi`(trip_sec, price);
SELECT max(fare) AS max_fare,payment_type FROM taxi_rides
GROUP BY payment_type
And the output,
Row max_fare payment_type
1 463.45 Cash
2 200.65 Credit Card
Notice that the procedure is saved within the dataset. Then we use CALL to call it and use its output (a temporary table) in the next select statement. I must point that every time the procedure is invoked, it executes the query.
Regarding your question about saved views: the view is updated every time you run it. Please refer to the documentation.
Finally, about the last question using parameters and globals in queries: you can use scripting in BigQuery in order to DECLARE and SET a variable. So, you can take advantages when changing filter parameters for example. Below there is an usage example using a public a public dataset,
DECLARE time_s timestamp;
SET time_s= timestamp(DATETIME "2016-01-01 15:30:00");
SELECT * FROM `bigquery-public-data.chicago_taxi_trips.taxi_trips`
WHERE trip_start_timestamp > time_s
LIMIT 10000
Pay attention that every time the filter needs to be changed, it is possible to do it from the SET statement.
Note: If you have any specific question, please open another thread or you can ask me in the comment section.
So, I am working with redshift and SQL for the first time. I have run into this problem due to my limted knowledge of SQL.
I have this first query which return me tables with the columnA. (TableX, TableY... etc)
SELECT tablename
FROM PG_TABLE_DEF
WHERE ("column" = 'columnA');
Also I have this second query which returns me all the rows from the table containig certain value of columnA.
SELECT *
FROM TableX
WHERE columnA='123934'
What I want to achieve is take the result from the first query which is list of tables, and for each table run the second query i.e. get the rows with value of columnA=123934 for each table returned from first query.
What you want to achieve is done using dynamic SQL. Dynamic queries let you run a query from a string.
I am not an Redshit user but to generate the SQL string you need to run you could use the following query:
SELECT 'SELECT * FROM '||tablename ||' WHERE ColumnA= ''123934''; '
FROM PG_TABLE_DEF
WHERE ("column" = 'columnA');
You can try running it manually.
I am using the following SQL query in SSRS:
select s.name, s.studentid, x.section_number
from students s
inner join sections x
on s.studentid=x.studentid
where section_number = :Section
This query produces a list of 10-20 students depending on which 'section' (class) is entered into the :Section parameter. In addition to this, I also want to produce a picture of each student dynamically that matches up with the list.
I have every student's picture on a webserver, and if there's just one value for student then I could use a parameter for StudentID and set the image expression as follows:
="http://website.com/img/" & Parameters!StudentID.Value & ".jpg"
However, I need to have a picture for every student that is returned in the results. Is this possible in SSRS?
Yes it is certainly possible. The problem is you need to have those pictures in the database, in order to return them in the result-set.
You can write a CLR stored procedure that goes and fetches the images, then inserts them into a table in the database. Or a stored procedure that calls a windows/web service that does that.
I have a art-report that does that:
SELECT
[...]
,
ISNULL
(
T_File.Thumbnail
,
CAST
(
0xff[... default image byte-array too large for SO...]
AS varbinary(MAX)
)
) AS RPT_Thumbnail
FROM T_Art
By the way, you can get the image mime-type from the file's extension in SQL
Coming from SQL Server and a little bit of MySQL, I'm not sure how to proceed on google's BigQuery web browser query tool.
There doesn't appear to be any way to create, use or Set/Declare variables. How are folks working around this? Or perhaps I have missed something obvious in the instructions or the nature of BigQuery? Java API?
It is now possible to declare and set variables using SQL. For more information, see the documentation, but here is an example:
-- Declare a variable to hold names as an array.
DECLARE top_names ARRAY<STRING>;
-- Build an array of the top 100 names from the year 2017.
SET top_names = (
SELECT ARRAY_AGG(name ORDER BY number DESC LIMIT 100)
FROM `bigquery-public-data`.usa_names.usa_1910_current
WHERE year = 2017
);
-- Which names appear as words in Shakespeare's plays?
SELECT
name AS shakespeare_name
FROM UNNEST(top_names) AS name
WHERE name IN (
SELECT word
FROM `bigquery-public-data`.samples.shakespeare
);
There is currently no way to set/declare variables in BigQuery. If you need variables, you'll need to cut and paste them where you need them. Feel free to file this as a feature request here.
Its not elegant, and its a a pain, but...
The way we handle it is using a python script that replaces a "variable placeholder" in our query and than sending the amended query via the API.
I have opened a feature request asking for "Dynamic SQL" capabilities.
If you want to avoid BQ scripting, you can sometimes use an idiom which utilizes WITH and CROSS JOIN.
In the example below:
the events table contains some timestamped events
the reports table contain occasional aggregate values of the events
the goal is to write a query that only generates incremental (non-duplicate) aggregate rows
This is achieved by
introducing a state temp table that looks at a target table for aggregate results
to determine parameters (params) for the actual query
the params are CROSS JOINed with the actual query
allowing the param row's columns to be used to constrain the query
this query will repeatably return the same results
until the results themselves are appended to the reports table
WTIH state AS (
SELECT
-- what was the newest report's ending time?
COALESCE(
SELECT MAX(report_end_ts) FROM `x.y.reports`,
TIMESTAMP("2019-01-01")
) AS latest_report_ts,
...
),
params AS (
SELECT
-- look for events since end of last report
latest_report_ts AS event_after_ts,
-- and go until now
CURRENT_TIMESTAMP() AS event_before_ts
)
SELECT
MIN(event_ts) AS report_begin_ts,
MAX(event_ts) AS report_end_ts
COUNT(1) AS event_count,
SUM(errors) AS error_total
FROM `x.y.events`
CROSS JOIN params
WHERE event_ts > event_after_ts
AND event_ts < event_before_ts
)
This approach is useful for bigquery scheduled queries.