Declare a value in Sparksql in Databricks - apache-spark-sql

I wanted to declare a fixed value before implementing various chunks of queries, and tried the following method. Unfortunately, no values are returned. Any advice will be much appreciated.
SET end_date = '2022-03-01';
—chunk 1
select * from p
where p.SEND_DATE between '2022-02-01' AND ${end_date}
—chunk 2
select * from p
where p.SEND_DATE between '2022-01-01' AND ${end_date}

You would use a widget:
CREATE WIDGET TEXT end_date DEFAULT '2022-03-01';
Then, you can use it in your query with the syntax you have in your question. If you want it to be a DateType instead of a string, then you can turn it into a date with to_date in your query.

Related

How to select just current month data in SQL?

I'm trying to make a query that makes me able to select just the data from the current month from a table, the date field that I'm using as reference is Datetime Type (e.g 2021-02-19 03:23:31), this field is called "Date". My try is this as follow:
Select *
FROM CORP.T_LOGI_RN_Planeacion_Entregas
Where Cast(CORP.T_LOGI_RN_Planeacion_Entregas.Date As Date) >= Cast(Month(getdate()) As Date)
It's not working, (im not pretty sure if I need to use "Cast" Actually) it throws me this error:
Cannot cast type int to date
But using something like this where I'm not specifying the Month, it works:
Select *
FROM CORP.T_LOGI_RN_Planeacion_Entregas
Where Cast(CORP.T_LOGI_RN_Planeacion_Entregas.Date As Date) >= Cast(getdate() As Date)
Any idea?
Thanks by the way and sorry if the solution is so obvious I'm little bit new on this, best regards.
Since you have not specified your DB Product but from your syntax It seems SQL Server. YOu can use MONTH function here -
Select *
FROM CORP.T_LOGI_RN_Planeacion_Entregas
Where MONTH(CORP.T_LOGI_RN_Planeacion_Entregas.Date) = MONTH(GETDATE())
AND YEAR(CORP.T_LOGI_RN_Planeacion_Entregas.Date) = YEAR(GETDATE());

Dynamic query using bigquery and data studio

I want to take out data for every date range in Data Studio without the need to change date range selectors in my BigQuery all the time. However, not sure if it is even possible to do so. The reasons I do this is to make sure that the queried data is only for 30 days, as later it do some kind of segmentation using that 30 days data.
Then I figured out that the Data Studio can use dynamic_date, however this way will never produce any datatable (datatable will be used to do other queries from it). Is it possible to do dynamic_date in BigQuery instead? like retrieving data from BigQuery using a date range not previously defined in the query.
From my point of view, code should be like :
SELECT
ID,
FROM `table`
WHERE DATE(Timestamp) between $DS_START_DATE and $DS_START_DATE + INTERVAL 30 DAY)
or
WHERE DATE(Timestamp) >= #DS_START_DATE
I believe in pure Bigquery you can use DECLARE clause for that purpose, defining variables of the specified type:
declare DS_START_DATE date default "2020-03-03";
declare DS_END_DATE date default "2020-03-04";
WITH sample AS (
SELECT '10001' AS id, cast('2020-03-01' AS timestamp) as date_id UNION ALL
SELECT '10002', cast('2020-03-02' AS timestamp) UNION ALL
SELECT '10003', cast('2020-03-03' AS timestamp) UNION ALL
SELECT '10004', cast('2020-03-04' AS timestamp) UNION ALL
SELECT '10005', cast('2020-03-05' AS timestamp) UNION ALL
SELECT '10006', cast('2020-03-06' AS timestamp)
)
select id, date_id from sample
where date(date_id) between DS_START_DATE and DS_END_DATE
Alternatively, you can take a look at parameterized queries, however as I mentioned in the comment, they are not supported in classic BigQuery web UI.

SQL statement between date

This is driving me crazy and not sure what I'm missing here..
so here is my data column looks like:
StartDateTime:
---------------
2012-01-17 11:13:46.530
2012-01-17 11:17:22.530
2012-02-17 11:31:22.223
here is my query trying to get:
select * from tablName
where convert(varchar(10), startDateTime, 101) between '2012-01-17' and '2012-01-17'
based on the above I should be getting TWO rows? but it does not, it return zero rows. what will be the correct way of doing?
PS:
I've looked at the MSDN site too:
Your query would only match dates that are between 2012-01-17 00:00:00 and 2012-01-17 00:00:00. So, the only matches would be when the date is exactly 2012-01-17 00:00:00.
Instead, I would do this:
declare #dateInput as DateTime
set #dateInput = '2012-01-17'
select *
from tablName
where startDateTime >= #dateInput
and startDateTime < dateadd(day, 1, #dateInput)
Note: SQL Server 2008+ has a new data type Date with no time component that can make these types of queries more readable.
There is now more information so I'll add a more appropriate answer.
The requirements are now a stored procedure passed a Date type parameter, not DateTime, and the desire is to return rows from a table based on criterion against a DateTime field named StartDateTime...
create procedure dbo.spGetEntriesForOneDay
#DesiredDate DateTime
as
SET NOCOUNT ON;
SET #DesiredDate = DATEADD(day, DATEDIFF(day, 0, #DesiredDate), 0)
SELECT Field1, Field2 -- see note 1
FROM dbo.TableName
WHERE StartDateTime >= #DesiredDate -- see note 2
AND StartDateTime < DATEADD(day, 1, #DesiredDate) -- see note 3
NOTE 1: Don't use * in production code, especially in a stored procedure. Besides being wasteful by returning columns you probably don't need and precluding the optimization of a covering index on a subset of the columns required you would need to recompile this stored procedure whenever the underlying table is altered in order to avoid unpredictable results.
NOTE 2: Avoid wrapping fields in functions. A field not wrapped in a function can potentially be matched by the optimizer to an index while a field wrapped in a function never will.
NOTE 3: #Martin Smith and #RedFilter are correct in that .997 precision assumes DateTime datatype forever; this approach is more future proof because is makes no assumptions of data type precision.
You're using a datetime field (I'm guessing).
Don't forget the time:
select * from tablName
where startDateTime between '2012-01-17' and '2012-01-17 23:59:59.997'
You can use the DateDiff function in the where clause. It would look like this:
select col1, col2 from tablName where DateDiff(day, startDateTime, #DesiredDate) = 0

Using a variable in an SQL query (Oracle DBMS)?

I am typing SQL queries to get some data from a software tool which is based on an Oracle database. I am using the typical SELECT-statement.
Now, in my SQL-query I am using at different places the date "02.05.2012". Is there a way to define a variable date_string at the beginning and then use at all relevant places this variable?
This would simplify things a lot. Thanks for all hints and tips!
You might try to rewrite your query to return the literal from an inline view ...
select
my_date,
...
from(
select to_date('02.05.2012','DD.MM.YYYY') my_date from dual),
table2
where
some_column <= my_date
What you look for is a bind variable.
select to-date(:date, 'dd.mm.yyyy') date1
, to-date(:date, 'dd.mm.yyyy') + 1 date2
from dual
On runtime you need to pass the value to the bind variable. It all depends on your programming language how to bind the variable, but there is plenty documentation for that.
DEFINE only works if you use sql*plus, and that's usually not the case inside a "software tool" :)
EDIT:
I'm beginning to understand now. It's just a textarea where you can enter a query and it will execute it and return the result. In that case you either write some complicated pl/sql code, or enter all the dates manually, or use a cross join with a select from dual:
with (select to_date('02.05.2012', 'dd.mm.yyyy') my_date from dual) d
select *
from some_table t
cross join d -- no ON required
If you want to select using the current date you can use sysdate.
Using SQLPLUS you can define your own variables:
SQL> define mydate ="01-05-2012"
SQL> select to_date('&mydate','DD-MM-YYYY') from dual;
01-MAY-12
try the following :
SELECT *
FROM table1
WHERE to_char(date1,'DD/MM/YYYY') = '&&date'
AND to_char(date2,'DD/MM/YYYY') = '&&date'
AND to_char(date3,'DD/MM/YYYY') = '&&date'
you will get a prompt to enter the value for the &&date , if you want to enter different values for each date, you should type &date instead of &&date
DEFINE is useful for your requirement.
DEFINE NAME="example"
access with &NAME

How do you update a DateTime field in T-SQL?

The following query does not update the datetime field:
update table
SET EndDate = '2009-05-25'
WHERE Id = 1
I also tried it with no dashes, but that does not work either.
When in doubt, be explicit about the data type conversion using CAST/CONVERT:
UPDATE TABLE
SET EndDate = CAST('2009-05-25' AS DATETIME)
WHERE Id = 1
Normally, it should work.
But can you try this? I don't have SQL on my home PC, I can't try myself
UPDATE table
SET EndDate = '2009-05-25 00:00:00.000'
WHERE Id = 1
The string literal is pased according to the current dateformat setting, see SET DATEFORMAT. One format which will always work is the '20090525' one.
Now, of course, you need to define 'does not work'. No records gets updated? Perhaps the Id=1 doesn't match any record...
If it says 'One record changed' then perhaps you need to show us how you verify...
Using a DateTime parameter is the best way.
However, if you still want to pass a DateTime as a string, then the CAST should not be necessary provided that a language agnostic format is used.
e.g.
Given a table created like :
create table t1 (id int, EndDate DATETIME)
insert t1 (id, EndDate) values (1, GETDATE())
The following should always work :
update t1 set EndDate = '20100525' where id = 1 -- YYYYMMDD is language agnostic
The following will work :
SET LANGUAGE us_english
update t1 set EndDate = '2010-05-25' where id = 1
However, this won't :
SET LANGUAGE british
update t1 set EndDate = '2010-05-25' where id = 1
This is because 'YYYY-MM-DD' is not a language agnostic format (from SQL server's point of view) .
The ISO 'YYYY-MM-DDThh:mm:ss' format is also language agnostic, and useful when you need to pass a non-zero time.
More info : http://karaszi.com/the-ultimate-guide-to-the-datetime-datatypes
UPDATE TABLE
SET EndDate = CAST('2017-12-31' AS DATE)
WHERE Id = '123'
If you aren't interested in specifying a time, you can also use the format 'DD/MM/YYYY', however I would stick to a Conversion method, and its relevant ISO format, as you really should avoid using default values.
Here's an example:
SET startDate = CONVERT(datetime,'2015-03-11T23:59:59.000',126)
WHERE custID = 'F24'
That should work, I'd put brackets around [Date] as it's a reserved keyword.
Is there maybe a trigger on the table setting it back?