We INSERT new records every day to a table with say id and created_on column.
How do i identify if records with a particular identifier existed every day in the last 7 days ?
This can be done with a Stored Procedure:
CREATE OR REPLACE PROCEDURE TIME_TRAVEL(QUERY TEXT, DAYS FLOAT)
RETURNS VARIANT LANGUAGE JAVASCRIPT AS
$$
function run_query(query, offset) {
try {
var sqlText = query.replace('"at"', " AT(OFFSET => " + (offset + 0) + ") ");
return (snowflake.execute({sqlText: sqlText})).next();
}
catch(e) { return false }
}
var days, result = [];
for (days = 0; days < DAYS; days++)
if (run_query(QUERY, -days * 86400)) result.push(days);
return result;
$$;
CALL TIME_TRAVEL('SELECT * FROM TASK_HISTORY "at" WHERE QUERY_ID = ''019024ef-002e-8f71-0000-05e10030a782''', 7);
For the time travel query replace to work, put in an "at" as a table alias.
The return value is an array of day offsets when the query returns any value.
This will only work beyond DAYS=2 if you have Snowflake Enterprise Edition.
I did it with the below query
select id, sum(present) as total_count
from
(select id,feed_date, count(1) as present
from catalog_rca
where feed_date between '2019-11-19' and '2019-11-25'
group by 1,2) as temp
group by 1 having total_count = 7;
Related
I created a stored procedure that sets the certain variable value and executes merge using this variable. Currently, it just returns a hardcoded message "Successfully executed.".
Is it possible to return the same result as the original merge query returns, like
number of rows inserted
number of rows updated
?
CREATE OR REPLACE PROCEDURE ALERTS_MERGE_PROCEDURE ()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
AS
$$
var sql_command = '
MERGE INTO tablename
..
WHEN MATCHED THEN
UPDATE SET ...
WHEN NOT MATCHED THEN
INSERT ...
);
'
snowflake.execute(
{
sqlText: sql_command
});
return "Successfully executed.";
$$;
You can iterate over the columns of the first row of the returned object from the execution:
create or replace temp table tablename as
select 1::int id, 'a'::string tx;
create or replace temp table tablesource as
select 1::int id, 'b'::string tx
union select 2, 'c';
CREATE OR REPLACE PROCEDURE ALERTS_MERGE_PROCEDURE ()
RETURNS STRING NOT NULL
LANGUAGE JAVASCRIPT
AS
$$
var sql_command = `
merge into tablename a
using tablesource b
on a.id = b.id
when matched then update set tx=b.tx
when not matched then insert (id, tx) values (b.id, b.tx);
`;
var x = snowflake.execute({sqlText: sql_command});
x.next();
var result = '';
for (i=1; i<=x.getColumnCount(); i++) {
result += x.getColumnName(i) + ': ' + x.getColumnValue(i) + '\n';
}
return result;
$$;
call alerts_merge_procedure();
Returns:
number of rows inserted: 1
number of rows updated: 1
Yes, look at the methods of the statement object. This is all documented here: https://docs.snowflake.com/en/sql-reference/stored-procedures-api.html#object-statement
I have a query that is select from an s3 bucket, but the value I need to select from changes each quarter - indicated below by {}. Is there anyway in snowflake I can write logic to be the most recent quarter
Select $1:date
from
'#lake.lake./s3key/{variable}/data.json.gzip' );
I would want to variable = 2021Q3, and then next quarter 2022Q1 ect
Is this possible? Or will I have to get python involved
I tried to use identifier for stages, but it does not seem to work.
I guess you can use Stored Procedure to achieve this.
create or replace procedure get_stage_data(quarter varchar)
returns string
language javascript
as
$$
var query = "Select $1::date from '#lake.lake./s3key/"+QUARTER+"/data.json.gzip";
var stmt = snowflake.createStatement({sqlText: query});
var res = stmt.execute();
var retVal = '';
while (res.next()) {
retVal += res.getColumnValue(1) + "\n"
}
return retVal;
$$;
Maybe write data to a table then you can analyze later.
We are currently pulling in data from SalesForce to SQL Database tables. There are 2 custom fields on different objects that were created for the Lead ID and a look up for which event/task is linked (this can be an account id, contact id, or lead id). Both of these are pulling over the 15 digit ID.
I am trying to find out if there is any SQL code or a SQL function that will allow me to convert that 15 digit to an 18 digit ID.
I need to have that 18 digit ID to join back to the other objects.
We have already tried using the CASESAFEID(Id) function in SalesForce, but with the API that was already set up and the visibility levels our particular ETL is not showing that field. Also, we would need to get a consultant to mess with the look up column.
I would like to take the 15 digit ID and convert it to the 18 digit code. If the SalesforceID is 0035000002tAzbu, how do I get the converted 18 digit value to be 0035000002tAzbuACC. I need to get that last 3 digits using SQL query or SQL function.
you could write a custom function in your sql database.
e.g. in snowflake, you can make a function like this
CREATE OR REPLACE FUNCTION dw.my_schema.f_sfdc_ch15_to_ch18("txt" string)
RETURNS string
LANGUAGE JAVASCRIPT
AS '
if ( txt == undefined || txt == "" || typeof txt == "undefined" || txt == null) {
return ;
} else {
var id15, id18;
if (txt.length == 18) {
return txt;
} else if (txt.length == 15) {
id15 = [txt.trim()];
} else {
return "";
}
for ( var x=0; x < id15.length; x++ ) {
var s = "";
for ( var i=0; i<3; i++) {
var f = 0;
for (var j=0; j<5; j++) {
var c = id15[x].charAt(i*5+j);
if (c>="A" && c<="Z") {
f+=1<<j;
}
}
s += "ABCDEFGHIJKLMNOPQRSTUVWXYZ012345".charAt(f);
}
id18 = id15[x]+s;
}
}
return id18.toString();
';
and use it like this
select dw.my_schema.f_sfdc_ch15_to_ch18(id15) from mytable;
This value can be computed. Check out this flowchart.
Source: https://stackoverflow.com/a/29299786/3135974
I have a user table with name, status and distance. I want to get the users under a particular distance and if the result is empty I want to check it again with an incremented value.
$user = User::where('status',1);
$i = 10;
while($i < 50)
{
$user->having('distance', '<=', $i);
if($user->get()->count())
break;
$i = $i+5;
};
In the first loop I got the query correctly as SELECT * from users where status = 1 and having distance <= 10. If the result is empty in the second loop I got the query as SELECT * from users where status = 1 and having distance <= 10 and having distance <= 15. I want to get the query as SELECT * from users where status = 1 and having distance <= 15. What change should I do for this?
You are working here on objects and you should use clone:
$user = User::where('status',1);
$i = 10;
while($i < 50)
{
$query = (clone $user)->having('distance', '<=', $i);
if($query->get()->count())
break;
$i = $i+5;
}
In addition you are using:
$query->get()->count()
what causes that you get all records matching query from database and calculate their amount. If you just need to have their count, it's much better to use just:
$query->count()
I want to compare dates in a TypoScript select.
Here's what I have (note that I commented the were clauses) :
lib.my_val = CONTENT
lib.my_val {
select {
pidInList = 100000
max = 1
#where = effective_date < CURDATE()
#where = TIMESTAMP(effective_date) < NOW()
orderBy = effective_date DESC
}
table = tx_my_table
renderObj = COA
renderObj {
5 = TEXT
5{
field = my_field
wrap = <span>|</span>
}
[...]
}
}
Which returns lines.
I tried to add a where statement any way I could with static dates or variables... without success. My understanding of the where clause is that everything after the = is dumped as is in the SQL query. But it seems I missed something.
Basically I want the TypoScript to generate a SQL Query smilar to this :
SELECT * FROM tx_my_table WHERE effective_date < NOW() ORDER BY effective_date DESC LIMIT 1;
This should be simple. Has anyone done this in the past?
Thanks!
Your TypoScript seems to be OK.
What happens if you enter the SQL Query directly into MySQL?
Note that with your code, only one record with pid=100000 is
selected.
Have you tried this:
--
lib.my_val {
select {
pidInList = 100000
max = 1
where = UNIX_TIMESTAMP(effective_date) < UNIX_TIMESTAMP()
orderBy = UNIX_TIMESTAMP(effective_date) DESC
}
table = tx_my_table
}
TYPO3 Wiki on select