I'm trying to use KEYS.KEYSET_CHAIN to obtain the a key from the KMS but I haven't figured out how to generate the first_level_keyset correctly. I'm getting the following message:
AEAD.ENCRYPT failed: Keyset deserialization failed: Error reading keyset data: Could not parse the input stream as a Keyset-proto.
I'm following the doc:
https://cloud.google.com/bigquery/docs/reference/standard-sql/aead_encryption_functions#keyskeyset_chain
My code:
DECLARE KMS_RESOURCE_NAME STRING;
DECLARE FIRST_LEVEL_KEYSET BYTES;
SET KMS_RESOURCE_NAME = 'gcp-kms://projects/XXX/locations/XXX/keyRings/XXX-key-ring/cryptoKeys/XXX-key';
SET FIRST_LEVEL_KEYSET = FROM_HEX("0a240... <suppressed>");
select AEAD.ENCRYPT(KEYS.KEYSET_CHAIN(KMS_RESOURCE_NAME, FIRST_LEVEL_KEYSET), b'string to encrypt', b'customer_id');
To generate the first_level_keyset, you will need to:
1.-Create a Key Management Service. Users need to have the cloudkms.cryptoKeyDecrypterViaDelegation role.
2.-Create a raw keyset.You need to execute the next command at cloud shell:
bq --project_id=my_project query --use_legacy_sql=false "SELECT KEYS.NEW_KEYSET('AEAD_AES_GCM_256') AS raw_keyset"
This command is going to return the raw keyset.
3.-Create and format a wrapped keyset.
echo "CK3245gBE... <suppressed>..." |base64 --decode > /tmp/decoded_key
gcloud kms encrypt --plaintext-file=/tmp/decoded_key --key=projects/my_project/locations/my_location/keyRings/my_keyring/cryptoKeys/my_key --ciphertext-file=/tmp/bankaccounts_wrapped
od -An --format=o1 /tmp/bankaccounts_wrapped |tr -d '\n'|tr ' ' '\'
This is going to return the array of bytes that you need to set at the first_level_keyset, such as:
DECLARE KMS_RESOURCE_NAME STRING;
DECLARE FIRST_LEVEL_KEYSET BYTES;
SET KMS_RESOURCE_NAME = 'gcp-kms://projects/my_project/locations/my_location/keyRings/my_keyring/cryptoKeys/my_key';
SET FIRST_LEVEL_KEYSET = b'\012\044\000\150\267\053\201 <suppressed>';
select AEAD.ENCRYPT(KEYS.KEYSET_CHAIN(KMS_RESOURCE_NAME, FIRST_LEVEL_KEYSET), b'string to encrypt', b'customer_id');
The result of this:
Related
I believe the problem is not specific to unloading data to S3, but how can we use variables in the names.
I open an S3 stage as following
CREATE OR REPLACE STAGE s3_stage
URL = 's3://some-foo-bar-url'
CREDENTIALS = (aws_role='arn:aws:iam::<some-number>:role/<some-foo-bar-role-with-relevant-permissions>')
;
Then I want to use it to unload my table data as
SET DATE_TODAY = TO_DATE(CURRENT_TIMESTAMP(2));
COPY INTO #crm.s3_stage/toProcess/{$TODAY_DATE}/restaurants
FROM (my_restaurants_table)
file_format = (FIELD_OPTIONALLY_ENCLOSED_BY='"')
include_query_id = TRUE
header = TRUE
max_file_size = 2000000
;
I want to $TODAY_DATE in S3 bucket's directory path
What should I change to make it work?
I believe you need to use a Stored Procedure / Snowflake Scripting, the below works
execute immediate
$$
declare
date_today varchar default TO_DATE(CURRENT_TIMESTAMP(2));
copy_into varchar default 'COPY INTO #<stage>/' || date_today || '/filename FROM (select * from <table> limit 10) overwrite = true';
begin
execute immediate :copy_into;
return date_today;
end;
$$;
Is there any way to pass a query to SQL*Plus through a variable?
I'm aware SQL*Plus has the capability to execute a file like:
sqlplus user/pass#db #filename
Within a kornshell script I'm trying to do:
query="select * from dual;"
sqlplus user/pass#db $query
There might have solution to do that BUT I can achieve the same goal using the following method.
[oracle#myserver Desktop]$ $ORACLE_HOME/bin/sqlplus -s jay/passsword#db <<!
select * from dual;
exit
!
D
-
X
Update, you can store the returned result in a variable as shown below.
query="select * from dual;"
var=$($ORACLE_HOME/bin/sqlplus -s jay/pass#db <<!
set pages 0
set head off
set feed off
$query
!
);
I want to pass shell variables to sql statement. Both shell script and SQL statement are present in the same script file.
I want the values of the variables retMonth, retLastDay and retPrvYear in the SQL statement.
Below is the code.
If I execute this, it prints - " partition_date between '01--' and '--' \ 0 0] 1 1] 12-DEC-14 1"
How can I have values of retMonth, retLastDay and retPrvYear in SQL statement?
echo $retMonth //This prints 07
echo $retLastDay //This prints 31
echo $retPrvYear //This prints 2015
count=$(sqlplus -s ${DBA_ORACLE_USER}/${DBA_ORACLE_PWORD}#${ORACLE_SID} <<END
#connect ${DBA_ORACLE_USER}/${DBA_ORACLE_PWORD}#${ORACLE_SID}
set serveroutput on
set linesize 1000
set heading off
set feedback off
define lastMonth=$retMonth
define lastYear=$retPrvYear
define lastDay=$retLastDay
SELECT count(1)
FROM MYTABLE
WHERE partition_date between '01-$lastMonth-$lastYear' and '$lastDay-$lastMonth-$lastYear'
);
END
)
Try using quoted shell variables directly without using define directives:
count=$(sqlplus -s "${DBA_ORACLE_USER}/${DBA_ORACLE_PWORD}#${ORACLE_SID}" <<END
set serveroutput on
set linesize 1000
set heading off
set feedback off
SELECT count(1)
FROM MYTABLE
WHERE partition_date between
"01-$retMonth-$retPrvYear" and "$retLastDay-$retMonth-$retPrvYear";
END
)
suppose below is my query.
DECLARE #INT INT
SET #INT = (SELECT COUNT(1) FROM MyTable)
IF (#INT = 0)
RAISERROR('Fail',16,1)
I want to use this query and create a SQL Server job using "Operating System (CmdExec)" type.
I want to SQL Job step to be successful if #INT <> 0 and fail if #INT = 0.
How can I write the cmd?
My try (and it doesn't work)...the server is different than where I will be setting up the job.
sqlcmd -S 123.45.67.890 -E -V15 -d MyDB -Q "DECLARE #INT INT
SET #INT = (SELECT COUNT(1) FROM MyTable)
IF (#INT = 0)
RAISERROR('Fail',16,1)" -b
Job failure msg.
Executed as user: LocalSVR\USER. Msg 105, Level 15, State 1, Server RemoteSVR, Line 1 Unclosed quotation mark after the character string 'DECLARE #INT INT '. Process Exit Code 15. The step failed.
Please note, the step fails for the same reason when MyTable has 1+ record(s).
Update: As per Scott's suggestion, I changed the formatting (removed line breaks) and it seems to work. #Scott, not sure how to mark your comment as answer. If you can write a response, I will mark it as answer. Cheers!!!
You're trying to execute multiple lines as opposed to a single query. You can get round that by using IF NOT EXISTS
SQLCMD -b -S"MyServer" -d"MyDatabase" -Q"IF NOT EXISTS(SELECT * FROM MyTable) RAISERROR('Fail',16,1)"
Note that the -b (On error batch abort) option means that SQLCMD itself will fail giving you a Trappable ErrorLevel
I have a select statement inside a bash script that returns the latest date in the DB. I run this query 4 times so I want to define it just once and assing the text to a variable.
#!/bin/bash
linux commands;
database_date=$(sqlplus -s/nolog $USER/$USER#BRMDPP <<END
set pagesize 0 feedback off verify off heading off echo off;
SELECT ...
exit;
END
)
commands that change the database date;
last_date=$(sqlplus -s/nolog $USER/$USER#BRMDPP <<END
set pagesize 0 feedback off verify off heading off echo off;
SELECT ...
exit;
END
)
commands that change the database date;
How can I store this big string $(sqlplus ... into one variable and use it again?
Thank you
One way would be to make use of a function:
foo() {
sqlplus -s/nolog $USER/$USER#BRMDPP <<END
set pagesize 0 feedback off verify off heading off echo off;
SELECT ...
exit;
END
}
and later invoke it by saying:
value=$(foo)
In order to get the value returned by the function, say echo "$value" (note that quoting variables is important).