I'm stuck with some poor database design where I have to query tables that are named by date.
The following query works when the table names are hard coded with relevant dates.
SELECT
ajob.ORDER_ID
, ajob.JOB_NAME
, abim.SERVICE_ID
, shist.SERVICE_NAME
FROM
obscuredschema.A190129001_AJOB ajob --hardcoded YYMMDD table name
INNER JOIN obscuredschema.A190129001_ABIMSVC abim --hardcoded YYMMDD table name
ON (ajob.ORDER_ID = abim.ORDER_ID)
INNER JOIN obscuredschema.SERVICE_HIST shist
ON (abim.SERVICE_ID = shist.SERVICE_KEY)
WHERE shist.SERVICE_NAME LIKE '%BIM'
AND shist.BIM_AUTH_ID > 0
;
Noting the two hardcoded table names (along with aliases)
How would I execute this same query using dynamic table names? (There's two)
The code for the dynamic date: TO_CHAR(trunc(sysdate - 7), 'YYMMDD')
If the first table name were a string, here's how I would build it:
'A'||TO_CHAR(trunc(sysdate - 7), 'YYMMDD')||'001_AJOB'
If the second table name were a string, here's how I would build it:
'A'||TO_CHAR(trunc(sysdate - 7), 'YYMMDD')||'001_ABIMSVC'
I don't think you can write a plain SQL query with dynamic table names.
You can write a PL/SQL procedure which uses execute immediate and returns a cursor or something; somebody asked about that just yesterday. If you're just trying to write this query to interact with some data, that might be your best bet.
In addition, you could modify that by turning your PL/SQL procedure into a pipelined function, and then you could call it from a SQL query using TABLE().
If it were me, I'd consider creating a synonym (or a standard view which just selects from the dynamically-named-tables), and scheduling a job to re-create it every time new tables are created. That might be simpler than dealing with pipelined functions.
Related
How do I drop a few tables (e.g. 1 - 3) using the output of a SELECT statement for the table names? This is probably standard SQL, but specifically I'm using Apache Impala SQL accessed via Apache Zeppelin.
So I have a table called tables_to_drop with a single column called "table_name". This will have one to a few entries in it, each with the name of another temporary table that was generated as the result of other processes. As part of my cleanup I need to drop these temporary tables whose names are listed in the "tables_to_drop" table.
Conceptually I was thinking of an SQL command like:
DROP TABLE (SELECT table_name FROM tables_to_drop);
or:
WITH subquery1 AS (SELECT table_name FROM tables_to_drop) DROP TABLE * FROM subquery1;
Neither of these work (syntax errors). Any ideas please?
even in standard sql this is not possible to do it the way you showed.
in standard sql usually you can use dynamic sql which impala doesn't support.
however you can write an impala script and run it in impala shell but it's going to be complicated for such task, I would prepare the drop statement using select and run it manually if this is one-time thing:
select concat('DROP TABLE IF EXISTS ',table_name) dropstatements
from tables_to_drop
My SQL is a bit rusty, so I don't know whether the following is even possible:
I have multiple tables t_a, t_b, t_c with the same column layout and I want to apply the same operation to them, namely output some aggregation into another table. For a table t_x this would look like this:
CREATE TABLE t_x_aggregate (
<here the col definitions which are the same for all new tables t_[abc]_aggregate>
);
INSERT INTO t_x_aggregate(id, ...)
SELECT id, SUM(factor*amount)
FROM t_x
WHERE some fixed condition
GROUP BY id;
I now want to execute something like a FOR loop around this:
for t_x in t_a, t_b, t_c
CREATE TABLE ...
INSERT INTO ...
end for
Is this possible in SQL? Or would I need to build a wrapper in another language for this?
So, the result of that operation would be 3 new tables? T_A_AGGREGATE, T_B_AGGREGATE and T_C_AGGREGATE?
I think that the fastest way is to write 3 separate CREATE TABLE statements, e.g.
create table t_a_aggregate as
select id, sum(factor * amount) suma
from t_a
where some_condition
group by id;
create table t_b_aggregate as
select id, sum(factor * amount) suma
from t_b
where some_condition
group by id;
create table t_c_aggregate as
select id, sum(factor * amount) suma
from t_c
where some_condition
group by id;
OK; I understand that queries aren't that simple, but nothing much changes - only table names in CREATE and FROM (perhaps somewhere else, but that's more or less "it"). Any decent text editor's search/replace capabilities should be able to do it quickly.
If you want to do it dynamically in a loop (read: PL/SQL), you can - but dynamic SQL doesn't scale, is difficult to maintain, is painful to debug. Therefore, if you're doing it only once, consider running 3 separate statements.
How to do it dynamically?
You'd have to create a string (we usually put them into a locally declared variable) which contains the whole DDL statement. Why? Because you can't execute DDL from a PL/SQL otherwise.
If there are multiple tables and/or columns involved, you'll have to combine "fixed" parts of the statement (like create table, select, from, order by) concatenated with "dynamic" parts - such as column names. Note that in between you have to concatenate commas as separators. Pay attention to usage of multiple single quotes as you have to escape them (or use the q-quoting mechanism).
Also, for multiple columns you'll probably have to do it in a loop, concatenating each new column to previously composed string.
It (the statement stored into the varirable) is the executed by EXECUTE IMMEDIATE. If it is correctly written, it'll succeed. Otherwise, it'll fail, but it won't tell you why it failed (that's why I said difficult debugging").
So, instead of executing it, we usually display that string (using dbms_output.put_line) so that we see how it looks like and - using copy/paste - try to execute it.
Basically, it can be quite complex and - as I said - difficult to maintain and debug.
For the FOR loop you need to use PL/SQL like this:(*)
declare
type array_t is table of varchar2(10);
array array_t := array_t('a', 'b', 'c');
lo_stmt varchar2(2000);
begin
lo_stmt :=
'CREATE TABLE t_'||array(i)||'_aggregate ('||
' <here the col definitions which are the same for all new tables t_[abc]_aggregate>'||
');'||
''||
'INSERT INTO t_'||array(i)||'_aggregate(id, ...)'||
'SELECT id, SUM(factor*amount)'||
'FROM t_'||array(i)||
'WHERE some fixed condition'||
'GROUP BY id;'||
execute immediate lo_stmt;
end loop;
end;
/
Look also at this SO question: How to use Oracle PL/SQL to create...
(*) #Littlefoot describes in the 2nd part of his answer valuable background to this program.
We have a large amount of data in an Oracle 11g server. Most of the engineers use Tableau for visualizing data, but there is currently not a great solution for visualizing straight from the Oracle server because of the structure of the database. Unfortunately, this cannot be changed, as it's very deeply integrated with the rest of our systems. There is a "dictionary" table, let's call it tab_keys:
name | key
---------------
AB-7 | 19756
BG-0 | 76519
FY-10 | 79513
JB-2 | 18765
...
...
And there are also the tables actually containing the data. Each entry in tab_keys has a corresponding data table named by prefixing the key with an identifier, in this case, we'll use "dat_". So AB-7 will store all its data in a table called dat_19756. These keys are not known to the user, and are only used for tracking "behind the scenes". The user only knows the AB-7 moniker.
Tableau allows communication with Oracle servers using standard SQL select statements, but because the user doesn't know the key value, they cannot write a SQL statement to query the data.
Tableau recently added the ability for users to query Oracle Table Functions, so I started going down the road of writing a table function to query for the key, and return a table of the results for Tableau to use. The problem is that each dat_ table is basically unique with a different numbers of columns, labels, number of records, and datatypes from the next dat_ table.
What is the right way to handle this problem? Can I:
1) Write a function (which tableau can call inline in regular SQL) to return a bonified table name which is dynamically generated? I tried this:
create or replace FUNCTION TEST_FUNC
(
V_NAME IN VARCHAR2
) RETURN user_tables.table_name%type AS
V_KEY VARCHAR(100);
V_TABLE user_tables.table_name%type;
BEGIN
select KEY into V_KEY from my_schema.tab_keys where NAME = V_NAME;
V_TABLE := dbms_assert.sql_object_name('my_schema.dat_' || V_KEY);
RETURN V_TABLE;
END TEST_FUNC;
and then SELECT * from TABLE(TEST_FUNC('AB-7')); but I get:
ORA-22905: cannot access rows from a non-nested table item
22905. 00000 - "cannot access rows from a non-nested table item"
*Cause: attempt to access rows of an item whose type is not known at
parse time or that is not of a nested table type
*Action: use CAST to cast the item to a nested table type
I couldn't figure out a good way to CAST the table as the table type I needed. Could this be done in the function before returning?
2) Write a table function? Tableau can supposedly query these like tables, but then I run into the problem of dynamically generating types (which I understand isn't easy) but with the added complication of this needing to be used by multiple users simultaneously, so each user would need a data type generated for them each time they connect to a table (if I'm understanding this correctly).
I have to think I'm missing something simple. How do I cast the return of this query as this other table's datatype?
There is no simple way to have a single generic function return a dynamically configurable nested table. With other products you could use a Ref Cursor (which maps to ODBC or JDBC ResultSet object) but my understanding is Tableau does not support that option.
One thing you can do is generate views from your data dictionary. You can use this query to produce a one-off script.
select 'create or replace view "' || name || '" as select * from dat_' || key || ';'
from tab_keys;
The double-quotes are necessary because AB-7 is not a valid object name in Oracle, due to the dash.
This would allow your users to query their data like this:
select * from "AB-7";
Note they would have to use the double-quotes too.
Obviously, any time you inserted a row in tab_keys you'd need to create the required view. That could be done through a trigger.
You can build dynamic SQL in SQL using the open source program Method4:
select * from table(method4.dynamic_query(
q'[
select 'select * from dat_'||key
from tab_keys
where name = 'AB-7'
]'
));
A
-
1
The program combines Oracle Data Cartridge Interface with ANYDATASET to create a function that can return dynamic types.
There might be a way to further simplify the interface but I haven't figured it out yet. These Oracle Data Cartridge Interface functions are very picky and are not easy to repackage.
Here's the sample schema I used:
create table tab_keys(name varchar2(100), key varchar2(100));
insert into tab_keys
select 'AB-7' , '19756' from dual union all
select 'BG-0' , '76519' from dual union all
select 'FY-10', '79513' from dual union all
select 'JB-2' , '18765' from dual;
create table dat_19756 as select 1 a from dual;
create table dat_76519 as select 2 b from dual;
create table dat_79513 as select 3 c from dual;
create table dat_18765 as select 4 d from dual;
I have lots of experience with T-SQL (MS SQL Server).
There it is quite common to first select some set of records into a
table variable or say temp table t, and then work with this t
throughout the whole SP body using it just like a regular table
(for JOINS, sub-queries, etc.).
Now I am trying the same thing in Oracle but it's a pain.
I get errors all the way and it keeps saying
that it does not recognize my table (i.e. my table variable).
Error(28,7): PL/SQL: SQL Statement ignored
Error(30,28): PL/SQL: ORA-00942: table or view does not exist
I start thinking what at all is possible to do with this
table variable and what not (in the SP body) ?
I have this declaration:
TYPE V_CAMPAIGN_TYPE IS TABLE OF V_CAMPAIGN%ROWTYPE;
tc V_CAMPAIGN_TYPE;
What on Earth can I do with this tc now in my SP?!
This is what I am trying to do in the body of the SP.
UPDATE ( SELECT t1.STATUS_ID, t2.CAMPAIGN_ID
FROM V_CAMPAIGN t1
INNER JOIN tc t2 ON t1.CAMPAIGN_ID = t2.CAMPAIGN_ID
) z
SET z.STATUS_ID = 4;
V_CAMPAIGN is a DB view, tc is my table variable
Presumably you are trying to update a subset of the V_CAMPAIGN records.
While in SQLServer it may be useful to define a 'temporary' table containing the subset and then operate on that it isn't necessary in Oracle.
Simply update the table with the where clause you would have used to define the temp table.
E.g.
UPDATE v_campaign z
SET z.status_id = 4
WHERE z.column_name = 'a value'
AND z.status <> 4
I assume that the technique you are familiar with is to minimise the effect of read locks that are taken while selecting the data.
Oracle uses a different locking strategy so the technique is mostly unnecessary.
Echoing a comment above - tell us what you want to achieve in Oracle and you will get suggestions for the best way forward.
I am currently writing a VBA-based Excel add-in that's heavily based on a Jet database backend (I use the Office 2003 suite -- the problem would be the same with a more recent version of Office anyway).
During the initialization of my app, I create stored procedures that are defined in a text file. Those procedures are called by my app when needed.
Let me take a simple example to describe my issue: suppose that my app allows end-users to select the identifiers of orders for which they'd like details. Here's the table definition:
Table tblOrders: OrderID LONG, OrderDate DATE, (other fields)
The end-user may select one or more OrderIDs, displayed in a form - s/he just has to tick the checkbox of the relevant OrderIDs for which s/he'd like details (OrderDate, etc).
Because I don't know in advance how many OrderID s/he will select, I could dynamically create the SQL query in the VBA code by cascading WHERE clauses based on the choices made on the form:
SELECT * FROM tblOrders WHERE OrderID = 1 OR OrderID = 2 OR OrderID = 3
or, much simpler, by using the IN keyword:
SELECT * FROM tblOrders WHERE OrderID IN (1,2,3)
Now if I turn this simple query into a stored procedure so that I can dynamically pass list of OrderIDs I want to be displayed, how should I do? I already tried things like:
CREATE PROCEDURE spTest (#OrderList varchar) AS
SELECT * FROM tblOrders WHERE OrderID IN (#OrderList)
But this does not work (I was expecting that), because #OrderList is interpreted as a string (e.g. "1,2,3") and not as a list of long values. (I adapted from code found here: Passing a list/array to SQL Server stored procedure)
I'd like to avoid dealing with this issue via pure VBA code (i.e. dynamically assigning list of values to a query that is hardcoded in my application) as much as possible. I'd understand if ever this is not possible.
Any clue?
You can create the query-statement string dynamically. In SQL Server you can have a function whose return value is a TABLE, and invoke that function inline as if it were a table. Or in JET you could also create a kludge -- a temporary table (or persistent table that serves the function of a temporary table) that contains the values in your in-list, one per row, and join on that table. The query would thus be a two-step process: 1) populate temp table with INLIST values, then 2) execute the query joining on the temp table.
MYTEMPTABLE
autoincrementing id
QueryID [some value to identify the current query, perhaps a GUID]
myvalue one of the values in your in-list, string
select * from foo
inner join MYTEMPTABLE on foo.column = MYTEMPTABLE.myvalue and MYTEMPTABLE.QueryId = ?
[cannot recall if JET allows ANDs in INNER JOIN as SQL Server does --
if not, adjust syntax accordingly]
instead of
select * from foo where foo.column IN (... )
In this way you could have the same table handle multiple queries concurrently, because each query would have a unique identifier. You could delete the in-list rows after you're finished with them:
DELETE FROM MYTEMPTABLE where QueryID = ?
P.S. There would be several ways of handling data type issues for the join. You could cast the string value in MYTEMPTABLE as required, or you could have multiple columns in MYTEMPTABLE of varying datatypes, inserting into and joining on the correct column:
MYTEMPTABLE
id
queryid
mytextvalue
myintvalue
mymoneyvalue
etc