I've found an interesting library libpg_query, which allows to parse any arbitrary Postgres SQL query into an internal parse tree, the way Postgres would actually understand the query. It acts offline, it's a rewrite of Postgres' source code. For some, it may be a positive thing, but for me it's not. You wouldn't actually know, which columns would be fetched in a starred expression (SELECT * FROM user), their types, etc.
And it doesn't compile for windows, haha.
But the Postgres client does compile, and this code is in there, but how could it be reached? Does Postgres have any command, which allows to parse queries and return the resulting tree? Kind of like EXPLAIN, but one parsing step earlier. I couldn't find one in the docs, and everyone else just redirects to the forementioned library.
You could create a temporary view from the query, then get the query tree from the pg_rewrite system table:
CREATE TEMPORARY VIEW _temp AS [your query];
SELECT ev_action FROM pg_rewrite WHERE rulename='_RETURN' AND ev_class='_temp'::regclass;
That returns a transformed pg_node_tree.
You can use sqlglot, a pure python SQL parser and transpiler that I wrote.
import sqlglot
sqlglot.parse(sql)
Related
I'm working in a project where I have been explicitly required to not use T-SQL syntax. The application we are using supports T-SQL but we are not allowed to use it to avoid potential migration issues.
My question is: is the SELECT ... INTO statement T-SQL or SQL? If it is T-SQL, is there a specific SQL query to copy an existing table into a new one? (I have tried with CREATE TABLE AS .. FROM but it doesn't work).
Sounds like a very basic question but I haven't been able to find the answer anywhere. Thus, in addition to the question above, it would be very helpful to know if there is a guide/dictionary/website that collects only the standard SQL syntax.
Thanks!
I think they recommend you to use ANSI SQL, instead of T-SQL (SQL Server) or PL-SQL (ORACLE). Considering it as common requirement, every database vendor provide their own way of implementing this requirement. When you use ANSI SQL, you will not have migration issues, when you move from one database vendor to another database vendor.
SQL SERVER
SELECT * INTO new_table
FROM existing_table
ORACLE & ANSI-SQL
CREATE TABLE new_table
AS SELECT * FROM existing_table
is SELECT INTO TSQL or SQL?
Neither. The MySQL documentation claims that SELECT INTO is a Sybase extension to standard sql. As such I don't think you can accurately say it's either of these, but you can say that it's neither. It is indeed used in T-SQL, as well as some other database vendor products, to create a table from a query. The SQL standard says that queries with that goal should be formed as CREATE TABLE blah AS SELECT .... Oracle/MySQL, for example, use the standard form though you can see them use SELECT INTO in a different context, to assign data to variables in stored procedures
If you want to avoid use of this non standard syntax when creating and populating a table then you'll have to:
CREATE TABLE blah (column spec to match query output)
INSERT blah (select query here)
But then you run into nuances like "sqlserver calls it datetime/datetime2 but oracle calls it date/timestamp"
And ultimately you'll probably get into a situation where you just can't use one form of sql to do all you want..
I'd imagine most libraries that do data access on multiple underlying databases have mechanisms to use vendor specific terminology where required
From the answers, it appears you might need to specify which SELECT INTO you're talking about. The other answers seem to suggest there exists some kind of SELECT ... INTO <table-name> when there is also a kind of SELECT ... INTO <local-variable-name list>. The latter is used in embedded SQL for making SQL interact with variables of the host language program. I'm not certain but that variant may also be used in the part of the SQL language that deals with procedures written in SQL (the SQL/PSM part of the standard).
A "reference" that covers "only the standard SQL syntax" is, in principle, the ISO standard document itself, only available by purchase from ISO (and yes, it's ISO not ANSI - ANSI does nothing more than rubberstamping the ISO document after removing all the names of non-US contributors). And not the easiest kind of literature. There are "draft" versions floating around on the internet that might deviate from the published final standards. E.g. http://www.wiscorp.com/sql200n.zip. Note that this is a SQL:2008 draft. Current standard version is SQL:2011. And it's several thousands of pages, so I guess that covers your question "Is all the syntax covered in w3schools standard SQL". (Hint : no)
I'm interested to know if anyone has come across a tool that can generate TypeScript type definitions based on the expected result of a SQL query? That is to say, is there a CLI that accepts a SQL schema and .sql file and outputs a .ts file based on the expected result of the query?
Such a tool already exists for GraphQL queries and my team has found it extremely useful because it completely removes errors associated with hand-rolled type definitions.
Yes, PgTyped is a new tool that does that.
It allows you to generate TypeScript interfaces for raw SQL queries.
It works similarly to apollo-codegen, but instead of the gql tag you need to use a sql tag for your SQL queries.
It only supports PostgreSQL and is still in beta stage, but I am actively working on it and any contributions are welcome.
sql-code-generator is another option.
It does:
generating type definitions from SQL resources (e.g., tables, views, functions, procedures)
generating type definitions from SQL queries (e.g., select * from table)
generating typed functions that execute SQL queries from SQL queries (e.g., const sqlQueryFindAllUsersByName = async ({ input: InputType }): Promise)
I have been given a requirement to create a pl/sql procedure which will accept select statement as an input parameter. All data must be fetched from the query and printed in DBMS_OUTPUT.
I've researched native dynamic SQL and DBMS_SQL but was unable to figure out how to fetch and process data from a table whose structure is unknown.
Since the table name will be provided during run time, i just want to know how to store the data fetched from the query because i cant define variables or collections since the structure of table is unknown
First off, the requirement seems incredibly dubious. You should never depend on data that is written to the DBMS_OUTPUT buffer-- it is entirely up to the client to enable a buffer, to ensure that the buffer is large enough, and to display the data from the buffer to the user. By default, none of that will happen. And writing a procedure to manipulate a table whose structure is completely unknown would be incredibly unusual.
If you are really determined, however, you would likely want to take Tom Kyte's SQL Unloader which uses DBMS_SQL to write data from an arbitrary query to a flat file and modify it to write it to DBMS_OUTPUT instead.
There is an open-source utility package for generating Excel readable files within PL/SQL.
https://code.google.com/p/plsql-utils/
However, I would recommend you look into using a more general purpose language for your tool if at all possible. PL/SQL can be incredibly useful for database logic, but for interacting with the outside world, I expect you will achieve a more maintainable solution using something like Python or Java.
Although, as always YMMV :-)
I'm new at working on an as400 and I have a query the joins across 4 tables. The query itself is fine, it runs in STRSQL and displays the results.
What I am in struggling with is getting the query to be able to run programmatically (it will eventually be run from a scheduled CL script).
I tried have creating a physical file that contains the query running it with RUNQRY, but it simply displays the query itself, not the actual result set.
Does anyone know what I am doing wrong?
UPDATE
Thanks everyone for the direction and the resources, with them I was able to reach my goal. In case it helps anyone, this is what I ended up doing (all of this was done in it's own library, ALLOCATE):
Created a source physical file (using CRTSRCPF): QSQLSRC, and created a member named SQLLEAGSEA, with the type of TXT, that contains the SQL statement.
Created another source physical file: QCLSRC, and created a member named POPLEAGSEA, with the type of CLP, that changes the current library to ALLOCATE then runs the query using RUNSQLSTM (more detail on this below). Here is the actual command:
RUNSQLSTM SRCFILE(QSQLSRC) SRCMBR(SQLLEAGSEA) COMMIT(*NONE) NAMING(*SYS)
Added the CLP to the scheduled jobs (using ADDJOBSCDE), running the following command:
CALL PGM(ALLOCATE/POPLEAGSEA)
With regard to RUNSQLSTM, my research indicated that I wasn't going to be able to use this function, because it didn't support SELECT statements. What I didn't indicate in my question was what I needed to do with the the result - I was going to be inserting the resultant data into another table (had I done that I'm sure the help could have figured that out a lot quicker). So effectively, I wasn't going to be doing an SELECT, my end result is actually an INSERT. So my SQL statement (in SQLLEAGSEA) begins with:
INSERT INTO
ALLOCATE/LEAGSEAS
SELECT
...
BLAH BLAH BLAH
...
From my research, I gather that RUNSQLSTM doesn't support SELECT because it doesn't have a mechanism to do anything with the results. Once I stopped taking baby steps and realized I needed to SELECT AND INSERT in the same statement, it solved my main problem.
Thanks again everyone!
The command is RUNSQLSTM to run a static SQL statement in a physical file member or stream file.
It is a non-interactive command so it will not execute sql statements that attempt to return a result set.
If you want more control, including the ability to run interactive statements, see the Qshell db2 utility.
For example:
QSH CMD('db2 -f /QSYS.LIB/MYLIB.LIB/MYSRCFILE.FILE/MYSQL.MBR')
Note that the db2 utility only accepts the *SQL naming convention.
QM Query
If all the SQL you need is the single complex SQL statement, and this is what it sounds like, then your best bet is to use Query Management Query (see QM Query manual here).
The results can be directed to a display, a spool file, or a physical file (ie a DB2 table). The default output when run interactively is to the screen, but when run in a (scheduled) batch job it will default to a spool file report.
You can create the QM Query interactively via WRKQMQRY, in prompted mode (much like Query/400) or in SQL mode. Or you can compile the QM Query from source, with the CRTQMQRY command.
To run your QM Query, STRQMQRY command.
RUNSQL cmd
If you are using a system that has IBM i 7.1 fully up-to-date, and has Technology Refresh 4 (TR4) installed, then you could also use the new RUNSQL command to execute a single statement. (see discussion in developerWorks)
SQL Scripting w/ RUNSQLSTM cmd
From CL you can run SQL scripts of multiple SQL statements from a source file member. There is no standard default source file name for this, but QSQLSRC is commonly used. The source member can contain multiple non-interactive SQL statements. This means you cannot use a SELECT statement (directly) since theoretically it will not know where to send the results. CL commands are even allowed if given a CL: prefix. Both SQL and CL statements should be terminated with a semicolon ;. While the SQL statements cannot display data directly to the screen, the same restriction does not apply to the scripted CL commands.
The STRQMQRY command can be embedded in the RUNSQLSTM script, by placing the prefix "CL: " in front of the command. Since STRQMQRY can direct output to the screen, a report, or an output table, this can come in very useful.
Remember that to direct your output from a SELECT query to a file you can use either the INSERT or CREATE TABLE statements.
CREATE TABLE newtbl AS
( full-select )
WITH DATA;
Or, to put the results into a table you create in your job's QTEMP library:
DECLARE GLOBAL TEMPORARY TABLE AS
( full-select )
WITH DATA;
[Note: If you create the source to be used by CRTQMQRY, you are advised to create it as CRTSRCPF yourlib/QQMQRYSRC RCDLEN(91), since the compiler will only use 79 columns of your source data (adding 12 for sequence and change date =91). However for QM Forms, which can be used to provide additional formatting, the CRTQMFORM compiler will use 81 columns so RCDLEN(93) is advised for QQMFORMSRC.]
RUNQRY is a utility that lets you execute a query that was created by another utility named WRKQRY. If you really want to process SQL statements held in a file try RUNSQLSTM. It uses a source physical file to store the statements, not a database file. The standard name for that source physical file is QQMQRYSRC. To create that file, CRTSRCPF yourlib/QQMQRYSRC. Then you can use PDM to work with that source PF. WRKMBRPDM yourlib/QQMQRYSRC. Use F6 to create a new source member. Make it source type TXT. Then use option 2 to will start an editor called SEU. Copy/paste your SQL statements into this editor. F3 to save the source. Once the source is saved, use RUNSQLSTM to execute it.
It is (now) possible to run SQL directly in a CL program without using QM Query, RUNSQLSTM or QShell.
Here is an article that discusses the RUNSQL statement in CL programs...
http://www.mcpressonline.com/cl/the-cl-corner-introducing-the-new-run-sql-command.html
The article contains information on what OS levels are supported as well as clear examples of several ways to use the RUNSQL statement.
This will work in two steps:
RUNSQL SQL('CREATE TABLE QTEMP/REPORT AS (SELECT +
EXTRACT_DATE , SYSTEM, ODLBNM, SUM( +
OBJSIZE_MB ) AS LIB_SIZE FROM +
ZSYSCOM/DISKRPTHST WHERE ODLBNM LIKE +
''SIS%'' GROUP BY EXTRACT_DATE, SYSTEM, +
ODLBNM ORDER BY LIB_SIZE DESC) WITH +
DATA') COMMIT(*NONE) DATFMT(*USA) DATSEP(/)
RUNQRY QRYFILE((QTEMP/REPORT)) OUTTYPE(*PRINTER) +
OUTFORM(*DETAIL) PRTDFN(*NO) PRTDEV(*PRINT)
The first step creates a temporary table result in qtemp and the second step/line runs an adhoc query over just the temporary table to a spool file.
Thanks,
Michael Frilot
There is of course a totally different solution: You could write and compile a program containing the statement. It requires some longer reading into, especially if you are new to the platform, but it should give you most flexibility over what you do with results. You can use SQL in C, C++, RPG, RPG/LE, REXX, PL (of which I don't know, what it is) and COBOL. Doing that, you can react in any processable way on results from one query and start/create other queries based on what you get.
Although some oldfashioned RPG-programmers try everything to deny SQL in RPG exists, it is possible today for many cases, to write RPG-programs with SQL only and no direct file access (without F-Specs, for those who know RPG).
If your solution works for you, perfect. If you need to do something else, try a look into this pdf: http://publib.boulder.ibm.com/infocenter/iseries/v5r3/topic/rzajp/rzajp.pdf
The integration into RPG is not too bad. It works with the normal program flow. Would look something like this (in free form):
/free
// init search values:
searchval = 'Someguy';
// so the sql query:
exec sql
SELECT colum1, colum2
INTO :var1, :var2
FROM somelib/somefile
WHERE keycol=:searchval;
// now do something with the values:
some_proc(var1);
/end-free
In this, var1, var2, and searchval are ordinary RPG-variables. No quoting needed. Works also with datastructures (externally defined e.g., the record format of the file itself fits well). You can work with cursors and loops, too, of course. I feel that RPG-programs tend to be easier to read with this.
The website i worked was recently attempted to be hacked by the following SQL injection script
boys' and 3=8 union
select 1,
concat(0x232425,ifnull(`table_name`,0x30),char(9),ifnull(`table_rows`,0x30), char(9),0x252423),
3,4,5,6,7,8,9
from `information_schema`.`tables`
where table_schema=0x62646B3032 limit 44,1 -- And '8'='8
This injection returned the mysql table name. This was reported by the error reporting system on that website and we managed to fix that part however I am not able to understand what does the above injection mean?
Anyone can explain this?
Penuel
They're using a select from the Information Schema views in mysql server :
http://dev.mysql.com/doc/refman/5.0/en/information-schema.html
They use some clever hacks to rout out simple sql injection prevention techniques.
According to this the MySQL concat()
Returns the string that results from
concatenating the arguments. May have
one or more arguments. If all
arguments are nonbinary strings, the
result is a nonbinary string. If the
arguments include any binary strings,
the result is a binary string. A
numeric argument is converted to its
equivalent binary string form
So 0x232425 is converted to #$% which is simply added to the begining and end of the table_name field. Maybe just to make it easier for them to pull out the Table names later using Regex.
Later on the char(9) is equivalent to a tab as you can see here and is just there to format the output nicer.
The 3,4,5,6,7,8,9 is just there so that the columns match the boys table that they are performing the Union on.
This injection returned the mysql table name.
Do you mean that your website displayed the table name when you gave it this input, or that the query returns that when run from the mysql client? If it showed on your website, then the attacker has the ability to inject much more harmful queries. Check your data.