How can I perform a Query to my Database (using tOracleInput), like a Select, and use Strings that are already defined as parameters in other components, for example in a 'tFlowToIterate' ?
For example: "SELECT * from TABLE_X where FIELD_X= ? ;"
My '?' is the variable that comes from my tFlowToIterate component (foo). I already tried with (String)globalMap.get("foo"), and other similar forms...
Thanks
[Talend Open Studio for Data Integration v5.3.1;
DB: Oracle]
You answered by yourself. tOracleInput component accepts the query as parameter. This is a very boring java String, no more, no less. This means that if you want to use a globalMap element inside a query, you just need to do a java String concatenation. Something like that:
"SELECT * from TABLE_X where FIELD_X='" + (String)globalMap.get("foo") + "'"
but this won't work (look carefully at the quotes):
"SELECT * from TABLE_X where FIELD_X='(String)globalMap.get("foo")'"
Keep in mind that if you write a query using string concatenation and external vars, the query editor will probably going to mess all the quotes, generating a broken query.
As a general advice, I never suggest to use the "*" operator inside a database input component like tOracleInput. Talend has a fixed-scheme structure that is generated at compile time. This means that if one day you'll add a column to TABLE_X, your ETL will going to fail.
A more robust solution is the following:
Write down your query with the * operator
Click "Guess Schema" to retrieve the table schema and put in your component metadata
Now click "Guess Query" to explicitely rewrite your SELECT
Fix the query (ie. WHERE conditions,...) if needed
You just need to concatenate it with your variable.
So in your case it would look like:
"SELECT *
FROM TABLE_X
WHERE FIELD_X = '" + (String)globalMap.get("foo") + "'"
Related
Whenver I go to my pgAdmin query console and fire a query then I am not able to query the tables created by TypeOrm
But whenever I am adding a where clause it does not allow me to query
Can someone guide me why can't I query or what mistake am I making here?
You are mixing up " and '. You have to use " for identifiers (like tablenames, "User"), but normal strings have to be surrounded with ' ('Hey').
So the correct version of you statement is:
SELECT *
FROM "User"
WHERE name = 'Hey';
Small sidenote: It is mostly seen as bad practice to use upper-case identifiers, as you need to quote them with " always. Usually, the tablename would be user and you can refer to it using user or User.
From https://en.wikipedia.org/wiki/Code_injection#Preventing_problems
To prevent code injection problems, utilize secure input and output handling, such as:
Using APIs that, if used properly, are secure against all input characters. Parameterized queries (also known as "Compiled queries", "prepared statements", "bound variables") allows for moving user data out of string to be interpreted. Additionally Criteria API[7] and similar APIs move away from the concept of command strings to be created and interpreted.
I was wondering how and why "parameterized queries (also known as "Compiled queries", "prepared statements", "bound variables") allows for moving user data out of string to be interpreted" and prevent or mitigate code injection problems?
Can you also provide some examples in explanation?
Thanks.
Compiled queries use special syntax that the database understands. They usually add placeholders for parameters such as in:
select * from applicant where name = ?
select * from applicant where name = :name
The exact syntax depends on the specific technology: JDBC, ODBC, etc.
Now, once those queries are sent to the database (without the specific parameter values), the database "saves" them. Later on (usually in the same database session), you can run them many times, by just providing the parameter values each time.
SQL Injection Safety
They are also safe against SQL injection. For example, if in the previous query instead of a simple value such as Mary you used the value x'; delete from applicant; -- the database will work safely. It would run something like:
select * from applicant where name = 'x; delete from applicant; --'
This query won't probably find anything and will be safe.
If instead you didn't use compiled query, but just decided to concatenate the SQL as a string you would do something like:
String sql = "select * from applicant where name = '" + param1 + "'";
And would end up with the UNSAFE query:
select * from applicant where name = 'x'; delete from applicant; --
This one would run two queries. The second one will delete all the information from your table. Probably not what you want.
I'm reading sql code which has a line that looks like this:
SELECT INTO _user tag FROM login.user WHERE id = util.uuid_to_int(_user_id)::oid;
What exactly does this do? The usual way to use SELECT INTO requires specifying the columns to select after the SELECT token, e.g.
SELECT * INTO _my_new_table WHERE ...;
The database is postgresql.
This line must appear inside of a PL/pgSQL function. In that context the value from column tag is assigned to variable _user.
According to the documentation:
Tip: Note that this interpretation of SELECT with INTO is quite different from PostgreSQL's regular SELECT INTO command, wherein the INTO target is a newly created table.
and
The INTO clause can appear almost anywhere in the SQL command. Customarily it is written either just before or just after the list of select_expressions in a SELECT command, or at the end of the command for other command types. It is recommended that you follow this convention in case the PL/pgSQL parser becomes stricter in future versions.
I have a simple query but am running into problems using LIKE in VBA. My SQL string in VBA is:
stsql1 = "Select Top 25 data.* from data where data.Description Like ('*') "
When I run this sql string in my VBA code I get no records returned, but if I copy/paste the same string into a query in SQL View in MS Access, the query returns the values I expect. Is there a trick to using the "Like" syntax in VBA?
I can provide additional code and a small version of the database if that would help.
For SQL, the database engine will accept either single or double quotes as text delimiters. So either of these 2 WHERE clauses will work.
WHERE some_field Like '*'
WHERE some_field Like "*"
VBA however only accepts double quotes as text delimiters, so you would have to use the second form.
Two other points about your SELECT statement:
Select Top 25 data.* from data where data.Description Like ('*')
TOP [number] is arbitrary without an ORDER BY clause
You don't need parentheses surrounding your Like pattern ... you can use Like "*"
If your VBA code is using ADO with that SELECT statement, you must change the wild card character from * to % ...
WHERE data.Description Like '%'
In ADO/VBA, you have to use % instead of * as the wildcard. I ran into this a couple times in the past ....
Realize that there are at least 2 (yes two!) LIKE operators here.
One is the LIKE operator of VBA.
The other is the LIKE operator of the SQL of the database you are attached to.
The usual wildcards in SQL are % (for any # of any characters) and _ (for one of any character).
Know also that MS Access can open databases that aren't Access; it could be Microsoft SQL Server, or Oracle or IBM DB2. (BTW, the database that is normal for Access is called Microsoft JET.) You may be sheltered from that truth when you create a Query object in Access - in that circumstance, you are using JET SQL even when it's a linked table you are querying.
However, under VBA, when using either DAO or ADO, you're talking directly to whatever the database system happens to be, in which case you MUST use the SQL of that specific system.
OK, short answer: Use % like cularis said.
I can't add a comment, but I think it would be worth noting that you have to use % even if you are querying MS Access.
(example: Outlook VBA runs query on an Access database. The proper query is select * where user like '%bob%', even though this query would not work if plugged directly into an MS Access query).
I'm unsure how someone would break my SQL if I simply replace all incoming single quotes with double quotes. Can someone enlighten me for both Oracle and SQL Server examples? Thanks.
string sql1 = "select * from users where user_id = '" + "O'Reily".Replace("'", "''").Replace("\", "") + "'";
==> "select * from users where user_id = 'O''Reily'
string sql2 = "select * from users where user_id = '" + "O'''Reily".Replace("'", "''").Replace("\", "") + "'";
==> "select * from users where user_id = 'O''''''Reily"
UPDATE: the slash '\' is a restricted character in the application and will be stripped out before it is used in the query. A double dash can just as easily be added to this list of restricted characters.
Parameterize your variables. Seriously. All modern environments have facilities to do so and you don't have to worry about escape sequences like \' which will turn into \'' with your scheme (in Oracle) which becomes an escaped quote and a regular (terminating) quote.
There are plenty of other tricks to pull this off which I'm not enumerating as they aren't helpful.
Again: Parameterize your variables. Seriously. If you won't learn how to use the parameterization you will become another hacked statistic.
EDIT: Read the links in Paul's answer and here is another: http://unixwiz.net/techtips/sql-injection.html
No matter how clever you think your sanitation of strings is, you are doing it wrong. Especially if you have to handle multiple back ends.
Composing queries out of strings is one of the few things I will flat out fire people for... the risk such a programmer poses to the company is greater than just about anything else they bring to the table (especially after we make it very clear we won't accept such code on day one and provide an entity framework that makes such things unnecessary).
To prevent SQL injection, what you really should do is use bound positional or named parameters instead of constructing your SQL as a string with the user input inlined. How this is done depends on how your application accesses the database. For example, here is what it would look like in Java using JDBC:
Bad:
String updateString = "UPDATE COFFEES SET SALES = 75 " +
"WHERE COF_NAME LIKE 'Colombian'";
stmt.executeUpdate(updateString);
Good:
PreparedStatement updateSales = con.prepareStatement(
"UPDATE COFFEES SET SALES = ? WHERE COF_NAME LIKE ? ");
updateSales.setInt(1, 75);
updateSales.setString(2, "Colombian");
updateSales.executeUpdate():
I borrowed the example from here:
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html
Your proposed solution is vulnerable to the inclusion of the \' string, which would end your quoted section and allow the injection of other commands.
You want to use SQL prepared statements wherever possible, which should be everywhere. Basically, you write your sql with specific placeholders for your data, and then pass that data via a separate, non-interpreted channel to the sql server.
A few links:
http://dev.mysql.com/tech-resources/articles/4.1/prepared-statements.html
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html
http://mattbango.com/notebook/web-development/prepared-statements-in-php-and-mysqli/
There are some tricks involving the attacker taking advantage of the application truncating input :
http://www.rampant-books.com/t_super_sql_154_ideas_prevent_injection.htm