I am sending query to apache drill from apache spark. I am getting the following error:
java.sql.SQLException: Failed to create prepared statement: PARSE
ERROR: Encountered "\"" at line 1, column 23.
When traced, I found I need to write a custom sql dialect. The problem I do not find any examples for pyspark. All the examples are for scala or java. Any help is highly appreciated.!
Here is the pyspark code :
`dataframe_mysql = spark.read.format("jdbc").option("url", "jdbc:drill:zk=ip:2181;schema=dfs").option("driver","org.apache.drill.jdbc.Driver").option("dbtable","dfs.`/user/titanic_data/test.csv`").load()`
Looks like you have used a double quote in your SQL query (please share your SQL).
By default Drill uses back tick for quoting identifiers - `
But you can change it by setting the system/session option (when you are already connected to Drill by JDBC for example) or you can specify it in JDBC connecting string. You can find more information here:
https://drill.apache.org/docs/lexical-structure/#identifier-quotes
I navigated to the drill web ui and updated the planner.parser.quoting_identifiers parameter to ". Then I edited my query as below:
dataframe_mysql = spark.read.format("jdbc").option("url", "jdbc:drill:zk=ip:2181;schema=dfs;").option("driver","org.apache.drill.jdbc.Driver").option("dbtable","dfs.\"/user/titanic_data/test.csv\"").load()
And it worked like charm!
Related
In BigQuery console I created a UDF function (language js) and now trying to call it from a saved query. I tried referencing the UDF with projectID.dataset.UDF_Name (same as I am using the for referencing vies/tables). When I click Run in UI I got an error:
"The project XXX has not enabled BigQuery"
I checked the BigQuery API and it says enabled.
When I only used dataset.UDF_Name for reference the query worked but I can save it as view getting another error: Bad routine reference "dataset.UDF_Name()"; routine references in standard SQL views require explicit project IDs
So clearly, the right approach is to use the projectID.dataset.UDF_Name() format but I can't figure out how to get rid of the "The project XXX has not enabled BigQuery" error.
Any help, much appreciated.
I've been trying to do a SQL Injection in BWAPP application level SQL Injection (GET/Search)
while setting the security level to medium and high
here is a screenshot of the level:
can anyone show me how to do an injection here..
try parameter (-) and carry on to find your payload
You should check the log.
In my case, the variables failed. My solution was to modify the file /var/www/html/bWAPP/sqli_1.php and add msqli on all lines containing mysql, and also changed the line: $recordset = mysqli_query ($link, $sql);
I hope it will help you
I am using the prepared statement like this
PreparedStatement pstmt = getConnection().prepareStatement(INSERT_QUERY);
pstmt.setInt(1,userDetails.getUsersId());
log.debug("SQL for inserting child transactions " + pstmt.toString());
I want to log the exact SQL statement after binding into the log file but this thing is not working. It is logging it something like SQLServerPreparedStatement:7. I searched on internet but did not get the satisfactory answer. Any help will be appreciated.
You can try to enable the JDBC internal logging by setting a PrintWriter on the DriverManager. Log4j2 provides an IO Streams module so you can include the output in your normal log file. Sample code:
PrintWriter logger = IoBuilder.forLogger(DriverManager.class)
.setLevel(Level.DEBUG)
.buildPrintWriter();
DriverManager.setLogWriter(logger);
In addition, individual JDBC drivers often provide proprietary logging mechanisms, usually enabled with a system property. You will need to consult the documentation for your specific driver for the details.
You can write a utility function which will replace the '?' with the bind variables.
I have an UPDATE pass through query saved in Access 2007. When I double-click on the pass through query it runs successfully. How can I get this query to run from VBA? I'd like it to run when my "splash screen" loads.
I'm currently using the following code:
CurrentDb.Execute "Q_UPDATE_PASSTHROUGH", dbSQLPassThrough
But I get the following message:
The pass-through query contains all the connection information and I've confirmed the SQL syntax is correct by running it multiple times, so not sure what I'm missing in my VBA call.
Use the QueryDef's Execute method:
CurrentDb.QueryDefs("Q_UPDATE_PASSTHROUGH").Execute
I don't think you should need to explicitly include the dbSQLPassThrough option here, but you can try like this if you want it:
CurrentDb.QueryDefs("Q_UPDATE_PASSTHROUGH").Execute dbSQLPassThrough
I recently ran into the same problem. While the above mentioned Execute method is working for most cases, some people (me included) experiencing a Run-time error '3001': Invalid Argument when using the parameter dbSQLPassThrough. This was also addressed in the answer above me and happens even in the simplest SQL-statements.
For those who are having the same problem, I recommend using the OpenQuery method as alternative.
A valid substitution for the following code
CurrentDb.QueryDefs("Q_UPDATE_PASSTHROUGH").Execute
would be
DoCmd.OpenQuery "Q_UPDATE_PASSTHROUGH"
I know this thread is 4 years old, however, searching for a solution for the not working Execute method on Google brings you directly to this thread which is why I thought it would be useful to add an alternative solution which solved this problem for me.
I confirm that the QueryDef's Execute method is the recommended way to achieve your goal.
CurrentDb.QueryDefs("Q_UPDATE_PASSTHROUGH").Execute
However, I can point out that in a similar case with Access 2010, using dbSQLPassThrough for the Options parameter caused a Run-time error '3001': Invalid Argument.
we've got a real confusing problem. We're trying to test an SQL Bulk Load using a little app we've written that passes in the datafile XML, the schema, and the SQL database connection string.
It's a very straight-forward app, here's the main part of the code:
SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class();
objBL.ConnectionString = "provider=sqloledb;Data Source=SERVER\\SERVER; Database=Main;User Id=Username;Password=password;";
objBL.BulkLoad = true;
objBL.CheckConstraints = true;
objBL.ErrorLogFile = "error.xml";
objBL.KeepIdentity = false;
objBL.Execute("schema.xml", "data.xml");
As you can see, it's very simple but we're getting the following error from the library we're passing this stuff to: Interop.SQLXMLBULKLOADLib.dll.
The message reads:
Failure: Attempted to read or write protected memory. This is often an indication that other memory has been corrupted
We have no idea what's causing it or what it even means.
Before this we first had an error because SQLXML4.0 wasn't installed, so that was easy to fix. Then there was an error because it couldn't connect to the database (wrong connection string) - fixed. Now there's this and we are just baffled.
Thanks for any help. We're really scratching our heads!
I am not familiar with this particular utility (Interop.SQLXMLBULKLOADLib.dll), but have you checked that your XML validates to its schema .xsd file? Perhaps the dll could have issues with loading the xml data file into memory structures if it is invalid?
I try to understand your problem ,but i have more doubt in that,
If u have time try access the below link ,i think it will definitely useful for you
link text
I know I did something that raised this error message once, but (as often happens) the problem ended up having nothing to do with the error message. Not much help, alas.
Some troubleshooting ideas: try to determine the actual SQL command being generated and submitted by the application to SQL Server (SQL Profiler should help here), and run it as "close" to the database as possible--from within SSMS, using SQLCMD, direct BCP call, whatever is appropriate. Detailing all tests you make and the results you get may help.