Error: ('HY000', 'The driver did not supply an error!') - with string - pandas

I am trying to push a pandas DataFrame from python to Impala but am getting a very uninformative error. The code I am using looks as such:
cursor = connection.cursor()
cursor.fast_executemany = True
cursor.executemany(
f"INSERT INTO table({', '.join(df.columns.tolist())}) VALUES ({('?,' * len(df.columns))[:-1]})",
list(df.itertuples(index=False, name=None))
)
cursor.commit()
connection.close()
This works for the first 23 rows and then suddenly throws this error:
Error: ('HY000', 'The driver did not supply an error!')
This doesn't help me locate the issue at all. I've turned all Na values to None so there is compatibility, Unfortunatly I can't share the data.
Does anyone have any ideas/ leads as to how I would solve this. Thanks

Related

Pandas diff-function: NotImplementedError

When I use the diff function in my snippet:
for customer_id, cus in tqdm(df.groupby(['customer_ID'])):
# Get differences
diff_df1 = cus[num_features].diff(1, axis = 0).iloc[[-1]].values.astype(np.float32)
I get:
NotImplementedError
The exact same code did run without any error before (on Colab), whereas now I'm using an Azure DSVM via JupyterHub and I get this error.
I already found this
pandas pd.DataFrame.diff(axis=1) NotImplementationError
but the solution doesnt work for me as I dont have any Date types. Also I did upgrade pandas but it didnt change anything.
EDIT:
I have found that the error occurs when the datatype is 'int16' or 'int8'. Converting the dtypes to 'int64' solves it.
However I leave the question open in case someone can explain it or show a solution that works with int8/int16.

Sparklyr : sql temporary error : argument is not interpretable as logical

Hi I'm new to sparklyr and I'm essentially running a query to create a temporary object in spark.
The code is something like
ts_data<-tbl(sc,"db.table") %>% filter(condition) %>% compute("ts_data")
sc is my spark connection.
I have run the same code before and it works but now I get the following error.
Error in if (temporary) sql("TEMPORARY ") : argument is not
interpretable as logical
I have tried changing filters, tried it with new tables, R versions and snapshots. Yet it still gives the same exact error. I am positive there are no syntax errors
Can someone help me understand how to fix this?

I ran into the same problem. Changing compute("x") to compute(name = "x") fixed it for me.
This was a bug of sparklyr, and is fixed with version 1.7.0. So either use matching by argument (name = x) or update your sparklyr version.

Error in data frame creation in R in Spark using as.data.frame

I am trying to convert SparkDataFrame to R data frame.
%python
temp_df.createOrReplaceTempView("temp_df_r")
%r
temp_sql = sql("select * from temp_df_r")
temp_r = as.data.frame(temp_sql)
Error in as.data.frame.default(temp_sql) :
cannot coerce class ‘structure("SparkDataFrame", package = "SparkR")’ to a data.frame
Sometimes I get error, it's still unknown why I get error sometimes and sometimes not.

I need more details. What environment do you use?

pandas with pyodbc - Nan error: [42S22] ERROR: Attribute 'QNAN' not found (31) (SQLExecDirectW)

While running pyodbc execute() with data from pandas DataFrame (x1 is number type)
sql = "select * from t where x1=?"
cursor.execute(sql, df[row,'x1'])
got strange looking error:
('42S22', "[42S22] ERROR: Attribute 'QNAN' not found (31) (SQLExecDirectW)")
How to read the error message like that?

Took me a while to put pieces together.
Here is how to read the error:
pyodbc expects x1 number but got Nan and translates it to string value 'QNAN'
sql became "select * from t where x1=QNAN"
parsing sql request by ODBC bridge SQLExecDirect function returns error status 42S22
42S22 - ODBC bridge error code for "invalid column name"
Note, don't confuse pyodbc with ODBC bridge software. ODBC bridge is installed as part of your OS packages.

R-SQL Invalid value from generic function ‘fetch’, class “try-error”, expected “data.frame”

I am having a problem to fetch some data from database using ROracle. Everything works perfect (I am getting the data from different tables without any problem), but one of the tables throws an error:
library(ROracle)
con <- dbConnect(dbDriver("Oracle"),"xxx/x",username="user",password="pwd")
spalten<- dbListFields(con, name="xyz", schema = "x") # i still get the name of the columns for this table
rs <- dbSendQuery(con, "Select * From x.xyz") # no error
data <- fetch(rs) # this line throws an error
dbDisconnect(con)
Fehler in .valueClassTest(ans, "data.frame", "fetch") : invalid
value from generic function ‘fetch’, class “try-error”, expected
“data.frame”
I followed this question: on stackoverflow, and i selected the columns
rs <- dbSendQuery(con, "Select a From x.xyz")
but none of it worked and gave me the same error.
Any ideas what am I doing wrong?
P.S. I have checked the sql query in Oracle SQL Developer, and I do get the data table there
Update:
If anyone can help me to locate/query my Oracle error log, then perhaps I can find out what is actually happening on the database server with my troublesome query.

This is for debugging purposes only. Try running your code in the following tryCatch construct. It will display all warnings and errors which are happening.
result <- tryCatch({
con <- dbConnect(dbDriver("Oracle"),"xxx/x",username="user",password="pwd")
spalten <- dbListFields(con, name="xyz", schema = "x")
rs <- dbSendQuery(con, "Select * From x.xyz") # no error
data <- fetch(rs) # this line throws an error
dbDisconnect(con)
}, warning = function(war) {
print(paste("warning: ",war))
}, error = function(err) {
print(paste("error: ",err))
})
print(paste("result =",result))

I know I'm late to the game on this question, but I had a similar issue and discovered the problem: My query also ran fine in SQL Developer, but that only fetches 50 rows at a time. ROracle fetches the whole data set. So, an issue that appears later in the data set won't show immediately in SQL Developer. Once I pages through results in SQL Developer, it threw and error at a certain point because there was a problem with the actual value stored in the table. Not sure how an invalid value got there, but fixing it fixed the problem in ROracle.