I'm trying to save a R dataframe back to a sql database with the following code:
channel <- odbcConnect("db")
sqlSave(db, new_data, '[mydb].[dbo].mytable', fast=T, rownames=F, append=TRUE)
However, this returns the error "table not found on channel", while simultaneously creating an empty table with column names. Rerunning the code returns the error "There is already an object named 'mytable' in the database". This continues in a loop - can someone spot the error?
Is this about what your data set looks like?
MemberNum x t.x T.cal m.x T.star h.x h.m.x e.trans e.spend
1 2.910165e+12 0 0 205 8.77 52 0 0 0.0449161
I've had this exact problem a few times. It has nothing to do with a table not being found on the channel. From my experience, sqlSave has trouble with dates and scientific notation. Try converting x to a factor:
new_data$x = as.factor(new_data$x)
and then sqlSave. If that doesn't work, try as.numeric and even as.character (even though this isn't the format that you want.
As a first shot try to run sqlTables(db) to check the tables in the db and their correct names.
You could then potentially use this functions return values as the input to sqlSave(...)
It seems you are trying to write to a SQL Server. If you specify the database name in the ODBC connection, and then refer to the table as "dbo.mytable" it might help.
I could do it changing the connection in the driver odbc. When you open it, you can do it for one db or in general for all dbs. When you opened it for one db, you will not have a problem with sqlSave().
Related
I know this topic has come up many times but still here I am. Data processing location seems consistent (dataset, US; query: US) and I am using backticks & long format in the FROM clause
Below are two sequences of code. The first one works perfectly:
SELECT station_id
FROM `bigquery-public-data.austin_bikeshare.bikeshare_stations`
Whereas the following returns an error message:
SELECT bikeshare_stations.station_id
FROM `bigquery-public-data.austin_bikeshare`
Not found: Dataset glassy-droplet-347618:bigquery-public-data was not found in location US
My question, thus, is why do the first lines of text work while the second doesn't?
You need to understand the different parts of the backticks:
bigquery-public-data is the name of the project;
austin_bikeshare is the name of the schema (aka dataset in BQ); and
bikeshare_stations is the name of the table/view.
Therefore, the shorter format you are looking for is: austin_bikeshare.bikeshare_stations (instead of bigquery-public-data.austin_bikeshare).
Using bigquery-public-data.austin_bikeshare means that you have a schema called bigquery-public-data that contains a table called austin_bikeshare , when this is not true.
I'm using npgsql as a nuget package in visual studio 2017 with visual basic.
Various commands do work very well but an ExecuteScalar allways returns 'nothing' although it should give a result.
The command looks like this:
Dim ser As Integer
Dim find = New NpgsqlCommand("SELECT serial from dbo.foreigncode WHERE code = '#code';", conn)
Dim fcode = New NpgsqlParameter("code", NpgsqlTypes.NpgsqlDbType.Varchar)
find.Parameters.Add(fcode)
find.Prepare()
fcode.Value = "XYZ"
ser = find.ExecuteScalar() ==> nothing
When the command string is copied as a value during debugging and pasted into the query tool of PGADMIN it delivers the correct result. The row is definitely there.
Different Commands executed with ExecuteNonQuery() work well, including ones performing UPDATE statements on the row in question.
When I look into the properties of the parameter fcode immediately before the ExecuteScalar it shows 'fcode.DataTypeName' caused an exception 'System.NotImplementedException'.
If I change my prepared statement to "SELECT #code" and set the value of the parameter to an arbitrary value just this value is returned. There is no access to the table taking place because the table name is not part of the SELECT in this case. If I remove the WHERE CLAUSE in the SELECT and just select one column, I would also expect that something has to be returned. But again it is nothing.
Yes there is a column named serial. It is of type bigint and can not contain NULL.
A Query shows that there is no single row that contains NULL in any column.
Latest findings:
I queried a different table where the search column and the result column happen to have the same datatype. It works, so syntax, passing of parameter, prepare etc. seems to work in principal.
The System.NotImplementedException in the DataTypeName property of the parameter occurs as well but it works anyway.
I rebuilt the index of the table in question. No change.
Still: when I copy/paste the CommandText and execute it in PGAdmin it shows the correct result.
Modifying the Command and using plain text there without parameter and without prepare still does yield nothing. The plain text CommandText was copy/pasted from PGAdmin where it was successfully executed before.
Very strange.
Reverting search column and result column also gives nothing as a result.
Please try these two alternatives and post back your results:
' Alternative 1: fetch the entire row, see what's returned
Dim dr = find.ExecuteReader()
While (dr.Read())
Console.Write("{0}\t{1} \n", dr[0], dr[1])
End While
' Alternative 2: Check if "ExecuteScalar()" returns something other than an int
Dim result = find.ExecuteScalar()
... and (I just noticed Honeyboy Wilson's response!) ...
Fix your syntax:
' Try this first: remove the single quotes around "#code"!
Dim find = New NpgsqlCommand("SELECT serial from dbo.foreigncode WHERE code = #code;", conn)
Update 1
Please try this:
Dim find = New NpgsqlCommand("SELECT * from dbo.foreigncode;", conn)
Q: Does this return anything?
Dim dr = find.ExecuteReader()
While (dr.Read())
Console.Write("{0}\t{1} \n", dr[0], dr[1])
End While
Q: Does this?
Dim result = find.ExecuteScalar()
Q: Do you happen to have a column named "serial"? What is it's data type? Is it non-null for the row(s) with 'XYZ'?
Please update your original post with this information.
Update 2
You seem to be doing ":everything right":
You've confirmed that you can connect,
You've confirmed that non-query updates to the same table work (with npgsql),
You've confirmed that the SQL queries themselves are valid (by copying/pasting the same SQL into PGAdmin and getting valid results).
As Shay Rojansky said, "System.NotImplementedException in the DataTypeName property" is a known issue stepping through the debugger. It has nothing to do with your problem: https://github.com/npgsql/npgsql/issues/2520
SUGGESTIONS (I'm grasping at straws)::
Double-check "permissions" on your database and your table.
Consider installing a different version of npgsql.
Be sure your code is detecting any/all error returns and exceptions (it sounds like you're probably already doing this, but it never hurts to ask)
... and ...
Enable verbose logging, both client- and server-side:
https://www.npgsql.org/doc/logging.html
https://www.postgresql.org/docs/9.0/runtime-config-logging.html
... Finally ...
Q: Can you make ANY query, from ANY table, using ANY query method (ExecuteReader(), ExecuteScalar(), ... ANYTHING) from your npgsql/.Net client AT ALL?
I finally found it. It's often the small things that can have a big impact.
When the value was assigned to the parameter a substring index was incorect.
Now it works perfectly.
Thanks to everybody who spent his time on this.
I try to use SELECT FROM #itab like explained here in SAP docs.
I have never used this feature, but think this is great. You can query a internal data structure which just exists in the RAM of the interpreter like it would be a real table in the database. I am impressed.
Here is the ABAP code:
data: lt_get_auth_values TYPE STANDARD TABLE OF US335.
CALL FUNCTION 'GET_AUTH_VALUES'
EXPORTING
OBJECT1 = 'Z:FOO'
USER = sy-uname
TABLES
VALUES = lt_get_auth_values.
SELECT highval from #lt_get_auth_values as mytab WHERE field = 'WERKS'
INTO TABLE #DATA(static_perm_filter_fields).
I can't active the function because "from #lt_get_auth_values" is a syntax error according to my system.
What's wrong with this line?
SAP Version: 740 (sorry, it first I thought it was 752)
SELECT ... FROM #itab appeared in 7.52 so it should work.
On my 7.52 system it works but you must indicate a table alias. There's an example in the ABAP documentation (cf first link above).
Here is the context
I'm building a library in VB.net able to quickly handle SQL Database datas, in windows form based front end. I'm using ADODB Connections and recordsets.
I managed to link front end to Access databases, MS SQL server, MySQL and, recently I'm working on SQLite databases to provide quick and portables SQL providers.
Here is the problem
As I understand it, SQLite stores single/double/float with IEEE-7 standards, and for exemple a stored value of 9.95 will be read as 9,94999980926514.
So when I load the record again, I can edit it (other fields), and update it. but if I try to edit the float value (lets say 9,94999980926514 > 10) then update it, then I've got an error see sample code
Dim LocRs as ADODB.Recordset
LocRs.Open("SELECT ID_Montant,Mont_Value,Mont_Date FROM tMontants",SQLConnection)
LocRs.addNew
LocRs.("ID_Montant").Value =666
LocRs.("Mont_Value").Value =9.95
LocRs.("Mont_Date").Value =Date.Today
LocRs.Update
LocRs.close
'No Problems'
LocRs.open("SELECT ID_Montant,Mont_Value,Mont_Date FROM tMontants WHERE ID_Montant=666",SQLConnection)
LocRs.Mont_Date.Value=Date.Today.AddDays(-2)
Console.WriteLine(LocRs.("Mont_Value").Value) 'Returns 9,94999980926514'
LocRs.Update
LocRs.close
'No Problem again'
LocRs.open("SELECT ID_Montant,Mont_Value,Mont_Date FROM tMontants WHERE ID_Montant=666",SQLConnection)
LocRs.("Mont_Value").Value=10
LocRs.Update
'Error : Row cannot be located for updating. Some values may have been changed since it was last read. Help Code -2147217864'
The error code seems to be of little help.
I'm using
locRs.LockType = LockTypeEnum.adLockOptimistic
locRs.CursorType = CursorTypeEnum.adOpenKeyset
locRs.CursorLocation = CursorLocationEnum.adUseClient
But I tried a lot of combination without success.
As for the provider I'm using Werner ODBC Provider
I Had a similar problem with Date fields.
When editing a field with an incomplete date like 2012-03-01 12:00:00 instead of 2012-03-01 12:00:00.000, But I corrected the problem by formating the date in it's complete form.
Now I'm stuck with this because the SQLite database stores approximates float value whether I rounded it or not before storing it with an update.
I'm guessing it's a provider problem while computing the update because I can do
"UPDATE tMontants SET Mont_Value = 10 WHERE ID_Montant = 666"
Without any problem
But I'd really like to force the recordset to work in order to integrate SQLlite solutions to every software I've deployed so far.
Thanks for your time.
I am using the RODBC package to query a text column from a database. The database is built on Microsoft SQL Server 2008 R2. The data type of the column in SQL is nvarchar(max).
However, when I run:
# Set up ODBC connection to CCWEB5 production server
# Note: default database is set to "CCSalary"
ccweb5.prod <- odbcConnect("ccweb5")
# Read in some job ad text
job.text <- sqlQuery(ccweb5.prod,"
SELECT TOP 100
ja.JobTitle,
ja.JobText as 'JobText',
LEN(ja.JobText) as 'JobTextLength'
FROM JobStore.dbo.JobAd as ja (NOLOCK)
")
Within SQL, I am expecting (for the top row):
JobTitle JobText JobTextLength
IT Field Service Technician <text goes here...> 2742
However, when I do: nchar(as.character(job.text[1,2]))
It returns: 255.
So my question is, what is causing this truncation and how do I avoid it? Thanks!!
OK, so it seems that I have found a work-around to this. After some more Google'ing, I found that:
One thing to consider with the SQL Native Client ODBC driver is that VARCHAR(MAX) has does not have fixed size and the ODBC driver
represents this by returning a max column size of 0. This can confuse
your application if it doesn't check for 0 as a special case. See the
bottom section of this article:
http://msdn.microsoft.com/en-us/library/ms130896.aspx But in general I
have not seen this happen with any of my .NET applications as it is
handled properly in ADO.NET.
Source: http://bytes.com/topic/sql-server/answers/808461-cannot-read-varchar-max
So, in my case, the following did the trick:
job.text <- sqlQuery(ccweb5.prod,"
SELECT DISTINCT TOP 100
ja.JobTitle,
[JobText] = CAST(ja.JobText AS varchar(8000)), -- note the data-type re-cast
[JobTextLength] = LEN(ja.JobText)
FROM JobStore.dbo.JobAd as ja (NOLOCK)
")
Such that nchar(as.character(job.text[1,2])) now returns 2742 (as it should).
I didn't see any similar questions on StackOverflow so I'll leave this up. Hope this helps somebody!
A solution would be to cast the nvarchar(max) field to ntext
job.text <- sqlQuery(ccweb5.prod,"
SELECT TOP 100
ja.JobTitle,
CAST(ja.JobText AS ntext) as 'JobText',
LEN(ja.JobText) as 'JobTextLength'
FROM JobStore.dbo.JobAd as ja (NOLOCK)
")