I'm trying to load data from an Access databse using PYODBC.
The code I wrote is this:
#Connection Parameters
MDB = 'C:/Users/xxxx/Documents/EugeneVideo.mdb'
DRV = '{Microsoft Access Driver (*.mdb, *.accdb)}'
PWD = 'PW'
pyodbc.pooling = False
con = pyodbc.connect('DRIVER={};DBQ={};PWD={}'.format(DRV,MDB,PWD))
SQL = """SELECT * FROM Customers WHERE c_state = 'CA' """
df = pd.read_sql_query(SQL, con)
csr = con.cursor()
csr.close()
con.close()
At the first time, it runs nicely.
The data is stored in a dataframe.
But if I try to run the script again, changing, for instance the filter parameter, python crashes.
I noticed that the LDB file is not being deleted after the first run, so, apparently, the connection is not being closed even I explicitly ask for that.
Where are the error? Is there a better way to assure that the connection will be closed?
I've already have tried the class methods, with similar results.
Cheers
I solved this issue by enabling VB in File -> Options -> Trust Center.
Related
I am trying to upload a dataframe to a database on Azure SQL Server Database using SQLAlchemy and pyobdc. I have established connection but when uploading I get an error that says
(pyodbc.Error) ('IM010', '[IM010] [Microsoft][ODBC Driver Manager] Data source name too long (0) (SQLDriverConnect)')
I'm not sure where this error is coming from since I've used sqlalchemy before without a problem. I've attached my code below, can anybody help me diagnose the problem?
username = 'bcadmin'
password = 'N#ncyR2D2'
endpoint = 'bio-powerbi-bigdata.database.windows.net'
engine = sqlalchemy.create_engine(f'mssql+pyodbc://{username}:{password}#{endpoint}')
df.to_sql("result_management_report",engine,if_exists='append',index=False)
I know of other ETL methods like Data Factory and SSMS but I'd prefer to use pandas as the ETL process.
Please help me with this error.
Three issues here:
If a username or password might contain an # character then it needs to be escaped in the connection URL.
For the mssql+pyodbc dialect, the database name must be included in the URL in order for SQLAlchemy to recognize a "hostname" connection (as opposed to a "DSN" connection).
Also for mssql+pyodbc hostname connections, the ODBC driver name must be supplied using the driver attribute.
The easiest way to build a proper connection URL is to use the URL.create() method:
from sqlalchemy import create_engine
from sqlalchemy.engine import URL
my_uid = "bcadmin"
my_pwd = "N#ncyR2D2"
my_host = "bio-powerbi-bigdata.database.windows.net"
my_db = "master"
my_odbc_driver = "ODBC Driver 17 for SQL Server"
connection_url = URL.create(
"mssql+pyodbc",
username=my_uid,
password=my_pwd,
host=my_host,
database=my_db, # required; not an empty string
query={"driver": my_odbc_driver},
)
print(connection_url)
"""console output:
mssql+pyodbc://bcadmin:N%40ncyR2D2#bio-powerbi-bigdata.database.windows.net/master?driver=ODBC+Driver+17+for+SQL+Server
"""
engine = create_engine(connection_url, fast_executemany=True)
Simply trying to find out the correct syntax for my connection string. Before anyone asks, yes I did look at other SO answers, and no they did not work for me. Here a couple of attempts I made from looking at other SO questions like the one I am asking
Server=(local);Database=SalesOrdersExample;Integrated Security= true
Data Source=(local);Database=SalesOrdersExample;Integrated Security=SSPI
Server=.\\SQLEXPRESS;Database=SalesOrdersExampleDataSet;Integrated Security=true
None of them worked (I have a Console.WriteLine("test"); thrown in there and it works up until I try conn.Open() (opening the connection to database) so I'm assuming that it must be my connection string since nothing gets written after conn.Open())
Console.WriteLine("test"); // works
SqlConnection conn = new SqlConnection("Server=.\\SQLEXPRESS;Database=SalesOrdersExampleDataSet;Integrated Security=true");
Console.WriteLine("test"); // works
conn.Open();
Console.WriteLine("test"); // does not work
So some information about the database is that it's local under my 'Data Connections' in my Server Explorer. I also have the .xsd file in my project so I have linked the Data Set to the current project I am on. Here is a picture representation to confirm that I have both the Data Connection and the Data Set.
EDIT: SO does not allow me to post pictures until I have 10 rep so here is direct link to picture:
DB Screenshot
Any help is appreciated thank you.
Visual Studio comes with LocalDB database, which is not exactly SQL Server Express database.
Try something like this:
Server=(localdb)\v11.0;Integrated Security=true;
or
Data Source=(LocalDB)\v11.0; AttachDbFileName=|DataDirectory|\DatabaseFileName.mdf; InitialCatalog=DatabaseName;Integrated Security=True;MultipleActiveResultSets=True
If using in c# code, you can use # to avoid problems with backslash characters:
SqlConnection conn =
new SqlConnection(#"Server=(localdb)\v11.0;Integrated Security=true;");
I am trying to connect to an OpenOffice Base database from Java and execute a query, and have not been able to.
These are the steps I followed:
1) Created a Database 'TestDB.odb' in OpenOffice, and a table 'Movies' with columns (ID, Name, Director)
2) Downloaded hsqldb jar file and inclued in project build path
3) Used the following code to connect to it:
String file_name_prefix = "C:/Documents and Settings/327701/My Documents/TestDB.odb";
Connection con = null;
Class.forName("org.hsqldb.jdbcDriver");
con = DriverManager.getConnection("jdbc:hsqldb:file:" + file_name_prefix, "sa","");
Statement statement = con.createStatement();
String query1 = "SELECT * FROM \"Movies\"";
ResultSet rs = statement.executeQuery(query1);
Althoug I'm able to connect to the Database, it throws the following exception on trying to execute the query:
org.hsqldb.HsqlException: user lacks privilege or object not found: Movies
Tried googling, but have not been able to resolve my problem. I'm stuck and it would be great if someone could guide me on how to fix this issue?
You cannot connect to an .odb database. The database you have connected to is in fact a separeate set of files with names such as TestDB.odb.script, etc.
Check http://user.services.openoffice.org/en/forum/viewtopic.php?f=83&t=17567 on how to use an HSQLDB database externally from OOo in server mode. You can connect to such databases with the HSQLDB jar.
OLD thread.
I lost 2 days of my life until I changed the property:
spring.jpa.properties.hibernate.globally_quoted_identifiers = false
I was using mysql before and then I changed to hsqldb in order to run some tests. I kinda copied and pasted this property without looking and then you know - Murphy's law ...
I hope it helps.
I have a wcf 4 service that is trying to add a record to a visual fox pro dbf file on the same server. When it tries to do the insert I get "Cannot update the cursor USER, since it is read-only."
I assume that this is a permissions problem. What do I have to do to give permission to the wcf service to update my dbf file?
This works on my develpment machine.
Here is some of the code:
OleDbConnection oConn = new OleDbConnection("provider=vfpoledb;Data Source="\\data\\tt.dbc");
oConn.Open();
OleDbCommand oCommand = new OleDbCommand();
oCommand.Connection = oConn;
oCommand.CommandText = "SET NULL OFF\r\nSET DELETED ON";
oCommand.ExecuteNonQuery();
OleDbCommand mycmd = new OleDbCommand("insert into user (lastname,firstname) values ('Doe','John')", oConn);
mycmd.CommandType = CommandType.Text;
lnRet = mycmd.ExecuteNonQuery();
It's not a permissions problem - that would generate an access denied error.
Two things to check:
Is the file marked as read-only?
Is the file being used by any other process?
I think #2 is more likely - I haven't done FoxPro, but I've done a lot of dBase with Clipper, and at least under dBase the file has to be opened exclusively (i.e., not shared) to do inserts/updates/deletions.
You need to verify that the credentials that are used to run the WCF service have Read/Write access to the data UNC path.
I am working on some small project for the local firm and the following code runs fine on my machine, but it produces errors on their server. Currently I don't have access to that server, and this is not a field that I know a lot about, so I have to ask you guys.
The page is written in the classic ASP (javascript for scripting).
The logic goes like this:
conn.Open("myconnection");
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn.close();
As I said this runs fine on my machine, but it returns some error (the worst thing is that I don't even know what error) on company's server if bigQuery returns more than ~20 rows. Is there something wrong with my code (this really isn't my field, but I guess it is ok to gather data in the loop like this?), or is there something wrong with their IIS server.
Thanks.
edit:
More info:
It 's Access database. Everything is pretty standard:
conn=Server.CreateObject("ADODB.Connection");
conn.Provider="Microsoft.Jet.OLEDB.4.0";
conn.Open("D:/db/testingDb.mdb");
Queries are bit long, so I wont post them. They are totally ordinary selects so they aren't the issue.
I had a legacy Classic ASP application which I inherited that was very similar (big queries with other queries running within the loop retrieving the first query's results) that ran fine until forty or more records were returned. The way I "solved" the problem was to instantiate another connection object to run the other query. So using your pseudo code, try --
conn.Open("myconnection");
conn2.Open("myconnection")
bigQuery = "...";
rs = conn.execute(bigQuery);
while (!rs.eof) {
...
smallQuery = "..."
rssmall = conn2.execute(smallQuery);
...
rssmall.close();
...
rs.movenext();
}
rs.close();
conn2.close();
conn.close();
What server are they actually running?
Most newer versions of Windows Server don't actually come with the Jet 4.0 driver for 64 bit at all so you can't use an access database with that driver if your app runs as a 64 bit app. You can try running as 32 bit which might solve the problem.
There is an alternative driver packaged as an office component which may be an option.
Try writing a simple test page that literally opens and closes the database connection like so:
<%
Dim conn
Set conn = Server.CreateObject("ADODB.Connection")
conn.Provider = "Microsoft.Jet.OLEDB.4.0"
conn.Open("D:/db/testingDb.mdb")
Response.Write("Database Opened OK")
conn.Close()
%>
Run this on the production server and if you see Database Opened OK then you'll know it's definitely the query rather than the database causing the issue.
If you get an error trying to open the database then you need to changed to using the newer driver or try the app in 32 bit mode
In the case that it is the query causing the issue then it may be that you'll need to use the various additional arguments to the Open() method to try using a different cursor (forward only if you only need to iterate over the results once) which will change how ADODB retrieves the data and hopefully mediate any performance bottleneck related to the query.
Edit
If you want to try debugging the code a bit more add the following at the start of the file. This causes the ASP script processor to continue even if it hits an error
On Error Resume Next
Then at intervals throughout the file where you expect an error might have happened do
If Err <> 0 Then
Response.Write(Err.Number & " - " & Err.Description)
End If
See this article from ASP 101 for the basics of error handling in ASP and VBScript