I am trying to connect to an Vertica DB from R using "RODBC" package. Also, the machine I am using is an remote server which doesn't have direct internet access so I basically "transfer" all source files from my local to the remote server to build the system. So, in order to give you an clear context, I am listing all my steps in attending of installing "RODBC" package below:
Step1 - I downloaded the RODBC_1.3-13.tar.gz source file for RODBC and then tried to directly install it with "R CMD INSTALL". However, I encountered error as "ODBC headers sql.h and sqlext.h not found".
Step2 - After a few researches, I found that the installation of "unixodbc-dev" would potentially solve this issue. Therefore, I downloaded all needed dependencies for "unixodbc-dev" and transferred them to the server. As you can see the list:
Therefore, I also successfully installed "unixodbc-dev":
However, another error message appears when I tried to re-install the "RODBC" using "sudo R CMD INSTALL /home/mli/RODBC_1.3-13.tar.gz" in which it returns error "no ODBC driver manager found":
As the message indicates, the installation program can't locate my ODBC driver manager. So, I downloaded "vertica-client-7.2.3-0.x86_64.tar.gz" and unzipped it on the server:
So, now my question is how can I customize the "R CMD INSTALL" command say, using some parameter handles to direct the installation program to locate the driver manager? Or am I trying this in a right direction? Please let me know. Any help would be really appreciated!!! :)
ADDITION:
I have also tried it with JDBC in which the I successfully loaded the "RJDBC" package in R and used the JDBC driver from vertica-client-7.2.3-0.x86_64.tar.gz. Also, I have already had "rJava" installed. However, I have still got an error when I tried to make the connection. I am listing my result below:
I successfully installed the "RJDBC" with "$R CMD INSTALL RJDBC_0.2-5.tar.gz --library=/usr/local/lib/R/site-library/" and then I tried the following scripts in R. All the lines are successfully executed except on the line 16:
Based on the error message, I assumed the version of the JDBC driver that I was using is too new for the Vertica server. So, I was trying to use an older version JDBC driver instead, like the "vertica-jdk5-6.1.0-0.jar" which I have downloaded from this link:http://www.java2s.com/Code/Jar/v/Downloadverticajdk56100jar.htm
So, I moved the file "vertica-jdk5-6.1.0-0.jar" to my home directory on the server and then changed the JDBC driver path in the R script:
As you can see, it still returns error "FATAL: Unsupported frontend protocol 3.6: server supports 3.0 to 3.5". Am I doing it right? Or is there an issue with the new driver that I downloaded? How can make it works? Please, any help will be really appreciated! Thanks!!!
A few things:
First, just do sudo apt-get install r-cran-rodbc. The package was created (by yours truly) in no small part because dealing with unixODBC or iODBC is not fun. But even once you have that, you still need the ODBC driver for Linux from Vertica. And that part is filly.
Second, I just did something similar the other day but just used JDBC, which worked. You do of course need sudo apt-get install r-cran-rjava which has its own can of worms (but I already mentioned Java...) Still, maybe try that instead?
Third, you can cheat and just use psql pointed to the Vertica port (usually one above the PostgreSQL port).
Related
I'm having issues getting the ODBC driver for Snowflake to work on an M1 Apple Silicon Mac running Big Sur.
Successfully following the instructions on Snowflake's website gets me to the point where testing the driver from the command line (using iodbctest) using the DSN results in the following error:
1: SQLDriverConnect = [iODBC][Driver Manager]dlopen(/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib, 6): no suitable image found. Did find:
/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib: no matching architecture in universal wrapper
/opt/snowfl (0) SQLSTATE=00000
2: SQLDriverConnect = [iODBC][Driver Manager]Specified driver could not be loaded (0) SQLSTATE=IM003
My Snowflake driver is installed to /opt/snowflake/snowflakeodbc, so that is correct -- I'm suspicious that this is specifically an M1 problem. I'm using the 2.24.1 version of the driver available from the download mirror here, and the path to the driver in /etc/odbcinst.ini is /opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib (which exists and seems, from all my research, that it should be right).
When I run a connection via DBI in R, I get a completely different error:
Error: nanodbc/nanodbc.cpp:1021: 00000:
[Snowflake][ODBC] (11560) Unable to locate SQLGetPrivateProfileString function.
In other StackOverflow posts, people have referenced the above error meaning that there is a missing library of some kind (IODBC isn't configured correctly?), but I've tried quite a few things to no avail. Any guidance would be great.
Tinkered with this a bit more and realized it's an artifact of the installation pathways for the .dmgs & the preset paths in simba.snowflake.ini.
You need to point the Snowflake driver towards the iODBC dylib (as per a sideswiping statement in the docs) -- the driver is originally configured to look for the ODBC dylib (not iODBC) in a folder that's on the path.
When you install the iODBC driver, verify that it is installed to /usr/local/iODBC (this was where my Silicon Mac installed it to) -- and that /usr/local/iODBC/lib has libiodbc.dylib in it. If so, navigate to your installed snowflake driver directory (should be /etc/snowflake) and alter the simba.snowflake.ini file (/etc/snowflake/snowflake/snowflakeodbc/universal/simba.snowflake.ini). You want to uncomment & alter the last line to be both uncommented & point with a full path towards the iODBC dylib (instead of the default, which is the ODBC dylib).
# Darwin specific ODBCInstLib
# iODBC
ODBCInstLib=/usr/local/iODBC/lib/libiodbcinst.dylib
Make sure to comment out any other ODBCInstLib line so that only one is configured. That should enable you to get your connection to snowflake up and running on an M1 Mac.
Big Sur is macOS v11.n
Snowflake supports macOS 10.14 and 10.15 Supported OSs
So what you are trying to do is not supported and is unlikely to work
None of the other solutions worked for me but #kiran-kumawat 's answer set me down a path that worked.
It seems like the core of the issue is that the odbc code is looking for arm64 architecture drivers but Snowflake is providing it in x86_64 architecture. By installing an x86_64 versions of odbc we are able to have it successfully talk to the driver.
First I uninstalled R and Rstudio. (it may be possible to sim-link or change things behind the scenes to make this work with existing installs but I am not sure).
Then install rosetta (apples software for translating between architectures) and a version of homebrew built with it. I am leaving my main version of homebrew in place.
softwareupdate --install-rosetta
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Then use that version of homebrew to install odbc, R, and Rstudio.
arch -x86_64 /usr/local/Homebrew/bin/brew install unixodbc
arch -x86_64 /usr/local/Homebrew/bin/brew install --cask rstudio
arch -x86_64 /usr/local/Homebrew/bin/brew install --cask r
We then need to install the snowflake driver: https://sfc-repo.snowflakecomputing.com/odbc/mac64/latest/index.html
Click through all the install prompts.
Modify your files
/usr/local/etc/odbcinst.ini:
[Snowflake Driver]
Driver = /opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib
/usr/local/etc/odbc.ini
[Snowflake]
Driver = Snowflake Driver
uid = <uid>
server = <server>
role = <role>
warehouse = <warehouse>
authenticator = externalbrowser
We also need to modify the simba.snowflake.ini file.
It is somewhat locked down so run:
sudo chmod 646 /opt/snowflake/snowflakeodbc/lib/universal/simba.snowflake.ini
Then
vim /opt/snowflake/snowflakeodbc/lib/universal/simba.snowflake.ini
And find the ODBCInstLib line that is uncommented and change it to:
ODBCInstLib=/usr/local/Cellar/unixodbc/2.3.9_1/lib/libodbcinst.dylib
After setting this up I was able to use this connection successfully:
install.packages("DBI")
install.packages("odbc")
con <- DBI::dbConnect(odbc::odbc(), "Snowflake")
one of our team member suggested below steps and it worked for us for Apple M1 series
Install the latest snowflake driver
Uninstall m1 based homebrew using cmd
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall.sh)"
Install intel based homebrew - restart terminal when done
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Re-install unixodbc
arch -x86_64 brew install unixodbc
Test
isql -v Pattern
in your database.yml file for connection to snowflake make following change-
replace "dsn: <DSN_NAME>" with following
conn_str: "Driver={PATH};Locale=en-US;uid={USER_NAME};pwd= {PASSWORD};server=<yours>.snowflakecomputing.com;role=<ROLE>;charset=UTF-8;warehouse=<WAREHOUSE>;database=<DATABASE>;schema=<SCHEMA>;"
Has anyone gotten this to work? I use excel w odbc to refresh snowflake files and have tried multiple ways to move the drivers etc and followed snowflake instructions but never works. I did get parallels to work running windows arm but would prefer to just do this in Mac OS
I also have a M1 (version Monterey 12.0) and I ran into similar issues when I tested the driver. Nevertheless, when I tried the "real connection" it worked like a charm. So, maybe it would be good for you to go and test the "real connection" to avoid a wasting of time using such testing. Hope you find this useful.
I have been consistently receiving the same error while trying to create a connection between R and Snowflake using an ODBC driver. The error that I'm receiving says:
'''Error during wrapup: nanodbc/nanodbc.cpp:1021: 00000: [unixODBC][Driver Manager]Can't open lib 'Snowflake' : file not found
Error: no more error handlers available (recursive errors?); invoking 'abort' restart'''
The instructions given by Snowflake to connect a driver are rather cut and clear - but provides little insight on what to do with errors. Can anyone lead me as to where to go given this error?
This smells like a configuration issue. Your driver manager, unixODBC, isn't able to locate the Snowflake driver. (The Snowflake documentation says on MacOS to use iODBC. Don't. It won't work with the odbc R package.)
Check that unixODBC is findable. Run odbcinst -j in a terminal. If it works, you will know that you have unixODBC properly installed, and it will give you the paths to your various configuration files.
On to checking configuration. This is the documentation for configuring on Linux using unixODBC. If you are using MacOS the same general instructions apply, but the file extensions will change from .so to .dylib . Since it's saying it can't find the file, I'm thinking that using full paths might resolve this for you. It's also possible that there's some issue with how you are specifying the driver.
Also, it looks like it's searching for a file named 'Snowflake'. I'm thinking you've got Driver=Snowflake somewhere in one of your config files. Best change that to Driver=<path>/<to>/<driver>/libSnowflake.dylib (or .so if you're on Linux). Do this in all the places where you have Driver=Snowflake.
So I'm relatively new to using r-studio and I'm having a problem installing RMySQL.
I'm running RStudio 0.98.501 and R 3.0.2 and trying to connect R to a database. However, whenever I try to install RMySQL I get the error message "package ‘RMySQL’ is not available (for R version 3.0.2)". When I searched I found this thread: http://r.789695.n4.nabble.com/RMySQL-with-Windows-7-td4684805.html which explains how I could be downloading packages to Program Files. I checked using the .libPaths() function and this was confirmed ("C:/Program Files/R/R-3.0.2/library"). I guess my question is how do I change the library path so that I can install RMySQL? Or am I going about this all wrong?
Try to follow the instructions here posted here.
Basically what you have to do is:
install MySQL from the Oracle homepage
install RTools
change your system variable "MYSQL_HOME" to match your MySQL installation
(in some cases ibmysql.dll which is part of your MySQL installation had to be copied from lib folder to bin folder so it could be found)
That should (hopefully) do the trick although I did not stumble upon any remark stating RMySQL installation (especially under Windows) is easy.
Trying setup mercurial SVM on my windows server (2008 RC) from last couple of hours. I am stuck on this error when I try to clone my repo from the client machine.
Error: no suitable response from remote hg
The server that I am running has SSH access (SSH running on port 1667). I also have a remote access to it.
I tried to clone using command as well as with the help of tortoisehg gui client. Commands I tried is:
hg clone ssh://myuser#myremoteip:1667//D:/Mercurial Projects/testproj E:\Mercurial\testproj-clone
hg clone --remotecmd D:/Program Files/TortoiseHg/hg --verbose -- ssh://myuser#myremoteip:1667//D:/Mercurial Projects/testproj E:\Mercurial\testproj-clone
but no success so far.
I also added following line in global setting at client side to give remote path of hg on server but no luck:
[ui]
remotecmd = D:/Program Files/TortoiseHg/hg
Please help me...
I had a similar problem and in my case it was that the computer had both TortoiseSVN and TortoiseHG installed. Both TortoiseHG and TortoiseSVN have a command TortoisePlink.exe that they use. However, due to the PATH, TortoiseHG was using TortoiseSVN's TortoisePlink.exe.
Uninstalling TortoiseSVN solved the problem for me.
You may open a "cmd" window and type:
where TortoisePlink.exe
to check what TortoisePlink.exe is used.
I think the problem was that my Python version was older than the one I needed. I was trying to set it up with Python 2.6. I followed another tutorial with Python 2.7 and latest Mercurial version (2.8.1)
Anyone with Windows Server 2008 and IIS 7+ should follow this tutorial.
I run into this problem after updating TortoiseHg. It turned out the location of TortoisePlink.exe has changed. I had it set explicitly to C:\Program Files\TortoiseHg\TortoisePlink.exe in mercurial.ini and I had to change it to C:\Program Files\TortoiseHg\lib\TortoisePlink.exe.
I'm following the instructions found here.
When I try to run $ heroku pg:psql or $ heroku pg:psql HEROKU POSTGRESQL_BROWN I recieve the following error message:
! The local psql command could not be located ! For help
installing psql, see local-postgresql
I can't find anything useful on the link it gives me (it just links to the instructions I was already using, but further down the page) nor can I find this error anywhere else.
If I've missed anything you need to know to answer this, just let me know. I'm rather new to all this and teaching myself as I go.
I had same error even after installing Postgres locally.
But after seeing this
I saw that "pqsl" was not in the PATH so I then did
PATH=%PATH%;C:\Program Files\PostgreSQL\9.2\bin
which worked for me
I have since solved this myself. When I ran heroku pg:info it says the version number is 9.1.8, I was locally running 9.2
installing 9.1.8 and ensuring Path pointed to the appropriate folder solved the problem.
After you change the path, make sure to restart the terminal!
Set the PATH. To find out the PATH of your psql script (on mac) open the sql shell script from your finder in Applications/Postgres installation. This will give you a hint as to where it is installed. That opened a window which told me it is located here: /Library/PostgreSQL/8.4/scripts/runpsql.sh
Then, I set the PATH variable from the terminal window by typing:
$ PATH="/Library/PostgreSQL/8.4/bin:$PATH"
(depends on the location of your PostgreSQL installation, find your bin path first, another exp: /usr/local/Cellar/postgresql#9.6/9.6.8/bin)
OR.....
You can also connect to the shell by opening the shell directly from your postgres installation folder. Then enter the credentials. If you don't know the credentials, here is how to find them out:
$ heroku pg:info
=== HEROKU_POSTGRESQL_RED_URL (DATABASE_URL)
$ heroku pg:credentials HEROKU_POSTGRESQL_RED_URL
Top answer wouldn't work for me oddly, my system would not add the Path via cmd with administrator access (Not sure why).
So check this > Windows key > environment variables > system variables
And add the last line (your version may differ in the path)
Make sure you've installed the toolbelt as psql is installed by default.
However you also need to ensure you've installed a local copy of PostgreSQL; if you don't the toolbelt will be unable to find the native psql client.
Assuming you have installed a local copy of PostgreSQL, make sure you can execute psql from the command line directly (i.e make sure you PATH is set correctly ). If the command does not execute, check your PATH, if it does execute see if you can connect via the PSQL connection string provided in the Heroku control panel. If you can connect reinstall the toolbelt, if you are unable to connect provision another dev database and try again.
If there are still issues, I would suggest contacting Heroku support for assistance after verifying no API issues are listed on the status page located here.
I got rid if this annoying message on Windows by adding a path element without the spaces, i.e.
C:\Progra~1\PostgreSQL\9.4\data
instead of
“C:\Program Files\PostgreSQL\9.4\data”
I followed the instructions here: http://www.computerhope.com/issues/ch000549.htm, which worked for me if you prefer to go the point-and-click configuration of the PATH variable.
This type of error usually appears in the Windows environment, because if you do not update the PATH after installing Postgresql, heroku pg:psql command does not work.
So you need to update your PATH environment variable to add the bin directory of your Postgres installation. The directory will look like this:
C:\Program Files\PostgreSQL\<VERSION>\bin.
For more information, go to the Heroku in Local setup website:
heroku-postgresql: Local setup
I had the same problem and discovered that Heroku doesn't seem to provision the latest version of PostgreSQL by default. Where the Heroku Getting Started instructions said
heroku addons:create heroku-postgresql:hobby-dev
That provisioned a v10 database for some reason (which you can check by clicking on Heroku Postgres in the Add-ons tab of your dashboard). I deleted that database and provisioned a new database using the --version flag:
heroku addons:create heroku-postgresql:hobby-dev --version 11
As of now, at least, you can find the latest version of Postgres supported by Heroku at this link: https://devcenter.heroku.com/articles/heroku-postgresql#version-support-and-legacy-infrastructure
I'm writing this in early 2019, but according to the PostgreSQL website the next version (12) is "tentatively scheduled" for third quarter of 2019 so if you're reading this in late 2019 potentially the same problem will come up for v12 instead
On Mac you can use the following:
export PATH="/Library/PostgreSQL/12/bin/:$PATH"
The only solution that I found on Windows:
go to advanced system settings
go to environment variables
select Path variable and click Edit
add a new line and enter your bin directory path (C:\Program Files\PostgreSQL<version>\bin) and click ok
restart your terminal
enter your psql command (heroku pg:psql)