I'm having issues getting the ODBC driver for Snowflake to work on an M1 Apple Silicon Mac running Big Sur.
Successfully following the instructions on Snowflake's website gets me to the point where testing the driver from the command line (using iodbctest) using the DSN results in the following error:
1: SQLDriverConnect = [iODBC][Driver Manager]dlopen(/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib, 6): no suitable image found. Did find:
/opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib: no matching architecture in universal wrapper
/opt/snowfl (0) SQLSTATE=00000
2: SQLDriverConnect = [iODBC][Driver Manager]Specified driver could not be loaded (0) SQLSTATE=IM003
My Snowflake driver is installed to /opt/snowflake/snowflakeodbc, so that is correct -- I'm suspicious that this is specifically an M1 problem. I'm using the 2.24.1 version of the driver available from the download mirror here, and the path to the driver in /etc/odbcinst.ini is /opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib (which exists and seems, from all my research, that it should be right).
When I run a connection via DBI in R, I get a completely different error:
Error: nanodbc/nanodbc.cpp:1021: 00000:
[Snowflake][ODBC] (11560) Unable to locate SQLGetPrivateProfileString function.
In other StackOverflow posts, people have referenced the above error meaning that there is a missing library of some kind (IODBC isn't configured correctly?), but I've tried quite a few things to no avail. Any guidance would be great.
Tinkered with this a bit more and realized it's an artifact of the installation pathways for the .dmgs & the preset paths in simba.snowflake.ini.
You need to point the Snowflake driver towards the iODBC dylib (as per a sideswiping statement in the docs) -- the driver is originally configured to look for the ODBC dylib (not iODBC) in a folder that's on the path.
When you install the iODBC driver, verify that it is installed to /usr/local/iODBC (this was where my Silicon Mac installed it to) -- and that /usr/local/iODBC/lib has libiodbc.dylib in it. If so, navigate to your installed snowflake driver directory (should be /etc/snowflake) and alter the simba.snowflake.ini file (/etc/snowflake/snowflake/snowflakeodbc/universal/simba.snowflake.ini). You want to uncomment & alter the last line to be both uncommented & point with a full path towards the iODBC dylib (instead of the default, which is the ODBC dylib).
# Darwin specific ODBCInstLib
# iODBC
ODBCInstLib=/usr/local/iODBC/lib/libiodbcinst.dylib
Make sure to comment out any other ODBCInstLib line so that only one is configured. That should enable you to get your connection to snowflake up and running on an M1 Mac.
Big Sur is macOS v11.n
Snowflake supports macOS 10.14 and 10.15 Supported OSs
So what you are trying to do is not supported and is unlikely to work
None of the other solutions worked for me but #kiran-kumawat 's answer set me down a path that worked.
It seems like the core of the issue is that the odbc code is looking for arm64 architecture drivers but Snowflake is providing it in x86_64 architecture. By installing an x86_64 versions of odbc we are able to have it successfully talk to the driver.
First I uninstalled R and Rstudio. (it may be possible to sim-link or change things behind the scenes to make this work with existing installs but I am not sure).
Then install rosetta (apples software for translating between architectures) and a version of homebrew built with it. I am leaving my main version of homebrew in place.
softwareupdate --install-rosetta
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Then use that version of homebrew to install odbc, R, and Rstudio.
arch -x86_64 /usr/local/Homebrew/bin/brew install unixodbc
arch -x86_64 /usr/local/Homebrew/bin/brew install --cask rstudio
arch -x86_64 /usr/local/Homebrew/bin/brew install --cask r
We then need to install the snowflake driver: https://sfc-repo.snowflakecomputing.com/odbc/mac64/latest/index.html
Click through all the install prompts.
Modify your files
/usr/local/etc/odbcinst.ini:
[Snowflake Driver]
Driver = /opt/snowflake/snowflakeodbc/lib/universal/libSnowflake.dylib
/usr/local/etc/odbc.ini
[Snowflake]
Driver = Snowflake Driver
uid = <uid>
server = <server>
role = <role>
warehouse = <warehouse>
authenticator = externalbrowser
We also need to modify the simba.snowflake.ini file.
It is somewhat locked down so run:
sudo chmod 646 /opt/snowflake/snowflakeodbc/lib/universal/simba.snowflake.ini
Then
vim /opt/snowflake/snowflakeodbc/lib/universal/simba.snowflake.ini
And find the ODBCInstLib line that is uncommented and change it to:
ODBCInstLib=/usr/local/Cellar/unixodbc/2.3.9_1/lib/libodbcinst.dylib
After setting this up I was able to use this connection successfully:
install.packages("DBI")
install.packages("odbc")
con <- DBI::dbConnect(odbc::odbc(), "Snowflake")
one of our team member suggested below steps and it worked for us for Apple M1 series
Install the latest snowflake driver
Uninstall m1 based homebrew using cmd
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall.sh)"
Install intel based homebrew - restart terminal when done
arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Re-install unixodbc
arch -x86_64 brew install unixodbc
Test
isql -v Pattern
in your database.yml file for connection to snowflake make following change-
replace "dsn: <DSN_NAME>" with following
conn_str: "Driver={PATH};Locale=en-US;uid={USER_NAME};pwd= {PASSWORD};server=<yours>.snowflakecomputing.com;role=<ROLE>;charset=UTF-8;warehouse=<WAREHOUSE>;database=<DATABASE>;schema=<SCHEMA>;"
Has anyone gotten this to work? I use excel w odbc to refresh snowflake files and have tried multiple ways to move the drivers etc and followed snowflake instructions but never works. I did get parallels to work running windows arm but would prefer to just do this in Mac OS
I also have a M1 (version Monterey 12.0) and I ran into similar issues when I tested the driver. Nevertheless, when I tried the "real connection" it worked like a charm. So, maybe it would be good for you to go and test the "real connection" to avoid a wasting of time using such testing. Hope you find this useful.
Related
After searching the web for hours i didnt yet find an answer to my problem. I am using Python 3.6 and i cant import selenium. I always get the message "No module named 'selenium''
I tried everything, i first downloaded selenium from this website https://pypi.python.org/pypi/selenium/3.6.0 .
Then I tried python -m pip install -U selenium and didnt work either. I tried some other things that people said but they didnt work either.
Im using windows 10.
Any help?
As you mentioned you are using Python 3.6 following the steps :
Open Command Line Interface (CLI) and issue the command python to check if Python is properly installed :
C:\Users\username>python
Python 3.6.1 (v3.6.1:69c0db5, Jan 16 2018, 17:54:52) [MSC v.1900 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
Ensure pip is working properly :
C:\Users\username>pip
Usage:
pip <command> [options]
Commands:
install Install packages.
download Download packages.
uninstall Uninstall packages.
freeze Output installed packages in requirements format.
list List installed packages.
show Show information about installed packages.
check Verify installed packages have compatible dependencies.
search Search PyPI for packages.
wheel Build wheels from your requirements.
hash Compute hashes of package archives.
completion A helper command used for command completion.
help Show help for commands.
General Options:
-h, --help Show help.
--isolated Run pip in an isolated mode, ignoring environment variables and user configuration.
-v, --verbose Give more output. Option is additive, and can be used up to 3 times.
-V, --version Show version and exit.
-q, --quiet Give less output. Option is additive, and can be used up to 3 times (corresponding to WARNING, ERROR, and CRITICAL logging levels).
--log <path> Path to a verbose appending log.
--proxy <proxy> Specify a proxy in the form [user:passwd#]proxy.server:port.
--retries <retries> Maximum number of retries each connection should attempt (default 5 times).
--timeout <sec> Set the socket timeout (default 15 seconds).
--exists-action <action> Default action when a path already exists: (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.
--trusted-host <hostname> Mark this host as trusted, even though it does not have valid or any HTTPS.
--cert <path> Path to alternate CA bundle.
--client-cert <path> Path to SSL client certificate, a single file containing the private key and the certificate in PEM format.
--cache-dir <dir> Store the cache data in <dir>.
--no-cache-dir Disable the cache.
--disable-pip-version-check
Don't periodically check PyPI to determine
whether a new version of pip is available for
download. Implied with --no-index.
Install latest selenium through pip :
C:\Users\username>pip install -U selenium
Collecting selenium
Downloading selenium-3.8.1-py2.py3-none-any.whl (931kB)
100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 942kB 322kB/s
Installing collected packages: selenium
Successfully installed selenium-3.8.1
Confirm that Selenium is installed :
C:\Users\username>pip freeze
selenium==3.8.1
Open an IDE (e.g Eclipse, PyCharm) and write a simple program as follows :
from selenium import webdriver
driver = webdriver.Firefox(executable_path="C:\\path\\to\\geckodriver.exe")
driver.get('https://stackoverflow.com')
Execute the program on which Firefox Quantum Browser will be initiated and the url https://stackoverflow.com will be accessed.
Python Download Location (Windows) :
Python (for Windows) can be download from the following location :
https://www.python.org/downloads/
I'm on VS Code in Windows 10, and here's how I solved it.
You need to pay attention to where the Python is located (in my case),
1) C:\Users\_Me_\AppData\Local\Programs\Python\Python38\
and where the Python looks for libraries/packages, including the ones installed using pip (again, in my case),
2) C:\Users\_Me_\AppData\Roaming\Python\Python38\
I don't know why these two locations are different (gotta fix it at some point). It seemed that Python was running from the first location, but it was looking for libraries in the second!:/
Anyway, since I have limited experience in Python , I just copied the \Lib\site-packages from the first location (including selenium folders) to \site-packages in second one in hopes of solving the issue, which worked out for me!
How to check for there locations
1) Open Python CLI, typed the following command:
which python
2) Open Python CLI, typed the following commands (From this answer):
>>> import site
>>> site.USER_SITE
EDIT
Since this seems a temporary solution, I uninstalled Python and reinstalled it again in a proper directory (other than the default install directory), and now which python and which pip point to the same folder! Problem solved!
If these solutions did not work for you, try looking into which Python interpreter your IDE is using (in my case I was using VSCode - you can find the interpreter in the bottom bar). Worked for me.
Easiest way is to copy all files of "venv" Lib, Scripts, Selenium and other folder into your main project folder.
This issue occur as pycharm directly take from virtual environment venv.
Hope this will resolve your problem :)
I am trying to install tensorflow-gpu by running pip install tensorflow-gpu Windows, inside an Anaconda enviornment, but I am getting the following error:
Could not install packages due to an EnvironmentError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:1977)
I also tried doing it in a command prompt with administrator access, but it still didn't work.
C:\WINDOWS\system32>pip install tensorflow-gpu
Collecting tensorflow-gpu
Downloading https://files.pythonhosted.org/packages/2f/84/b6dfafe3282101f7d3a9410652ab4e6dc73f981fd63a40be0b47ff3bac3a/tensorflow_gpu-1.9.0-cp35-cp35m-win_amd64.whl (103.3MB)
19% |###### | 19.9MB 2.6MB/s eta 0:00:32
Could not install packages due to an EnvironmentError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:1977)
All other network-related activities(such as browsing the web) work properly on the computer. I also have the correct CUDA and cuDNN installed, along with the latest NVIDIA drivers.
I fixed this problem by purging my Anaconda installation and reinstalling it, and then installing tensorflow-gpu. However, I am still curious as to why this error even happened in the first place.
It's related to network connection. I faced the same error in my Ubuntu system. Hence solved it by changing my network(WiFi) connection.
Use the following syntax while installation packages:
pip install --user packagename
I had the same problem with PyCharm to install tensorflow. As Dimitri Bolt described, I started the CMD as administrator and used the sytax described by Devendra Kanade. I got the error again, but each time I started the command, the progress became bigger. After three or four times, the download was successful and I was able to import tensorflow in PyCharm.
You should note "Downloading" (19%, not finished yet) in the pip-msg just before the error. For pip, this means 99% that this is a download error.
The quick fix is to increase <retries> in the pip option :
--retries <retries> Maximum number of retries each connection should attempt (default 5 times).
NB: I've never tried this.
The general fix using normal download (I tried this to install PyQt5) :
a) Find the name of the problematic web-file in your pip-errmsg, enclosed in quotes, then google the whole (including quotes). In your case, that will be "tensorflow_gpu-1.9.0-cp35-cp35m-win_amd64.whl" .
b) Choose a reliable site and download that file. Windows 10 may give you erroneous msg about some .part-file but just ignore that msg and download again.
c) Load your browser list for downloaded files. For Firefox, ctrl-J will load that list (library). If failure, click to repeat download on the list (not site). You may need to repeat the clicking 10 times for the download to be successful !!!
d) Suppose that "tensorflow_gpu-1.9.0-cp35-cp35m-win_amd64.whl" is now in the local folder c:\Users\uuuu\Downloads\ .
Create in this folder an ansi-text file named "example-requirements.txt" which has 3 lines(2 empty lines) as below:
tensorflow_gpu-1.9.0-cp35-cp35m-win_amd64.whl
e) Now in a normal command prompt, issue 3 commands as below:
c:
cd c:\Users\uuuu\Downloads\
pip install --requirement example-requirements.txt --no-cache-dir
NB: you can copy and paste, very easy in Win 10.
f) If successful, done now !
g) If you again have pip-problem with downloading another web-file then you must repeat a) , b) , c) , d) and edit the old "example-requirements.txt" to contain 4 lines(2 empty lines) as below:
tensorflow_gpu-1.9.0-cp35-cp35m-win_amd64.whl
another-file-name-with-extension
h) Repeat e) , f) , g) ... ...
NB: DOC for pip install-options are found on https://pip.pypa.io/en/stable/cli/pip_install/#
(search for "Example Requirements File" on the page).
stumbled upon the same error while installing via conda, updating conda solved the problem (btw, the new version downloaded the packages sequentially, whilst the older attempted a parallel download)
Reinstall the library again!
I face this problem for several libraries like tensorflow-gpu and matplotlib
I have no idea why but if i found something i will share.
While installing a kvm via virt-install I have used following attribute os_variant=rhel7.2. While installing I am getting following error :
distro 'rhel7.2' does not exist in our dictionary
When I do uname -r I am getting output as
3.10.0-327.el7.x86_64
It is a RHEL KVM host.
Running osinfo-query os|grep 'Red Hat Enterprise Linux 7.2' returns following:
rhel7.1 | Red Hat Enterprise Linux 7.2 | 7.2 | http://redhat.com/rhel/7.2
What can be solution to this problem?
You could create a custom config file to define a RHEL-7.2 distro, but it is honestly not that important from virt-install's POV. The distro is used to lookup the optimized drivers to use for disk & network primarily. Just using the 'rhel7.1' distro type when installing 'rhel-7.2' will work just fine from this POV.
virt-install get's it's os information out of osinfo-db
If your os does not ship a recent version of osinfo-db you can manually download it from https://releases.pagure.org/libosinfo/ and import it.
e.g.
wget https://releases.pagure.org/libosinfo/osinfo-db-20200325.tar.xz
osinfo-db-import -v osinfo-db-20200325.tar.xz
-v will display all imported os'es, I believe your choice has to match one of the xml files in this list.
I am trying to connect to an Vertica DB from R using "RODBC" package. Also, the machine I am using is an remote server which doesn't have direct internet access so I basically "transfer" all source files from my local to the remote server to build the system. So, in order to give you an clear context, I am listing all my steps in attending of installing "RODBC" package below:
Step1 - I downloaded the RODBC_1.3-13.tar.gz source file for RODBC and then tried to directly install it with "R CMD INSTALL". However, I encountered error as "ODBC headers sql.h and sqlext.h not found".
Step2 - After a few researches, I found that the installation of "unixodbc-dev" would potentially solve this issue. Therefore, I downloaded all needed dependencies for "unixodbc-dev" and transferred them to the server. As you can see the list:
Therefore, I also successfully installed "unixodbc-dev":
However, another error message appears when I tried to re-install the "RODBC" using "sudo R CMD INSTALL /home/mli/RODBC_1.3-13.tar.gz" in which it returns error "no ODBC driver manager found":
As the message indicates, the installation program can't locate my ODBC driver manager. So, I downloaded "vertica-client-7.2.3-0.x86_64.tar.gz" and unzipped it on the server:
So, now my question is how can I customize the "R CMD INSTALL" command say, using some parameter handles to direct the installation program to locate the driver manager? Or am I trying this in a right direction? Please let me know. Any help would be really appreciated!!! :)
ADDITION:
I have also tried it with JDBC in which the I successfully loaded the "RJDBC" package in R and used the JDBC driver from vertica-client-7.2.3-0.x86_64.tar.gz. Also, I have already had "rJava" installed. However, I have still got an error when I tried to make the connection. I am listing my result below:
I successfully installed the "RJDBC" with "$R CMD INSTALL RJDBC_0.2-5.tar.gz --library=/usr/local/lib/R/site-library/" and then I tried the following scripts in R. All the lines are successfully executed except on the line 16:
Based on the error message, I assumed the version of the JDBC driver that I was using is too new for the Vertica server. So, I was trying to use an older version JDBC driver instead, like the "vertica-jdk5-6.1.0-0.jar" which I have downloaded from this link:http://www.java2s.com/Code/Jar/v/Downloadverticajdk56100jar.htm
So, I moved the file "vertica-jdk5-6.1.0-0.jar" to my home directory on the server and then changed the JDBC driver path in the R script:
As you can see, it still returns error "FATAL: Unsupported frontend protocol 3.6: server supports 3.0 to 3.5". Am I doing it right? Or is there an issue with the new driver that I downloaded? How can make it works? Please, any help will be really appreciated! Thanks!!!
A few things:
First, just do sudo apt-get install r-cran-rodbc. The package was created (by yours truly) in no small part because dealing with unixODBC or iODBC is not fun. But even once you have that, you still need the ODBC driver for Linux from Vertica. And that part is filly.
Second, I just did something similar the other day but just used JDBC, which worked. You do of course need sudo apt-get install r-cran-rjava which has its own can of worms (but I already mentioned Java...) Still, maybe try that instead?
Third, you can cheat and just use psql pointed to the Vertica port (usually one above the PostgreSQL port).
I am tying to work/setup the Systemtap tool for profiling OS procesess, on a Virtual Linux. I am using VirtualBox to run the image. Via
rpm -q kernel
and
cat /proc/version
The version obtained is:
Linux version 2.6.32-5-686 (Debian 2.6.32-48squeeze4)
I have correctly downloaded and installed the tool and wrote a simple program (.stp). However I keep getting the same error, which I have searched information in many places without success:
After executing:
sudo stap my_profiler.stp
I get:
semantic error: libdwfl failure (all kernel modules found): no error
Pass 3: translation failed. Try again with another '--vp 001' option.
According to https://sourceware.org/systemtap/SystemTap_Beginners_Guide/errors.html
semantic error: libdwfl failure
There was a problem processing the debugging information. In most cases, this error results from the installation of a kernel-debuginfo package whose version does not match the probed kernel exactly. The installed kernel-debuginfo package itself may have some consistency or correctness problems.
I have found no relevant information on the "kernel-debuginfo" package. I have also tried the verbose option without benefit. I even tried with an old Snapshot of the VM. Any ideas?
The code of the .stp program I ran:
probe timer.profile{
printf("Process: %s\n", execname())
printf("Process ID: %d\n", pid())
}
Found the problem!!!! It seemed that I was using the wrong version of the Linux Kernel. I was using the default kernel supplied by the version I wrote in the question. It seems that that version (the 2.6.32-5-686 one) has problems with the debug-info so all I did was try the same with another version (the Linux version 3.9.6 with gcc version 4.7.2 Debian 4.7.2-5) and it worked without trouble :)