Superset - Hive Connection Issue - hive

I can connect to HIVE using sqlalchemy without any issues :
oguz#dikanka:~$ python
Python 2.7.15+ (default, Jul 9 2019, 16:51:35)
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from sqlalchemy import create_engine
>>> engine = create_engine('hive://dikanka:10000')
>>> result = engine.execute("select movieid from default.ratings")
>>> result.first();
(242,)
But when I try to connect Superset to Hive, I receive the following error :
ERROR: {"error": "Connection failed!\n\nThe error message returned was:\nCan't load plugin: sqlalchemy.dialects:hive"}
Details are as follows :
I have a single-node HDP-3.1.4.0 setup on my laptop.
I have the following services up and running : HDFS 3.1.1, Hive 3.1.0, Zookeeper 3.4.6, Superset 0.23.0
Python version is Python 2.7.15+.
Hive command line interface works smoothly.
I've installed the following python modules:
pip install thrift
sudo apt-get install libsasl2-dev
pip install sasl
pip install thrift_sasl
pip install pyhive
Last but not least, I have a KUbuntu 18.04.
On Superset, I try to add a database named "default", and I set the SQLAlchemy URI to
hive://dikanka:10000
which is exactly the same URI I used with create_engine above.
Then I click "Test Connection", with all default values for the rest of the form.
I get the error as mentioned above.

In order to instal the hive sqlalchemy driver you need to do
pip install 'pyhive[hive]'
That should fix the error you got.

Related

Can't install packages via pip, with error: ERROR: Could not find a version that satisfies the requirement, but can import via python console?

I am getting essentially the same errors on multiple packages install via pip on Ubuntu 20.04 using python3.8.10:
ERROR: Could not find a version that satisfies the requirement (from versions: none)
ERROR: No matching distribution found for
However, if I run python3 in the console, i can import these packages that are not shown up on pip list and can't be installed. How to fix?:
python3
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> import logging
>>> import re
>>> import traceback
>>> import subprocess
>>> dir(subprocess)
['CalledProcessError', 'CompletedProcess', 'DEVNULL', 'PIPE', 'Popen', 'STDOUT', 'SubprocessError', 'TimeoutExpired', '_PIPE_BUF', '_PopenSelector', '_USE_POSIX_SPAWN', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_active', '_args_from_interpreter_flags', '_cleanup', '_mswindows', '_optim_args_from_interpreter_flags', '_posixsubprocess', '_time', '_use_posix_spawn', 'builtins', 'call', 'check_call', 'check_output', 'contextlib', 'errno', 'getoutput', 'getstatusoutput', 'io', 'list2cmdline', 'os', 'run', 'select', 'selectors', 'signal', 'sys', 'threading', 'time', 'warnings']
>>>
Please advise on how to fix it so Pip would synch with what i am able to import at runtime, ie. python console.
Do i need to set PYTHONPATH variable?

Pyodbc Azure sql database

I am using a macOs BigSur, and I would like to use python to connect to azure sql database.
I followed Microsoft documentation:
https://learn.microsoft.com/en-us/python/api/overview/azure/sql?view=azure-python
to set all the configuration and installed all the requirement for the Mac OS following this documentation.
https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/install-microsoft-odbc-driver-sql-server-macos?view=sql-server-ver15
but when I run my python script I get this error:
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 17 for SQL Server' : file not found (0) (SQLDriverConnect)")
I install mssql-tools and unixodbc
and if I run the command odbcinst -j
I get back this output:
unixODBC 2.3.9
DRIVERS............: /etc/odbcinst.ini
SYSTEM DATA SOURCES: /etc/odbc.ini
FILE DATA SOURCES..: /etc/ODBCDataSources
USER DATA SOURCES..: /Users/<my-user>/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
Anyone has a clue about this error? and please if you need more infos just ask me. Thank you so much
EDIT:
Following the documentation, Microsoft suggests to make a sim link out of the odic.ini and odbcinst.ini.
I run the code to create those symlinks but I realised that if I try to open those files, they are empty, and if I try to go to the folder and open them manually rather than with terminal, I get the error that it can't be done because the source doesn't exist.
Did anyone ever occurred in this error?
LATEST UPDATE:
I installed FreeTDS and updated my .odbc.ini with the following configuration:
[my_server]
Description = my_server
TDS_Version = 7.4
Driver = /usr/local/lib/libtdsodbc.so
Server = YOUR.SERVERNAME.HERE.com
Port = 1433
When I run my python script to connect to the database, I get the following Traceback:
Traceback (most recent call last):
File "database.py", line 11, in <module>
cnxn = pyodbc.connect('dsn=my_server;'
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/usr/local/lib/libtdsodbc.so' : file not found (0) (SQLDriverConnect)")
And in fact there is no libtdsodbc.so in that location.
In another topic they suggest to install freeTDS with the flag --with-unixodbc but when I try to do so, I get the error that the command doesn't exist
UPDATE
If your macos is on the x64 architecture, you can use below method to solve issues.
If you use M1, it's not support now.
Solution for x64 architecture.
Run below code, it should useful to you.
brew update
brew install unixodbc
brew install FreeTDS
Then update the .odbc.ini file.
For more details, please refer below blogs.
How to set up ODBC in Mac OS to connect to MS SQL Server for use with Python and R
Starting ODBC version 17.8, Apple M1 is supported.
You can install ODBC 18 on macOS with the following commands:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release
brew update
HOMEBREW_NO_ENV_FILTERING=1 ACCEPT_EULA=Y brew install msodbcsql18 mssql-tools18

installing PySocks in Jython

I'm trying to develop software using Jython and SOCKS5, so I installed PySocks, but I'm receiving the following import error:
Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11)
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_31
Type "help", "copyright", "credits" or "license" for more information.
>>> import socks
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "socks.py", line 117, in <module>
TypeError: Error when calling the metaclass bases
function() argument 1 must be code, not str
How can I correctly use PySocks in Jython?
“The Jython project strives to make all Python modules run on the JVM, but there are a few differences between the implementations. Perhaps the major difference between the two implementations is that Jython does not work with C extensions. Therefore, most of the Python modules will run without changes under Jython, but if they use C extensions then they will probably not work.”
According to the jython doc, it seems jython doesn't support C entensions.
you can install this lib by this cmd:
# sudo apt-get install python-socksipy
I found the response in :
How do I install Socks / SocksIPy on Ubuntu?

Mac Lion Python install: Matplotlib ends in mysql error

I am setting up python/numpy/scipy/matplotlib/mysql on my Mac Lion. I first installed Python 2.7.3 and mysqldb. I then updated to XCode 4.3.3 and used the ScipySuperpack to install numpy, scipy, etc.
at this point I am able to import several packages in python without incident but matplotlib dies as follows:
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr 9 2012, 20:52:43)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> import scipy
>>> import _mysql
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg/_mysql.py:3:
UserWarning: Module _mysql was already imported from
/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-
packages/MySQL_python-1.2.3-py2.7-macosx-10.6-intel.egg/_mysql.pyc, but
/Users/myname/pypy is being added to sys.path
>>> import matplotlib
Error 2002: Can't connect to local MySQL server through socket
'/tmp/mysql.sock' (2)
Sorry if the question is basic but I don't understand the warning, or what it may have to
do with the eventual error...
Strange. There is probably some internal namespace collision going on (given that this doesn't happen if you are in the mysql build directory). Matplotlib does not use mysql in any way, I checked this from a clone of the matplotlib source with:
$> grep -ir mysql *
And got no hits. But I guess the obvious answer is not to import matplotlib from within the mysql build directory...

Eye of Gnome Python plugins won't autogen because check for PYGTK fails

Presenting symptom: autogen disables the build of slideshowshuffle and pythonconsole, reporting "no python support." Platform is Ubuntu 9.04, Jaunty Jackalope; Gnome 2.26.1.
Log extract:
checking for a Python interpreter with version >= 2.3... python
checking for python... /usr/bin/python
checking for python version... 2.6
checking for python platform... linux2
checking for python script directory... ${prefix}/lib/python2.6/site-packages
checking for python extension module directory... ${exec_prefix}/lib/python2.6/site-packages
checking for PYGTK... no
configure: WARNING: Python not found, disabling python support
Evidence that both python and pygtk are installed:
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygtk
>>>
I note the capitalization of PYGTK, which is common for environment variables. There is no PYGTK environment variable.
Your search - "PYGTK environment
variable" - did not match any
documents.
A grep for PYGTK in the tree rooted from /usr/share/doc/python-gtk2-doc/html returned no rows.
Try installing "python-gtk2-dev" package. You can make sure you have it with
pkg-config --list-all | grep pygtk-2.0
I think, the one you're using from python is "python-gtk2".