I want to connect my Airflow and Microsoft SQL Server. I configured my connection under 'connections' bar in 'Admin' box as mentioned in the following link:
http://airflow.apache.org/howto/manage-connections.html
But when I run my Dag task that is related to SQL server immedatly fails by following error:
[2019-03-28 16:16:07,439] {models.py:1788} ERROR - (18456, "Login failed for user 'XXXX'.DB-Lib error message 20018, severity 14:\nGeneral SQL Server error: Check messages from the SQL Server\nDB-Lib error message 20002, severity 9:\nAdaptive Server connection failed (***.***.***.28:1433)\n")
My code from DAG for Micrososft Sql Connection is following:
sql_command = """
select * from [sys].[tables]
"""
t3 = MsSqlOperator( task_id = 'run_test_proc',
mssql_conn_id = 'FIConnection',
sql = sql_command,
dag = dag)
I verified ip address and port number kind of configuration things by establishing connection through pymssql library from my local computer. Test code is following:
pymssql.connect(server="***.***.***.28:1433",
user="XXXX",
password="XXXXXX"
) as conn:
df = pd.read_sql("SELECT * FROM [sys].[tables]", conn)
print(df)
Could you please share if you have experienced this issue?
By the way I am using VirtualBox in Ubuntu 16.04 LTS
I had the same problem because freetds-dev was missing on linux:
apt-get install freetds-dev
Related
I am just learning how to use R to connect to SQL database. When I run my code, I am getting a message box says:
library(odbc)
con <- dbConnect(odbc(),Driver = "ODBC Driver 17 for SQL Serve",
Server ="DESKTOP-SHXXX",
Database ="BikeStores",
UID="robin",
PWD = rstudioapi::askForPassword("xxx"),
Trusted_Connection ="yes")
# Error: nanodbc/nanodbc.cpp:1021: 08001: [Microsoft][ODBC Driver 17 for SQL
# Server]Named Pipes Provider: Could not open a connection to SQL Server [2].
# [Microsoft][ODBC Driver 17 for SQL Server]Login timeout expired
# [Microsoft][ODBC Driver 17 for SQL Server]A network-related or instance-
# specific error has occurred while establishing a connection to SQL Server.
# Server is not found or not accessible. Check if instance name is correct
# and if SQL Server is configured to allow remote connections. For more
# information see SQL Server Books Online.
Any help?
I have a table in the default spark pool which i need to load into the dedicated sql pool in the azure synapse. Below are the codes I implemented, however it is not loading.
%%pyspark
spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "true")
new_df = spark.createDataFrame(segmentation_output)
new_df.write.mode("overwrite").saveAsTable("default.segmentation_output")
%%pyspark
new_df.createOrReplaceTempView("pysparkdftemptable")
%%spark
val scala_df = spark.sqlContext.sql ("select * from pysparkdftemptable")
scala_df.write.synapsesql("eana.bi.xim_CustomerSegment", Constants.INTERNAL)
Error : StructuredStream-spark package version: 2.4.5-1.3.1
StructuredStream-spark package version: 2.4.5-1.3.1
StructuredStream-spark package version: 2.4.5-1.3.1
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host eec8e890e9d5--0.tr624.northeurope1-a.worker.database.windows.net (redirected from emna-dv-ibsanalytics-wco-id-euno-sqs.database.windows.net), port 11030 has failed. Error: "connect timed out. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
I'm trying to connect impala to superset, and when I test the connection prints: "Seems OK!", and when I try to see databases on impala with the SQL Editor in the left side it shows all databases without problems.
Preview of Databases/Tables
But when i write a query and click on "Run Query", it gives the error: "Could not start SASL: b'Error in sasl_client_start (-1) SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket expired)'"
Error running query
I'm running superset with SSL and in production mode (with Gunicorn) and Impala with SSL in a Kerberized Hadoop Cluster, and my impala database config is:
Impala Config
And in the extras I put:
{
"metadata_params": {},
"engine_params": {
"connect_args": {
"port": 21050,
"use_ssl": "True",
"ca_cert": "path/to/my/ca_cert.pem",
"auth_mechanism": "GSSAPI"
}
},
"metadata_cache_timeout": {},
"schemas_allowed_for_csv_upload": []
}
How can I solve this error? In my superset log it only shows:
Triggering query_id: 65
INFO:superset.views.core:Triggering query_id: 65
Query 65: Running query on a Celery worker
INFO:superset.views.core:Query 65: Running query on a Celery worker
Versions: Superset 0.36.0, Impyla 0.16.2
I was able to fix this error doing this steps:
1 - Created service user for celery-worker, created a kerberos ticket for him and created a crontab to renew the ticket.
2 - Runned celery worker from this service user, instead running from root.
3 - Killed an celery-worker that was running in another machine of my cluster
4 - Restarted Impala and Superset
I think this error ocurred because in some queries instead of use the celery worker in my superset machine, it was using the celery worker that was in another machine without a valid kerberos ticket. I could fix this error because when I was reading celery-worker log , it showed that a connection with the celery worker in other machine failed in a query running.
I need to create a format file from a local database table but I get a connection error when I try to run the command in cmd. Here is the command I am using:
bcp OrderDatabase.dbo.Orders format nul -c -f D:\Format\Orders.fmt -T
I get the following errors:
SQLState = 08001, NativeError = 2
Error = [Microsoft][SQL Server Native Client 11.0]Named Pipes Provider: Could not open a connection to SQL Server [2].
SQLState = 08001, NativeError = 2
Error = [Microsoft][SQL Server Native Client 11.0]A network-related or instance-
specific error has occurred while establishing a connection to SQL Server.
Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.
SQLState = S1T00, NativeError = 0
Error = [Microsoft][SQL Server Native Client 11.0]Login timeout expired
I have checked Services.msc and SQL Server is indeed "Started". I have also checked Sql Server Configuration Manager and the SQL Server state is "Running" and Start Mode is "Automatic".
The connection string to the local database is:
Data Source=(LocalDB)\v11.0;AttachDbFilename=D:\Tools_TextileMagazine\CSharp_Template\Tools_TextileMagazine\OrderDatabase.mdf;Integrated Security=True
Any ideas to what could be wrong?
Add the -S parameter like specified here:
http://msdn.microsoft.com/it-it/library/ms162802.aspx
So your connection string will be:
bcp OrderDatabase.dbo.Orders format nul -c -f D:\Format\Orders.fmt -T -S localhost\V11.0
I assume from your connection string that you named istance: V11.0
I'm getting an error when running this query in SQL Server 2008:
EXEC master..xp_cmdshell'bcp
"select Row_ID, Service_Code,Skill from Account_Service"
queryout "d:\text.txt" -c -T -x'
Error is:
SQLState = 08001, NativeError = 2
Error = [Microsoft][SQL Server Native Client 10.0]Named Pipes Provider: Could not open a connection to SQL Server [2].
SQLState = 08001, NativeError = 2
Error = [Microsoft][SQL Server Native Client 10.0]A network-related or instance-specific error has occurred while
establishing a connection to SQL Server. Server is not found or not
accessible. Check if instance name is correct and if SQL Server is
configured to allow remote connections. For more information see
SQL Server Books Online.
SQLState = S1T00, NativeError = 0
Error = [Microsoft][SQL Server Native Client 10.0]
Login timeout expired
NOTE: Shared Memory, Named Pipes, TCP/IP are all enabled in SQL Server Configuration Manager
Look into the SQL Configuration Manager.
I guess TCP/IP Protocoll is not enabled, so you can Access your SQL Server via. Shared Memory.
So you would be able to Connect if you are on the SQL Server it self but can not Access it from another Maschine.
Otherwise I would say it is an Network Problem as you see the Connection times out. Maybe a Firewall Software is blocking or your Server can not be reached.
The most likely Cause I expririenced, fitting to oyur discrition is disabled TCP/IP Protocol for SQL Server.