I want use the python library ncclient 0.6.6 with Python 2.7.15 to connect to a NETCONF server (netopeer2) and read out the running config.
I tried to follow the example from the manual, running this code in the console:
with manager.connect(host="*the IP adress*", port=*the port*, timeout=None, username="*user*", password="*pwd*") as m:
c = m.get_config(source='running').data_xml
with open("%s.xml" % host, 'w') as f:
f.write(c)
As written in the manual, I try to disable public-key authentification with allow_agent and look_for_keys as False. Unfortunately, this does not work properly, because I get the error message:
File "<stdin>", line 1, in <module>
File "/home/sisc/.local/lib/python2.7/site-packages/ncclient/manager.py", line 177, in connect
return connect_ssh(*args, **kwds)
File "/home/sisc/.local/lib/python2.7/site-packages/ncclient/manager.py", line 143, in connect_ssh
session.connect(*args, **kwds)
File "/home/sisc/.local/lib/python2.7/site-packages/ncclient/transport/ssh.py", line 481, in connect
raise SSHUnknownHostError(known_hosts_lookup, fingerprint)
ncclient.transport.errors.SSHUnknownHostError: Unknown host key [e3:8d:35:a9:43:f9:3c:8a:f4:d3:88:5b:a9:36:93:59] for [[192.168.56.2]:1831]
I do not get why it still complains about the unknown host key, even though I explicitly disabled public-key authentification.
The netopeer NETCONF server is definitely running, for I get a "Hello" Message as soon as I try to SSH into it from out of the terminal.
Did I miss something?
m = manager.connect(host="172.17.0.2", port=830, username="netconf", password="netconf", hostkey_verify=False)
Did the trick. Hostkey_verify has to be false.
Related
Airflow version: 2.0.0
When I use the sftp command to manually connect to the host from any airflow worker everything works fine. Here is the error log from when I try to use the operator which under the hood uses the paramiko library to transfer files:
{ssh.py:202} WARNING - No Host Key Verification. This wont protect against Man-In-The-Middle attacks
{transport.py:1819} INFO - Connected (version 2.0, client 1.91)
{transport.py:1819} INFO - Auth banner: b'MOMENTUM SYSTEMS - SSH Server\nAuthentication Methods Supported:\nPUBLICKEY, PASSWORD'
{transport.py:1819} INFO - Authentication continues...
{transport.py:1819} INFO - Disconnect (code 2): unexpected service request
{taskinstance.py:1396} ERROR - Authentication failed.
Traceback (most recent call last):
File "/home/centos/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1086, in _run_raw_task
self._prepare_and_execute_task_with_callbacks(context, task)
File "/home/centos/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1260, in _prepare_and_execute_task_with_callbacks
result = self._execute_task(context, task_copy)
File "/home/centos/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1300, in _execute_task
result = task_copy.execute(context=context)
File "/home/centos/airflow-dags/utils/operators/s3_to_sftp.py", line 76, in execute
sftp_client = ssh_hook.get_conn().open_sftp()
File "/home/centos/.local/lib/python3.7/site-packages/airflow/providers/ssh/hooks/ssh.py", line 225, in get_conn
client.connect(**connect_kwargs)
File "/home/centos/.local/lib/python3.7/site-packages/paramiko/client.py", line 446, in connect
passphrase,
File "/home/centos/.local/lib/python3.7/site-packages/paramiko/client.py", line 764, in _auth
raise saved_exception
File "/home/centos/.local/lib/python3.7/site-packages/paramiko/client.py", line 751, in _auth
self._transport.auth_password(username, password)
File "/home/centos/.local/lib/python3.7/site-packages/paramiko/transport.py", line 1509, in auth_password
return self.auth_handler.wait_for_response(my_event)
File "/home/centos/.local/lib/python3.7/site-packages/paramiko/auth_handler.py", line 236, in wait_for_response
raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.
The Airflow connection that I use has the password and no additional options in extra.
The answer provided to the linked question worked for my use case:
Multi-factor authentication (password and key) with Paramiko
TL;DR: When I try to use flask db migrate with a remote database, I get: MySQLdb._exceptions.OperationalError: (1045, "Access denied for user ''#'localhost' (using password: YES)"). But I'm certain that the username and password are correct, the remote user has the SELECT privilege, and I don't actually have any models declared yet.
I have an SSH tunnel set up for access to a remote database.
$ ssh -L <local_port>:localhost:<remote_internal_port> <mysql_user>#<server> -p <remote_external_port>
I have credentials for a read-only user on the remote MySQL DB (really MariaDB). This is not the database the users are stored in, so I'm using SQLALCHEMY_BINDS to add the second database.
My environment looks like this:
DATABASE_URL=sqlite:////tmp/dev.db
DATABASE_BINDS="remote=mysql+mysqldb://<mysql_user>:<pass>#localhost:<local_port>/<db_name>"
Then in the config file:
from environs import Env
env = Env()
env.read_env()
SQLALCHEMY_DATABASE_URI = env.str("DATABASE_URL") # local
SQLALCHEMY_BINDS = env.dict("DATABASE_BINDS") # remote, etc
I don't presently have any models declared for the remote database at all. I'm planning to use reflection to load them.
The initial run of flask db init --multidb works fine. It creates the migration folder. But when I run flask db migrate -m "Initial DB", I get this:
Traceback (most recent call last):
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2275, in _wrap_pool_connect
return fn()
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 760, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 492, in checkout
rec = pool._do_get()
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/impl.py", line 238, in _do_get
return self._create_connection()
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/pool/base.py", line 639, in __connect
connection = pool._invoke_creator(self)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/path/to/project/venv/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/path/to/project/venv/lib/python3.7/site-packages/MySQLdb/__init__.py", line 84, in Connect
return Connection(*args, **kwargs)
File "/path/to/project/venv/lib/python3.7/site-packages/MySQLdb/connections.py", line 166, in __init__
super(Connection, self).__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (1045, "Access denied for user '<mysql_user>'#'localhost' (using password: YES)")
I've checked and double-checked that the username and password I've declared in the environment are correct. I can successfully use a GUI client with these settings to browse the remote MySQL database via the SSH tunnel. I can use the MySQL CLI client on the remote server to USE and SELECT on the appropriate database.
I'm at a loss for where to look here. I think the SSH tunnel is a red herring, since it's working well with the GUI client, but I don't know how to get more logging to see what the problem really is. This answer implies that the issue might in fact be one of permissions, not with the l/p themselves. Does Flask-Migrate require something other than the SELECT privilege?
The problem was that I was using "localhost" instead of "127.0.0.1".
I think my university recently changed a firewall or filtering protocol with regards to Twitter's streaming API. However, for me to provide them more information, I'm going to have to figure out the details.
Ok, below is the simplest possible program to collect data from Twitter's stream:
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener
ckey="[OMITED]"
csecret="[OMITED]"
atoken="[OMITED]"
asecret="[OMITED]"
class listener(StreamListener):
def on_data(self, data):
print(data)
return(True)
def on_error(self, status):
print status
auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)
twitterStream = Stream(auth, listener())
twitterStream.filter(track=["car"])
When I run this program on my laptop/home desktop, it works fine. However, I have a university computer setup for data collection and it produces the following error:
Traceback (most recent call last):
File "test.py", line 25, in <module>
twitterStream.filter(track=["car"])
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 228, in filter
self._start(async)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 172, in _start
self._run()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tweepy/streaming.py", line 106, in _run
conn.connect()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1278, in connect
server_hostname=server_hostname)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 352, in wrap_socket
_context=self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 579, in __init__
self.do_handshake()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 808, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Now, the thing is, I can open up Safari and browse the web. Further, data collection like this use to work (as of a month ago). The error says the SSL cert is failing verification (but strangely the original OAuth seems to work -- it is the filter command kicking up the error). Is there a way to get it to spit out if that is because an intermediate cert has been put in place? Or maybe a port is being blocked? Basically, is there a way to get a lot more details so I can go the university and say: 'you are blocking X'?
Any suggestions?
I've resolved the situation. It was an error relating to SSL sites that were signed with Digicerts.
I encountered the following perculiar behaviour today.
The following code works on Python 3.3:
smtp = smtplib.SMTP()
smtp.connect(host="smtp.gmail.com", port=587)
smtp.ehlo()
smtp.starttls()
In Python 3.4 the above code doesn't work, instead the following error is encountered:
File "smtp_test.py", line 10, in <module>
smtp.starttls()
File "/usr/lib/python3.4/smtplib.py", line 676, in starttls
server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 344, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 540, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 767, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_DECODE_ERROR] tlsv1 alert decode error (_ssl.c:598)
If the above code is modified to specify the host and port in the constructor and not use the connect method, as in the code below, then it works.
smtp = smtplib.SMTP(host="smtp.gmail.com", port=587)
smtp.ehlo()
smtp.starttls()
The above behaviour occurs with OpenSSL version 1.0.1f and OpenSSL 1.0.1g
Could someone explain this behaviour to me ?
According to a tcpdump the code in 3.4 sends in SNI extension with an empty target name. SNI (Server Name Indication) is used when having different certificates behind the same IP address. I consider this a bug: if it does not have a name it should not send the SNI extension instead of sending an extension with a zero-length name in it.
My config file:
Host server
User new_user
HostName 10.0.1.193
Port 55555
LocalForward 3000 10.0.1.193:6000
IdentityFile ~/.ssh/server
Client.py
import xmlrpclib
s = xmlrpclib.ServerProxy('http://localhost:3000')
print s.pow(2,3) # Returns 2**3 = 8
print s.add(2,3) # Returns 5
print s.div(5,2) # Returns 5//2 = 2
# Print list of available methods
print s.system.listMethods()
Server.py
from SimpleXMLRPCServer import SimpleXMLRPCServer
from SimpleXMLRPCServer import SimpleXMLRPCRequestHandler
# Restrict to a particular path.
class RequestHandler(SimpleXMLRPCRequestHandler):
rpc_paths = ('/RPC2',)
# Create server
server = SimpleXMLRPCServer(("localhost", 6000),
requestHandler=RequestHandler)
server.register_introspection_functions()
# Register pow() function; this will use the value of
# pow.__name__ as the name, which is just 'pow'.
server.register_function(pow)
# Register a function under a different name
def adder_function(x,y):
return x + y
server.register_function(adder_function, 'add')
# Register an instance; all the methods of the instance are
# published as XML-RPC methods (in this case, just 'div').
class MyFuncs:
def div(self, x, y):
return x // y
server.register_instance(MyFuncs())
# Run the server's main loop
server.serve_forever()
My server.py is running fine, but when I run my client.py, it gives the following error:
Traceback (most recent call last):
File "client.py", line 4, in <module>
print s.pow(2,3) # Returns 2**3 = 8
File "/usr/lib/python2.7/xmlrpclib.py", line 1224, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1578, in __request
verbose=self.__verbose
File "/usr/lib/python2.7/xmlrpclib.py", line 1264, in request
return self.single_request(host, handler, request_body, verbose)
File "/usr/lib/python2.7/xmlrpclib.py", line 1292, in single_request
self.send_content(h, request_body)
File "/usr/lib/python2.7/xmlrpclib.py", line 1439, in send_content
connection.endheaders(request_body)
File "/usr/lib/python2.7/httplib.py", line 954, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 814, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 776, in send
self.connect()
File "/usr/lib/python2.7/httplib.py", line 757, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 571, in create_connection
raise err
socket.error: [Errno 111] Connection refused
I have checked that my ssh if working and I can ssh into remote server with the given configuration i.e.
ssh server
works find. Can anyone explain what might be going wrong?
Your server runs and perhaps it does not complain, but this does not mean it "runs correctly" or more pointedly it doesn't mean the server is in a working state that the client expects.
The above is somewhat cryptic for a reason: something unknown has gone wrong, and even though you don't know yet what's broken, you want to start testing things you know should work and verify they are in fact working. This is a useful debugging skill even if the error is meaningless to you.
In this case, the client error message is "connection refused", meaning "refused [at the server]".
Try this:
on your "client" PC in a Terminal/DOS window, run:
telnet [your server ip] [your server port]
You should expect the same error - a connection refused. Perhaps the server is not actually opening the port. Or perhaps the server opened the port, but you can not see it remotely on another host due to a firewall on the server.
Also, running both client and server code on the same host can sometime reveal more clues (it should work but if it doesn't then there's maybe more than 1 problem).