NiFi cannot connect to itself via RPG in an secured mode - ssl

i have running some NiFi stand alone instances in secured mode. connecting each other vie site-to-site is working fine, i'm not able to get a connection to itself.
let's assume some nifi instances
- https://my-nifi-1.local:9443/nifi
- https://my-nifi-2.local:9443/nifi
- https://my-nifi-3.local:9443/nifi
doing a remote connection between https://my-nifi-1.local:9443/nifi and https://my-nifi-2.local:9443/nifi and https://my-nifi-3.local:9443/nifi works properly.
if my flow sends some data via "Remote connections Output Port" to a Remot-Process-Group on the top level of the flow, then its not working.
ErrorMessage: forbidden - Site-to-Site is not secure.
i tried to configure my RPG using https://my-nifi-1.local:9443/nifi or https://localhost:9443/nifi.
THX for any help

found the solution
Each instances CN has to be added to the users and also there are some policies to apply
Thanks to Bryan Bende I could figure it out -> https://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site

Related

My website running in Oracle Cloud is down, how can I repair it?

I have a website that is running in Oracle Cloud, the website now is down and I tried to access using SSH in Putty but it cannot be accessed I only see the terminal screen.
What should I do?
Could you please check if the instance is running or not. If the instance is running please check for the ingress/egress firewall rules at security list/NSG/OS firewall for connectivity to the instance. Make sure you are not connected to any VPN or any corporate network. Once you are able to connect to the instance check for the status of the service. It should be Active.
Could you please confirm, were you able to connect to the instance before? Since when you are facing this issue?
For your reference: connecting to your instance.

Apache Airflow unable to establish connect to remote host via FTP/SFTP

I am new to Apache Airflow and so far, I have been able to work my way through problems I have encountered.
I have hit a wall now. I need to transfer files to a remote server via sftp. I have not had any luck doing this. So far, I have gotten S3 and Postgres/Redshift connections via their respective hooks to work in various DAGs. I have been able to use the FTPHook with success testing on my local FTP server, but have not been able to figure out how to use SFTP to connect to a remote host.
I am able to connect to the remote host via SFTP with FileZilla, so I know my credentials are correct.
Through Google searching I have found the SFTPOperator, but am not able to figure out how to use it. I have also found FTPSHook, but still I have not been able to get it to work.
I keep getting the error nodename nor servname provided, or not known or a general Operation timed out in my Airflow logs.
Can someone point me in the right direction? Should I be using the FTPSHook with SSH or FTP Airflow Conn Type? Or do I need to utilize the SFTPOperator? I am also confused as to how I am supposed to setup the credentials in my Airflow connections. Do I use the SSH profile or FTP?
If I can provide any more additional info that may help, please let me know.
Cheers!
SFTPOperator is using ssh_hook underhood to open sftp transport channel that serves as a basis for file transfer. You can either configure ssh_hook by yourself or provide connection id via ssh_conn_id.
op = SFTPOperator(
task_id="test_sftp",
ssh_conn_id="my_ssh_connection",
local_filepath="",
remote_filepath="",
operation=SFTPOperation.PUT,
dag=dag
)

Cannot connect to RDS SQL Server Database using Management Studio

I created a SQL Server RDS Instance in AWS and it seems to be up and running, but if I try to connect to it using Management Studio I get this error:
Here is the text of the error:
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: TCP Provider, error: 0 - The wait operation timed out.) (Microsoft SQL Server, Error: 258)
I initially tried with the default security group that was created with the instance, but when that didn't work I created a new security group and modified the instance to use it.
Here you can see the details:
I tried this connection setup to connect:
Server Type: Database Engine
Server Name: valuationdlsdev.ck1qvjqhglyg.us-west-2.rds.amazonaws.com,1433
Authentication: SQL Server Authentication
Login: the Master User Login I created when creating the RDS Instance
Password: the Master User Password I created when creating the RDS Instance
I was kinda at my wits end and so I changed the setting on the Security Group to All traffic just to see if that would work, so here are all the settings on the security group:
At this point I'm wondering if port 1433 is not open, because I feel like I've tried everything. Could someone please help me.
Thanks.
In my case I opened the VPC Security group associated with my database
In the EC2 Security groups dashboard I selected Edit Inbound Rules from the actions dropdown and chose edit inbound rules.
At first, I looked at the inbound rules and thought everything was OK since this was the current setup
After all - if it was allowing all traffic, then what could possibly be wrong?
On a whim I added a rule for TCP port 1433. Ending up with this simple setup
Then it immediately started working for me.
Make sure it is publicly accessible, there is a radio button you have to check to make it publicly accessible.
Also add MS SQL inbound rule in inbound tab.After making the change wait for sometime so that the settings are updated in the instance.
In my experience this was counter-intuitive. With the options I selected, all ports and IPs seemed to be open, but after editing the inbound and outbound rules in the security group to have MS SQL for anywhere, I was able to connect.
For inbound rules, go to the VPC Security group of your database instance
In Inbound tab click modify
In column source change ip 0.0.0.0 by your IP by "My IP" or "Anywhere"
I had the same issue.
I ended up deleting the security group inbound rules, and just added a new inbound rule for port 1433, source being: 0.0.0.0
Image attached.
inbound rules
Thanks for the discussion here. Just post my finding in case anyone needs help in the future.
I initially followed this guide https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToMicrosoftSQLServerInstance.html.
then, I got some ideas from this post and figure out my particular issue in the end. https://forums.aws.amazon.com/thread.jspa?messageID=845682 The poster really did wonderful troubleshooting steps which could help fix most of the general Error 258 problems already. In the end I used the suggestion from the answerer to find out my problem.
In terms of my case of encountering error 258, I tried to connect to RDS SQL server 2016 inside a secure network from my workplace. When I switched to use the public network served by some Telecomm vendor, the connecting was succeeded.
If you want to access from different network were the instance was created, you'll need to open access to the IP range of where you want to access, by going to the "security group" assigned to your DB instance, and then adding the rule for your IP range.
PD. AWS by default only allow access from the IP range of the machine where you activated "public access" to the instance.
I was also not able to access it from my office laptop, but I was able to access it from my personal laptop. I think it is because of some company firewall rules.
In case anyone comes across this post looking for an answer, I just wanted to updated and make sure it's there if anyone needs it. The issue here turned out to be that I misunderstood the way "Publicly Accessible" works and set it to "Yes". Apparently it should have been set to "No". "Yes", however does work for the SQL Server Express version.

Microsoft Azure VPN WebApp not communicating with external SQL

The problem I have is that we're trying to use our WebApp in Microsoft Azure to connect to an external SQL-database (not our own) through a VPN. The SQL-database is only allowing connections from our local IP-addresses that we put up as a Network (for example 176.0.0.0/24).
We are now connected to the same virtual private network, and through our Azure-VM we can now connect to the SQL-Server through SQL Server Management Studio.
Now we want to do the same with a WebApp, but we're not getting through to the server. It gets "Not authenticated" before reaching the SQL-Server (probably the server isn't accepting our IP from the WebApp).
The different problems I have tried to look through is:
Do we only try to connect through our Outbound IPs?
Is the WebApp not connected to the VPN?
I have unfortunatley not found any real answers, and neither any solutions to my problem. If you have any ideas of how to solve our problem, or maybe know how I could tunnel all of the SQL-calls through the VM, the help would be very much appreciated!
Hybrid connection is one option. What you can also do is enable point-to-site in your VPN. Once you do that, you can directly integrate your web app to the vnet and your connections will work. (Go to your web app -> Settings -> Networking -> VNet Integration)
If your Vnet is V1(older way of creating VNs) then enabling point to site is very straightforward. You can do it through portal. For V2 Vnets you have to do it through powershell commands.
Here is a link for the documentation which explains both the options.
https://azure.microsoft.com/en-in/documentation/articles/web-sites-integrate-with-vnet/
There's a way to "tunnel all of the SQL-calls through the VM". You may want to use hybrid connections (cf https://azure.microsoft.com/en-us/documentation/articles/integration-hybrid-connection-overview/).
The principle is to have an agent installed on the VM that can access the database with the correct IP address.
Suppose you can access the SQL DB as mysqldbsrv from the VM. You add an hybrid connection associated to your web app, you install the agent on the VM. Then, when you connect to mysqldbsrv from the Web App, you go through the VM.

openfire connect clients other than spark

I have openfire installed on my windows 7 and I am able to add users in the web admin interface .I am also able to connect clients using spark (locally on localhost).When I try to connect with other clients the connection fails .I have tried Psi (http://psi-im.org/) and the gui example for QXmpp (https://github.com/qxmpp-project/qxmpp/tree/master/examples/GuiClient) .Psi fails saying it is problem with certificates: There was an error communicating with the server.Details: Authentication error: Not authorized. I looked into the security settings and can't find anything that lets the QXmpp gui client connect successfuly .
What can I do to minimize security concerns so my QXmpp clients can connect successfully?I might enable those later as I get more comfortable with xmpp programming. I would appreciate any piece of help.
I don't know if openfire disables other clients to connect by default ,but by installing the Client Control plugin(http://www.igniterealtime.org/projects/openfire/plugins/clientControl.jar) and allowing all clients to connect I was able to connect my QXmpp clients.Hope this helps somebody some day.