I'm serving my site through nginx. For securing it, I have added ssl certificate and made it compatible with https protocol.
Now when I do request data from the site through browser while keeping ssl proxying on, whole request body and response body are showing there, so there is some loophole in my configuration and if it's not a loophole, I want it to be like giant company's site - facebook, apple etc. Where these ssl proxy tool can not parse the request and response.
If the client doesnt explicit show itself as a proxy (aka via X-Forwarded headers), is very hard to know for a server if any connection establishes proxied, Of course, out there are sophisticated methods to find these connections, like blacklists with common proxy sites, AI traffic algorithms, etc. but you will need massive amounts of data (that giant companies have) or specialized traffic services like cloudflare.
Related
When I run an IOS App through proxy using tools such as Charles, Burp suite and Proxyman I'm not able to see the full request (receiving 403 when SSL is enabled) of the final end point after loged in, and also the App just stop working(Just work when SSL is disabled). I would like to see what the full request looks like to do it using Postman and HttpClient in Java. Is there anything I could do in order to get status 200 like when SSl is disabled ?
Any help to try bypass it is appreciated.
This is probably due to TLS fingerprinting.
Platforms like Cloudflare offer services to block bots and other non-browser traffic, which analyse low-level details of how a client makes connections, including the TLS fingerprint, and use this to spot unusual traffic. Because proxies like the ones you're using create separate upstream TLS connections when they intercept incoming connections, they can often end up sending HTTPS traffic where the TLS fingerprint the server sees doesn't match the HTTP headers, which is sufficient for the connection to be blocked as 'unusual'.
Defeating this is not easy. I also maintain an HTTPS debugging proxy called HTTP Toolkit, where I've done some work to defeat this, by optimizing individual fields of TLS handshakes to avoid common blocks (some more details: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/). You might have more success using HTTP Toolkit in this case, since it can successfully avoid 90% of these kinds of blocks at the moment.
(Note that HTTP Toolkit doesn't have automated setup for iOS yet, but you can still use it manually like any other debugging proxy - you just need to trust the certificate & set your proxy settings)
That said, this still won't work 100% of the time. This is a cat & mouse game between bots and site-scrapers and bot-blocking services, where the criteria are constantly changing & tightening, so there is no perfect solution. Your best bet is to keep trying different tools, and see which ones can make themselves look least suspicious while proxying your traffic.
Ant-Media-Server is running on an IPAdress without any domains. We just set up this server to be used for streaming in order to use it from different domains pointing to different servers.
Since all of our domains use ssl, we face the typical connection problem:
mixed Content: The page at 'https://SOMEDOMAIN.com/QUERY' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://1.2.3.4:56'. This request has been blocked; this endpoint must be available over WSS.
Ant-Media already offers tutorials on how to install a Let's Encrypt SSL Certificate but sadly it is not available for pure IP-Addresses.
Apart from the Ant-Media Service, the server doesn't has any NGINX, NodeJS, Apache or other http Servers installed - the plan was just to use it for streaming by calling the IP-Address.
Do you have any ideas on how to solve that problem?
Unfortunately, this is not possible.
The goal of having a SSL is ensure you are requesting the right domain name besides encrypting the content between your users and your server.
Here are some alternatives:
create an endpoint in your own app that proxies data to your server.
Instead of playing the IP address, you can play:
/your-proxy-url?stream=http://yourIp.com:port/....
Note that using a proxy will make all the traffic pass through your web app.
As a reference, if you are using PHP on your website, you can have some ideas from here: https://gist.github.com/iovar/9091078
Create a reverse-proxy in front of your web app that redirects the traffic to your IP address.
Both solutions does not change your Ant Media Server, just adds a new resource between your users and your streaming server - adding the SSL on it.
I read an awesome post on application server vs. webserver at What is the difference between application server and web server?. Moreover, Difference between proxy server and reverse proxy server nicely explains what a proxy server is.
I also learned that some web servers, such as Apache, have reverse proxy built-in. (Source). Also, Wikipedia (https://en.wikipedia.org/wiki/Reverse_proxy) has an image that shows webserver and reverse proxy as separate entities.
Source: https://en.wikipedia.org/wiki/Reverse_proxy (image originally via Privacy Canada, now CC0, license info)
So, I am not sure about the difference between webserver and reverse proxy. Can someone please shed the light?
A web server listens for HTTP requests and reacts to them by sending back an HTTP response.
A reverse proxy is a web server which determines what response to make by also implementing an HTTP client.
Client A makes an HTTP request to the reverse proxy. The reverse proxy makes an HTTP request to Server B. Server B sends an HTTP response to the reverse proxy. The reverse proxy sends that data as its HTTP response to client A.
A proxy server is a go‑between or intermediary server that forwards requests for content from multiple clients to different servers across the Internet. A reverse proxy server is a type of proxy server that typically sits behind the firewall in a private network and directs client requests to the appropriate backend server. A reverse proxy provides an additional level of abstraction and control to ensure the smooth flow of network traffic between clients and servers.
Common uses for a reverse proxy server include:
Load balancing – A reverse proxy server can act as a “traffic cop,” sitting in front of your backend servers and distributing client requests across a group of servers in a manner that maximizes speed and capacity utilization while ensuring no one server is overloaded, which can degrade performance. If a server goes down, the load balancer redirects traffic to the remaining online servers.
Web acceleration – Reverse proxies can compress inbound and outbound data, as well as cache commonly requested content, both of which speed up the flow of traffic between clients and servers. They can also perform additional tasks such as SSL encryption to take load off of your web servers, thereby boosting their performance.
Security and anonymity – By intercepting requests headed for your backend servers, a reverse proxy server protects their identities and acts as an additional defense against security attacks. It also ensures that multiple servers can be accessed from a single record locator or URL regardless of the structure of your local area network.
A web server stores and delivers the content for a website – such as text, images, video, and application data – to clients that request it. The most common type of client is a web browser program, which requests data from your website when a user clicks on a link or downloads a document on a page displayed in the browser.
A web server communicates with a web browser using the Hypertext Transfer Protocol (HTTP). The content of most web pages is encoded in Hypertext Markup Language (HTML). The content can be static (for example, text and images) or dynamic (for example, a computed price or the list of items a customer has marked for purchase). To deliver dynamic content, most web servers support server‑side scripting languages to encode business logic into the communication. Commonly supported languages include Active Server Pages (ASP), Javascript, PHP, Python, and Ruby.
A web server might also cache content to speed delivery of commonly requested content. This process is also known as web acceleration.
A web server can host a single website or multiple websites using the same software and hardware resources, which is known as virtual hosting. Web servers can also limit the speed of response to different clients so as to prevent a single client from dominating resources that are better used to satisfy requests from a large number of clients.
While web servers typically host websites that are accessible on the Internet, they can also be used to communicate between web clients and servers in local area networks such as a company’s intranet. A web server can even be embedded in a device such a digital camera so that the users can communicate with the device via any commonly available Web browser.
We are currently using Apache to handle incoming SSL requests. These are two-way SSL connections. Apache accepts the https connection and pass the request on as http connection to the application server. This works well for us.
We would like to use the same kind of centralized mechanism for outgoing two-way SSL connections. Is there a way do this with Apache or another product? To complicate things the client certificate needed to identify out client can vary depending on the destination.
In short:
- Internal clients connect through http to Apache or another product.
- Apache or another product knows based on a rule (?) that a two-way ssl connection is required and sets this up with the destination.
- Depending on the destination the correct certificate is sent to identify our client.
Regards,
Nidkil
What you're talking about is, or course, an HTTP proxy server. In the first scenario you are using it as a transparent proxy to provide SSL support for connections to a set of web pages. In the second scenario you want to use it to provide connections to secure-only pages on behalf of clients speaking HTTP.
You can do this with the Squid proxy, which is free and open-source, provided that your machine sits between the clients and the Internet. Look for "SSLBump". You do need a certificate which the clients would consider valid for all web pages to be accessed (otherwise they will notice what you are doing, which is basically a man-in-the-middle attack).
However, I would strongly recommend against this - if a site requires SSL, it is likely to do so for a reason. It is almost certainly not OK to have internal clients connecting to an online banking site and have you bumping down their encryption so that you can monitor their traffic or whatever...
What is difference between https protocol and SSL Certificate that we use in web browser?
Aren't both of these used to encrypt communication between client (browser) and server?
HTTPS is HTTP (HyperText Transfer Protocol) plus SSL (Secure Socket Layer). You need a certificate to use any protocol that uses SSL.
SSL allows arbitrary protocols to be communicated securely. It enables clients to (a) verify that they are indeed communicating with the server they expect and not a man-in-the-middle and (b) encrypt the network traffic so that parties other than the client and server cannot see the communication.
An SSL certificate contains a public key and certificate issuer. Not only can clients use the certificate to communicate with a server, clients can verify that the certificate was cryptographically signed by an official Certificate Authority. For example, if your browser trusts the VeriSign Certificate Authority, and VeriSign signs my SSL certificate, your browser will inherently trust my SSL certificate.
There's some good reading here: http://en.wikipedia.org/wiki/Transport_Layer_Security
Two pieces of one solution.
https is the protocol that defines how the client and server are going to negotiate a secure connection.
The SSL Certificate is the document that they will use to agree upon the servers authenticity.
HTPS is the new HTTPS.
HTTPS is highly vulnerable to SSL Stripping / MITM (man in the middle).
to quote adam langley's (google) blog imperial violet:
"HTTPS tends to cause people to give talks mocking certificate security and the ecosystem around it. "
The problem is that the page isn't served over HTTPS. It should have been, but when a user types a hostname into a browser, the default scheme is HTTP. The server may attempt to redirect users to HTTPS, but that redirect is insecure: a MITM attacker can rewrite it and keep the user on HTTP, spoofing the real site the whole time. The attacker can now intercept all the traffic to this perfectly well configured and secure website.
This is called SSL stripping and it's terribly simple and devastatingly effective. We probably don't see it very often because it's not something that corporate proxies need to do, so it's not in off-the-shelf devices. But that respite is unlikely to last very long and maybe it's already over: how would we even know if it was being used?
In order to stop SSL stripping, we need to make HTTPS the only protocol. We can't do that for the whole Internet, but we can do it site-by-site with HTTP Strict Transport Security (HSTS).
HSTS tells browsers to always make requests over HTTPS to HSTS sites. Sites become HSTS either by being built into the browser, or by advertising a header:
Strict-Transport-Security: max-age=8640000; includeSubDomains
The header is in force for the given number of seconds and may also apply to all subdomains. The header must be received over a clean HTTPS connection.
Once the browser knows that a site is HTTPS only, the user typing mail.google.com is safe: the initial request uses HTTPS and there's no hole for an attacker to exploit.
(mail.google.com and a number of other sites are already built into Chrome as HSTS sites so it's not actually possible to access accounts.google.com over HTTP with Chrome - I had to doctor that image! If you want to be included in Chrome's built-in HSTS list, email me.)
HSTS can also protect you, the webmaster, from making silly mistakes. Let's assume that you've told your mother that she should always type https:// before going to her banking site or maybe you setup a bookmark for her. That's honestly more than we can, or should, expect of our users. But let's say that our supererogatory user... ]
because of obstructing/very stupid link-rules for new users on stackoverflow i cannot give you the rest of adam's answer and you'll have to visit adam langley's blog yourself at
https://www.imperialviolet.org/2012/07/19/hope9talk.html
"Adam Langley works on both Google's HTTPS serving infrastructure and Google Chrome's network stack."
HTTPS is an application layer protocol. It can provide non-repudiation of individual requests or responses through digital signatures.
SSL is a lower level protocol and does not have this capability. SSL is a transport level encryption.
HTTPS is more flexible than SSL: an application can configure the level of security it needs. SSL has fewer options so it is easier to setup and administer.