Disable Content Security Policy in PhantomJS - phantomjs

I am trying to use a WebSocket in a page.evaulate() call in PhantomJS (2.1.1). When trying to connect to the WebSocket server, I get the following error:
SecurityError: DOM Exception 18: An attempt was made to break through the security policy of the user agent.
I was able to disable the CSP in Firefox by going to about:config and setting security.csp.enable to false. After doing this, I could run the same code from the PhantomJS script by pasting it into the console in Firefox with no problems.
Is it possible to disable Content Security Policy in PhantomJS? (I have tried setting page.webSecurityEnabled = false and page.localToRemoteUrlAccessEnabled = true to no avail.)

PhantomJS latest stable version 2.1.1 has a bug when it tries to get a webpage that has the following response header:
content-security-policy: default-src 'none'
The Fix:
Branch 2.1.1, File: \phantomjs\src\qt\qtwebkit\Source\WebCore\page\ContentSecurityPolicy.cpp, Change line 354:
, m_allowEval(false)
To:
, m_allowEval(true)
Then build.py -r ... Enjoy!

Related

How to solve flutter web api cors error only with dart code?

It seems like CORS error is well-known issue in the web field. But I tried flutter web for the first time ever and I faced critical error.
The code below worked well in app version when it was running on iOS device, but when i tested the same code on Chrome with web debugging from beta channel, it encountered CORS error.
Other stackoverflow answers explained how to solve the CORS issue with serverside files of their projects. But I have totally no idea what is server thing and how to deal with their answers. The error message from Chrome console was below
[ Access to XMLHttpRequest at 'https://kapi.kakao.com/v1/payment/ready' from origin 'http://localhost:52700' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. ]
So, what i want to do is to solve above 'Access-Control-Allow-Origin header' issue ONLY WITH DART CODE! Code below is what i've tried to solve these issues only with my main.dart.
onPressed: () async {
var res =
await http.post('https://kapi.kakao.com/v1/payment/ready', encoding: Encoding.getByName('utf8'), headers: {
'Authorization': 'KakaoAK $_ADMIN_KEY',
HttpHeaders.authorizationHeader: 'KakaoAK $_ADMIN_KEY',
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS, PUT, DELETE, HEAD",
}, body: {
'cid': 'TC0ONETIME',
'partner_order_id': 'partner_order_id',
'partner_user_id': 'partner_user_id',
'item_name': 'cool_beer',
'quantity': '1',
'total_amount': '22222',
'vat_amount': '2222',
'tax_free_amount': '0',
'approval_url': '$_URL/kakaopayment',
'fail_url': '$_URL/kakaopayment',
'cancel_url': '$_URL/kakaopayment'
});
Map<String, dynamic> result = json.decode(res.body);
print(result);
},
Even though i actually had the header "Access-Control-Allow-Origin": "*" which most other answers recommended, the Chrome console printed same error message. Weird thing is that the same code made successful request in mobileApp version. So I think this is only problem with flutter WEB VERSION.
Hope somebody can figure it out and suggest only-dart code to resolve the issue in my main.dart!! Thank you for reading [:
1- Go to flutter\bin\cache and remove a file named: flutter_tools.stamp
2- Go to flutter\packages\flutter_tools\lib\src\web and open the file chrome.dart.
3- Find '--disable-extensions'
4- Add '--disable-web-security'
Since Flutter 3.3.0
I implemented the option to add any browser flag to the flutter command.
flutter run -d chrome --web-browser-flag "--disable-web-security"
Or for drive command:
flutter drive --driver=test_driver/integration_test.dart --target=integration_test/app_test.dart -d web-server --web-browser-flag="--disable-web-security"
Note: This is just for development and testing. Flutter is executed explicitly on the client's browser! You should NOT and you can NOT disable it in production (as stated by #Tommy), as it is a security feature of the browser, and not meant to be changed in dart code. You have to enable CORS on your web server, which is providing the resources of your Flutter app, to ensure it works for everyone.
If you use dart language without Flutter on the server side with shelf, then see this response.
I think disabling web security as suggested will make you jump over the current error for now but when you go for production or testing on other devices the problem will persist because it is just a workaround, the correct solution is from the server side to allow CORS from the requesting domain and allow the needed methods, and credentials if needed.
run/compile your Flutter web project using web-renderer. This should solve the issue both locally and remotely:
flutter run -d chrome --web-renderer html
flutter build web --web-renderer html
This is a CORS (cross-origin resource sharing) issue and you do not have to delete/modify anything. You just have to enable the CORS request from your server-side and it will work fine.
In my case, I have created a server with node.js and express.js, so I just added this middleware function that will run for every request.
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Methods", "GET,PUT,PATCH,POST,DELETE");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
And BOOOOM! I received the data.
You just have to look at the settings to enable CORS for your server.
Server side engine like node js or django is really needed to work with flutter web with bunch of external apis. Actually there's high possibility of same CORS error when we try to use internal api because of the CORS mechanism related to port number difference.
There are bunch of steps and answers from SO contributors that recommend to use chrome extensions to avoid CORS errors, but that is actually not cool for users. All the users should download the browser extensions to use the single website from us, which wouldn't be there if we used true server engines.
CORS is from browser as far as i know, so our flutter ios and android apps with same api code don't give those CORS errors. First time i encountered this error with flutter web, i believed i can deal with CORS in my app code lines. But that is actually not healthy way for users and long term dev plans.
Hope all flutter web newbies understand that web is quite a wild field for us. Even though i'm also newbie here, i highly recommend all the flutter web devs from 1.22.n stable to learn server side engines like node js. It is worth try.
And if u came so far down to this line of my self-answer, here's a simple guide for flutter web with node js. Flutter web is on stable channel but all those necessary infra are not fully ready for newbies like me. So be careful when you first dive into web field, and hope you re-check all the conditions and requirements to find out if you really need web version of your flutter app, and also if you really need to do this work with flutter. And my answer was yes lol
https://blog.logrocket.com/flutter-web-app-node-js/
If you run a Spring Boot server, add "#CrossOrigin" to your Controller or to your service method.
#CrossOrigin
#PostMapping(path="/upload")
public #ResponseBody ResponseEntity<Void> upload(#RequestBody Object object) {
// ...
}
I know the question explicitly asked for a solution "with dart code" only, but I was not able to fix the exception with dart code (for example by changing the header).
The disabling web security approaches work well in development, but probably not so well in production. An approach that worked for me in production dart code involves avoiding the pre-flight CORS check entirely by keeping the web request simple. In my case this meant changing the request header to contain:
'Content-Type': 'text/plain'
Even though I'm actually sending json, setting it to text/plain avoids the pre-flight CORS check. The lambda function I'm calling didn't support pre-flight OPTIONS requests.
Here's some info on other ways to keep a request simple and avoid a pre-flight request
https://docs.flutter.dev/development/platform-integration/web-images
flutter run -d chrome --web-renderer html
flutter build web --web-renderer html
This official solution worked for me on Chrome only (Source). But I had to run it first every time.
flutter run -d chrome --web-renderer html
And disabling web security also worked (Source). But the browsers will show a warning banner.
But In case you are running on a different browser than Chrome (e.g. Edge) and you want to keep 'web security' enabled.
You can change the default web renderer in settings in VS Code
File ==> Preferences ==> Settings ==> Enter 'Flutter Web' in the Search Bar ==> Set the default web renderer to html
After hours of testing, the following works perfectly for me.
Add the following to the PHP file:
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
This allow the correct connection with the HTTP GET POST with no issue from flutter for me.
I discovered this in the following discussion:
XMLHttpRequest error Flutter
I am getting the same error with php api so i add the php code these lines ;
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
I think you may not doing this in right way.
The cors headers should be added in HTTP response header while you added them in you reuqest header obviously.
for more information check out the documentation https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#what_requests_use_cors
The below solution is great if you are only communicating with a local NodeJS server.
Install NodeJS
Create a basic NodeJS express project
Create a folder to put you NodeJS project in
ex: C:\node_project\
in PowerShell run: npm init in the folder
fill in your desired values
entry point: must be app.js for this example to work
in PowerShell run: npm install express in the folder
create a app.js file in the folder
// init express
const express = require("express");
const app = express();
// set the path to the web build folder
app.use(express.static("C:/Users/your_username/path_to_flutter_app/build/web"));
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log(`Server listening on port ${PORT}...`);
});
The value "C:/Users/your_username/path_to_flutter_app/build/web" must be changed to the web build folder in your flutter app.
The app can be accessed through your browser once the app is built, the node server is running, and the browser is at the correct address
Build the app
open PowerShell and navigate to the flutter project's root ex: C:/Users/your_username/path_to_flutter_app/
run flutter build web
turn on the node server
open PowerShell and navigate to the NodeJS server folder ex: C:\node_project\
run: node app.js
Open in your browser
Enter http://localhost:8080/ into the browser
Note that everytime you change your flutter app's dart code you will need to re-run flutter build web
Wrong Server on Target Port 🤦
I feel silly for even admitting this, but I had some other local server running on the targeted port. I've no clue why the server seemed to boot on the same port, or why the iOS app seemed to work, but now that I'm hitting the actual server it's working fine.
I was also getting some 404's mixed in, but originally thought that was due to the CORs error.
Maybe someone else has this same issue and this helps them.
In my case, The problem was in laravel backend code which did not support CORS, So I added the CORS into backend project then it worked successfully in test and live.
The 5th step of Osmans answer should be to add the option
'--disable-site-isolation-trials',
Only this works for me.
Chrome version 106.0.5249.119
Update:
I recommend to use User Rebo's answer. It is now possible to pass --disable-web-security as a browser flag to run & drive commands!
Original outdated answer:
Alternative solution for MacOS & Android Studio (without modifying Flutter source)
We use a similar approach as Osman Tuzcu. Instead of modifying the Flutter source code, we add the --disable-web-security argument in a shell script and just forward all other arguments that were set by Flutter. It might look overly complicated but it takes just a minute and there is no need to repeat it for every Flutter version.
1. Run this in your terminal
echo '#!/bin/zsh
# See also https://stackoverflow.com/a/31150244/410996
trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT
set -e ; /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --test-type --disable-web-security "$#" ; set +e &
PID=$!
wait $PID
trap - SIGINT SIGTERM EXIT
wait $PID
' > ~/chrome_launcher
chmod 755 ~/chrome_launcher
This adds a chrome_launcher script to your user folder and marks it executable.
2. Add this line to your .zshrc (or .bashrc etc.):
export CHROME_EXECUTABLE=~/chrome_launcher
3. Restart Android Studio
If a simple restart does not work, use Invalidate Caches / Restart in Android Studio to force loading of changes.
Notes
The script also adds the --test-type flag to suppress the ugly warning about the disabled security features. Be aware that this option might also suppress other error messages!
The CHROME_EXECUTABLE takes only the path to an executable file it is not possible to set arguments there.
Without trapping exit signals and killing the process group, the Google Chrome instance was not killed when you hit the Stop Button in Android Studio.
If you are using FVM, I suggest to use flutter_cors package
dart pub global activate flutter_cors
fluttercors --disable
If you face
zsh: command not found: fluttercors
You need to add it to PATH. In my case, I'm using zsh, I add it to .zshrc by
vim ~/.zshrc
Press I to start editing and paste export PATH="$PATH":"$HOME/.pub-cache/bin" to the top of the file
Then press ESC and type :wq to save the .zshrc file.
Now you're good to go
Now, just need to run your flutter web normally. It will trigger Chrome without CORS.
For me none of the solutions above worked on production as it was expected. Altough there is one solution I can suggest which uses CORS proxy to avoid CORS issues on flutter web on production. You can find CORS proxies on this website.
Basically you bypass all the unnecessary headers which your browser appends to your requests, so you may not encounter the same CORS issues when making request to another API. Hope it helps!
It Worked With Me By The Following Code :
in conn.php file put like this :
<?php
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
$connect = new mysqli("localhost","db_user","db_password","db_name");
if($connect){
}else{
echo "Connection Failed";
exit();
}
This is a CORS (cross-origin resource sharing) issue and you just need to enable the CORS request from your server-side.
In my case it is Asp.Net MVC Web API and adding below code to Application_BeginRequest at Global.asax worked for me:
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Origin", "http://localhost:7777/");
if (HttpContext.Current.Request.HttpMethod == "OPTIONS")
{
//These headers are handling the "pre-flight" OPTIONS call sent by the browser
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Methods", "POST, GET");
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Headers", "content-type");
HttpContext.Current.Response.End();
}
Use desired urls , Methods and Headers
Also There is no need to change anything in Web.config
If anyone looking for an equivalent of the accepted answer (Osman's) when working with dart web (webdev), here's what worked for me on Dart 2.17.6 (a bit more complex but in case you needed a quick fix, it might be handful).
Find webdev executable (this helps) then you see something like this:
The snapshot file (generated if not exist, as you see) is executed when you want to run app in browser. It contains the code that dart runs when launching chrome (using browser_launcher dart package).
Backup and remove the snapshot file (location in the screenshot above) so it can be regenerated in next run.
Locate browser_launcher package in your pub cache (also you might find location of browser_launcher by searching in the snapshot file) and edit lib\src\chrome.dart, find '--disable-extensions' and add '--disable-web-security'.
Run your app and remove the backup created in step 2.
If you are working with django in the side of the service, you can configure CORS with 'corsheaders', in this link you can find whole the documentation to setup your back end and recognice your requests.
https://pypi.org/project/django-cors-headers/
Go to flutter\bin\cache and remove a file named: flutter_tools.stamp
Go to flutter\packages\flutter_tools\lib\src\web and open the file chrome.dart.
Find '--disable-extensions'
Add '--disable-web-security'

Meteor Velocity/Jasmine fails to start Karma through proxy

I am attempting to test a meteor application using Jasmine built on Velocity. When I test the application behind our corporate proxy, Karma does not start, the tests don't run, and I get this error in my Jasmine logs:
stream error Error during WebSocket handshake: Unexpected response code: 301
Presumably because it cannot connect to Karma.
I have tried this outside of our intranet (with no proxy), and it works just fine. Unfortunately, it isn't really feasible to run tests outside of our network.
Is there a setting that I am missing that would make this work?
Edit:
I have answered my own question just in case it is helpful to anyone who is having the same problem, but I'm open to other solutions. I will accept another solution if it solves this problem.
I am not sure if this is the best way to handle this, but I have gotten karma to launch and the tests to run by unsetting my http proxy environment variable (setting it to "" also works).
$ unset HTTP_PROXY
$ meteor
or start meteor with an inline env variable
$ HTTP_PROXY="" meteor

PhantomJS "SSL handshake failed" on HTTPS site, even when using all the flags

PhantomJS will fail to load many HTTPS urls by default, but you can get many of them to work by adding some command line flags like --ignore-ssl-errors=true and --ssl-protocol=any. However I'm finding some sites that fail even when these flags are enabled; for example try the following code snippet:
var page = require('webpage').create();
page.onResourceError = function(e){
console.log("Page resource failed to load. Reason: "+e.errorString);
}
page.open("https://www.bluehaven.com/pool-remodeling/", function(status){
console.log("Load status: "+status);
});
When I run this script with those two flags enabled, I get the following output:
$ phantomjs --ignore-ssl-errors=true --ssl-protocol=any lib/test_phantom.js
Page resource failed to load. Reason: SSL handshake failed
Load status: fail
I've googled quite a bit and the discussions around PhantomJS and SSL failures all recommend using the above two command flags, or suggest that this is due to a bug in PhantomJS that isn't likely to get fixed anytime soon.
Does anyone here on SO have different ideas on how I can get PhantomJS to load the above URL (and any other sites where the same HTTPS errors occur)? My plan B is to try out SlimerJS as I understand it doesn't share this SSL error, but that will involve some risk and I'd rather not do it unless absolutely needed.
The program works fine on Ubuntu 14.04 with phantomjs 1.9.0.
Since the server only supports ECDHE ciphers I would assume that you are using an old version of OpenSSL with no support for these ciphers on your system.

Unable to set proxy :selenium.common.exceptions.WebDriverException:Access Denied

I want to use Selenium Webdriver and I am unable to do so because when I run my code, I get the following exception.
My code is very basic and as follows.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
Exception Message
selenium.common.exceptions.WebDriverException: Message: '<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD>\n<BODY>\n<FONT face="Helvetica">\n<big><strong></strong></big><BR>\n</FONT>\n<blockquote>\n<TABLE border=0 cellPadding=1 width="80%">\n<TR><TD>\n<FONT face="Helvetica">\n<big>Access Denied (authentication_failed)</big>\n<BR>\n<BR>\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nYour credentials could not be authenticated: "Credentials are missing.". You will not be permitted access until your credentials can be verified.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nThis is typically caused by an incorrect username and/or password, but could also be caused by network problems.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica" SIZE=2>\n<BR>\nFor assistance, contact your network support team.\n</FONT>\n</TD></TR>\n</TABLE>\n</blockquote>\n</FONT>\n</BODY></HTML>\n'
It opens firefox but after that it is unable to connect to google or any other local sites.
The exception is at driver = webdriver.Firefox()
I googled around and I followed the link on SO.
But unfortunately I still get the same error.
I cannot run as the root user. I changed my proxy settings and set No Proxy element for localhost as well as mentioned in the link.
I am using Python 2.7 and have installed selenium 2.31 version.
I also tried setting proxy.
myProxy = "*********:8080"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': myProxy,
'ftpProxy': myProxy,
'sslProxy': myProxy,
'noProxy': 'localhost,127.0.0.1,*.abc'
})
driver = webdriver.Firefox(proxy=proxy)
I also tried to set the proxy to system's proxy i.e., in the above code, 'proxyType': ProxyType.SYSTEM
But it again gives the above exception message.
is there a place where I have to set my username and password?
Any help would be appreciated!
Remove proxy settings from all the browsers on the system manually. I had IE, Firefox and Google Chrome.
When I removed the proxy settings of all the browsers and enabled the proxy only on Firefox,it worked without giving any errors. I do not know the exact reason why this works like this, may be got to do with the registry settings on windows which I am not sure about.
After doing the above said, I ran the basic code and it worked fine.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
I didn't set the proxy explicitly also. By default, it had taken the system's proxy settings. Hope this would help others facing similar issue.

ERROR bad Request-Line - Force ssl - rails

I set force_ssl to true by accident and then when reverting to false and running server I get the following error:
ERROR bad Request-Line
WEBrick 1.3.1
INFO ruby 1.9.3 (2012-02-16) [x86_64-darwin11.4.0]
WEBrick::HTTPServer#start: pid=472 port=3000
ERROR bad Request-Line `\x16\x03\x01\x00?\x01\x00\x00?\x03\x02P?Mf\x00??C?'.
How can I fix this?
When I hit http url on local host it redirects to https and gives me this error:
SSL connection error
Unable to make a secure connection to the server. This may be a problem with the server, or it may be requiring a client authentication certificate that you don't have.
Error 107 (net::ERR_SSL_PROTOCOL_ERROR): SSL protocol error.
Answer can be found here: How to use deactivate Webrick's SSL
you can fix it by clearing the browser cookies and it works again. It seems the authentication information is saved in the cookie and keeps reverting our requests to HTTPS but does not authenticate correctly so you get that error.
Maybe you can try:
1) stop rails server
2) rake tmp:clear on the command line
Then start again and see if the problem persists.
UPDATE:
Maybe you can try to clean any cache content / history content on your browser too.
I usually end up using a different explorer, i.e.: Safari, Firefox, etc. As clearing the cookies do not solve the problem for me (while using Chrome at least)