Express.js: how to get assets gzipped - express

I use compress() middleware, put it the first in configure().
app.configure('all', function(){
app.use(express.compress());
...
app.use(express.static('public'), { maxAge: oneMonth });
})
How do I check that my content is gzipped? I've got a fricking strange situation:
1) On my dev machine: I reqeust localhost:4000/mystyle.css - DON'T see Content-encoding: gzip
2) When I deploy it on production if I request the file it self mydomain.com/mystyle.css - I SEE there see Content-encoding: gzip
3) I request mydomain.com and see in Network in chrome dev tools, find there mystyle.css and there I DON'T see Content-encoding: gzip
4) I use different services to check if my content is gzipped some says that it IS, some that it IS NOT.
WTF? Can some one explain?

Your issue is your use of app.configure. This is largely deprecated, but you're specifically using it such that you're looking for an all environment.
The documentation explains: "This method remains for legacy reason, and is effectively an if statement as illustrated in the following snippets."
Instead, just use the app.use without wrapping them in a configure statement.

Related

How to solve flutter web api cors error only with dart code?

It seems like CORS error is well-known issue in the web field. But I tried flutter web for the first time ever and I faced critical error.
The code below worked well in app version when it was running on iOS device, but when i tested the same code on Chrome with web debugging from beta channel, it encountered CORS error.
Other stackoverflow answers explained how to solve the CORS issue with serverside files of their projects. But I have totally no idea what is server thing and how to deal with their answers. The error message from Chrome console was below
[ Access to XMLHttpRequest at 'https://kapi.kakao.com/v1/payment/ready' from origin 'http://localhost:52700' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. ]
So, what i want to do is to solve above 'Access-Control-Allow-Origin header' issue ONLY WITH DART CODE! Code below is what i've tried to solve these issues only with my main.dart.
onPressed: () async {
var res =
await http.post('https://kapi.kakao.com/v1/payment/ready', encoding: Encoding.getByName('utf8'), headers: {
'Authorization': 'KakaoAK $_ADMIN_KEY',
HttpHeaders.authorizationHeader: 'KakaoAK $_ADMIN_KEY',
"Access-Control-Allow-Origin": "*",
"Access-Control-Allow-Methods": "POST, GET, OPTIONS, PUT, DELETE, HEAD",
}, body: {
'cid': 'TC0ONETIME',
'partner_order_id': 'partner_order_id',
'partner_user_id': 'partner_user_id',
'item_name': 'cool_beer',
'quantity': '1',
'total_amount': '22222',
'vat_amount': '2222',
'tax_free_amount': '0',
'approval_url': '$_URL/kakaopayment',
'fail_url': '$_URL/kakaopayment',
'cancel_url': '$_URL/kakaopayment'
});
Map<String, dynamic> result = json.decode(res.body);
print(result);
},
Even though i actually had the header "Access-Control-Allow-Origin": "*" which most other answers recommended, the Chrome console printed same error message. Weird thing is that the same code made successful request in mobileApp version. So I think this is only problem with flutter WEB VERSION.
Hope somebody can figure it out and suggest only-dart code to resolve the issue in my main.dart!! Thank you for reading [:
1- Go to flutter\bin\cache and remove a file named: flutter_tools.stamp
2- Go to flutter\packages\flutter_tools\lib\src\web and open the file chrome.dart.
3- Find '--disable-extensions'
4- Add '--disable-web-security'
Since Flutter 3.3.0
I implemented the option to add any browser flag to the flutter command.
flutter run -d chrome --web-browser-flag "--disable-web-security"
Or for drive command:
flutter drive --driver=test_driver/integration_test.dart --target=integration_test/app_test.dart -d web-server --web-browser-flag="--disable-web-security"
Note: This is just for development and testing. Flutter is executed explicitly on the client's browser! You should NOT and you can NOT disable it in production (as stated by #Tommy), as it is a security feature of the browser, and not meant to be changed in dart code. You have to enable CORS on your web server, which is providing the resources of your Flutter app, to ensure it works for everyone.
If you use dart language without Flutter on the server side with shelf, then see this response.
I think disabling web security as suggested will make you jump over the current error for now but when you go for production or testing on other devices the problem will persist because it is just a workaround, the correct solution is from the server side to allow CORS from the requesting domain and allow the needed methods, and credentials if needed.
run/compile your Flutter web project using web-renderer. This should solve the issue both locally and remotely:
flutter run -d chrome --web-renderer html
flutter build web --web-renderer html
This is a CORS (cross-origin resource sharing) issue and you do not have to delete/modify anything. You just have to enable the CORS request from your server-side and it will work fine.
In my case, I have created a server with node.js and express.js, so I just added this middleware function that will run for every request.
app.use(function(req, res, next) {
res.header("Access-Control-Allow-Origin", "*");
res.header("Access-Control-Allow-Methods", "GET,PUT,PATCH,POST,DELETE");
res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
next();
});
And BOOOOM! I received the data.
You just have to look at the settings to enable CORS for your server.
Server side engine like node js or django is really needed to work with flutter web with bunch of external apis. Actually there's high possibility of same CORS error when we try to use internal api because of the CORS mechanism related to port number difference.
There are bunch of steps and answers from SO contributors that recommend to use chrome extensions to avoid CORS errors, but that is actually not cool for users. All the users should download the browser extensions to use the single website from us, which wouldn't be there if we used true server engines.
CORS is from browser as far as i know, so our flutter ios and android apps with same api code don't give those CORS errors. First time i encountered this error with flutter web, i believed i can deal with CORS in my app code lines. But that is actually not healthy way for users and long term dev plans.
Hope all flutter web newbies understand that web is quite a wild field for us. Even though i'm also newbie here, i highly recommend all the flutter web devs from 1.22.n stable to learn server side engines like node js. It is worth try.
And if u came so far down to this line of my self-answer, here's a simple guide for flutter web with node js. Flutter web is on stable channel but all those necessary infra are not fully ready for newbies like me. So be careful when you first dive into web field, and hope you re-check all the conditions and requirements to find out if you really need web version of your flutter app, and also if you really need to do this work with flutter. And my answer was yes lol
https://blog.logrocket.com/flutter-web-app-node-js/
If you run a Spring Boot server, add "#CrossOrigin" to your Controller or to your service method.
#CrossOrigin
#PostMapping(path="/upload")
public #ResponseBody ResponseEntity<Void> upload(#RequestBody Object object) {
// ...
}
I know the question explicitly asked for a solution "with dart code" only, but I was not able to fix the exception with dart code (for example by changing the header).
The disabling web security approaches work well in development, but probably not so well in production. An approach that worked for me in production dart code involves avoiding the pre-flight CORS check entirely by keeping the web request simple. In my case this meant changing the request header to contain:
'Content-Type': 'text/plain'
Even though I'm actually sending json, setting it to text/plain avoids the pre-flight CORS check. The lambda function I'm calling didn't support pre-flight OPTIONS requests.
Here's some info on other ways to keep a request simple and avoid a pre-flight request
https://docs.flutter.dev/development/platform-integration/web-images
flutter run -d chrome --web-renderer html
flutter build web --web-renderer html
This official solution worked for me on Chrome only (Source). But I had to run it first every time.
flutter run -d chrome --web-renderer html
And disabling web security also worked (Source). But the browsers will show a warning banner.
But In case you are running on a different browser than Chrome (e.g. Edge) and you want to keep 'web security' enabled.
You can change the default web renderer in settings in VS Code
File ==> Preferences ==> Settings ==> Enter 'Flutter Web' in the Search Bar ==> Set the default web renderer to html
After hours of testing, the following works perfectly for me.
Add the following to the PHP file:
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
This allow the correct connection with the HTTP GET POST with no issue from flutter for me.
I discovered this in the following discussion:
XMLHttpRequest error Flutter
I am getting the same error with php api so i add the php code these lines ;
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
I think you may not doing this in right way.
The cors headers should be added in HTTP response header while you added them in you reuqest header obviously.
for more information check out the documentation https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS#what_requests_use_cors
The below solution is great if you are only communicating with a local NodeJS server.
Install NodeJS
Create a basic NodeJS express project
Create a folder to put you NodeJS project in
ex: C:\node_project\
in PowerShell run: npm init in the folder
fill in your desired values
entry point: must be app.js for this example to work
in PowerShell run: npm install express in the folder
create a app.js file in the folder
// init express
const express = require("express");
const app = express();
// set the path to the web build folder
app.use(express.static("C:/Users/your_username/path_to_flutter_app/build/web"));
const PORT = process.env.PORT || 8080;
app.listen(PORT, () => {
console.log(`Server listening on port ${PORT}...`);
});
The value "C:/Users/your_username/path_to_flutter_app/build/web" must be changed to the web build folder in your flutter app.
The app can be accessed through your browser once the app is built, the node server is running, and the browser is at the correct address
Build the app
open PowerShell and navigate to the flutter project's root ex: C:/Users/your_username/path_to_flutter_app/
run flutter build web
turn on the node server
open PowerShell and navigate to the NodeJS server folder ex: C:\node_project\
run: node app.js
Open in your browser
Enter http://localhost:8080/ into the browser
Note that everytime you change your flutter app's dart code you will need to re-run flutter build web
Wrong Server on Target Port 🤦
I feel silly for even admitting this, but I had some other local server running on the targeted port. I've no clue why the server seemed to boot on the same port, or why the iOS app seemed to work, but now that I'm hitting the actual server it's working fine.
I was also getting some 404's mixed in, but originally thought that was due to the CORs error.
Maybe someone else has this same issue and this helps them.
In my case, The problem was in laravel backend code which did not support CORS, So I added the CORS into backend project then it worked successfully in test and live.
The 5th step of Osmans answer should be to add the option
'--disable-site-isolation-trials',
Only this works for me.
Chrome version 106.0.5249.119
Update:
I recommend to use User Rebo's answer. It is now possible to pass --disable-web-security as a browser flag to run & drive commands!
Original outdated answer:
Alternative solution for MacOS & Android Studio (without modifying Flutter source)
We use a similar approach as Osman Tuzcu. Instead of modifying the Flutter source code, we add the --disable-web-security argument in a shell script and just forward all other arguments that were set by Flutter. It might look overly complicated but it takes just a minute and there is no need to repeat it for every Flutter version.
1. Run this in your terminal
echo '#!/bin/zsh
# See also https://stackoverflow.com/a/31150244/410996
trap "trap - SIGTERM && kill -- -$$" SIGINT SIGTERM EXIT
set -e ; /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --test-type --disable-web-security "$#" ; set +e &
PID=$!
wait $PID
trap - SIGINT SIGTERM EXIT
wait $PID
' > ~/chrome_launcher
chmod 755 ~/chrome_launcher
This adds a chrome_launcher script to your user folder and marks it executable.
2. Add this line to your .zshrc (or .bashrc etc.):
export CHROME_EXECUTABLE=~/chrome_launcher
3. Restart Android Studio
If a simple restart does not work, use Invalidate Caches / Restart in Android Studio to force loading of changes.
Notes
The script also adds the --test-type flag to suppress the ugly warning about the disabled security features. Be aware that this option might also suppress other error messages!
The CHROME_EXECUTABLE takes only the path to an executable file it is not possible to set arguments there.
Without trapping exit signals and killing the process group, the Google Chrome instance was not killed when you hit the Stop Button in Android Studio.
If you are using FVM, I suggest to use flutter_cors package
dart pub global activate flutter_cors
fluttercors --disable
If you face
zsh: command not found: fluttercors
You need to add it to PATH. In my case, I'm using zsh, I add it to .zshrc by
vim ~/.zshrc
Press I to start editing and paste export PATH="$PATH":"$HOME/.pub-cache/bin" to the top of the file
Then press ESC and type :wq to save the .zshrc file.
Now you're good to go
Now, just need to run your flutter web normally. It will trigger Chrome without CORS.
For me none of the solutions above worked on production as it was expected. Altough there is one solution I can suggest which uses CORS proxy to avoid CORS issues on flutter web on production. You can find CORS proxies on this website.
Basically you bypass all the unnecessary headers which your browser appends to your requests, so you may not encounter the same CORS issues when making request to another API. Hope it helps!
It Worked With Me By The Following Code :
in conn.php file put like this :
<?php
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Methods: GET, POST');
header("Access-Control-Allow-Headers: X-Requested-With");
$connect = new mysqli("localhost","db_user","db_password","db_name");
if($connect){
}else{
echo "Connection Failed";
exit();
}
This is a CORS (cross-origin resource sharing) issue and you just need to enable the CORS request from your server-side.
In my case it is Asp.Net MVC Web API and adding below code to Application_BeginRequest at Global.asax worked for me:
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Origin", "http://localhost:7777/");
if (HttpContext.Current.Request.HttpMethod == "OPTIONS")
{
//These headers are handling the "pre-flight" OPTIONS call sent by the browser
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Methods", "POST, GET");
HttpContext.Current.Response.AddHeader("Access-Control-Allow-Headers", "content-type");
HttpContext.Current.Response.End();
}
Use desired urls , Methods and Headers
Also There is no need to change anything in Web.config
If anyone looking for an equivalent of the accepted answer (Osman's) when working with dart web (webdev), here's what worked for me on Dart 2.17.6 (a bit more complex but in case you needed a quick fix, it might be handful).
Find webdev executable (this helps) then you see something like this:
The snapshot file (generated if not exist, as you see) is executed when you want to run app in browser. It contains the code that dart runs when launching chrome (using browser_launcher dart package).
Backup and remove the snapshot file (location in the screenshot above) so it can be regenerated in next run.
Locate browser_launcher package in your pub cache (also you might find location of browser_launcher by searching in the snapshot file) and edit lib\src\chrome.dart, find '--disable-extensions' and add '--disable-web-security'.
Run your app and remove the backup created in step 2.
If you are working with django in the side of the service, you can configure CORS with 'corsheaders', in this link you can find whole the documentation to setup your back end and recognice your requests.
https://pypi.org/project/django-cors-headers/
Go to flutter\bin\cache and remove a file named: flutter_tools.stamp
Go to flutter\packages\flutter_tools\lib\src\web and open the file chrome.dart.
Find '--disable-extensions'
Add '--disable-web-security'

Spring Boot, Apache CXF 3.2.5 with MTOM sends empty attachment

I'm having a weird issue with Apache CXF and large (375MB) MTOM attachments are empty.
Running it locally in Eclipse produces the desired results, but deploying it to our server just gives an empty attachment.
The server is written in .NET and doesn't support chunking. With Chunking enabled the client works, but when i disable cunking it fails.
Sadly i'm unable to debug on the server, so i'm restricted to trace logging.
I've tried every trick i've been able to google.
Disable schema validation (CXF-4551) (CXF-7758)
Manually copying the file to java.io.tmpdir before sending, to ensure it can be read.
Custom DataSource
Disable WS-Security
Disable logging interceptor
Nothing seems to make a difference.
Every run i just get something like the following
</soap:Body></soap:Envelope>
--uuid:40ef745b-ac3c-4013-bbe7-a9cc28880423
Content-Type: application/xml
Content-Transfer-Encoding: binary
Content-ID: <7611ca0a-22f8-4637-b4f7-a5dfe7f20b81-3#www.somewhere.dk>
Content-Disposition: attachment;name="32_2018-03-28_output.xml"
--uuid:40ef745b-ac3c-4013-bbe7-a9cc28880423
Trying with a smaller (2KB) file on the server works just fine. A 75MB file gets attached correctly, but results in a HTTP 400 from the receiver (which i suspect is because the file is not fully transferred)
Does anyone have any ideas as to what might be causing this ?
After much trial & error, i finally managed to "solve" this. I enabled schema validation, and the data now appears. This is the exact issue that both bugs in my original question claims to fix.
Client client = ClientProxy.getClient(port);
BindingProvider bp = ((BindingProvider) port);
bp.getRequestContext().put("schema-validation-enabled", "true");
I can't add a comment so I'm posting this as an answer.
Jimmy could you perhaps comment on the latest CXF issue and provide some more details? Which version of CXF, what kind of client you are using, real code samples ideally, client logs?

UTF8 characters come out as windows-1252, ExpressJS running on Electron

A little bit of introduction - the (wannabe) software I'm working on is a desktop application built on Github Electron. It is designed to have two modules - a UI and a backend with an API, the second "living" in a hidden browser window and running on ExpressJS. The idea is for the UI and the API to communicate over HTTP so they could be decoupled if needed. All files are in UTF-8.
Now the problem itself - the main API route looks like:
router.get('/', (request, result) => {
let message = 'Здрасти, коко!';
console.log('Answering with ' + message);
result.json(message);
});
When being called (from a browser, or Postman, or whatever), the answer looks like this:
{"message":"ЗдраÑти, коко!"}
...with those headers:
HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 64
ETag: W/"40-3JawFDiTNEinvN6xFO6T9g"
Date: Tue, 20 Dec 2016 06:47:53 GMT
Connection: keep-alive
Using the 2cyr tool I found out that the source encoding is UTF-8 but it gets properly displayed only as windows-1252 which confuses me a lot.
To narrow down the possibilities, I added the console.log() to the route handler and (a bit surprising to me) I'm getting the same "broken" result in the Chromium debugger. I suspected the file's encoding, but this is what I get about it:
Petars-Mac:api petar$ file -I api.js
api.js: text/x-c++; charset=utf-8
The last thing that came to my mind was actually decoupling the API from Electron. When I run it in the terminal with node, I actually get the proper result - both in the log message in the terminal and in the JSON answer in the browser.
What am I doing wrong and what further debugging could I possibly do?
So here we go, right before posting an issue in the Electron repository - it's the most stupid error I could imagine ever making in this situation.
TL;DR:
<meta charset="utf-8">
What I thought was that opening a second browser window for the backend and putting some JavaScript that runs in it would be enough. What I forgot was that it actually remains a browser window and therefore it needs just a little tiny bit of HTML to let it know that it serves UTF-8 content.
Maybe it's not me, maybe I was right expecting Express to serve UTF-8 over HTTP but nope. Anyway, it all works now.

IE8 attachment download issue over SSL

While downloading excel file on IE8 over SSL, we are getting error: "Unable to download Document.ashx from MyHostName.Unable to open this Internet site. The requested site is either unavailable or cannot be found. Please try again later."
We tried couple of solutions, but none of them worked except regedit(which is not viable).
1)Tried the registry edit as per http://support.microsoft.com/kb/323308 and it worked. But, this is not a viable solution.
2) Tried the solution as per this link http://support.microsoft.com/kb/815313 where they say the files will download if you ‘copy shortcut’ and run it on via the browser URL. But, this solution was not consistent and worked sometimes but mostly did not work.
3) Update our java controller code to response.setHeader("Cache-Control", " must-revalidate, private "); response.setDateHeader("Expires", -1) but still when we look at the header it shows as "cache-control: no-cache". this solution is also not working
4) Un-Check the Option to ‘Not Save Encrypted files to Disk” on IE 8 Browser. But that also not working. http://blogs.msdn.com/b/ieinternals/archive/2010/04/21/internet-explorer-may-bypass-cache-for-cross-domain-https-content.aspx and
http://support.microsoft.com/kb/2549423
also searched on internet with various options suggested as above, but none of them are working full proof.
if anyone has solution other than listed above. please do share it.
Thanks
I had the same problem and I discovered, that a header:
Pragma: private
will fix this.
Make sure that "no-store" and "no-cache" occur in your header in that order. IE8 pukes if it's reversed.
Check out the update at the end of this post: Internet Explorer Cannot Download https://something
I found that I needed to also remove the "pragma" header to make it work. Apparently, pragma is not intended for responses although I don't know if any old browsers out there require it.
Source: Difference between Pragma and Cache-control headers?
The problem is that if the server sends to the browser an http header that disables caching, Explorer gives an error. We also tried all the solutions you said without luck, but analyzing the http response we saw that the application server added no-cache= "Set-Cookie" http header automatically and it disabled IE8 caching... There is a setting that can be made on the AS but in our case we could not do it, so we changed our cookie-management code server side and we resolved the problem.
I have faced this issue and following entry resolve this problem
response.setContentType("application/vnd.ms-xls; charset=utf-8");
response.setHeader("Content-disposition","attachment;filename="+filename);
response.setHeader("Pragma", "public");
response.setHeader("Cache-Control", "public, max-age=3800");

Background Intelligent Transfer Service and Amazon S3

I'm using SharpBITS to download file from AmazonS3.
> // Create new download job. BitsJob
> job = this._bitsManager.CreateJob(jobName, JobType.Download);
> // Add file to job.
> job.AddFile(downloadFile.RemoteUrl, downloadFile.LocalDestination);
> // Resume
> job.Resume();
It works for files which do no need authentication. However as soon as I add authentication query string for AmazonS3 file request the response from server is http state 403 -unauthorized. Url works file in browser.
Here is the HTTP request from BIT service:
HEAD /mybucket/6a66aeba-0acf-11df-aff6-7d44dc82f95a-000001/5809b987-0f65-11df-9942-f2c504c2c389/v10/summary.doc?AWSAccessKeyId=AAAAZ5SQ76RPQQAAAAA&Expires=1265489615&Signature=VboaRsOCMWWO7VparK3Z0SWE%2FiQ%3D HTTP/1.1
Accept: */*
Accept-Encoding: identity
User-Agent: Microsoft BITS/7.5
Connection: Keep-Alive
Host: s3.amazonaws.com
The only difference between the one from a web browser is the request type. Firefox makes a GET request and BITS makes a HEAD request. Are there any issues with Amazon S3 HEAD requests and query string authentication?
Regards, Blaz
You are probably right that a proxy is the only way around this. BITS uses the HEAD request to get a content length and decide whether or not it wants to chunk the file download. It then does the GET request to actually retrieve the file - sometimes as a whole if the file is small enough, otherwise with range headers.
If you can use a proxy or some other trick to give it any kind of response to the HEAD request, it should get unstuck. Even if the HEAD request is faked with a fictitious content length, BITS will move on to a GET. You may see duplicate GET requests in a case like this, because if the first GET request returns a content length longer than the original HEAD request, BITS may decide "oh crap, I better chunk this after all."
Given that, I'm kind of surprised it's not smart enough to recover from a 403 error on the HEAD request and still move on to the GET. What is the actual behaviour of the job? Have you tried watching it with bitsadmin /monitor? If the job is sitting in a transient error state, it may do that for around 20 mins and then ultimately recover.
Before beginning a download, BITS sends an HTTP HEAD request to the server in order to figure out the remote file's size, timestamp, etc. This is especially important for BranchCache-based BITS transfers and is the reason why server-side HTTP HEAD support is listed as an HTTP requirement for BITS downloads.
That being said, BITS bypasses the HTTP HEAD request phase, issuing an HTTP GET request right away, if either of the following conditions is true:
The BITS job is configured with the BITS_JOB_PROPERTY_DYNAMIC_CONTENT flag.
BranchCache is disabled AND the BITS job contains a single file.
Workaround (1) is the most appropriate, since it doesn't affect other BITS transfers in the system.
For workaround (2), BranchCache can be disabled through BITS' DisableBranchCache group policy. You'll need to do "gpupdate" from an elevated command prompt after making any Group Policy changes, or it will take ~90 minutes for the changes to take effect.