I'm trying to use sharp on a couple of AWS lambda#edge. The idea is to resize and cache an image when requested (see this).
I'm also using serverless with serverless-webpack to deploy the lambdas.
I can deploy the lambdas and everything goes well if I test them in AWS console.
However, these are lamda#edge and they will be used as cloudwatch request/response triggers. Therefore, the maximum lambda size is 1Mb.
My problem is I can't seem to get even near that size, the best I could achieve was 11.6Mb. And, it seems it's possible as seen in that first link.
This is the serverless configuration which results in 34.7Mb lambda:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions:
scripts:
- rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp
package:
exclude:
- .env
- .git/**
- .gitlab-ci.yml
- tests*
excludeDevDependencies: true
individually: true
And with this I got 11.6Mb:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions:
scripts:
- npm rebuild sharp --target=10.15.0 --target_arch=x64 --target_platform=linux
package:
exclude:
- .env
- .git/**
- .gitlab-ci.yml
- tests*
excludeDevDependencies: true
individually: true
I've also played around with the package.exclude, but with no luck:
- node_modules/**
- '!node_modules/sharp/**'
and this is my webpack config:
const path = require('path');
const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const entries = {};
Object.keys(slsw.lib.entries).forEach(key => (entries[key] = ['./source-map-install.js', slsw.lib.entries[key]]));
module.exports = {
mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
entry: slsw.lib.entries,
devtool: 'source-map',
resolve: {
extensions: ['.js', '.jsx', '.json', '.ts', '.tsx'],
},
// externals: ['sharp'], #tried that too
externals: [nodeExternals()],
output: {
libraryTarget: 'commonjs',
path: path.join(__dirname, '.webpack'),
filename: '[name].js',
},
target: 'node',
module: {
rules: [
// all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
{ test: /\.ts?$/, loader: 'ts-loader', options: { happyPackMode: true } },
],
},
};
When running locally, I can see what it's packaging... the node_modules folder has sharp and its dependencies, it seems. But the biggest folder is sharp.
I suspect I'm packaging stuff inside sharp folder that I don't need... but I can't seem to understand what.
Any help?
Thanks
UPDATE:
Reading more carefully, it seems the function where I need sharp (origin-response) size limit is 5Mb.
I just need to find a way to package sharp only for that function. Webpack seems to put it in both, even though I don't need it on the other function (viewer request).
Any help on this?
I ended up running a script in custom.webpack.packagerOptions.scripts that will ignore sharp where it's not needed.
This is the script I used:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions: # uncomment this block if invoking locally
scripts:
- if [ -f "src/handlers/myfunction.js" ]; then rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp; else rm -rf node_modules; fi
Related
I'm using VUE CLI 3 and I need to remove the console.log and code comments from the production built. This is my Terser setup:
webpack.config.js in src folder
module.exports = {
mode: 'production',
optimization: {
minimize: true,
minimizer: [
new TerserPlugin({
terserOptions: {
ecma: undefined,
warnings: false,
parse: {},
compress: {drop_debugger},
mangle: true, // Note `mangle.properties` is `false` by default.
module: false,
output: null,
toplevel: false,
nameCache: null,
ie8: false,
keep_classnames: undefined,
keep_fnames: false,
safari10: false,
},
}),
],
},
};
My production workflow: Run npm run build -> cd dist -> npm run serve
The production build still outputs all console.log statements and shows code comments (<!-- -->).
What do I need to change to remove them?
First of all: make sure you are configuring Terser correctly:
terserOptions: {
ecma: 6,
compress: { drop_console: true },
output: { comments: false, beautify: false }
}
npm run serve
is usually a shortcut for:
vue-cli-service
vue-cli-service --help
Usage: vue-cli-service <command> [options]
Commands:
serve start development server
build build for production
inspect inspect internal webpack config
lint lint and fix source files
So when your workflow is the above mentioned npm run build -> cd dist -> npm run serve then you are actually starting the dev server, which does not apply Terser.
In order to run the production build you can use any webserver capable of serving static content:
NodeJs examples:
npm install -g serve
serve -s dist
or
npm install -g node-static
static dist
I am making a js that I aim to publish npm. It will be used on both web and node.
I read webpack doc and I use this following config. Bundled and minified, it produces a 20kb package, which, IMHO, is pretty big for what it does.
Should I bundle it with webpack that way ?
{
mode: 'production',
entry: {
mylib: './src/mylib_browser.ts', // same for node
"mylib.min": './src/mylib_browser.ts'
},
watch: true,
target: 'web', // node for node
devtool: 'source-map',
externals: [nodeExternals()],
module: {
rules: [
{
test: /\.tsx?$/,
use: 'ts-loader',
exclude: /node_modules/
}
]
},
resolve: {
extensions: ['.tsx', '.ts', '.js']
},
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name]_browser.js',
libraryTarget: 'umd',
library: 'MyLib',
umdNamedDefine: true
},
plugins: [
new UglifyJsPlugin({
sourceMap: true,
include: /\.min\.js$/,
})
],
};
And I have the same config for node.
Is it the right config to do it? How could I decrease size ?
Also, when I look at other npm package, a lot are just vanilla js. Should I just let the user of my npm package, package it as a normal dependency ?
Are you sure the minification is happening? You have UglifyJs only minifying already minified files, which seems wrong?
new UglifyJsPlugin({
sourceMap: true,
include: /\.min\.js$/, //<= remove this line
})
Also, when I look at other npm package, a lot are just vanilla js. Should I just let the user of my npm package, package it as a normal dependency ?
If your library will be packaged by consumers then I would distribute with both minified and unminified sources. This allows consumers to include the minified library via one of the npm CDNs (e.g. unpkg) when hacking around (e.g. jsfiddle) and the unminified when using the library as part of a bundled application, e.g. via webpack.
If your library is only meant for the Node environment then minification is generally considered unnecessary and even a burden should the consumer encounter a bug in your library and wish to debug it.
Running grunt - I get this error message:
Running "sass:all" (sass) task
Fatal error: The implementation option must be passed to the Sass task
I've tried re-installing grunt, node, npm, dependencies - but I always come back to this error I can't get past.
Should I post my Gruntfile.js? Frankly, this was set up by a third-party and we don't use it often - I'm thinking maybe we should start from the ground up because it is from about 4 years ago originally... but wondering if anyone has seen this error before and knows of a fix/workaround.
With the update to grunt-sass 3, you have to choose whether you want to use node-sass or dart-sass to compile
For node-sass you need to install the module with:
$ npm install --save-dev node-sass
In you gruntfile, you than need to add node-sass as requirement and add the define constant as implementation option:
const sass = require('node-sass');
require('load-grunt-tasks')(grunt);
grunt.initConfig({
sass: {
options: {
implementation: sass,
sourceMap: true
},
dist: {
files: {
'main.css': 'main.scss'
}
}
}
});
See also official page for more details: https://www.npmjs.com/package/grunt-sass
use this
**const sass = require("node-sass");**
**grunt.initConfig({
sass: {
options: {
implementation: sass,
sourceMap: true,
},
dist: {
files: {
"css/styles.css": "css/styles.css",
},
},
},
});
This will help you solve the problem
UPDATE: Only works for grunt-sass 2.x
I had this problem when upgrading from grunt-sass 1.x to 2.x. This solved it for me:
Add implementation: 'sass' to your sass.options object in Gruntfile.js like so:
options: {
implementation: 'sass',
outputStyle: 'expanded',
sourceMap: true,
quiet: true // stop depreciation errors
},
I've been trying to solve this problem from morning and more than half of my day is wasted trying to re-configure webpack, remove and reinstall packages to see why it's going wrong. I'm not really sure what's happening but I get this new "package-lock.json" file every time I try to configure.
I started to get familiar with webpack as early as yesterday and I was really happy when I used it along with react and react-bootstrap yesterday.
I really feel like the problem is with npm but I didn't uninstall it.
I uninstalled/deleted the node_modules folder several times and reinstalled them back but it doesn't work either!
[EDIT]
I just found out that there's something weird going on, I have this package-lock.json file and it seems like this file is preventing from any npm module installation. I checked my node_modules folder and I find only fsevents module and nothing else. I tried to run npm install but nothing happens.
What I have done from yesterday
- update npm
- clean cache i guess.
Here's my error
And this is my webpack configuration file
var webpack = require('webpack');
module.exports = {
entry: "./app/index.js",
output : {
path: __dirname + '/build',
filename: 'bundle.js'
},
module: {
rules: [
{
test: /\.(js|jsx)$/,
exclude: /node_modules/,
use: 'babel-loader'
}
,{
test: /\.(sass|scss)$/,
exclude: /node_modules/,
use: [
{
loader: 'style-loader'
},{
loader: 'css-loader'
},{
loader: 'sass-loader'
}
]
}
]
},
devServer: {
contentBase: __dirname + '/build',
port: 3000
}
}
Please help me through this!
Error while running webpack-dev-server
ERROR in ./css/app.scss
Module build failed: Error: Parameter 'dependency' must be a Dependency
at Compilation.process [as _addModuleChain] (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compilation.js:347:9)
at Compilation.process [as addEntry] (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compilation.js:423:7)
at SingleEntryPlugin.<anonymous> (/Users/roshankarki/Desktop/del_del/wp_tst/node_modules/webpack/lib/SingleEntryPlugin.js:22:15)
at Compiler.applyPluginsParallel (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/node_modules/tapable/lib/Tapable.js:107:14)
at Compiler.compile (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compiler.js:394:7)
at Compiler.runAsChild (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compiler.js:203:7)
at Object.module.exports.pitch (/Users/roshankarki/Desktop/del_del/wp_tst/node_modules/extract-text-webpack-plugin/loader.js:81:18)
webpack: bundle is now VALID.
My webpack.config.js looks like this:
var path = require('path');
var ExtractTextPlugin = require("extract-text-webpack-plugin");
module.exports = {
/* this defins path where entry files are to be found */
context: path.resolve('js'),
entry: ["./utils","./app"],
output: {
/* this is output directory of bundle.js */
path: path.resolve('build/'),
/* this is path of directory where bundle will be served in server */
publicPath: '/public/assets/',
filename: "bundle.js"
},
/* this plugin helps to generate seperate styles.css file rather in once single bundle.js file */
plugins: [
new ExtractTextPlugin("styles.css")
],
/* when server is loaded it tells where to look for index file accessed from root */
devServer: {
contentBase: 'public'
},
module: {
preLoaders: [
{
test: /\.js$/,
exclude: /node_modules/,
loader: "jshint-loader"
}
],
loaders: [
{
test: /\.es6$/,
exclude: /node_modules/,
loader: "babel-loader"
},
{
test: /\.css$/,
exclude: /node_modules/ ,
loader: ExtractTextPlugin.extract("style-loader", "css-loader")
},
{
test: /\.scss$/,
exclude: /node_modules/ ,
loader: ExtractTextPlugin.extract("style-loader", "css-loader!sass-loader")
}
]
},
/* this will resolve file in common js style require without extension */
resolve: {
extensions: ['', '.js', '.es6']
},
watch: true
}
Before using this plugin everything was working fine. As i intend to output different bundle for css, i need to fix this issue.
Found issue in github but it did not help:
https://github.com/webpack/extract-text-webpack-plugin/issues/132
well updating node worked for me. I did this by running "brew update" and "brew upgrade" command, as i am using homebrew as a package manage in mac.
Following instruction in the git link provide in my question resulted in error as i was installing webpack and webpack-dev-server globally. I removed it and only had local copy which did the trick at the end.
command to list global copy of node modules installed:
npm list -g --depth=0
I use this command to remove global copy:
npm uninstall -g {replace_with_package_name}
command to check local copy:
npm list --depth=0