I'm trying to use sharp on a couple of AWS lambda#edge. The idea is to resize and cache an image when requested (see this).
I'm also using serverless with serverless-webpack to deploy the lambdas.
I can deploy the lambdas and everything goes well if I test them in AWS console.
However, these are lamda#edge and they will be used as cloudwatch request/response triggers. Therefore, the maximum lambda size is 1Mb.
My problem is I can't seem to get even near that size, the best I could achieve was 11.6Mb. And, it seems it's possible as seen in that first link.
This is the serverless configuration which results in 34.7Mb lambda:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions:
scripts:
- rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp
package:
exclude:
- .env
- .git/**
- .gitlab-ci.yml
- tests*
excludeDevDependencies: true
individually: true
And with this I got 11.6Mb:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions:
scripts:
- npm rebuild sharp --target=10.15.0 --target_arch=x64 --target_platform=linux
package:
exclude:
- .env
- .git/**
- .gitlab-ci.yml
- tests*
excludeDevDependencies: true
individually: true
I've also played around with the package.exclude, but with no luck:
- node_modules/**
- '!node_modules/sharp/**'
and this is my webpack config:
const path = require('path');
const slsw = require('serverless-webpack');
const nodeExternals = require('webpack-node-externals');
const entries = {};
Object.keys(slsw.lib.entries).forEach(key => (entries[key] = ['./source-map-install.js', slsw.lib.entries[key]]));
module.exports = {
mode: slsw.lib.webpack.isLocal ? 'development' : 'production',
entry: slsw.lib.entries,
devtool: 'source-map',
resolve: {
extensions: ['.js', '.jsx', '.json', '.ts', '.tsx'],
},
// externals: ['sharp'], #tried that too
externals: [nodeExternals()],
output: {
libraryTarget: 'commonjs',
path: path.join(__dirname, '.webpack'),
filename: '[name].js',
},
target: 'node',
module: {
rules: [
// all files with a `.ts` or `.tsx` extension will be handled by `ts-loader`
{ test: /\.ts?$/, loader: 'ts-loader', options: { happyPackMode: true } },
],
},
};
When running locally, I can see what it's packaging... the node_modules folder has sharp and its dependencies, it seems. But the biggest folder is sharp.
I suspect I'm packaging stuff inside sharp folder that I don't need... but I can't seem to understand what.
Any help?
Thanks
UPDATE:
Reading more carefully, it seems the function where I need sharp (origin-response) size limit is 5Mb.
I just need to find a way to package sharp only for that function. Webpack seems to put it in both, even though I don't need it on the other function (viewer request).
Any help on this?
I ended up running a script in custom.webpack.packagerOptions.scripts that will ignore sharp where it's not needed.
This is the script I used:
custom:
webpack:
includeModules:
forceExclude:
- aws-sdk
packagerOptions: # uncomment this block if invoking locally
scripts:
- if [ -f "src/handlers/myfunction.js" ]; then rm -rf node_modules/sharp && docker run -v "$PWD":/var/task lambci/lambda:build-nodejs10.x npm install sharp; else rm -rf node_modules; fi
I am making a js that I aim to publish npm. It will be used on both web and node.
I read webpack doc and I use this following config. Bundled and minified, it produces a 20kb package, which, IMHO, is pretty big for what it does.
Should I bundle it with webpack that way ?
{
mode: 'production',
entry: {
mylib: './src/mylib_browser.ts', // same for node
"mylib.min": './src/mylib_browser.ts'
},
watch: true,
target: 'web', // node for node
devtool: 'source-map',
externals: [nodeExternals()],
module: {
rules: [
{
test: /\.tsx?$/,
use: 'ts-loader',
exclude: /node_modules/
}
]
},
resolve: {
extensions: ['.tsx', '.ts', '.js']
},
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name]_browser.js',
libraryTarget: 'umd',
library: 'MyLib',
umdNamedDefine: true
},
plugins: [
new UglifyJsPlugin({
sourceMap: true,
include: /\.min\.js$/,
})
],
};
And I have the same config for node.
Is it the right config to do it? How could I decrease size ?
Also, when I look at other npm package, a lot are just vanilla js. Should I just let the user of my npm package, package it as a normal dependency ?
Are you sure the minification is happening? You have UglifyJs only minifying already minified files, which seems wrong?
new UglifyJsPlugin({
sourceMap: true,
include: /\.min\.js$/, //<= remove this line
})
Also, when I look at other npm package, a lot are just vanilla js. Should I just let the user of my npm package, package it as a normal dependency ?
If your library will be packaged by consumers then I would distribute with both minified and unminified sources. This allows consumers to include the minified library via one of the npm CDNs (e.g. unpkg) when hacking around (e.g. jsfiddle) and the unminified when using the library as part of a bundled application, e.g. via webpack.
If your library is only meant for the Node environment then minification is generally considered unnecessary and even a burden should the consumer encounter a bug in your library and wish to debug it.
I've started using webpack2 (to be precise, v2.3.2) and after re-creating my config I keep running into an issue I can't seem to solve I get (sorry in advance for ugly dump):
ERROR in ./src/main.js
Module not found: Error: Can't resolve 'components/DoISuportIt' in '[absolute path to my repo]/src'
resolve 'components/DoISuportIt' in '[absolute path to my repo]/src'
Parsed request is a module
using description file: [absolute path to my repo]/package.json (relative path: ./src)
Field 'browser' doesn't contain a valid alias configuration
aliased with mapping 'components': '[absolute path to my repo]/src/components' to '[absolute path to my repo]/src/components/DoISuportIt'
using description file: [absolute path to my repo]/package.json (relative path: ./src)
Field 'browser' doesn't contain a valid alias configuration
after using description file: [absolute path to my repo]/package.json (relative path: ./src)
using description file: [absolute path to my repo]/package.json (relative path: ./src/components/DoISuportIt)
as directory
[absolute path to my repo]/src/components/DoISuportIt doesn't exist
no extension
Field 'browser' doesn't contain a valid alias configuration
[absolute path to my repo]/src/components/DoISuportIt doesn't exist
.js
Field 'browser' doesn't contain a valid alias configuration
[absolute path to my repo]/src/components/DoISuportIt.js doesn't exist
.jsx
Field 'browser' doesn't contain a valid alias configuration
[absolute path to my repo]/src/components/DoISuportIt.jsx doesn't exist
[[absolute path to my repo]/src/components/DoISuportIt]
[[absolute path to my repo]/src/components/DoISuportIt]
[[absolute path to my repo]/src/components/DoISuportIt.js]
[[absolute path to my repo]/src/components/DoISuportIt.jsx]
package.json
{
"version": "1.0.0",
"main": "./src/main.js",
"scripts": {
"build": "webpack --progress --display-error-details"
},
"devDependencies": {
...
},
"dependencies": {
...
}
}
In terms of the browser field it's complaining about, the documentation I've been able to find on this is: package-browser-field-spec. There is also webpack documentation for it, but it seems to have it turned on by default: aliasFields: ["browser"]. I tried adding a browser field to my package.json but that didn't seem to do any good.
webpack.config.js
import path from 'path';
const source = path.resolve(__dirname, 'src');
export default {
context: __dirname,
entry: './src/main.js',
output: {
path: path.resolve(__dirname, 'dist'),
filename: '[name].js',
},
resolve: {
alias: {
components: path.resolve(__dirname, 'src/components'),
},
extensions: ['.js', '.jsx'],
},
module: {
rules: [
{
test: /\.(js|jsx)$/,
include: source,
use: {
loader: 'babel-loader',
query: {
cacheDirectory: true,
},
},
},
{
test: /\.css$/,
include: source,
use: [
{ loader: 'style-loader' },
{
loader: 'css-loader',
query: {
importLoader: 1,
localIdentName: '[path]___[name]__[local]___[hash:base64:5]',
modules: true,
},
},
],
},
],
},
};
src/main.js
import DoISuportIt from 'components/DoISuportIt';
src/components/DoISuportIt/index.jsx
export default function() { ... }
For completeness, .babelrc
{
"presets": [
"latest",
"react"
],
"plugins": [
"react-css-modules"
],
"env": {
"production": {
"compact": true,
"comments": false,
"minified": true
}
},
"sourceMaps": true
}
What am I doing wrong/missing?
Turned out to be an issue with Webpack just not resolving an import - talk about horrible horrible error messages :(
// I Had to change:
import DoISuportIt from 'components/DoISuportIt';
// to (notice the missing `./`)
import DoISuportIt from './components/DoISuportIt';
Just for record, because I had similiar problem, and maybe this answer will help someone: in my case I was using library which was using .js files and I didn't had such extension in webpack resolve extensions. Adding proper extension fixed problem:
module.exports = {
(...)
resolve: {
extensions: ['.ts', '.js'],
}
}
I'm building a React server-side renderer and found this can also occur when building a separate server config from scratch. If you're seeing this error, try the following:
Make sure your entry value is properly pathed relative to your context value. Mine was missing the preceeding ./ before the entry file name.
Make sure you have your resolve value included. Your imports on anything in node_modules will default to looking in your context folder, otherwise.
Example:
const serverConfig = {
name: 'server',
context: path.join(__dirname, 'src'),
entry: {serverEntry: ['./server-entry.js']},
output: {
path: path.join(__dirname, 'public'),
filename: 'server.js',
publicPath: 'public/',
libraryTarget: 'commonjs2'
},
module: {
rules: [/*...*/]
},
resolveLoader: {
modules: [
path.join(__dirname, 'node_modules')
]
},
resolve: {
modules: [
path.join(__dirname, 'node_modules')
]
}
};
I encountered this error in a TypeScript project. In my webpack.config.js file I was only resolving TypeScript files i.e.
resolve: {
extensions: [".ts"],
}
However I noticed that the node_module which was causing the error:
Field 'browser' doesn't contain a valid alias configuration
did not have any ".ts" files (which is understandable as the module has been converted to vanilla JS. Doh!).
So to fix the issue I updated the resolve declaration to:
resolve: {
extensions: [".ts", ".js"],
}
I had the same issue, but mine was because of wrong casing in path:
// Wrong - uppercase C in /pathCoordinate/
./path/pathCoordinate/pathCoordinateForm.component
// Correct - lowercase c in /pathcoordinate/
./path/pathcoordinate/pathCoordinateForm.component
Add this to your package.json:
"browser": {
"[module-name]": false
},
Changed my entry to
entry: path.resolve(__dirname, './src/js/index.js'),
and it worked.
This also occurs when the webpack.config.js is simply missing (dockerignore 🤦♂️)
In my case it was a package that was installed as a dependency in package.json with a relative path like this:
"dependencies": {
...
"phoenix_html": "file:../deps/phoenix_html"
},
and imported in js/app.js with import "phoenix_html"
This had worked but after an update of node, npm, etc... it failed with the above error-message.
Changing the import line to import "../../deps/phoenix_html" fixed it.
My case was rather embarrassing: I added a typescript binding for a JS library without adding the library itself.
So if you do:
npm install --save #types/lucene
Don't forget to do:
npm install --save lucene
Kinda obvious, but I just totally forgot and that cost me quite some time.
In my case, to the very end of the webpack.config.js, where I should exports the config, there was a typo: export(should be exports), which led to failure with loading webpack.config.js at all.
const path = require('path');
const config = {
mode: 'development',
entry: "./lib/components/Index.js",
output: {
path: path.resolve(__dirname, 'public'),
filename: 'bundle.js'
},
module: {
rules: [
{
test: /\.js$/,
loader: 'babel-loader',
exclude: path.resolve(__dirname, "node_modules")
}
]
}
}
// pay attention to "export!s!" here
module.exports = config;
I had aliases into tsconfig.json:
{
"compilerOptions": {
"paths": {
"#store/*": ["./src/store/*"]
}
},
}
So I solved this issue by adding aliases to webpack.config also:
module.exports = {
//...
resolve: {
alias: {
'#store': path.resolve(__dirname, '../src/store'),
},
},
};
I got same problem and fixed with adding file extension.
// Old:
import RadioInput from './components/RadioInput'
// New:
import RadioInput from './components/RadioInput.vue'
Also, if you still want to use without extensions, you can add this webpack config: (Thanx for #matthew-herbst for the info)
module.exports = {
//...
resolve: {
extensions: ['.js', '.json', '.wasm'], // Add your extensions here.
},
};
For anyone building an ionic app and trying to upload it. Make sure you added at least one platform to the app. Otherwise you will get this error.
In my experience, this error was as a result of improper naming of aliases in Webpack.
In that I had an alias named redux and webpack tried looking for the redux that comes with the redux package in my alias path.
To fix this, I had to rename the alias to something different like Redux.
In my case, it was due to a broken symlink when trying to npm link a custom angular library to consuming app. After running npm link #authoring/canvas
"#authoring/canvas": "path/to/ui-authoring-canvas/dist"
It appear everything was OK but the module still couldn't be found:
When I corrected the import statement to something that the editor could find Link:
import {CirclePackComponent} from '#authoring/canvas/lib/circle-pack/circle-pack.component';
I received this which is mention in the overflow thread:
To fix this I had to:
cd /usr/local/lib/node_modules/packageName
cd ..
rm -rf packageName
In the root directory of the library, run:
a) rm -rf dist
b) npm run build
c) cd dist
d) npm link
In the consuming app, update the package.json with:
"packageName": "file:/path/to/local/node_module/packageName""
In the root directory of the consuming app run npm link packageName
In my case (lolz),
I was importing a local package (that I was developing, and building with rollup) via NPM/Yarn link, into another package I was developing. The imported package was a load of React components, and was configured to have a peerDependency of react and react-dom.
The consuming package was being built with Webpack and obviously wasn't correctly feeding the installed react and react-dom libraries into my local dependency as it was compiling it.
I adjusted my webpack configuration to indicate it should alias those peer dependencies to the correct dependencies in the consuming package:
/* ... */
resolve: {
extensions: [/* make sure you have them all correct here, as per other answers */],
alias: {
react: path.resolve('./node_modules/react'),
'react-dom': path.resolve('./node_modules/react-dom')
}
},
/* ... */
Obviously you need to import path in the webpack.config.js file in order to use the methods seen above.
A more detailed explanation can be found in this article
My case was similar to #witheng's answer.
At some point, I noticed some casing error in some file names in my development environment. For example the file name was
type.ts
and I renamed it to
Type.ts
In my Mac dev environment this didn't register as a change in git so this change didn't go to source control.
In the Linux-based build machine where the filenames are case-sensitive it wasn't able to find the file with different casing.
To avoid issues like this in the future, I ran this command in the repo:
git config core.ignorecase false
In my case, I imported library files like:
import { MyFile } from "my-library/public-api";
After I removed the public-api from the import everything worked fine:
import { MyFile } from "my-library";
MyFile is exported in the public-api file in the library.
In my case,
I have mistakenly removed a library ("mini-create-react-context") from package.json. I added that back, and did yarn install and build the app and it start working properly. So please take a look at your package.json file once.
In my case I had accidentally imported this package while trying to use process.env:
import * as process from 'process';
Removing it fixed the problem.
For everyone with Ionic:
Updating to the latest #ionic/app-scripts version gave a better error message.
npm install #ionic/app-scripts#latest --save-dev
It was a wrong path for styleUrls in a component to a non-existing file.
Strangely it gave no error in development.
In my situation, I did not have an export at the bottom of my webpack.config.js file. Simply adding
export default Config;
solved it.
In my case, it is due to a case-sensitivity typo in import path. For example,
Should be:
import Dashboard from './Dashboard/dashboard';
Instead of:
import Dashboard from './Dashboard/Dashboard';
In my case I was using invalid templateUrl.By correcting it problem solved.
#Component({
selector: 'app-edit-feather-object',
templateUrl: ''
})
I am using single-spa, and encountered this issue with the error
Module not found: Error: Can't resolve '/builds/**/**/src\main.single-spa.ts' in /builds/**/**'
I eventually figured out that in angular.json build options "main" was set to src\\main.single-spa.ts. Changing it to src/main.single-spa.ts fixed it.
Had the same issue with angular was importing
import { Injectable } from "#angular/core/core";
changed it to
import { Injectable } from "#angular/core";
I was getting this error when running a GitHub action. The issue was because I'd listed the package as a peer dependency instead of a dependency.
Since I'm using Rollup, the solution was to install the package both as a peer dependency and a dev dependency, and use rollup-plugin-peer-deps-external to remove the dev dependency from the final build.
For me the issue was, I was importing
.ts files into .js files
changing them to ts as well solved the issue.
In my case, I had a mixture of enum and interface in the index.d.ts file.
I extracted enums into another file and the issue resolved.
Error while running webpack-dev-server
ERROR in ./css/app.scss
Module build failed: Error: Parameter 'dependency' must be a Dependency
at Compilation.process [as _addModuleChain] (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compilation.js:347:9)
at Compilation.process [as addEntry] (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compilation.js:423:7)
at SingleEntryPlugin.<anonymous> (/Users/roshankarki/Desktop/del_del/wp_tst/node_modules/webpack/lib/SingleEntryPlugin.js:22:15)
at Compiler.applyPluginsParallel (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/node_modules/tapable/lib/Tapable.js:107:14)
at Compiler.compile (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compiler.js:394:7)
at Compiler.runAsChild (/usr/local/lib/node_modules/webpack-dev-server/node_modules/webpack/lib/Compiler.js:203:7)
at Object.module.exports.pitch (/Users/roshankarki/Desktop/del_del/wp_tst/node_modules/extract-text-webpack-plugin/loader.js:81:18)
webpack: bundle is now VALID.
My webpack.config.js looks like this:
var path = require('path');
var ExtractTextPlugin = require("extract-text-webpack-plugin");
module.exports = {
/* this defins path where entry files are to be found */
context: path.resolve('js'),
entry: ["./utils","./app"],
output: {
/* this is output directory of bundle.js */
path: path.resolve('build/'),
/* this is path of directory where bundle will be served in server */
publicPath: '/public/assets/',
filename: "bundle.js"
},
/* this plugin helps to generate seperate styles.css file rather in once single bundle.js file */
plugins: [
new ExtractTextPlugin("styles.css")
],
/* when server is loaded it tells where to look for index file accessed from root */
devServer: {
contentBase: 'public'
},
module: {
preLoaders: [
{
test: /\.js$/,
exclude: /node_modules/,
loader: "jshint-loader"
}
],
loaders: [
{
test: /\.es6$/,
exclude: /node_modules/,
loader: "babel-loader"
},
{
test: /\.css$/,
exclude: /node_modules/ ,
loader: ExtractTextPlugin.extract("style-loader", "css-loader")
},
{
test: /\.scss$/,
exclude: /node_modules/ ,
loader: ExtractTextPlugin.extract("style-loader", "css-loader!sass-loader")
}
]
},
/* this will resolve file in common js style require without extension */
resolve: {
extensions: ['', '.js', '.es6']
},
watch: true
}
Before using this plugin everything was working fine. As i intend to output different bundle for css, i need to fix this issue.
Found issue in github but it did not help:
https://github.com/webpack/extract-text-webpack-plugin/issues/132
well updating node worked for me. I did this by running "brew update" and "brew upgrade" command, as i am using homebrew as a package manage in mac.
Following instruction in the git link provide in my question resulted in error as i was installing webpack and webpack-dev-server globally. I removed it and only had local copy which did the trick at the end.
command to list global copy of node modules installed:
npm list -g --depth=0
I use this command to remove global copy:
npm uninstall -g {replace_with_package_name}
command to check local copy:
npm list --depth=0