No node_modules from netlify deploy - npm-install

I've got continuous deployment setup with Netlify and a git repo but so far no matter what I do, Netlify isn't npm-installing anything. There's no node_modules folder when I download a zip of the deploy, my site can't access node_modules, it's just not there. I've started with a random npm package (lodash) to try to get it to install but I've got nothing so far.
Netlify says that it automatically runs npm install. I've tried having no build commands and I've tried adding npm install as a build command with no results from either.
package.json:
{
"name": "netlify-test",
"version": "1.0.0",
"description": "stuff",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "git+https://github.com/electrovir/netlify-test.git"
},
"author": "electrovir",
"license": "MIT",
"bugs": {
"url": "https://github.com/electrovir/netlify-test/issues"
},
"homepage": "https://github.com/electrovir/netlify-test#readme",
"dependencies": {
"lodash": "^4.17.11"
}
}
HTML:
<!doctype html>
<html>
<head>
<title>
hi
</title>
<script src="node_modules/lodash/_apply.js"></script>
</head>
<body>
Why can't this find node_modules??
</body>
</html>

It took a while to figure this out but I discovered that Netlify does npm install into your repo root or Base directory when you have a package.json. This can be seen by changing the Build command to something like ls and reading the deploy console output (there will be a node_modules folder listed).
However, at some point after running the Build command and before deploying, this node_modules is deleted. Hence, if you use something similar to the following as a Build command, node_modules can be copied to your Publish directory:
rm -rf dist && mkdir dist && rsync -rv * dist --exclude ./dist
This copies the Base directory contents into dist, which I have set as my Publish directory, ignoring ./dist itself (it can't be copied into itself). It'd be even better if this were added as an npm script in package.json with npm run <script name> for your Build command.
Note that Base directory, Build command, and Publish directory are all under Deploys > Deploy settings or Settings > Build & deploy in Netlify.
Thanks to lastmjs for help with figuring this out.

It would be odd if node_modules WERE there. Netlify's continuous deployment does run npm install for you, if and only if there is a package.json in the root of your repository (or instead in the base directory if you set one in netlify.toml. However, it also uses a custom npm directory that is outside of the deployment directory (see how it is setup here: https://github.com/netlify/build-image/blob/master/run-build-functions.sh#L34), since deploying your node_modules shouldn't be needed for a static site at browse time - only at build time.
The intended path for a Netlify deploy is:
you have your dependency manager configuration checked into your git repo at the root of the repo (if no base directory is set in the UI or in the toml file) or in the base directory if set. This could be any/all of Gemfile.lock, package.json, yarn.lock, or requirements.txt as explained in this article about the build settings at Netlify
you do not store your node_modules in the repo. Many of those will be specific to the architecture of the build environment - and your local machine is almost certainly different than Netlify's. You'll want those generated at build time.
your build is intended to USE those modules DURING BUILD. So you use for instance dotenv to read your .env file while building, or you use gulp-cli to process your Gulpfile.js
your build completes, and you've used all the modules during build to generate standalone html/js/css/images, so you're done with them - your static html and other assets are generated in your publish directory, where the build process has intentionally not put your node modules, and thus they are not deployed. Your static website should not need the modules at runtime, so deploying the (thousands!) of files in there is not efficient or needed.
You could have a situation that requires some/one of them - e.g. a function or you could need a file from one of them even on your static site - and that's fine, feel free to copy it into the publish directory explicitly. But that is not the norm :)

Related

Issue pulling from Gitlab private package registry

We have a self-hosted GitLab (15.5.4) and I've configured everything we needed for publishing npm packages.
A CI/CD pipeline that properly creates the entry in the Package Registry.
The problem is that when I pull the package [npm i #scope/lib] (It doesn't change if I cast the auth token in the package.json or I pass through an environment variable as suggested in the documentation) the unwanted result is that the #scope/lib doesn't have the dist/ folder in it!! [node_module/#scope/lib/].
If I browse to the Package Registry and manually download the .tgz file I can see that the dist/ folder is present.
I've played around a bit with the .npmignore and "prepublish" script but I had no success and literally have no glue why this is happening.
Any tips would be very appreciated
To clarify:
The proper way is to tell npm to keep the dist/ folder, bypassing the .gitignore file (instead of defining an .npmignore article here ) is to define a files entry in the package.json :
{
"files": [
"dist",
"build",
...
]
}
Another unproper way to do get the result I needed is to use a postinstall command. But it is clearly an anti-pattern. Given that I am writing a typescript library, that is tested and then compiled by the CI, there's no need to recompile it within the postinstall command. But it could be an hacky solution when needed.
{
"scripts": {
"postinstall": "tsc src/index.ts"
}
}
To sum up, I think it was only an npm cache issue or more probably a server-side cache issue, because I've run npm cache clean --force different times.
Hope this helps.

npm, avoid publishing of src dir without using .npmignore

The npm publish
command creates a tarball (with src dir) and publish it to registry.
is there a way to exclude the src dir avoiding use of .npmignore ?
npm provides no other built-in feature to achieve that, so a custom solution is required.
If you really don't want to use .npmignore to keep the src directory out of your published package, then consider utilizing pre and post hooks in your npm scripts instead.
The pertinent hooks are:
prepublishOnly: Run BEFORE the package is prepared and packed, ONLY on npm publish ...
postpublish: Run AFTER the package is published.
For *nix platforms
Add a prepublishOnly script to the scripts section of your package.json that moves the src directory to another location outside of your project directory prior to publishing.
Also, add a postpublish script that moves the src directory back to the project directory when publishing has completed.
Run npm publish (as per normal) to publish your package.
For instance:
package.json
...
"scripts": {
"prepublishOnly": "mv src/ ../",
"postpublish": "mv ../src .",
...
},
...
Note: You'll need to ensure that no other src folder/directory exists at the path location you choose to temporarily move the src directory to via your prepublish script.
Cross platform:
For a cross-platform solution consider utilizing shx. This package includes a portable mv command. In which case, configure your prepublish and postpublish scripts something like the following instead:
package.json
...
"scripts": {
"prepublishOnly": "shx mv src/ ../",
"postpublish": "shx mv ../src .",
...
},
...
You can use the files property in your package.json to explicitly include the files you want to publish.
{
"files": [
"dist",
"index.js"
]
}
As #RobC answer, there is no other way then a custom solution for avoid using .npmignore.
Since I’m using the publish command in a Jenkins pipeline, a solution is to create a temporary .npmignore while the publish step directly in the Jenkinsfile:
echo "src/" >> .npmignore
echo "*.js" >> .npmignore
echo "*.json" >> .npmignore
echo "Jenkinsfile" >> .npmignore
curl --insecure -u ${USERPASS} 'https://my-repo/api/npm/auth' >> /home/jenkins/.npmrc
npm publish --registry https:// my-repo/api/npm/npm-local/

lerna publish and npm pack failing to package all the files in "dist" folder

I am trying to build my first Angular Component package using lerna and it was working pretty well until I realized I had to add "ng-packagr" to get all of the HTML bundled inline with the rest of the code. After adding that support and getting it to work now all of a sudden my files are not getting published into the tarball.
Here is my package.json
{
"name": "#custom/core",
"version": "0.0.7",
"description": "Test",
"main": "./dist/bundles/custom-core.umd.min.js",
"module": "./dist/esm2015/custom-core.js",
"typings": "./dist/index.d.ts",
"$schema": "./node_modules/ng-packagr/package.schema.json",
"ngPackage": {
"lib": {
"entryFile": "./src/index.ts"
},
"whitelistedNonPeerDependencies": [
"."
]
},
"scripts": {
"build": "ng-packagr -p package.json"
},
"files": [
"dist"
],
...
My dist folder contains all kinds of folders like this:
But then when I run a lerna publish or npm pack this is what happens:
As you can see only 1 file gets added to the tarball...
Does anyone know why this is happening all of a sudden? I've tried playing around with my .gitignore thinking maybe it was forcing the packaging to ignore these other files but it wasn't that.
UPDATE
Ok so I found that the culprit is ng-packagr. When I run my npm run build which uses ng-packagr -p package.json to build the different module packages, that CLI is also generating a package.json that goes inside of my dist folder. When the npm pack or lerna publish attempt to package everything using a package.json they must be looking at the generated on in dist rather than the one in the folder above it.
I'm not sure how I should be fixing this.
My solution was to abandon my plan to use the files key in the package.json file and to instead use a .npmignore file. Here is a copy of mine:
# Node generated files
node_modules
npm-debug.log
assets
package-lock.json
# aot files
aot
# OS generated files
Thumbs.db
.DS_Store
# Ignored files
*.ts
!*.d.ts
tsconfig.json
tsconfig-aot.json
tslint.json
*.tgz
config
src

Which command do I use to generate the build of a Vue app?

What should I do after developing a Vue app with vue-cli?
In Angular there was some command that bundle all the scripts into one single script.
Is there something the same in Vue?
I think you've created your project like this:
vue init webpack myproject
Well, now you can run
npm run build
Copy index.html and /dist/ folder into your website root directory. Done.
If you've created your project using:
vue init webpack myproject
You'd need to set your NODE_ENV to production and run, because the project has web pack configured for both development and production:
NODE_ENV=production npm run build
Copy dist/ directory into your website root directory.
If you're deploying with Docker, you'd need an express server, serving the dist/ directory.
Dockerfile
FROM node:carbon
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
ADD . /usr/src/app
RUN npm install
ENV NODE_ENV=production
RUN npm run build
# Remove unused directories
RUN rm -rf ./src
RUN rm -rf ./build
# Port to expose
EXPOSE 8080
CMD [ "npm", "start" ]
in your terminal
npm run build
and you host the dist folder. for more see this video
To deploy your application to prod environment add
"build": "vue-cli-service build --mode prod"
in your scripts in package.json file.
Open your main.js and add
Vue.config.productionTip = false;
right after your imports.
Then open your cli in the project folder and run this command
npm run build
This will make a dist folder in your project directory you may upload that dist folder in your host and your website will be live
If you run into problems with your path, maybe you need to change the assetPublicPath in your config/index.js file to your sub-directory:
http://vuejs-templates.github.io/webpack/backend.html
The vue documentation provides a lot of information on this on how you can deploy to different host providers.
npm run build
You can find this from the package json file. scripts section. It provides scripts for testing and development and building for production.
You can use services such as netlify which will bundle your project by linking up your github repo of the project from their site. It also provides information on how to deploy on other sites such as heroku.
You can find more details on this here
The commands for what specific codes to run are listed inside your package.json file under scripts. Here is an example of mine:
"scripts": {
"serve": "vue-cli-service serve",
"build": "vue-cli-service build",
"lint": "vue-cli-service lint"
},
If you are looking to run your site locally, you can test it with
npm serve
If you are looking to prep your site for production, you would use
npm build
This command will generate a dist folder that has a compressed version of your site.
THIS IS FOR DEPLOYING TO A CUSTOM FOLDER (if you wanted your app not in root, e.g.
URL/myApp/) - I looked for a longtime to find this answer...hope it helps someone.
Get the VUE CLI at https://cli.vuejs.org/guide/ and use the UI build to make it easy. Then in configuration you can change the public path to /whatever/ and link to it URL/whatever.
Check out this video which explains how to create a vue app using CLI if u need more help: https://www.youtube.com/watch?v=Wy9q22isx3U
For NPM => npm run Build
For Yarn => yarn run build
You also can check scripts in package.json file
You write down the below command being at the project root.
npm run build
First Install Vue Cli Globally
npm install -g #vue/cli
To create a new project, run:
vue create project-name
run vue
npm run serve
Vue CLI >= 3 uses the same vue binary, so it overwrites Vue CLI 2 (vue-cli). If you still need the legacy vue init functionality, you can install a global bridge:
Vue Init Globally
npm install -g #vue/cli-init
vue init now works exactly the same as vue-cli#2.x
Vue Create App
vue init webpack my-project
Run developer server
npm run dev
This command is for start the development server :
npm run dev
Where this command is for the production build :
npm run build
Make sure to look and go inside the generated folder called 'dist'.
Then start push all those files to your server.
One way to do this without using VUE-CLI is to bundle the all script files into one fat js file and then reference that big fat javascript file into main template file.
I prefer to use webpack as a bundler and create a webpack.conig.js in the root directory of project. All the configs such as entry point, output file, loaders, etc.. are all stored in that config file. After that, I add a script in package.json file that uses webpack.config.js file for webpack configs and start watching files and create a Js bundled file into mentioned location in webpack.config.js file.
I think you can use vue-cli
If you are using Vue CLI along with a backend framework that handles static assets as part of its deployment, all you need to do is making sure Vue CLI generates the built files in the correct location, and then follow the deployment instruction of your backend framework.
If you are developing your frontend app separately from your backend - i.e. your backend exposes an API for your frontend to talk to, then your frontend is essentially a purely static app. You can deploy the built content in the dist directory to any static file server, but make sure to set the correct baseUrl
npm run build - this will uglify and minify the codes
save index.html and dist folder in root directory of your website.
free hosting service that you might be interested in -- Firebase hosting.
if you used vue-cli and webpack when you created your project.
you can use just
npm run build command in command line, and it will create dist folder in your project. Just upload content of this folder to your ftp and done.
If you are using npm u can use npm run build but if you are using yarn you can simply run yarn build
If you want to create a build for a domain, you can use the $ npm run build command.
If you're going to build for a sub-domain, follow these instructions:
Create a file that's name is vue.config.js in the root
Write down the below code in the vue.config.js file:
module.export = {
publicPath: '/demo-project',
}
Now run $ npm run build
Note: Use your subdomain name instead of "/demo-project".
If you want to build and send to your remote server you can use cli-service (https://cli.vuejs.org/guide/cli-service.html) you can create tasks to serve, build and one to deploy with some specific plugins as vue-cli-plugin-s3-deploy

configure a different path per client deps in npm

In an asp.net core project all the client files must to be copied under the approot directory to be deployed correctly: jspm let you define a proper directory for client deps, but with npm I have to copy the files from node_modules
directory to the approot\node_modules using a gulp task.
Since I'm not interested in filtering or manipulating the files before the deploy, but I just what that the files in the installed module are deployed, there is some way to do this without use gulp ?
You can use npm to do that. Add a script to your package.json:
"scripts": {
"copy": "xcopy from to"
}
Then you can call npm run-script copy to have it executed. I used xcopy as an example, you can use whatever you like. There is some more documentation about scripts in npm.