I am trying to configure dbt_utils for dbt with github. I have the following in my packages.yml file:
packages:
- package: dbt-labs/dbt_utils
version: 0.8.0
https://github.com/actonmarketing/actondata/blob/main/packages.yml
And I have removed packages from my gitignore file.
When I run dbt deps I get a message saying no packages found in packages.yml.
What am I doing wrong?
packages.yml needs to be at the same hierarchy level as dbt_project.yml
moving packages.yml from project_name/dbt_packages/packages.yml to project_name/packages.yml worked for me.
--
If you want it to live in a subdirectory, see the dbt docs here :
https://docs.getdbt.com/docs/building-a-dbt-project/package-management#project-subdirectories
You might just need to save your packages.yml file
Related
This is probably a pretty basic question, but I can't find an answer:
If I have a project with a dependency in package.json listed as foobar: ^3.2.1, what version of that dependency will be installed when I run vite build, assuming that the latest available version of the package is 3.4.5?
First thing first, vite build won't change anything to your dependencies. I won't install ones nor update them. It will only build your project (i.e. compile / transpile / minify / bundle etc.) using your source code and the code it imports (likely within the node_modules).
It will build locally, so using your local dependencies in the node_modules folder.
To check the current package version you have installed, you can run:
npm list --depth=0 | grep foobar
(The grep part is optional)
You can also open your package-lock.json or yarn.lock file and search for your package to know to what version your package has been fixed to.
To understand about the semantic version with npm, read this documentation: https://docs.npmjs.com/about-semantic-versioning
Is there a way to check what packages are installed? I would expect something like dbt list packages?
The context is:
Until I run dbt deps the content of packages.yml gives me nothing. And there are some situations when the models could be triggered without running dbt deps
I would like to check the packages in runtime
I searched over google and dbt --help but I didn't find anything.
No, there is no dbt list packages command; there is no --list option for dbt deps; packages are not listed in dbt debug; installed packages are not listed in manifest.json; nor is there another officially supported analog of pip list to determine all of the currently-installed packages in your runtime.
You should open an issue for this!
Your current options (v1.3.0) are:
Run dbt clean and dbt deps before any other dbt command, to ensure your installed dependencies match what is in your packages.yml file
Run dbt compile and inspect manifest.json to ensure it has the macro(s) and/or model(s) you need from a package (note that dbt compile will fail if packages are listed in packages.yml and not instealled)
Inspect your project's dbt_packages subdirectory. Each installed package will have its own directory; if that package has any dependencies, they will be nested under subsequent dbt_packages directories
dbt clean and dbt deps are pretty fast, so the cost of (1) is low, and you don't have to manage the logic/complexity of various bad states if you go that route. But I think you should absolutely open an issue for this!
You should have a packages.yml file in your repo that contains all the packages that are installed via dbt deps.
Following the dbt deps, a directory called /dbt_modules (by default, can be changed in your dbt_project.yml) will appear in your repo (it is included, also by default, in your .gitignore file to avoid duplicating the source code for the package).
Recently I had the opportunity to work with packages closely and I published an article The Practical Guide to Utilizing DBT Packages for Data Transformation which explains dbt packages and its underhood
Here is the quote:
To verify that a package is installed in your dbt project, you can check the packages.yml file and run the dbt deps command.
Check the packages.yml file: This file lists all of the packages that are installed in your dbt project. Look for the name of the package you want to verify. If it is listed in the packages list, then it is installed.
Run the dbt deps command:
This command will show you a list of all of the packages that are installed in your dbt project. Look for the name of the package you want to verify. If it is listed, then it is installed.
In the root dbt project dir, you observe a new dir dbt_modules/ which contains the compiled packages that are ready to be used. NOTE: dir dbt_modules/ has to be added to .gitignore.
>>> tree -L 1 .
.
├── data
├── dbt_modules
├── dbt_project.yml
├── macros
├── models
├── packages
├── packages.yml
├── profiles.yml
├── snapshots
└── target
If your packages.yml file contains package that is not installed then you would not be able to run any dbt command:
>>> dbt list
Encountered an error:
Compilation Error
dbt found 1 package(s) specified in packages.yml, but only 0 package(s) installed in dbt_modules. Run dbt deps to install package dependencies.
So this is our guarantee that in runtime we would not have any issues related to the package installation.
When trying to run yarn dev and I am getting the following error:
Error: [BABEL] C:\Users\User\OtherFolders\src\index.js: Could not find plugin "proposal-private-methods". Ensure there is an entry in ./available-plugins.js for it.
I have "#babel/plugin-proposal-private-methods": "7.8.3" as a dependency on my package.json file. Also, the #babel/plugin-proposal-private-methods folder is present in the node_modules folder.
I had tried the following to no luck:
Adding a resolution: "#babel/compat-data": "7.10.1" to the package file
I remove the node_module folder, yarn-lock, and package-lock files before attempting to run yarn install. yarn install runs without issues
I also try $ npm i #babel/plugin-proposal-private-methods
Any ideas on what the error could be or potential solutions? Thanks in advance for any help!
Kudos to the Babel Project GitHub team to help to solve the issue. The issue solution is to change the version of #babel/preset-env from 7.8.6 to 7.8.7 or higher. The response to this ticket in GitHub is in this link.
I'm upgrading my dbt version from 0.16.0 to 0.17.2, and trying to update our codegen package as well.
My packages YML looks like this:
packages:
- package: fishtown-analytics/codegen
version: 0.2.0
The error I receive is this:
$ dbt deps
Running with dbt=0.17.2
Encountered an error:
Version error for package fishtown-analytics/codegen: Could not find a satisfactory version from options: ['=0.2.0', '=0.0.4', '=0.0.4']
Any Ideas?
I Also had some local packages within the packages.yml file which were dependent on a bad version of fishtown-analytics/codegen. After, I fixed all the dependencies in the downstream packages files everything imported correctly.
I am using Yarn v0.19.1 to install some dependencies. I deleted my node_modules folder completely and did a fresh yarn install.
I am trying to install the dependency leaflet using yarn add leaflet. The module installs successfully, except during the Cleaning Modules... phase, Yarn removes the images folder which would typically live within leaflet/dist/images. When I do a npm install leaflet this folder does not get removed.
During a yarn install, the images folder is present until the Cleaning modules phase happens.
Who/what is doing this? Is this something yarn does? Or is this something specified within the leaflet library? How could I resolve this?
I checked the package.json in the leaflet library and nothing seemed out of the ordinary there. It runs a jake file, but even within that file nothing is being deleted related to images.
Here is what the folder looks like, within my node_modules folder, for both package manager installs:
yarn
npm
There was a .yarnclean file in my project. This added some files/folders to ignore when installing dependencies. Running yarn clean added this file and I wasn't aware until I saw this issue. Viewing the docs also gave the same info.
I resolved this by removing images from the set of ignored directories.
Here is my .yarnclean file for an example:
# test directories
__tests__
test
tests
powered-test
# asset directories
docs
doc
website
assets
# examples
example
examples
# code coverage directories
coverage
.nyc_output
# build scripts
Makefile
Gulpfile.js
Gruntfile.js
# configs
.tern-project
.gitattributes
.editorconfig
.*ignore
.eslintrc
.jshintrc
.flowconfig
.documentup.json
.yarn-metadata.json
# misc
*.gz
*.md