I have an AWS CodeBuild process kicking off whenever I check code into master branch of my GitHub repo. My codebase is a dotnet core serverless application. I have a dependency on a private nuget package residing in GitHub Package Repository. My CodeBuild process is failing when trying to find the package:
error NU1101: Unable to find package datastop.io.data-dictionary.
No packages exist with this id in source(s): nuget.org
datastop.io.data-dictionary is a private nuget package residing in GitHub Package Repository, not in nuget.org. It appears from the error message the CodeBuild process is attempting to find 'datastop.io.data-dictionary' in nuget.
Here's my current buildspec.yml file:
version: 0.2
phases:
install:
runtime-versions:
dotnet: 2.1
commands:
- echo Entered the install phase...
- export PATH="$PATH:/root/.dotnet/tools"
- dotnet tool install -g Amazon.Lambda.Tools
pre_build:
commands:
- echo Entered the pre_build phase...
- dotnet restore
build:
commands:
- echo Entered the build phase...
- echo Build started on `date`
- cd $FOLDER
- dotnet lambda package --configuration release --framework netcoreapp2.1 -o ./$ZIPPED_APPLICATION
- aws cloudformation package --template-file infrastructure.yml --s3-bucket $S3_DEPLOYMENT_BUCKET --output-template-file packaged-infrastructure.yml --region us-east-2
- aws cloudformation deploy --template-file packaged-infrastructure.yml --stack-name $CLOUDFORMATION_STACK_NAME --capabilities CAPABILITY_IAM --region us-east-2
The build process is failing while running dotnet restore.
I'm assuming I need to modify my buildspec.yml file somehow to let the build process know to look at the correct package repository (our private GitHub Package Repository). How do I modify my buildspec.yml file to point to the appropriate GitHub Package Repository when trying to find datastop.io.data-dictionary package?
If you add a nuget.config file to your project, and add your GitHub repo to that config it will know where to look. From there it's a question of permissions, which is what I'm struggling with.
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="github" value="https://nuget.pkg.github.com/YOUR_ACCOUNT_NAME/index.json" />
</packageSources>
</configuration>
Related
Hi I have codepipeline to deploy my angular app, and in that app I am using my private github package. Everything is working locally etc. But on codeBuild I have no idea how to register into github package repository.
my buildspec looks like:
version: 0.2
env:
variables:
S3_BUCKET: "{{s3_bucket_url}}"
BUILD_ENV: "{{BUILD_ENV}}"
BUILD_FOLDER: "dist"
phases:
install:
runtime-versions:
nodejs: 14
pre_build:
commands:
- echo Installing source NPM dependencies...
- npm install
- npm install -g #angular/cli
build:
commands:
- echo Build started on `date` with $BUILD_ENV flag.
- ng build $BUILD_ENV
post_build:
commands:
- echo Build completed on `date`
artifacts:
files:
- '**/*'
base-directory: 'dist*'
if fails on npm install because 404 Not Found - GET https://registry.npmjs.org. For example in github actions I just simply define registry-url: 'https://npm.pkg.github.com' and thats correct.
Thanks for help :)
It fails because, in the execution context of the CodeBuild process, access to the repo containing the GitHub package is restricted, so it can't find the package because it doesn't have access to the repo's packages. You will need to authenticate to the GitHub Package API.
One way to authenticate is to create a Personal Access Token, include it in your CodeBuild Environment by linking a secret in the SecretsManager, then accessing that token in your buildspec script in the env section:
Create a personal access token: In GitHub, create a Personal access token with the read:packages permission. Here's a link to a tutorial on how to do that.
Register token as a secret in Secrets Manager: In SecretsManager, create a secret with one entry. Name the key of the entry GH_PERSONAL_ACCESS_TOKEN, and in the value field, provide the token that you created in step 1. Pick a descriptive name for your secret (something like codebuild/gh_token). Take note of the secret's name.
Authenticate to GitHub Packages using the Personal Access Token: In your buildspec script, you will need to retrieve the secret containing your Personal Access Token, then use that to authenticate before you run the npm install command:
env:
secrets-manager:
GH_PERSONAL_ACCESS_TOKEN: {SECRET_ARN}:PERSONAL_ACCESS_TOKEN # <- replace {SECRET_ARN} with arn of secret
phases:
#...
pre_build:
commands:
- echo Installing source NPM dependencies...
# this is needed to set the url where the package is located
- npm config set #OWNER:registry https://npm.pkg.github.com # <- replace OWNER with the organization/owner name
# this is needed to set the personal access token that we created
- npm config set //npm.pkg.github.com/:_authToken $GH_PERSONAL_ACCESS_TOKEN
- npm install
- npm install -g #angular/cli
Unable to generate scaffolding using aspnet-codegenerator, below is what I tried:
Created an ASP.Net RazorPages application using
dotnet new webapp
Did a dotnet build
Installed dotnet-aspnet-codegenerator using
dotnet tool install --global dotnet-aspnet-codegenerator --version 3.1.4
Ran dotnet aspnet-codegenerator --help
It says: No code generators are available in this project.Add Microsoft.VisualStudio.Web.CodeGeneration.Design package to the project as a NuGet package reference.
Added the package mentioned in step 4 using
dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design
Package added is:
<ItemGroup> <PackageReference Include="Microsoft.VisualStudio.Web.CodeGeneration.Design" Version="3.1.4" /> </ItemGroup>
Again ran: dotnet build
Final-Step
Ran dotnet aspnet-codegenerator --help
Again it says: No code generators are available in this project.Add Microsoft.VisualStudio.Web.CodeGeneration.Design package to the project as a NuGet package reference.
.Net core installed version: 3.1.401
Os: Ubuntu 20.04
This is already a few month old, but I ran into it recently for all my projects.
In case someone else lands on this page, the solution in my case(MAC OS), was to uninstall and reinstall dotnet-aspnet-codegenerator.
Run in terminal:
dotnet tool uninstall --global dotnet-aspnet-codegenerator
dotnet tool install --global dotnet-aspnet-codegenerator --version 3.1.4
I have same problem when upgrade from .net core 3.1.4 to 5.0.1.
My solution:
update global tools:
dotnet tool install --global dotnet-aspnet-codegenerator
dotnet tool update --global dotnet-aspnet-codegenerator
dotnet tool install --global dotnet-ef
dotnet tool update --global dotnet-ef
add/update links to packages
dotnet remove package Microsoft.VisualStudio.Web.CodeGeneration.Design
dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design
dotnet add package Microsoft.EntityFrameworkCore.Design
Replace string netcoreapp3.1.4 to net5.0 in .csproj and launch.json files
Clear directories
bin
obj
Rebuild project
Finally run:
dotnet aspnet-codegenerator --help
If all fine it will show available generators.
The root cause of this problem has been described in this issue as the .NET code expecting a case-insensitive filesystem which fails on a Linux host.
A suggested workaround would be to temporarily mount you ~/.nuget directory on a vfat filesystem.
To do so create a dummy file holding the filesystem
dd if=/dev/zero of=/tmp/mynuget bs=1024k count=2000
mkfs.vfat /tmp/mynuget
and then mount it
sudo mount -o uid=myuser,gid=myuser -t vfat /tmp/mynuget /home/myuser/.nuget
and use it
dotnet restore
dotnet aspnet-codegenerator ...
I test your command,and when I do this dotnet tool install --global dotnet-aspnet-codegenerator --version 3.1.4,I can get
and then Ran dotnet aspnet-codegenerator --help,I get the same message No code generators are available in this project.Add Microsoft.VisualStudio.Web.CodeGeneration.Design package to the project as a NuGet with you,
but when I do the following,I can run without the message:
I think maybe the package is not added correctly,you can try to remove and add it again.
dotnet remove package Microsoft.VisualStudio.Web.CodeGeneration.Design
dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design
I have found the SOLUTION to this issue:
The newest versions are giving some problems, so you will have to downgrade the version of the sdk to the 3.1.3 by writing this command on the terminal:
sudo apt install dotnet-sdk-3.1=3.1.301-1
After that you will need to install the following:
dotnet tool install --global dotnet-aspnet-codegenerator --version 3.1.3
After that you will need to add the right Web.CodeGeneration.Design Package by writing this on the terminal:
sudo apt install dotnet-sdk-3.1=3.1.301-1
dotnet tool install --global dotnet-aspnet-codegenerator --version 3.1.3
That has worked for me!!
I have an npm source code that needs to be built and push to npmjs repository. In details it looks like:
Build pipeline:
1) Get sources from Bitbucket repository
2) Get from package.json version number (ex. 0.0.3), increase it by 0.0.1 (0.0.4) and add this value to build variables $(version)
3) Make NPM install and build.
4) Take package-0.0.4.tgz and package.json to the artifacts folder and publish it.
Release pipeline:
1) Download artifacts
2) Extract package-0.0.4.tgz to npm-publish folder
3) Copy package.json to npm-publish folder
4) Publish npm folder to npmjs repository.
My question - is it possible to commit to Bitbucket repository updated package.json file with new version after publishing to npmjs repository?
It is possible to commit to Bitbucket repository. You only need to add a script task to execute the git commands.
For below example. I add a powershell task to run below commands in the pipeline to commit changes and push to the Bitbucket repo.
- powershell: |
git config --global user.email "you#example.com"
git config --global user.name "user.name"
#echo "some-text" > filename.txt
git add .
git commit -m "update package version"
git push https://username:password#bitbucket.org/name/repo.git HEAD:master -q
#if your password or username contain # replace it with %40
displayName: 'push to bitbucket'
I have an AWS Codebuild that gets the code from GitHub and after a successful build, it uploads the artefact to an s3 bucket. The bucket has versioning enabled. I want to create a new git release tag after every successful build which will contain the s3 version id of the artefact uploaded. How is that achievable?
My buildspec.yaml looks as follows
version: 0.2
phases:
install:
commands:
- echo install phase started on `date`
- npm i
build:
commands:
- echo build started on `date`
- npm run build
post_build:
commands:
- echo post_build started on `date`
artifacts:
files:
- '**/*'
I need to continually build a create-react-app application and deploy it to Amazon S3 bucket.
I have written the following CircleCi config.yml:
version: 2
jobs:
build:
docker:
- image: circleci/node:7.10
steps:
- checkout
- run: npm install
- run: npm run build
deployment:
prod:
branch: circle-config-test
commands:
- aws s3 sync build/ s3://http://www.typing-coacher.net.s3-website.eu-central-1.amazonaws.com/ --delete
What I think should happens:
I have a docker container, I install the application, build it and the files are resting ready in build folder.
I am running the command listed in CircleCi docs and the build files are moving from the docker machine to s3 bucket.
To deploy a project to S3, you can use the following command in the
deployment section of circle.yml:
aws s3 sync <path-to-files> s3://<bucket-URL> --delete
What actually happens:
Application is being install and build files are being created, but nothing happen with deployment. it doesn't even appear on the builds console.
What Am i missing?
disclaimer: CircleCI Developer Advocate
Everything from the deployment: line and down shouldn't be there. That's syntax for CircleCI 1.0 while the rest of your config file is CircleCI 2.0.
You can either:
Create a new step and check for the branch name with Bash. If it's circle-config-test, then run the deployment commands. You'll also need to install the AWS CLI in that build.
Using [CircleCI Workflows], create a deployment job with a branch filter for circle-config-test. You can use any image that contains the AWS CLI or install it yourself. The CI Builds: AWS Docker image contains this for you.