Is there a way to force NuGet to restore the latest package version during pipeline execution? - vb.net

We are using an internal NuGet feed in our Azure DevOps environment to host many different packages, which are consumed in many different projects. I would like to configure the pipeline so that whenever an internal package is referenced, it is always resolved to the latest version. All of the internal references are configured with wildcards in the PackageReference tag of the .vbproj file, like so:
<PackageReference Include="MyPackageName" Version="*" />
And the restore command in the YAML file is configured like so:
- task: NuGetCommand#2
condition: and(succeeded(), eq(variables['ModifiedProject'], 'true'))
displayName: 'NuGet restore'
inputs:
command: 'restore'
restoreSolution: '$(ModifiedProject.Directory)/$(ModifiedProject.Solution)'
feedsToUse: 'select'
vstsFeed: '[guid]/[guid]'
noCache: true
The projects build successfully, but they still end up using the oldest version of the package instead of restoring the newest version. Is there a way to force the restore task to default to the newest package version?

I ended up finding a workaround for this issue by using the Azure DevOps REST API. My objective was to get the pipeline to flow through these basic steps:
After all build steps have been completed successfully, push the new package version into the artifact feed.
Use the REST API to unlist the old package version, leaving only the newest version available.
Because of the use of floating versions in the PackageReference tags, any projects that depend on these packages will then automatically pull the newest version when they are built in their respective pipelines.
I was only comfortable going this route because "deleting" a package in Azure doesn't really delete it, it just gets moved to the Recycle Bin. In the event there is a breaking change that is discovered post-push, we can always restore a previous version to the feed. Given that this process is being used exclusively for our 50+ internal packages with their own testing processes, it definitely seems like a safe route to go, and far more efficient than any other option I can find. However, I don't think I will mark this as a definitive answer to the question, because it still feels a bit hacky and I'd prefer there was a legitimate option to force newest versions in Azure's NuGetCommand#2 task.
Code
As mentioned, I used Azure's well-documented REST API for these functions, particularly the areas governing artifacts. Although there is a page dedicated to deleting a package from a NuGet feed, I couldn't get their specification to work. I ended up inspecting the calls made from the UI and copying those, while still using my own token for authentication. This method does the history "trimming" I needed:
public void TrimPackageFeed(string feedName, string packageName)
{
var packageVersions = GetPackageVersions(feedName, packageName);
var deprecated = packageVersions.Where(x => !x.IsLatest && !x.IsDeleted)?.ToList();
if (deprecated != null && deprecated.Any())
{
foreach (var version in deprecated)
{
var url = $"{version.Links.Feed.Value.Replace("feeds.dev.azure.com", "pkgs.dev.azure.com")}/nuget/packagesBatch";
var payload = new AzurePackagePayload
{
Data = null,
Operation = 2,
Packages = new List<AzurePackagePayloadItem>
{
new AzurePackagePayloadItem
{
ID = packageName,
Version = version.Version
}
}
};
ApiRequest(url, Method.POST, null, JsonConvert.SerializeObject(payload));
}
}
}
I built this as a .NET Core 3.1 command line app, published as a self-contained executable within our build repo. I used C# because it's most familiar to me, but I'm confident this could be scripted in any language (probably even just PowerShell). I then appended the following task to the end of my YAML pipeline definition:
- task: CmdLine#2
condition: and(succeeded(), eq(variables['ModifiedProject'], 'true'))
displayName: 'Trim package feed'
inputs:
script: |
AzureApiClient -action trim-package-feed -feed FeedNameHere -package $(ModifiedProject.AssemblyName)
workingDirectory: 'Azure\AzureApiClient\Output'
failOnStderr: true
The package gets pushed to the feed, then the assembly name is passed to my API client which will trim the historical versions and leaves only the new version available to restore.

If you are using a self-hosted agent to run the pipeline, you can try to clear the local nuget cache, delete all the nuget packages in the global nuget cache under C:\Users\xxx\.nuget\packages or use clean nuget caches.
If the package to be consumed is just pushed into the feed, you need to wait for a while . There has to be a delay in populating packages on the feed.
In addition , you can try to use dotnet restore task to see if this issue still occurs . Here is a ticket with similar issue you can refer to .

Related

dbt- synapse models\example generating error

The problem I’m having is after the profiles.yml All checks passed! (the connection successful)
when i run dbt get this message
Configuration paths exist in your dbt_project.yml file which do not apply to any resources. There are 1 unused configuration paths:- models.dbt_project.example
when I run
dbt --version
Core: - installed: 1.4.1
- latest: 1.4.1 - Up to date!
Plugins: - sqlserver: 1.3.0 - Not compatible!
- synapse: 1.3.2 - Not compatible!
At least one plugin is out of date or incompatible with dbt-core.
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
The error message is telling you what is wrong. There is config in your dbt_project.yml file that does not apply to any files in your models directory.
When you run dbt init, it creates some example models (.sql files) in the models/example directory, like my_first_dbt_model.sql. It also adds some example config to dbt_project.yml that looks like this:
# towards the bottom...
models:
your_project_name:
# Config indicated by + and applies to all files under models/example/
example:
+materialized: view
You need to delete the example key, since you deleted that directory. You could also delete the keys above it, or keep them, since you'll probably add config at some point.
The OTHER error is probably because you installed dbt-core separately from dbt-synapse. You should delete your virtual environment and start over by just running pip install dbt-synapse, which will automatically install a compatible version of dbt-core. You should NEVER pin versions of dbt-core, only the version of your adapter (since the adapter will specify its compatible versions of dbt-core).

How to debug neovim lsp custom command

I am attempting to get the volar vue language server to work in place of vetur for neovim's native lsp.
Using both lspconfig and lspinstall I was able to create a working custom install for sumneko_lua (unrelated but had to manually build due to some issues with the built-in :LspInstall lua). Below is that code duplicated and modified for an attempt at using this new vue server:
local vue_config = require'lspinstall/util'.extract_config('vuels')
vue_config.default_config.cmd = {'node', './node_modules/vscode-vue-languageservice/out/index.js', '--stdio'}
require'lspinstall/servers'.newvue = vim.tbl_extend('error', vue_config, {
install_script = [[
! test -f package.json && npm init -y --scope=lspinstall || true
npm install vscode-vue-languageservice#latest
]],
uninstall_script = nil
})
Running :LspInstall newvue installs properly, however :LspInfo shows this language server is attached to the buffer (of a .vue file) but not active. I believe the issue is with this path: ./node_modules/vscode-vue-languageservice/out/index.js. It exists, but may not be the correct entry point? The default vue ls simply has vls as the command because it provides a binary. Am I missing something in this package? I have yet to come across another language server without a single binary to pick out.
Thanks!
Can you try an absolute path to the out.js file? In my pretty elaborate config for a custom Volar install I'm using something just /home/myuser/dev/volar/packages/server/out/index.js (where the volar folder is just the whole volar cloned github repo). My full config is here
I don't think you can use relative paths like you did. I know you're assuming that the "./node_modules" means "workspace directory" but it's hard to tell in which directory nvim-lspconfig opens up those executables.
I have yet to come across another language server without a single binary to pick out.
Volar also provides a binary, it's volar-server (after running npm i -g #volar/server), it's just with a custom install (ie. alongside the real volar) you can't use it, because I assume you want to use your local install with custom code.
As for more indepth debugging/logging, you can check ~/.cache/nvim/lsp.log to see why the language server dies in detail.

How can I test electron-builder auto-update flow?

I built an Electron app and I am now looking at how to distribute it.
I went with electron-builder to handle packaging etc.
For a bit of context, as a web developer, I am used to continuously deploy web apps on a web server but I have a hard time figuring out how to distribute a packaged one in Electron.
In electron-builder docs there is a brief mention about testing auto-update:
"Note that in order to develop/test UI/UX of updating without packaging the application you need to have a file named dev-app-update.yml in the root of your project, which matches your publish setting from electron-builder config (but in YAML format)"
But, it's rather vague...
So I actually have two questions:
1. How do I actually test the auto-update flow?
Do I need to actually publish a new version to trigger an update locally? Seems pretty unclear, it would be like developing against the production server.
2. Is it possible to have a fallback for unsigned code?
I don't have yet any certificate for code signing. So the OS/app will block the auto-update. But, I'd still want to tell the user that an update is available so they can go and download the app manually. Can I do that? (going back to point 1, I'd like to be able to test this flow)
I've just finished dealing with this. I also wanted to test against a non-production server and avoid having to package my app each time I iterated. To test downloads I had to sign my app, which slowed things down. But it sounds like you just need to check for updates. Which I think you can do as follows...
I created a dummy github repo, then created a a file dev-app-update.yml containing:
owner: <user or organization name>
repo: dev-auto-update-testing
provider: github
The path where this file is expected to be defaults to a place you can't access. Thankfully, you can override it like so:
if (isDev) {
// Useful for some dev/debugging tasks, but download can
// not be validated becuase dev app is not signed
autoUpdater.updateConfigPath = path.join(__dirname, 'dev-app-update.yml');
}
...that should be enough for your case -- since you don't need downloads.
If not, here are some other tips:
you can change the repo setting in your electron-builder config to point at your dummy repo then package your app. This will give you a packed, production build that points at your dummy repo -- this is how I did my download testing (though I have a cert, and signed my app)
you should be calling autoUpdate's checkForUpdates(), but if checkForUpdatesAndNotify() gives you a useful OS Notification then you should be able to set autoUpdater.autoDownload to false and end up with what you need.
Lastly, it sounds you could skip autoUpdater, since you won't be using the download feature anyway. Instead you could use github's releases api, assuming you use github to host your release. If not then your host should have something similar. Use that to check for updates then tell the user from within your App (could present them with a clickable URL too). If you want OS Notifications electron has a module for that.
We're using electron-updater with GitHub as a provider for auto-updates. Unfortunately, it breaks a lot and the electron-builder team doesn't support these issues well (1, 2, 3) (from my own experience, but you can find more examples on GitHub).
One way to test updates in dev mode:
Create a build of your app with an arbitrarily high version number
Create a public repo and publish the above build
Create a dev-app-update.yml next to your main entry point and configure it for the repo above (see)
In your main entry point:
import { autoUpdater } from "electron-updater";
...
if (process.env.NODE_ENV === "development") {
// Customize the test by toggling these lines
// autoUpdater.autoDownload = false
// autoUpdater.autoInstallOnAppQuit = false;
autoUpdater.checkForUpdates();
}
Then when running yarn dev you should see something like:
Checking for update
...
Found version 100.0.0 (url: <>.exe)
Downloading update from <>.exe
updaterCacheDirName is not specified in app-update.yml Was app build using at least electron-builder 20.34.0?
updater cache dir: C:\Users\<>\AppData\Local\Electron
New version 100.0.0 has been downloaded to C:\Users\<>\AppData\Local\Electron\pending\<>.exe
And it should install when you close the dev app.
This should give you some certainty but we still ran into issues in production. If you want to be sure, play through the full update flow with a test repo but packaged production apps just as you would do with the live one.

IntelliJ run vs running a jar, with a Springboot Kotlin, Multi module Gradle project with Social Oauth2

TL;DR: Why does everything run fine when started via IntelliJ, and why is it broken when call java -jar app.jar. And how do I fix this?
Alright, I have some issues with a backend I am trying to dockerize. I have an application created with Spring Boot (1.4.2.RELEASE) following the Spring Oauth (2.0.12.RELEASE) guide on their page. I follow the Gradle version, since I prefer Gradle over Maven. Also I am using Kotlin instead of Java. Everything is fine, I start via IntelliJ my backend with static front end, I can login via Facebook (and Google and Github), I receive a nice Principal witch holds al the information I need, and I can modify Spring Security to authorize and permit endpoints. So far so good.
Now for the bad part, when I run either ./gradlew clean build app:bootrun or ./gradlew clean build app:jar and run the jar via java -jar (like I will do in my Docker container), my backend comes up. My static front end pops up. Now I want to login via Facebook, I end up on the Facebook login page, I enter my credentials, and... nothing!
I end up back on my homepage, not logged in, no log messages that mean anything to me, just silence. The last thing I see in the log is:Getting user info from: https://graph.facebook.com/me
This Url will give me in my browser:
{
"error": {
"message": "An active access token must be used to query information about the current user.",
"type": "OAuthException",
"code": 2500,
"fbtrace_id": "GV/58H5f4fJ"
}
}
When going to this URL via an IntelliJ start, it will give me credential details. Obviously something is going wrong, but I have no clue what. Especially since a run from IntelliJ works fine. There is some difference between how the jar is started, and how IntelliJ's run config works, but I have no clue where to search for what. I could post trace logging, or all my Gradle files, but perhaps thats too much info to put in 1 question. I will defenitly update this question if someone needs some more details :)
The structure outline of this project is as follows:
root:
- api: is going to be opensourced later, contains rest definitions and DTOs.
- core: contains the meat. Also here is included in the gradle file
spring-boot-starter, -web, -security, spring-security-oauth2, and some jackson stuff.
- rest: contains versioned rest service implementations.
- app: contains angular webjars amongst others, the front end, and
my `#SpringBootApplication`, `#EnableOAuth2Client`
and the impl of `WebSecurityConfigurerAdapter`.
Why does everything run fine when started via IntelliJ, and why is it broken using bootRun or the jar artefact. And how do I fix this?
I found it, the problem was not Multi module Graldle, Spring boot, or Oauth2 related. In fact it was due to a src set config of Gradle, where Java was supposed to be in a Java src set folder, and Kotlin in a Java src set folder:
sourceSets {
main.java.srcDirs += 'src/main/java'
main.kotlin.srcDirs += 'src/main/kotlin'
}
As Will Humphreys stated in his comment above, IntelliJ takes all source sets, and runs the app. However, when building the jar via Gradle, these source sets are stricter. I had a Java file in my Kotlin src set, which is no problem for IntelliJ. But the jar created by Gradle takes into account the source sets as defined in the build.gralde file, which are stricter.
I found my missing bean issue with the code below:
#Bean
public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
return args -> {
System.out.println("Let's inspect the beans provided by Spring Boot:");
String[] beanNames = ctx.getBeanDefinitionNames();
Arrays.sort(beanNames);
for (String beanName : beanNames) {
System.out.println(beanName);
}
};
}
The Bean I missed was called AuthenticationController, which is a #RestController, and kinda crucial for my authentication code.

How to set up TeamCity to use own HTTP authenticated NuGet service?

a problem is that TeamCity is unable to authorize into NuGet service which runs on same TeamCity instance.
Below you can find how the Build configuration looks now:
I also added Build Feature for NuGet Credentials:
With this configuration I receive following errors in log:
Restoring NuGet package Nuget.Package.Name.0.1.0.41.
Please provide credentials for: http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc
UserName: GET http://www.nuget.org/Packages(Id='Nuget.Package.Name';,Version='0.1.0.41')
NotFound http://www.nuget.org/Packages(Id='Nuget.Package.Name';,Version='0.1.0.41') 194ms
GET http://www.nuget.org/FindPackagesById()?id='Nuget.Package.Name';
NotFound http://www.nuget.org/FindPackagesById()?id='Nuget.Package.Name'; 105ms
WARNING: Unable to find version '0.1.0.41' of package 'Nuget.Package.Name'.
C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache: Package 'Nuget.Package.Name.0.1.0.41' is not found on source 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache'.
C:\Windows\system32\config\systemprofile\.nuget\packages\: Package 'Nuget.Package.Name.0.1.0.41' is not found on source 'C:\Windows\system32\config\systemprofile\.nuget\packages\'.
http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc: Unable to load the service index for source http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc.
Cannot prompt for input in non-interactive mode.
http://nuget.org: The V2 feed at 'http://www.nuget.org/FindPackagesById()?id='Nuget.Package.Name'' returned an unexpected status code '404 Not Found'.
Errors in packages.config projects
Unable to find version '0.1.0.41' of package 'Nuget.Package.Name'.
C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache: Package 'Nuget.Package.Name.0.1.0.41' is not found on source 'C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache'.
C:\Windows\system32\config\systemprofile\.nuget\packages\: Package 'Nuget.Package.Name.0.1.0.41' is not found on source 'C:\Windows\system32\config\systemprofile\.nuget\packages\'.
http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc: Unable to load the service index for source http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc.
Cannot prompt for input in non-interactive mode.
http://nuget.org: The V2 feed at 'http://www.nuget.org/FindPackagesById()?id='Nuget.Package.Name'' returned an unexpected status code '404 Not Found'.
NuGet Config files used:
C:\Windows\system32\config\systemprofile\AppData\Roaming\NuGet\NuGet.Config
Feeds used:
C:\Windows\system32\config\systemprofile\AppData\Local\NuGet\Cache
C:\Windows\system32\config\systemprofile\.nuget\packages\
http://teamcity-nuget-server.cloudapp.azure.com/httpAuth/app/nuget/v1/FeedService.svc
http://nuget.org
I tried several other configurations, i.e using %teamcity.nuget.feed.server% and even %teamcity.nuget.feed.auth.server%. I also tried to remove build feature with NuGet credentials (since documentation says it doesn't need to authenticate with self hosted NuGet service). Tried to remove http://nuget.org source as well. Nothing works.
There are several issues in TeamCity: one with NuGet version 3.3.0 and higher: https://youtrack.jetbrains.com/issue/TW-44438 and the second with local agent: https://youtrack.jetbrains.com/issue/TW-44479. Please check the tickets linked. You can try to use earlier NuGet version or try the workaround.
I think your problem is that you are using NuGet 3 but the package source is pointing to the v2 Feed.
Update your "NuGet.config" (C:\Windows\system32\config\systemprofile\AppData\Roaming\NuGet\NuGet.Config) with the following package sources:
<packageSources>
<add key="NuGet official package source" value="https://api.nuget.org/v3/index.json" />
</packageSources>