I'm building a WebJob for Azure to run in an App Service using .Net Core 3.1.
The WebJob will be triggered via Timers (it's basically a cronjob).
Timer triggers require the AzureWebJobsStorage connection string as storage is required for Timer events.
When deployed to Azure App Service, I want the WebJob to read the AzureWebJobsStorage value from the properties on the App Service.
I have a Resource Manager template that deploys my infrastructure and sets the connection string on my App Service resource:
"connectionStrings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('_StoreAccountName'), ';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('_StoreAccountName')), '2019-04-01').keys[0].value,';EndpointSuffix=core.windows.net')]"
}
],
When testing my WebJob locally, I need to set that AzureWebJobsStorage value so that my local builds can connect to storage.
Since I re-deploy the infrastructure all the time as I make tweaks and changes to it, I do not want to manually maintain the long connection string in my appsettings.json or a local.settings.json file.
In Visual Studio, In theory, I can add a Service Dependency to the project for Azure Storage and that will store the connection string in my local Secrets.json file. Then, when I redeploy the infrastructure I can use the Visual Studio UI to edit the connection and re-connect it to the newly deployed storage account (i.e. it will create and update the connection string without me having to do it manually).
When I add Azure Storage as a connected service, Visual Studio adds a line like this in my Secrets.json file:
"ConnectionStrings:<LABEL>": "DefaultEndpointsProtocol=https;AccountName=<LABEL>;AccountKey=_____________;BlobEndpoint=https://<LABEL>.blob.core.windows.net/;TableEndpoint=https://<LABEL>.table.core.windows.net/;QueueEndpoint=https://<LABEL>.queue.core.windows.net/;FileEndpoint=https://<LABEL>.file.core.windows.net/",
and this in my ServiceDependencies/serviceDependencies.local.json:
"storage1": {
"resourceId": "/subscriptions/[parameters('subscriptionId')]/resourceGroups/[parameters('resourceGroupName')]/providers/Microsoft.Storage/storageAccounts/<LABEL>",
"type": "storage.azure",
"connectionId": "<LABEL>",
"secretStore": "LocalSecretsFile"
}
and this in my ServiceDependencies/serviceDependencies.json:
"storage1": {
"type": "storage",
"connectionId": "<LABEL>"
}
Where <LABEL> is the name of the Storage Account (in both JSON snippits).
When I run the WebJob locally, it loads the appsettings.json, appsettings.Development.json, secrets.json, and Environment Variables into the IConfiguration.
However, when I run the WebJob locally it dies with:
Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException: The listener for function 'Functions.Run' was unable to start.
---> System.ArgumentNullException: Value cannot be null. (Parameter 'connectionString')
at Microsoft.Azure.Storage.CloudStorageAccount.Parse(String connectionString)
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.get_TimerStatusDirectory() in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 77
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusBlobReference(String timerName) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 144
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusAsync(String timerName) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 93
at Microsoft.Azure.WebJobs.Extensions.Timers.Listeners.TimerListener.StartAsync(CancellationToken cancellationToken) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Listener\TimerListener.cs:line 99
at Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener.StartAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Singleton\SingletonListener.cs:line 72
at Microsoft.Azure.WebJobs.Host.Listeners.FunctionListener.StartAsync(CancellationToken cancellationToken, Boolean allowRetry) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\FunctionListener.cs:line 69
I have confirmed that if I add the ConnectionStrings:AzureWebJobsStorage value to my appsettings.json then the program runs fine.
So I know it's an issue with the loading of the AzureWebJobsStorage value.
Has anyone figured out how to get an Azure WebJob, running locally, to properly read the connection string that Visual Studio configures when adding the Azure Storage as a Connected Service?
What's the point of adding the Connected Service to the WebJob if it won't read the connection string?
(note: I realize that the WebJobs docs https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-how-to#webjobs-sdk-versions state that Because version 3.x uses the default .NET Core configuration APIs, there is no API to change connection string names. but it's unclear to me if that means the underlying WebJobs code also refuses to look in the Connected Services setup or if I'm just missing something)
I found a work-around, but I don't like it... basically check if there's a ConnectionStrings:AzureWebJobsStorage value at the end of my ConfigureAppConfiguration code and if not, try and read the one from the secrets.json file and set the ConnectionStrings:AzureWebJobsStorage to that value.
private const string baseAppSettingsFilename = "appsettings.json";
private const string defaultStorageAccountName = "<LABEL>";
...
IHostBuilder builder = new HostBuilder();
...
builder.ConfigureAppConfiguration(c =>
{
c.AddJsonFile(
path: baseAppSettingsFilename.Replace(".json", $".{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json"),
optional: true,
reloadOnChange: true);
if (Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") == "Development")
{
c.AddUserSecrets<Program>();
}
// Add Environment Variables even though they are already added because we want them to take priority over anything set in JSON files
c.AddEnvironmentVariables();
IConfiguration config = c.Build();
if (string.IsNullOrWhiteSpace(config["ConnectionStrings:AzureWebJobsStorage"]))
{
string storageConnectionString = config[$"ConnectionStrings:{defaultStorageAccountName}"];
if (string.IsNullOrWhiteSpace(storageConnectionString))
{
throw new ConfigurationErrorsException($"Could not find a ConnectionString for Azure Storage account in ConnectionStrings:AzureWebJobsStorage or ConnectionStrings:{defaultStorageAccountName}");
}
c.AddInMemoryCollection(new Dictionary<string, string>() {
{ "ConnectionStrings:AzureWebJobsStorage", storageConnectionString }
});
}
});
This seems exceedingly dumb but even looking at the Azure SDK source code I'm thinking it's just hard coded to a single key name and the Service Configuration in Visual Studio is simply not supported: https://github.com/Azure/azure-webjobs-sdk-extensions/blob/afb81d66749eb7bc93ef71c7304abfee8dbed875/src/WebJobs.Extensions/Extensions/Timers/Scheduling/StorageScheduleMonitor.cs#L77
I just ran into a similar problem where VS2019 automatically configured Function and Function1 with Connection = "ConnectionStrings:AzureWebJobsStorage" and it couldn't find that. Simply changing it to Connection = "AzureWebJobsStorage" worked like a charm.
FYI - I also had to change BlobTrigger("Path/{name}"... to BlobTrigger("path/{name}"...
re: Microsoft.Azure.StorageException: The specified resource name contains invalid characters
Related
I'm playing around in .NET Core 6 Web API's. But I can't seem to figure out how the connection string works now.
The first part here that is commented out works fine. But I need to be able to throw the program on different systems and change the connection string with appsettings.json.
The second part is what I attempted but that doesn't work.
Config connection string in .net core 6 is where I got it from.
//builder.Services.AddDbContext<TodoContext>(opt =>
// opt.UseSqlServer(#"Data Source=JOHANDRE\\SQL2017; Database=ToDoItems; User=xxx; Password=xxx;"));
builder.Services.AddDbContext<TodoContext>(opt =>
opt.UseSqlServer(builder.Configuration.GetConnectionString("ToDoItemsDatabase")));
My appsettings.json:
"ConnectionStrings": {
"ToDoItemsDatabase": "Server=JOHANDRE\\SQL2017; Database=ToDoItems; User=xxx; Password=xxx;"
},
I want to add that it does not throw errors. it just does not seem to find the connection.
Problem is how you start your Web API from the service. You are using Process without setting ProcessStartInfo.WorkingDirectory to the folder containing exe and configuration and the started process shares the working directory with parent one, so either move appsettings.json to the parent project folder or set the WorkingDirectory to match the directory containing the exe:
toDoTest.StartInfo.UseShellExecute = false;
toDoTest.StartInfo.WorkingDirectory = "C:\\Develop\\ToDoMVCtutorial\\bin\\Release\\net6.0\\publish\\";
Also you can try redirecting your Web API output to capture the logs.
I am trying to published my .Net Core app to IIS that uses Entity Framework. I can publish it fine but no database is included in the publish wizard.
I have followed different tutorials including this:
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/iis/?view=aspnetcore-3.0
However none tell you how to deploy your database or what needs to be done if using entity framework. I'm relatively new to published and hosting web apps so I'm not sure what to do.
At the moment my frontend of the web app loads on IIS but when I go to login it brings up a 500 error. I have included my connection string and added a user and gve correct permissions under SSMS.
It came to my attention when publishing it shows "no databases found int he project".
Would this effect me not being able to access the database and bringing up the 500 error when logging in and how do i fix this.
No databases found in the project
How did you created your database locally in the first place? Did you manually ran the database update command? I would suggest you add to your startup.cs file a code to ensure your database is created and any missing migrations were applied, you can achieve this within your Configure method:
using (var scope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
scope.ServiceProvider.GetRequiredService<DbContext>().Database.Migrate();
}
Or just EnsureCreated() instead of Migrate() if you don't to apply missing migrations on future loads.
It seems that you need to change your ConnectionString configuration in appsettings.json from the format
"Data": {
"DefaultConnection": {
"ConnectionString": "xxxxxx"
}
}
to:
"ConnectionStrings": {
"DefaultConnection": "xxxxxx"
}
And your startup with
services.AddDbContext<ApplicationDbContext>(opt => opt.UseSqlServer(Configuration["ConnectionStrings:DefaultConnection"]));
I have problem with build pipeline in Azure Devops, with variables from build pipeline not replacing empty configuration in appsettings.json. Below there is more details.
My current test project is build using asp.net core technology and is connected to SQL server. I also use Entity Framework Core and autofac.
To connect to SQL server I use appsettings.json configuration:
{
"ConnectionStrings": {
"AzureDbConnectionString": ""
}
}
but my credentials are stored in secrets.json
{
"ConnectionStrings": {
"AzureDbConnectionString": "Server=tcp:servername-db-srv.database.windows.net,1433;Initial Catalog=dbname-db;Persist Security Info=False;User ID=user;Password=Password;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
}
}
I've configured my build pipeline variable:
Name:
ConnectionStrings--AzureDbConnectionString
Value:
Server=tcp:servername-db-srv.database.windows.net,1433;Initial Catalog=dbname-db;Persist Security Info=False;User ID=user;Password=Password;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
The problem occurs while running Generate Migration Scripts in the build pipeline.
Autofac.Core.DependencyResolutionException: An exception was thrown while activating λ:Microsoft.EntityFrameworkCore.DbContextOptions[] -> λ:Microsoft.EntityFrameworkCore.DbContextOptions -> λ:Microsoft.EntityFrameworkCore.DbContextOptions`1[[AspNetAutofacAzure02.Data.SchoolContext, AspNetAutofacAzure02, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. ---> System.ArgumentException: The string argument 'connectionString' cannot be empty.
at Microsoft.EntityFrameworkCore.Utilities.Check.NotEmpty(String value, String parameterName)
Like I mentioned, it looks like the variable isn't used while generating the script.
Is there anything wrong I do here?
Did you try ConnectionStrings:AzureDbConnectionString? That's the normal format for overriding appsettings.json. Or ConnectionStrings__AzureDbConnectionString.
If the value is coming from keyvault in the format ConnectionStrings--AzureDbConnectionString, then just map a new variable: ConnectionStrings:AzureDbConnectionString = $(ConnectionStrings--AzureDbConnectionString)
You can add Set Json Property task to replace the ConnectionStrings. You may need to install this task to your organization first.
Usage:
First click the 3dots on the right side to locate your appsettings.json file.
Then set the Property path value to ConnectionStrings.AzureDbConnectionString.
Last set the Property value to your pipeline variable $(ConnectionStrings--AzureDbConnectionString)
I have some integration tests that I'd like to run against LocalDB.
My config.json file has the following section...
{
"ConnectionStrings": {
"DefaultConnection": ""
}
}
Is it possible to set this value in the build config?
You can achieve this in several ways. Like Powershell scripts to replace the values and ofcourse this replace token extension
You should define your variable like below
{
"ConnectionStrings": {
"DefaultConnection": "#{connectstring}#"
}
}
During the deployment it will get replaced with your actual values.
Refer this SO for more details
I solved it using the Configuration of Connection Strings in the target App Service for my Azure Web App.
Within the App startup I use this code to access it, and if running local for debugging it uses the config.json or secrets.json.
Configuration.GetConnectionString("My_ConnectionString_Name");
So you can have your local db mentioned in the secrets.json and set the productive database in the Azure App connection string (if applicable).
I have a .NetCore stateless WebAPI service running inside Service Fabric local cluster.
return Endpoint.Start(endpointConfiguration).GetAwaiter().GetResult();
When I'm trying to start NServiceBus endpoint, I'm getting this exception :
Access to the path 'C:\SfDevCluster\Data_App_Node_0\AppType_App10\App.APIPkg.Code.1.0.0.diagnostics' is denied.
How can it be solved ? VS is running under administrator.
The issue you are having is because the folder you are trying to write to is not supposed to be written by your application.
The package folder is used to store you application binaries and can be recreated dynamically whenever an application is hosted in the node.
Also, the binaries are reused by multiple service instances running on same node, so it might compete to use the files by different instances.
You should instead instruct your application to write to the WorkFolder,
public Stateless1(StatelessServiceContext context): base(context)
{
string workdir = context.CodePackageActivationContext.WorkDirectory;
}
The code above will give you a path like this:
'C:\SfDevCluster\Data_App_Node_0\AppType_App10\App.APIPkg.Code.1.0.0.diagnostics\work'
This folder is dynamic, will change depending on the node or instance of your application is running, when created, your application should already have permission to write to it.
For more info, see:
how-do-i-get-files-into-the-work-directory-of-a-stateless-service?forum=AzureServiceFabric
Open folder properties Security tab
Select ServiceFabricAllowedUsers
Add Write permission