Migration with two different connection strings in Entity Framework - asp.net-core

I have one DbContext and two databases, one for debugging / development (local) and one for production (online). I switch them with the appsetting.Development.json and appsettings.Production.json files.
Both just look like this:
{
"ConnectionStrings": {
"Default": "Server={myServer};Persist Security Info=False;User ID={MyUser};Password={myPassword};MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
}
So when I work on the Code and start it, it uses the localDb and when I publish it, it uses the other db.
Now when I change something in my Models I have always to delete the migration files, then do
dotnet ef migrations add InitialModel
and then
dotnet ef database update
after that delete the migrations file again, change the appsettings.Development.json entry to my online db and do the commands again.
So here are my questions:
Is there a way to choose what config should be used in the migration commands?
Like:
dotnet ef migrations add InitialModel --config Production
Is there a way to use the Migration Files with both databases so that I don't have to delete the migration files always?

I am not sure if there is a Migration command that let's you choose config seeing as the connection string is added in the Startup.cs file. However, I can share how i do it and you see if it works for you. Normally, i would have two connection strings in the appsettings.json file. Something like this
"ConnectionStrings": {"DevConnection": "Data source=databasename.db","ProdConnection":"Data source=databasename.db"},
and in the Startup.cs file, i specify the connection string in the dependency injection container. In my Program.cs file, i have this
var host = CreateHostBuilder(args).Build();
using(var scope = host.Services.CreateScope())
{
var services = scope.ServiceProvider;
try
{
var context = services.GetRequiredService<DataContext>();
var userManager = services.GetRequiredService<UserManager<AppUser>>(); //if you have users in your seed data
context.Database.Migrate();
Seed.SeedData(context, userManager).Wait(); //If you have seed data with test users
}
catch(Exception ex)
{
var logger = services.GetRequiredService<ILogger<Program>>();
logger.LogError(ex, "An error occurred during migration");
}
}
host.Run();
This piece of code ensures you don't need to run the Update database command. You just create your migration and start your application.
What you just need to do is after development, you switch the connection string in the Dependency Injection container. You don't need to run the Update Database command when you switch the connection string. You can also use Environment Variables. You can have a look at that and see how it works.

Related

How to get .Net Core 3.1 Azure WebJob to read the AzureWebJobsStorage connection string from the Connected Services setup?

I'm building a WebJob for Azure to run in an App Service using .Net Core 3.1.
The WebJob will be triggered via Timers (it's basically a cronjob).
Timer triggers require the AzureWebJobsStorage connection string as storage is required for Timer events.
When deployed to Azure App Service, I want the WebJob to read the AzureWebJobsStorage value from the properties on the App Service.
I have a Resource Manager template that deploys my infrastructure and sets the connection string on my App Service resource:
"connectionStrings": [
{
"name": "AzureWebJobsStorage",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('_StoreAccountName'), ';AccountKey=', listKeys(resourceId('Microsoft.Storage/storageAccounts', variables('_StoreAccountName')), '2019-04-01').keys[0].value,';EndpointSuffix=core.windows.net')]"
}
],
When testing my WebJob locally, I need to set that AzureWebJobsStorage value so that my local builds can connect to storage.
Since I re-deploy the infrastructure all the time as I make tweaks and changes to it, I do not want to manually maintain the long connection string in my appsettings.json or a local.settings.json file.
In Visual Studio, In theory, I can add a Service Dependency to the project for Azure Storage and that will store the connection string in my local Secrets.json file. Then, when I redeploy the infrastructure I can use the Visual Studio UI to edit the connection and re-connect it to the newly deployed storage account (i.e. it will create and update the connection string without me having to do it manually).
When I add Azure Storage as a connected service, Visual Studio adds a line like this in my Secrets.json file:
"ConnectionStrings:<LABEL>": "DefaultEndpointsProtocol=https;AccountName=<LABEL>;AccountKey=_____________;BlobEndpoint=https://<LABEL>.blob.core.windows.net/;TableEndpoint=https://<LABEL>.table.core.windows.net/;QueueEndpoint=https://<LABEL>.queue.core.windows.net/;FileEndpoint=https://<LABEL>.file.core.windows.net/",
and this in my ServiceDependencies/serviceDependencies.local.json:
"storage1": {
"resourceId": "/subscriptions/[parameters('subscriptionId')]/resourceGroups/[parameters('resourceGroupName')]/providers/Microsoft.Storage/storageAccounts/<LABEL>",
"type": "storage.azure",
"connectionId": "<LABEL>",
"secretStore": "LocalSecretsFile"
}
and this in my ServiceDependencies/serviceDependencies.json:
"storage1": {
"type": "storage",
"connectionId": "<LABEL>"
}
Where <LABEL> is the name of the Storage Account (in both JSON snippits).
When I run the WebJob locally, it loads the appsettings.json, appsettings.Development.json, secrets.json, and Environment Variables into the IConfiguration.
However, when I run the WebJob locally it dies with:
Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException: The listener for function 'Functions.Run' was unable to start.
---> System.ArgumentNullException: Value cannot be null. (Parameter 'connectionString')
at Microsoft.Azure.Storage.CloudStorageAccount.Parse(String connectionString)
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.get_TimerStatusDirectory() in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 77
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusBlobReference(String timerName) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 144
at Microsoft.Azure.WebJobs.Extensions.Timers.StorageScheduleMonitor.GetStatusAsync(String timerName) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Scheduling\StorageScheduleMonitor.cs:line 93
at Microsoft.Azure.WebJobs.Extensions.Timers.Listeners.TimerListener.StartAsync(CancellationToken cancellationToken) in C:\azure-webjobs-sdk-extensions\src\WebJobs.Extensions\Extensions\Timers\Listener\TimerListener.cs:line 99
at Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener.StartAsync(CancellationToken cancellationToken) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Singleton\SingletonListener.cs:line 72
at Microsoft.Azure.WebJobs.Host.Listeners.FunctionListener.StartAsync(CancellationToken cancellationToken, Boolean allowRetry) in C:\projects\azure-webjobs-sdk-rqm4t\src\Microsoft.Azure.WebJobs.Host\Listeners\FunctionListener.cs:line 69
I have confirmed that if I add the ConnectionStrings:AzureWebJobsStorage value to my appsettings.json then the program runs fine.
So I know it's an issue with the loading of the AzureWebJobsStorage value.
Has anyone figured out how to get an Azure WebJob, running locally, to properly read the connection string that Visual Studio configures when adding the Azure Storage as a Connected Service?
What's the point of adding the Connected Service to the WebJob if it won't read the connection string?
(note: I realize that the WebJobs docs https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-how-to#webjobs-sdk-versions state that Because version 3.x uses the default .NET Core configuration APIs, there is no API to change connection string names. but it's unclear to me if that means the underlying WebJobs code also refuses to look in the Connected Services setup or if I'm just missing something)
I found a work-around, but I don't like it... basically check if there's a ConnectionStrings:AzureWebJobsStorage value at the end of my ConfigureAppConfiguration code and if not, try and read the one from the secrets.json file and set the ConnectionStrings:AzureWebJobsStorage to that value.
private const string baseAppSettingsFilename = "appsettings.json";
private const string defaultStorageAccountName = "<LABEL>";
...
IHostBuilder builder = new HostBuilder();
...
builder.ConfigureAppConfiguration(c =>
{
c.AddJsonFile(
path: baseAppSettingsFilename.Replace(".json", $".{Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT")}.json"),
optional: true,
reloadOnChange: true);
if (Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") == "Development")
{
c.AddUserSecrets<Program>();
}
// Add Environment Variables even though they are already added because we want them to take priority over anything set in JSON files
c.AddEnvironmentVariables();
IConfiguration config = c.Build();
if (string.IsNullOrWhiteSpace(config["ConnectionStrings:AzureWebJobsStorage"]))
{
string storageConnectionString = config[$"ConnectionStrings:{defaultStorageAccountName}"];
if (string.IsNullOrWhiteSpace(storageConnectionString))
{
throw new ConfigurationErrorsException($"Could not find a ConnectionString for Azure Storage account in ConnectionStrings:AzureWebJobsStorage or ConnectionStrings:{defaultStorageAccountName}");
}
c.AddInMemoryCollection(new Dictionary<string, string>() {
{ "ConnectionStrings:AzureWebJobsStorage", storageConnectionString }
});
}
});
This seems exceedingly dumb but even looking at the Azure SDK source code I'm thinking it's just hard coded to a single key name and the Service Configuration in Visual Studio is simply not supported: https://github.com/Azure/azure-webjobs-sdk-extensions/blob/afb81d66749eb7bc93ef71c7304abfee8dbed875/src/WebJobs.Extensions/Extensions/Timers/Scheduling/StorageScheduleMonitor.cs#L77
I just ran into a similar problem where VS2019 automatically configured Function and Function1 with Connection = "ConnectionStrings:AzureWebJobsStorage" and it couldn't find that. Simply changing it to Connection = "AzureWebJobsStorage" worked like a charm.
FYI - I also had to change BlobTrigger("Path/{name}"... to BlobTrigger("path/{name}"...
re: Microsoft.Azure.StorageException: The specified resource name contains invalid characters

EF Core Environment Selection

We have recently started writing applications in .net core using EF Core in a code first model. Previously we would change the connection string in the web.config to point to the database that we wanted to update and then run Update-Database -Verbose, which worked well.
Now to select the environment so that the correct connection string is used we have to run $env:ASPNETCORE_ENVIRONMENT = "Development" and then run Update-Database -Verbose. This is where my problem comes in, I can run the command to set the environment to development, but it will still point to my test environment, I can then rerun the exact commands again and it will then point to my development environment.
Is this a known issue, or am I interacting with EF Core incorrectly? I don't want to get to the point that we have this app in production and accidentally updating the production db when attempting to update my dev db.
In an ASP .NET Core web app you could set up ConfigureServices in Startup.cs like so to connect to the appropriate DB depending on the current environment:
public void ConfigureServices(IServiceCollection services)
{
if (_env.IsDevelopment())
{
services.AddDbContext<MyDbContext>(options =>
options.UseThisDb(MyDevDbConnectionString)
);
}
else
{
// For production.
services.AddDbContext<MyDbContext>(options =>
options.UseThisDb(MyProdDbConnectionString)
);
}

Publish .Net Core app with Entity Framework to IIS

I am trying to published my .Net Core app to IIS that uses Entity Framework. I can publish it fine but no database is included in the publish wizard.
I have followed different tutorials including this:
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/iis/?view=aspnetcore-3.0
However none tell you how to deploy your database or what needs to be done if using entity framework. I'm relatively new to published and hosting web apps so I'm not sure what to do.
At the moment my frontend of the web app loads on IIS but when I go to login it brings up a 500 error. I have included my connection string and added a user and gve correct permissions under SSMS.
It came to my attention when publishing it shows "no databases found int he project".
Would this effect me not being able to access the database and bringing up the 500 error when logging in and how do i fix this.
No databases found in the project
How did you created your database locally in the first place? Did you manually ran the database update command? I would suggest you add to your startup.cs file a code to ensure your database is created and any missing migrations were applied, you can achieve this within your Configure method:
using (var scope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>().CreateScope())
{
scope.ServiceProvider.GetRequiredService<DbContext>().Database.Migrate();
}
Or just EnsureCreated() instead of Migrate() if you don't to apply missing migrations on future loads.
It seems that you need to change your ConnectionString configuration in appsettings.json from the format
"Data": {
"DefaultConnection": {
"ConnectionString": "xxxxxx"
}
}
to:
"ConnectionStrings": {
"DefaultConnection": "xxxxxx"
}
And your startup with
services.AddDbContext<ApplicationDbContext>(opt => opt.UseSqlServer(Configuration["ConnectionStrings:DefaultConnection"]));

How to fix: In Azure Devops connection string is null despite setting Variable in the build Pipeline

I have problem with build pipeline in Azure Devops, with variables from build pipeline not replacing empty configuration in appsettings.json. Below there is more details.
My current test project is build using asp.net core technology and is connected to SQL server. I also use Entity Framework Core and autofac.
To connect to SQL server I use appsettings.json configuration:
{
"ConnectionStrings": {
"AzureDbConnectionString": ""
}
}
but my credentials are stored in secrets.json
{
"ConnectionStrings": {
"AzureDbConnectionString": "Server=tcp:servername-db-srv.database.windows.net,1433;Initial Catalog=dbname-db;Persist Security Info=False;User ID=user;Password=Password;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;"
}
}
I've configured my build pipeline variable:
Name:
ConnectionStrings--AzureDbConnectionString
Value:
Server=tcp:servername-db-srv.database.windows.net,1433;Initial Catalog=dbname-db;Persist Security Info=False;User ID=user;Password=Password;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection Timeout=30;
The problem occurs while running Generate Migration Scripts in the build pipeline.
Autofac.Core.DependencyResolutionException: An exception was thrown while activating λ:Microsoft.EntityFrameworkCore.DbContextOptions[] -> λ:Microsoft.EntityFrameworkCore.DbContextOptions -> λ:Microsoft.EntityFrameworkCore.DbContextOptions`1[[AspNetAutofacAzure02.Data.SchoolContext, AspNetAutofacAzure02, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. ---> System.ArgumentException: The string argument 'connectionString' cannot be empty.
at Microsoft.EntityFrameworkCore.Utilities.Check.NotEmpty(String value, String parameterName)
Like I mentioned, it looks like the variable isn't used while generating the script.
Is there anything wrong I do here?
Did you try ConnectionStrings:AzureDbConnectionString? That's the normal format for overriding appsettings.json. Or ConnectionStrings__AzureDbConnectionString.
If the value is coming from keyvault in the format ConnectionStrings--AzureDbConnectionString, then just map a new variable: ConnectionStrings:AzureDbConnectionString = $(ConnectionStrings--AzureDbConnectionString)
You can add Set Json Property task to replace the ConnectionStrings. You may need to install this task to your organization first.
Usage:
First click the 3dots on the right side to locate your appsettings.json file.
Then set the Property path value to ConnectionStrings.AzureDbConnectionString.
Last set the Property value to your pipeline variable $(ConnectionStrings--AzureDbConnectionString)

Set connection string in Azure DevOps pipeline

I have some integration tests that I'd like to run against LocalDB.
My config.json file has the following section...
{
"ConnectionStrings": {
"DefaultConnection": ""
}
}
Is it possible to set this value in the build config?
You can achieve this in several ways. Like Powershell scripts to replace the values and ofcourse this replace token extension
You should define your variable like below
{
"ConnectionStrings": {
"DefaultConnection": "#{connectstring}#"
}
}
During the deployment it will get replaced with your actual values.
Refer this SO for more details
I solved it using the Configuration of Connection Strings in the target App Service for my Azure Web App.
Within the App startup I use this code to access it, and if running local for debugging it uses the config.json or secrets.json.
Configuration.GetConnectionString("My_ConnectionString_Name");
So you can have your local db mentioned in the secrets.json and set the productive database in the Azure App connection string (if applicable).