Background and problem
I have this open source repository that I have an AppVeyor build configuration for.
This configuration creates an artifact for a website that needs to get published. This is because it only seems to be possible for AppVeyor to do Web Deploy using an artifact, and not a path.
This poses a problem, because my website (before it gets deployed), needs to write some secret values (like API secrets for a Patreon API) down into a file before deploying to production using Web Deploy. But if I do this before creating the artifacts, the secrets will be part of the artifact as well.
The questions
How can I set specific configuration values that my website application can read without exposing it to the viewers of the build configuration and yet still deploy it to production using AppVeyor?
If I could deploy a path instead of an artifact I could mutate the files before deploying, but since an artifact is public to everyone, I don't want to do that. Is this possible?
Alternatively it would be great if I could hide artifacts from others or prevent them from being shown via permissions or something similar, but I haven't found anything that allows me to do that. Is this possible?
What I've tried and more technical details
I have already encrypted the values in my appveyor.yml file:
environment:
patreon_client_id:
secure: PLU/ujLWtFY+Tw/UN6vbHoUSgxeykAIa7dJfLeuHyAyLtnhMqJCARZjN7G6zhO3m9yjr2pClq+VRScJEL+4vSTcJSndZWCqBA5YLFhM6xeE=
patreon_client_secret:
secure: tHr/9QE88kYtxaqdLM332mB3xD+4QRNg8y06DY5qAWf155NtSqi7G4zNpjeFCiTPa86f0LDdPAAjyrWZsLEXoCKZmA7PDBxU5kcllrub2cE=
patreon_creators_access_token:
secure: viBR0QyoO8HxK9X/n93AHhF0SNPs9hG0BEqoQKWV688=
patreon_creators_refresh_token:
secure: qJzAlyrpLkpWxEb7zL17uYnC0HLAwU8M3xcxzI7vkGc=
Here's the part where I create my artifact.
- path: build\website\Website.zip
name: Website
type: WebDeployPackage
As you can see, a Website artifact is created. I then publish this artifact with Web Deploy:
- provider: WebDeploy
server: https://shapeshifter.scm.azurewebsites.net:443/msdeploy.axd?site=shapeshifter
website: shapeshifter
username: $shapeshifter
password:
secure: 5Urzbp6Aj24/wHED9+Q/CtH4EjN7nv9PGdCdBDr5XECq8wnDxQcHK5YoS246hOqcEBNCU2OZ4rq26LVWCRbfbw==
artifact: Website
aspnet_core: true
remove_files: true
app_offline: true
aspnet_core_force_restart: true
Please take a look at Web Deploy Parametrization
Related
I have some tiered ABP.IO application template project deployment questions - but they may be ASP.NET Core deployment questions.
Background
I'm a bit confused as to whether I need to create appsettings.Production.json files to mirror the appsettings.json files in my class library projects (MyProduct.Application, MyProduct.Application.Contracts, etc.) AND my four ASP.NET projects (MyProduct.HttpApi.Host, MyProduct.IdentityServer, MyProduct.Web, and MyProduct.Web.Public) OR whether I just need to create them for ONLY the four ASP.NET projects and make sure that the settings that are in the class library projects are represented in the ones for the ASP.NET projects.
Questions
Should I create appsettings.Production.json files in my class
library/DLL projects?
If yes to 1, will the launchSettings.json file be the right place to
ensure that the libraries are built with the production
configuration?
If yes to 2, are there any considerations when deploying to
production? I know I need to use an environment variable on the
server.
If no to 1 or 2, how do I build my libraries to use the production
configuration?
Is it possible to replace the client secrets wherever they may
appear? It would seem like it would be necessary but there's no help
on this in the documentation. Are there any considerations toward
doing this? Is a simple search and replace of all the default
secrets sufficient or are there code changes necessary?
Is it possible to replace all references to localhost with the FQDN
of the respective site (Host/API, IdentityServer, Web, Web.Public)?
The application template would require this, correct? I am doing an
IIS deployment currently - not a Docker or Kubernetes deployment.
What else am I missing?
Thanks for taking the time to comment. If you have a resource to share with me, please do. I cannot find a deployment guide or checklist on the ABP Framework site, ABP Commercial site, Community Forum, or Discord channel.
UPDATE
I have been through these two resources and I am a lot more educated about configuration in ASP.NET Core but I still cannot find the answer to my question about configuring class libraries in production. 1 - https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-6.0 2 - https://learn.microsoft.com/en-us/aspnet/core/fundamentals/environments?view=aspnetcore-6.0
FINAL UPDATE
Eventually I just had to figure things out but Omer's answers make a lot of sense in hindsight.
My solution was to add the appsettings.Production.json files to each of the deployable projects as suggested below. You can read Omer's answer for details. I pretty much did everything that Omer suggested but I had not thought about the one shot seeding of the Identity Server database tables. That was truly helpful. My final hurdle was figuring out a way to perform DB Migrations on my local DB instance and my remote servers with just a click.
Through various posts, I eventually figured out that I could use the Launch Profile editor buried under the Debug section of the DbMigrator project properties, to create myself two Launch Profiles. I have one for local development and one for production - although through this mechanism, I don't see why you couldn't create one for each part of your staging pipeline.
It should be noted that I deleted the default profile which was named using the project name/namespace.
Here is the Launch Profile editor screen for the Development profile:
And here is the Launch Profile editor screen for the Production profile:
Of primary importance is the ASPNETCORE_ENVIRONMENT=Development environment variable in development and the ASPNETCORE_ENVIRONMENT=Production environment variable in production.
Exiting the editor produces the Properties folder and the contained
launchSettings.json file.
You could create this folder and file yourself without going through the editor. Here is the text of that file:
{
"profiles": {
"EnvironmentConfiguration.Cli (Development)": {
"commandName": "Project",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},
"EnvironmentConfiguration.Cli (Production)": {
"commandName": "Project",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Production"
}
}
}
}
Now when I want to run a schema migration, I can simply select the DbMigrator project as the startup project...
...and I will have two launch profiles in my debug menu:
Does anyone know of a better way?
I am using ABP with Blazor Wasm and IdentityServer is not seperated. So I am publishing only .Host and .Blazor projects.
No, you only need them for published projects (.Host, .Web, .Blazor etc)
Should I create appsettings.Production.json files in my class
library/DLL projects?
Libraries are not standalone projects. They are used by other projects (By .Host project or they can be used by another library project) So, they will take these configuration settings from live application (.Host, .Web, .Blazor etc)
If no to 1 or 2, how do I build my libraries to use the production
configuration?
These keys are using by IdentityServer and I think they are seeded to Database on initial migration. If you want to change them, you need to change them from appsettings.json files and change value in database also. By the way, it is encrypted and you need to change value in DB with new encrypted value. (https://support.abp.io/QA/Questions/441/About-changing-client-secrets)
Is it possible to replace the client secrets wherever they may
appear? It would seem like it would be necessary but there's no help on this in the documentation. Are there any considerations toward doing this? Is a simple search and replace of all the default secrets sufficient or are there code changes necessary?
Change "localhost" values to your FQDN in all appsettings.json files. Also there should be some changes in database for IdentityServer. Because in the initial migration, it is written on DB.
[dbo].[IdentityServerClientCorsOrigins].[Origin]
[dbo].[IdentityServerClientPostLogoutRedirectUris].[PostLogoutRedirectUri] [dbo].[IdentityServerClientRedirectUris].[RedirectUri]
Is it possible to replace all references to localhost with the FQDN
of the respective site (Host/API, IdentityServer, Web, Web.Public)?
The application template would require this, correct? I am doing an
IIS deployment currently - not a Docker or Kubernetes deployment.
Do not forget to install SSL. If you are using Cloudflare disable SSL from Cloudflare (If you have also in server) Because it may conflict.
Another important thing is to remove Webdav if you are using IIS. Because Webdav occurs error for put request. (https://stackoverflow.com/a/59235862/2178028)
Also, I dont know why but for the first publish of Blazor projects, it gives 403 error for .dll files in ISS. Then I follow this link (https://www.eugenechiang.com/2021/12/12/failed-to-find-a-valid-digest-in-the-integrity-attribute-for-resource-in-blazor-app/) and problem is solved.
What else am I missing?
actually configuration of development mode different from production mode so to handle this you must use appsetting.production.json
answer for first question is no, because all projects use ui project settings by dependency injection
I have a sveltekit website that I deployed to cloudflare pages, the problem is that when I deploy the app with the static adapter and try to visit the site it says "No webpage was found for the web address" but when I use the cloudflare adapter it works successfully, so I was intending to use the cloudflare adapter but I noticed that the number of "Functions requests today" was increasing although my app does not have any functions (some how every request is counted as a server function), So what am I doing wrong here?
When you run npm run build does the build directory contain an index.html file? If not you may need to specify prerender.default = true like so:
import adapter from '#sveltejs/adapter-static';
/** #type {import('#sveltejs/kit').Config} */
const config = {
kit: {
adapter: adapter(),
prerender: {
default: true
}
}
};
export default config;
With that you should get a /build directory that contains index.html. Next just follow the instructions from Cloudflare Pages documentation for deploying your site https://developers.cloudflare.com/pages/framework-guides/deploy-anything/
These instructions include the following:
Deploying with Cloudflare Pages
Deploy your site to Pages by logging in to the Cloudflare dashboard > Account Home > Pages and selecting Create a project. Select the new GitHub repository that you created and, in the Set up builds and deployments section, provide the following information:
Configuration option
Value
Production branch
main
Build command (optional)
<YOUR_BUILD_COMMAND>
Build output directory
<YOUR_BUILD_DIR>
Unlike many of the framework guides, the build command and build directory for your site are going to be completely custom. If you do not need a build step, leave the Build command field empty and specify a Build output directory. The build output directory is where your application's content lives.
After configuring your site, you can begin your first deploy. Your custom build command (if provided) will run, and Pages will deploy your static site.
For the complete guide to deploying your first site to Cloudflare Pages, refer to the Get started guide.
After you have deployed your site, you will receive a unique subdomain for your project on *.pages.dev. Cloudflare Pages will automatically rebuild your project and deploy it. You will also get access to preview deployments on new pull requests, so you can preview how changes look to your site before deploying them to production.
From these instructions it looks like you only need to set Production branch to your main branch (or which ever branch you would like deployed) and Build output directory to build (unless otherwise specified in your svelte.config.json). Ensure that your .gitignore does not include the /build directory unless you want to use the Build command config then go ahead and do that.
So, I'm struggling to understand how Gatsby works. I'm using the https://www.gatsbyjs.org/starters/AlexanderProd/gatsby-shopify-starter/ which uses a Gatsby plugin called gatsby-source-shopify. The plugin takes two params: shopName and accessToken. It looks like this in gatsby-config.js:
{
resolve: `gatsby-source-shopify`,
options: {
// The domain name of your Shopify shop. This is required.
shopName: process.env.SHOP_NAME,
// An API access token to your Shopify shop. This is required.
accessToken: process.env.SHOPIFY_ACCESS_TOKEN,
},
},
Will the access token be available for people to look at when I deploy the app? Do I need to use something like Serverless functions to hide my API keys, or is this fine. Any general explanation of how this works in Gatsby would be awesome.
Thanks Gatsby fam!
As the code shows, it uses process.env.SHOP_NAME where SHOP_NAMEis the name of the environment variable. Those files are declared at the root of the project using some naming such as .env.domain1.com. In this file, you can store any desired variable to use it in your Gatsby configurations. When dealing with delicate variables (API keys, tokens, passwords, etc) it's recommended to use that way and ignore all .env files in your .gitignore.
When you trigger a command in Gatsby, you can pass it some variables, for example:
"develop": "GATSBY_ACTIVE_ENV=domain1.com gatsby develop"
In this case, GATSBY_ACTIVE_ENV var will have domain1.com as a value. Then, in you gataby-config.js, when you can use environment variables (above module.exports):
require("dotenv").config({
path: `.env.${process.env.NODE_ENV}`,
})
Then, you can create an environment file such as .env.domain1.com in your root project and store any desired variable:
SHOP_NAME: 12345
Taking into account the code you've provided if you run develop (with all I've explained) command it will take SHOP_NAME as 12345.
So, answering your question, you won't have access to that tokens. You need to store them in your local machine and in your deploy server, not in your repository.
From Gatsby docs:
Please note that you shouldn’t commit .env.* files to your source
control and rather use options given by your Continuous Deployment
(CD) provider...
Edit: Thanks to #Hans Martin Henken for providing the following article about Gatsby security
Trying to deploy spinnaker in Kubernetes using halyard.
All my custom configurations are under
~/.hal/default/service-settings
~/.hal/default/profile
So, running the below command deploys the configuration.
hal deploy apply
This reads my settings under default. Is it possible to have a folder other than default? if so, How can i change the config to use the config under the new folder as opposed to default.
Also, All the pods are using the test & local profiles while starting.
com.netflix.spinnaker.front50.Main : The following profiles are active: test,local
Is this only for test or local deployment? Is there any production profile for production grade spinnaker?
About the "default", this is called "Deployment" - see this: https://www.spinnaker.io/reference/halyard/#deployments
And, on the profile names, I would not worry too much... You add overrides to the "profiles" directory on these...
I'm trying to integrate serverless to my circleci workflow.
I tried first adding both, key and secret to AWS permissions, but that did not work.
Then, I added key and secret to Environment variables and in my config file:
sudo npm install -g serverless
sls config credentials --provider aws --key $AWS_ACCESS_KEY_ID --secret $AWS_SECRET_ACCESS_KEY
sls deploy -v
But I see the same error:
Serverless Error ---------------------------------------
You are not currently logged in. Follow instructions in http://slss.io/run-in-cicd to setup env vars for authentication.
Anyone had this issue? I could not find an answer or hint online. Thanks.
This likely only applies to those trying to use Serverless Enterprise with the monitoring & dashboards they have set up. #wintvelt's answer wouldn't work for me because if i deleted the org variable, it would likely break the connection needed for Enterprise. So steps for my CircleCI setup:
In CircleCI, create a Context for each environment with the AWS Key ID and Secret as environment variables (putting them in a context is a nice to have, you could use other methods of making Circle inject environment variables into builds).
In your Serverless Framework dashboard, create a new access key which you will use in Circle.
Create a new environment variable SERVERLESS_ACCESS_KEY with the value from step 2.
I got this idea from reading how Seed.run has users integrate with Serverless. For more info read this link: https://seed.run/docs/integrating-with-serverless-enterprise.
Just checked Circleci stopped supporting AWS Permissions as a configurable option in the settings page.
You need to set the credentials as environment variables for the projects. The credentials should be named exactly AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
that's all you need to do. you don't have to do any additional step. I tried this on my project and it worked.
Your deployment step should simply be
sls deploy
As a follow-up to the previous answer: I had exactly the same error.
I took the solution from the chat as a solution.
For me the fixes I applied:
In CircleCI project settings, under "AWS permissions" I added the AWS Access Key ID and Secret Access key
In CircleCI project settings, under "Environment variables", I also added the AWS Access Key ID and Secret Access key
From my serverless.yml file, I deleted the line with org variable
For me, 1. and 2. alone was not enough. I also had to remove the line from my yml file to make deployment via CircleCI work.
For those landing here with the same issue, hope this helps!