Branching strategy for ADF - azure-data-factory-2

We are facing challenges with ADF deployment with multiple teams working on same ADF for different use cases. When we move from Dev to Test to Prod we find difficult to deploy the code.
Teams working on their on working branches (feature)
This will go to collab branch and publish
When we move to SIT then we take it to another branch 'integration' branch
If team A push their code onto SIT it will be done form integration branch.
After some days team B also move to SIT, need to merge code to integration branch
But when team A moves to Prod later they also get the code from team B as well
How can we resolve this situation? We really do not want other teams code which is testing phase to move to Prod.
Are we missing something in branching strategy?enter image description here

keep your QA release on say Integration branch
keep your prod release on say master branch..
keep 2 hot fix branches for each team
prod bug fixes will then go from dev branch to.prod thru respective hotfix branches
also same bug fixes will go to integration branch

Related

How do I manually remove old release builds from an expired/deleted plan branch in Bamboo?

I use Bamboo regularly as a QA tester to deploy pull requests and feature branches/release branches, but I'm not a developer and have a layman's understanding of how it works.
Our Bamboo configuration is set up to remove inactive branches after a certain amount of time (2 weeks) which happens pretty regularly with longer-term projects, unfortunately. (When that happens, I do know how to configure a new plan and run a new build.) Often, with these larger projects, they've been deployed manually many times over the course of the project, resulting in a large list of possible "release" versions when I go to "Promote existing release to this environment."
Now, I have a brand-new build of a brand-new plan for a project I've been working on off and on for a year and I would like to delete all these old builds (releases?) that show up in the dropdown when I want to just deploy the current version of the current new build, but I can't figure out where to do it (neither can the devs I've asked, but it's NBD to them, whereas this is a constant annoyance for me).
All the advice I can find online says things like "all builds are automatically deleted when the branch expires ...." and that doesn't seem to be true, because these are definitely from old expired plan branches. They also explain how to delete things manually .... from an existing plan branch, which I don't have, because the older plan branches expired and were removed.
Am I using the wrong terminology here and these aren't "builds" and there's a separate way to delete them? Do we have a setup that's failing to delete them when it should? Do devs need to do something different with their branches? I obviously don't have access to global settings but I could put in a request if that's what needs to change.
To be clear, I'm talking about going to deployment preview, selecting "promote existing release to this environment," entering in the jira number/beginning of the branch name, and seeing a million of these (which all look identical because our branch names are hella long):
deployment preview screenshot
I have read through all the Bamboo documentation relating to plans, builds, branches, and deployment, and Googled various combinations of relevant keywords and haven't found a solution. I've also asked devs I work with and they don't know either.

Master Branch for Multi-Version Project (Git Flow)

We're working on an application that is being used by multiple customers, each having a different version of the app.
Our team is currently following the Git Flow branching strategy, but we've been hitting a wall on maintaining a single master branch.
For example:
Customer A has v1.1.2
Customer B has v1.1.4
We have fixed a vulnerability for Customer B in v1.1.4, and updated the version to v1.1.5 in the master branch.
Now Customer B has asked for the patch fix in their version, but does not wish to update to v1.1.4 as of yet.
This fix then gets versioned as v1.1.2.1 (still on its release branch).
We are currently maintaining multiple release branches and pulling in feature branches in as needed, but we feel this isn't scalable in the long-run.
Is there a recommended solution or branching strategy for this?
Maybe you should create release every x.x.x version also fix your bugs on x.x.x.x versions and your customers can use this versions.
So there can be many x.x.x version. You can think this x.x.x versions are kinda major.
Did you tried looking for oneflow approach where you create the new branch from previous release.
Try having a look in this https://www.endoflineblog.com/gitflow-considered-harmful
There are two main updates - new features and security updates. The new features can be maintained per customer as per the version numbers. However, the security updates can be given as a patch that is available from v1.x.x onwards. This would decouple the vulnerability fixes and the feature updates. You could put the patches in a specific branch and someone could merge their given branch with the latest on the security patches branch.
You could send an email everytime there are new security patches.
My suggestion is to move your Softwares components to multiple seperate repositories.
You can then include them as subrepositories in different states in the different branches/repositories (you would have tp create a branch or repo for each customer).

Is test automation on CI/CD really trustable?

I was running a project with test automation on CI/CD and suddenly I got a question.
Is test automation on CI/CD really trustable?
The reason why I get this question is below.
For example, let's assume that I edit some codes on branch A and other codes on branch B and I make PRs.
As I made a test automation, The codes on branch A and branch B would be tested automatically and let's say two tests are passed.
However what if do the two parts of edited codes have no merge conflicts but interact each other and make bugs?
Then how could I trust the test automation and click the merge button?
A good question here, or, at least a situation that I have seen before. Branch A works fine, Branch B works fine but the combination of Branch A and B does not work.
I am going to assume that the automated tests catch the issue. So essentially flow of builds looks like this:
Tip of master passes.
Branch A passes.
Branch B passes.
Branch A is merged and tip of master passes.
Branch B is merged and the tip of master fails.
In this case the automation in CI/CD is trustable. As it accurately reports that the problem was caused by merging B. You can quickly return to a working state by:
Reverting the changes from B (at which point master is now passing).
Then you can get branch B ready to merge again.
Rebasing branch B on the current master.
Fixing the issue.
Testing / Reviewing and Merging branch B.
I am sure you are thinking that if you are the owner of Branch B then this is a lot of work for something you hoped would be caught prior to merge. However, typically in this environment you are trying to protect the state of the master branch. I.E. above all else the master branch is in a working state. The inconvenience to one developer is outweighed by the rest of the team being able to use the working master branch.
In terms of should you trust the automation and click the merge button. I personally avoid clicking the merge button late in the day. I trust that the automation will report issues, however, I don't trust that it is perfect. I prefer to merge in the morning where I have time to react if the build after the merge fails.

TFS Builds, Project Files: Orphaned references to files not being pushed are causing endless build errors

We are using TFS 2010 (Visual Studio) for our deployments and have client code projects (.csproj files) and database projects (.dbproj files) We understand that when our Developers add files to our application there is a corresponding reference to these files in the Project file. If I push a changeset from Dev to QA that includes the project file, and the project file contains a reference to a file that's been added that is not in the changeset, I will receive a build error.
Once we started pushing just changesets (as opposed to performing full builds) this quickly became our number one bottleneck in doing TFS builds. I would deploy the database project and there would be 20 errors. The only way I could correct them was to navigate down the entire solution explorer tree and exclude each orphaned reference individually. This has proved far too time consuming and on the advice of our lead programmer we have returned to doing full builds of QA and UAT.
We are in the early stages of this product, and therefore we will be adding many files for some time. We need a better solution for this problem. Neither the manual exclusions nor asking developers to not check in code until it is ready for qa will suffice for us. Has anybody out there had any experience with this problem and if so how did you deal with it? Thanks!
Jon
Pushing changesets to QA selectively is known as cherry picking and causes the sorts of issues that you are experiencing. This is not the recommended practice, instead setup the Qa build so that successful build is part of checkin. This way that if a part of a fix is missed ( as it may be in multiple change sets ) the build will fail and the checkin cannot be performed.
Second have the developers do the second checkin to QA or merge the dev change sets to Qa and have the team lead coordinate changes to project files by watching for changes by turning on "notify changes made by others " or setting a policy for the dev team. Full builds should always be done as partials do not always pick up the complete pick up the dependency graph.

Mercurial - Delivering Isolated Features to Test and Live

We are going to swap to Mercurial.
A piece in our plan that is missing is how to manage branching/merging of build to Test Box (and LiveBox) so isolated features can be mixed with the StableRelease and built to TestBox.
For instance, it seems the predominant usage is to have
DefaultStableBanch
TestBanch
FeatureABranch
FeatureBBranch
Development on FeatureA and FeatureB, will happen at the same time. It looks like predominant usage is to have cloned repositories with branches for the above.
Scenario 1 : If we build to test, we would merge LiveCode+FeatureA+FeatureB. If all goes well then we can merge the changesets to upstream to DefaultStable branch and build to LiveBox with FeatureA and FeatureB. job done.
Scenario 2 : If we build to test, we would merge LiveCode+FeatureA+FeatureB, and QA shows there is a problem with FeatureB. We do not want to build FeatureB anymore. We do want to progress FeatureA. We want to re-test with FeatureA on it's own and let QA pass that. Then release that to Live and hence business agility.
Questions :
If FeatureB fails QA , we need to take out FeatureB changeset nodes from the Test Branch, build to TesBox again and then hopefully then merge upstream to DefaultStable branch to LiveBox.
What is the best way of removing the FeatureB changeset nodes, from the TestBranch, since 1. we need more dev on FeatureB, and the FeatureB nodeset is not finsihed.
2. We need to isolate DefaultStable+FeatureABranch and build that to test
How are other people managing this ?
There are a lot of great writeups up Mercurial workflows, including:
http://stevelosh.com/blog/2010/02/mercurial-workflows-branch-as-needed/
http://stevelosh.com/blog/2010/05/mercurial-workflows-stable-default/
https://www.mercurial-scm.org/wiki/StandardBranching
All of those use Named Branches very minimally -- definitely not one per feature, which with clones as branches sounds like the work mode you (and I) prefer.
Hitting your specific question, if the combination LiveCode+FeatureA+FeatureB is failing tests the the best way to handle it is to just keep repairing FeatureB and then merging those changes down into FeatureA + Feature B. However, before you get to that stage it's a good idea to have QA hit LiveCode+FeatureA and LiveCode+FeatureB separately too, it's slightly more work for them (more test targets) but having each feature in isolation helps find the cause of the defect more quickly.
Once LiveCode+FeatureA and LiveCode+FeatureB are passing QA, then you merge them into LiveCode+FeatureA+FeatureB and if that still passes tests merge the whole thing into DefaultStable. There should be no need to remove FeatureB from a LiveCode+FeatureA+FeatureB because you can always just create a new clone of LiveCode and merge in only FeatureA if you want it.
Here's a great writeup of a Mercurial (Kiln) based QA/release process:
http://blog.bitquabit.com/2011/03/10/when-things-go-well/
Create FeatureA and FeatureB in feature clone branches from stable. Test is merely a temporary area for QA/Test to work from, so I would treat it as 'throwaway' from day one.
When FeatureA and FeatureB are developed enough, create a clone of either one, and pull the other into QA/Test. Do the build for QA, and when they provide feedback, make appropriate changes to FeatureB.
If FeatureA is acceptable for promotion, pull it into/push it to Stable, and merge into stable.
Is that clearer than my original post?