How to Publish Azure Data Factory Changes using PowerShell? - azure-data-factory-2

How to Publish Azure Data Factory Changes using PowerShell? Using portal, we click on the "Publish" button and save the ADF changes. How to achieve same thing using Azure PowerShell/REST API/ARM template?

You can use the PowerShell module azure.datafactory.tools, created specifically for that goal. It is free and open-source located in the following GitHub repo: https://github.com/SQLPlayer/azure.datafactory.tools
PS module is available via PowerShell Gallery.
It's very well documented, so I'm not gonna copy/paste the documentation here, just worth to mention you need just one command to publish ADF from selected location:
Publish-AdfV2FromJson -RootFolder "$RootFolder" -ResourceGroupName "$ResourceGroupName" -DataFactoryName "$DataFactoryName" -Location "$Location"
Disclaimer:
I'm the author of the module which does not deploy ADF using ARM Template. If you want to understand better what's the differences, read this blog post:
Two methods of deployment Azure Data Factory

We use the exact same method Publish-AdfV2FromJson but any updates we do gets deployed in the development branch but not in other environments through CI.
We are also using -Option $opt where $opt is defined as:
$opt = New-AdfPublishOption
$opt.Excludes.Add("integrationruntime.*", "")
$opt.Excludes.Add("trigger.*", "")
$opt.DeployGlobalParams = $false
$opt.StopStartTriggers = $false
My hunch says the new ARM template is not getting overwritten but not sure how to fix that.

Related

How to Test defined process in TFS which is not default to any teams?

I Have created copy(Scrum Process Backup)of my current process(Scrum Process) being followed in TFS, I have added few fields and rules, before I make copied process(Scrum Process Backup)as default process. I want to test how it is working, Please let me know how to test and where to fine option to test
Thanks in Advance
I think you have to use some test environment:
Install second TFS, upload your process and use any data.
Create a new project collection (Manage project collections in Team Foundation Server).

How to import code from private GitHub repo into snips?

I understand that the normal way to use complex action code in Snips is to place the code in GitHub and pull it from there via Action Type "GitHub".
Is there any way to access a private GitHub repo for this purpose?
If this is not possible, what would be the normal way to use complex action code that cannot be published?
Regarding the "Code Snippets" the documentation says "It is meant for quick testing, or simple interactions." In what respect can Code Snippets support only "simple interactions" ?
You need to make your repo public in order to link it with your console assistant, and code snippets are limited to Python action code. If these are not an option for you you could also make your code run locally by either:
executing it manually (just launch your script, which listens to the MQTT communications via your code or by using Hermes)
placing it in /var/snips/skills and making it executable if you're on Linux. The snips-skill-server will look into this directory at boot and execute any executable file that starts with "action-". See the documentation

Running a JSON graph locally

I've been using Flowhub.io to do my development on the nodejs device. Now that the GUI-based design is done, I'm ready to take it offline and run the code via the command line. How would do I do this? I have the JSON file corresponding to the graph I created online, but not sure how to use the noflo nodejs module.
Could someone help me by showing me an example of how to load a graph using the noflo module, please? Thanks!
f you want to run an existing graph, you can use the --graph option.
noflo-nodejs --graph graphs/MyMainGraph.json
If you also want the process to exit when the network stops, you can pass --batch.
PS: I added this to the noflo-nodejs README.

SP2010 Migration - A feature with ID 14/5c935448-ed11-4bae-bfff-ef8b307f38ac has already been installed in this farm

I am attempting a migration of SP2010 to SP2013, so far what I have accomplished is below:
1) created a backup of the contentDB from SP2010 and restored it on to SP2013
2) Added all the wsp's exported from SP2010 solution store to SP2013 soltuion store.
3) When I try to deploy a solution I get a error message saying "A feature with ID 14/5c935448-ed11-4bae-bfff-ef8b307f38ac has already been installed in this farm. Use the force attribute to explicitly re-install the feature."
most of them suggest to turn on force attribute on the feature and then do the deployment in my case I do not have code for the wsp, so unable to recompile them to turn on the force attribute.
Have used the featureadmin for SP2013 it does not find any faulty feature in the farm, it doesnot list any feature with Id 5c935448-ed11-4bae-bfff-ef8b307f38ac.
Executing select fullurl, description from features join AllWebs on
(features.webid = AllWebs.id) where featureid = '5c935448-ed11-4bae-bfff-ef8b307f38ac' lists rows of data from db but I can't find the feature folder on 14/15 hive.
Stuck at the moment trying to find a way to get the solutions deployed and perform the db upgrade. Any pointers welcome.
Thanks in advance.
You can define -force with power shell command as well.
Salam Santhosh
I face this problem before , and I resolved by adding the AlwaysForceInstall="TRUE" to the feature.xml for the WSP, after that I went to the central admin and I do uninstall and re-install to the WSP, after that you can activate the feature using STSADM or powershell normally
this is an example of the feature

"Execute Code First Migrations" checkbox disappeared from my publish profile

I'm using web deploy to deploy an MVC4 application using EF5 code first. I made a publish profile called "development" that uses web deploy for application and database using the 'Execute Code First Migrations' checkbox to run migrations on application startup. The publishing worked great for a while. At some point I added a new publish profile called "test" to deploy to another server, which uses the ftp method of deploy and no automatic migrations. This works fine too. However, when I tried to use my old "development" publish profile again, VS changes the settings automatically to the dbDacFx way ('Update database' checkbox instead of 'Execute Code First Migrations') and I can't get it back to the way it was.
There were some other changes to the project while using the 'Test' profile, but nothing that seems to me like it could cause this. Does anyone know why VS thinks my project doesn't use code first anymore?
Since asking this question, I've run into this problem several more times in multiple versions of MVC and EF (all code first though of cource). When it happens, don't save the changes to your publish profile, make sure you are on the 'debug' build configuration, clean solution and rebuild solution. And if all that fails, restart Visual Studio and try again.
So far, this has always solved the problem for me.
Here is the right answer - is solved the problem for me.
Just rename your connection string exactly as your DbContext, e.g.
public ApplicationDbContext()
: base("Vocabulary.Domain.ApplicationDbContext", throwIfV1Schema: false)
{
}
Note that name should include namespace like "Vocabulary.Domain.ApplicationDbContext".
Don't forget to update .pubxml files:
<ObjectGroup Name="Vocabulary.Domain.ApplicationDbContext" Order="1" Enabled="True">
This article describes when publish dialog has or has no the tick "Execute Code First Migrations (runs on application start)"
This happened to me in VS2013. Neither cleaning the solution, nor restarting it worked. I tried to "re-enable" migrations and it worked. Even though I got an error message that migrations have already been enabled in project, the "Execute Code First Migrations" checkbox appeared again.
Since nothing mentioned above worked.
I downloaded the publish profile from azure and it worked flawlessly the first time
I had this problem and for me it had a different cause to any I've found documented elsewhere.
My application is spread over many projects in a solution. As I use interfaces and a separate View/Presentation model to wrap calls to the database, my front web project had no direct reference to the project containing the DbContext class. Note that this posed no problem to my running the site locally. I do this so that developers in the front end web project are not tempted to bypass the Presentation layer and go more directly to the database.
Adding this reference immediately fixed the problem and now I see the 'Execute Code First Migrations' checkbox in my Publish Profile dialog.
Just to add to the mix, don't call your Connection Class DbConn didn't seem to like that.