I want to create a parameter that contains a list of string (list of hub codes). This list of string is created by reading an external csv file (this list could contain the different codes depending on the hub codes in the CSV file)
What I want is to find a easy auto way to perform batch runs by each hub code in the list.
So this question is:
1) how to add and set a new parameter directly from the code (during the initialization when reading the CSV) instead of GUI parameter panel?
2) how to avoid manual configuration of hub list in the batch run configuration
Something like this for adding the parameters should work in your ContextBuilder.
Parameters params = RunEnvironment.getInstance().getParameters();
((DefaultParameters)params).addParameter("foo", "Big Foo", Integer.class, 3, false);
You would read the csv file to get the parameter name and value.
I'm not sure I completely understand the batch run configuration question, but each batch run has a run number associated with it
RunState.getInstance().getRunInfo().getRunNumber()
If you can associate line numbers in your csv parameter file with run number (e.g. run number 1 should use line 1, and so on), then each batch run would use a different parameter line.
I have tried using the below however what I am trying to achieve is being able to pass this through PowerShell and it asking me for a value for -companyCode.
"\Program Files (x86)\Home\Business Intelligence\ScriptoSql\"ScriptoSqldb1.exe -companyCode="" -sqlserver=SQLSERVER1\INS1,8071 -sqldatabase=StagingDB
Now the command works if I pass a value with it in the string but I want to make it dynamic so when it is called it will ask for that value and it is then entered by the user and the script runs.
Wrap the command invocation in a custom function with a mandatory parameter -CompanyCode. That way PowerShell will prompt for a missing parameter when the function is called.
function Invoke-ScriptToSqlDB {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[string]$CompanyCode
)
& "${env:ProgramFiles(x86)}\...\ScriptoSqldb1.exe" -companyCode=$CompanyCode ...
}
Invoke-ScriptToSqlDB -CompanyCode 'whatever' # pass company code
Invoke-ScriptToSqlDB # prompt for company code
I have alot of Bamboo variables defined due the fact that i have a system with alot of legacy and config at places where it does not belong. Getting rid of all this will take a bit longer on the roadmap so i need to find a way to auto replace all these values.
The number im talking about is that there are 8 customer config files with each about 100 variables. Indeed, there was a maniac who added all of those in Bamboo because as you might thought most of them are variable for each environment.
At this moment i want to automate the deployment process and all is going fine exact the fact that i need to replace 100 variables and i dont want to maintain it in my script itself all the time.
I am looking for a way to retrieve all the variables in an array so i can just iterate through all the keys and try to replace them at the config files.
echo "${bamboo.application.myvalue}" will replace the value as expected. The only problem is, how can i get all the keys under bamboo.*
I tried it with the following functions but all without success:
printenv
env
declare
All above without success. How can i retrieve a list of all those variables as inline script in Bamboo.
Thanks alot
I think it is not possible to change the value of the variables on the fly. Instead, you can use the "Inject Bamboo variables" task in order to be able to change the variable value.
This task reads a file to create the variables. So, all you have to do is to create this file with the values you need, and then use this variables.
E.g.: Creating a file from a powershell script:
$path = 'bambooVariaveis.properties'
$connectionstringX = 'connectionstring="Data Source=XXXX;"'
$Utf8NoBomEncoding = New-Object System.Text.UTF8Encoding($False)
[System.IO.File]::WriteAllLines($path, $connectionstringX, $Utf8NoBomEncoding)
E.g: Inject Bamboo Variables config
Using it (in a subsequent script task):
echo ${bamboo.inject.connectionstring}
I am using jdbc connection and i am passing parameters with example ${sample_db_connection} and that parameters has been defined in server in a text file as sample_db_connection=localhost and i want to pass the text file in the job step so that whenever the job ran and it found this parameter ,automatically it will take the value defined in text file.
You need to create a KTR file using "Property Input" as the input step and "Modified Java Script" Step to define the key value mapping. Check the image below:
Define your filename in the input step. In the JS step, you can use "setVariable" function to define the key-value mapping.
Once this job is executed at the start, pentaho will set the variables for all the connection.
Hope i have understood the question correctly and this is what you are looking for !! :)
One question for you is here ;)
I have this function:
function Set-DbFile {
param(
[Parameter(ValueFromPipeline=$true)]
[System.IO.FileInfo[]]
$InputObject,
[Parameter(ValueFromPipelineByPropertyName=$true)]
[scriptblock]
$Properties
)
process {
$InputObject | % {
Write-Host `nInside. Storing $_.Name
$props = & $Properties
Write-Host ' properties for the file are: ' -nonew
write-Host ($props.GetEnumerator()| %{"{0}-{1}" -f $_.key,$_.Value})
}
}
}
Look at the $Properties. It should be evaluated for each file and then the file and the properties should be processed further.
Example how to use it might be:
Get-ChildItem c:\windows |
? { !$_.PsIsContainer } |
Set-DbFile -prop {
Write-Host Creating properties for $_.FullName
#{Name=$_.Name } # any other properties based on the file
}
When I copy & paste function Set-dbFile to command line and run the example snippet, everything is fine.
However, when I store the function in a module, import it and run the example, the $_ variable is empty. Does anybody know why? And how to solve it? (other solutions are welcome as well)
Results for function defined in a script/typed in commandline:
Inside. Storing adsvw.ini
Creating properties for C:\windows\adsvw.ini
properties for the file are: Name-adsvw.ini
Inside. Storing ARJ.PIF
Creating properties for C:\windows\ARJ.PIF
properties for the file are: Name-ARJ.PIF
....
Results for function defined in module:
Inside. Storing adsvw.ini
Creating properties for
properties for the file are: Name-
Inside. Storing ARJ.PIF
Creating properties for
properties for the file are: Name-
....
The problem here is down to scope hierarchy. If you define two functions like...
function F1{
$test="Hello"
F2
}
function F2{
$test
}
Then F2 will inherit the variable scope of F1 since it's called from F1's scope. If you define function F2 in a module and export the function the $test variable is not available since the module has it's own scope tree. See the Powershell Language Specification (Section 3.5.6):
In your case the current node variable is defined in the local scope and hence it will not survive into the module scope since it's in a different tree with a different scope root (apart from global variables).
To quote the text on the GetNewClosure() method in the Powershell Language Specification (Section 4.3.7):
Retrieves a script block that is bound
to a module.Any local variables that
are in the context of the caller will
be copied into the module.
...hence GetNewClosure() works a treat since it bridges the local scope/module divide. I hope this helps.
Looks like GetNewClosure() is as good a work around as any, but it changes the way the script block sees those variables. Passing $_ to the scriptblock as an argument works, too.
It has nothing to do with normal scope issues (e.g., global vs local), but it appears like that at first. Here's my very simplified reproduction and some explanation following:
script.ps1 for normal dot-sourcing:
function test-script([scriptblock]$myscript){
$message = "inside"
&{write-host "`$message from $message"}
&$myscript
}
Module\MyTest\MyTest.psm1 for importing:
function test-module([scriptblock]$myscript){
$message = "inside"
&{write-host "`$message from $message"}
&$myscript
}
function test-module-with-closure([scriptblock]$myscript){
$message = "inside"
&{write-host "`$message from $message"}
&$myscript.getnewclosure()
}
Calls and output:
» . .\script.ps1
» import-module mytest
» $message = "outside"
» $block = {write-host "`$message from $message (inside?)"}
» test-script $block
$message from inside
$message from inside (inside?)
» test-module $block
$message from inside
$message from outside (inside?)
» test-module-with-closure $block
$message from inside
$message from inside (inside?)
So I started hunting around since this piqued my curiosity, and I found a few interesting things.
This Q&A, which also features a link to this bug report is pretty much the exact same topic, as are some other blog articles I ran across. But while it was reported as a bug, I disagree.
The about_Scopes page has this to say (w:
...
Restricting Without Scope
A few Windows PowerShell concepts are similar to scope or interact with
scope. These concepts may be confused with scope or the behavior of scope.
Sessions, modules, and nested prompts are self-contained environments,
but they are not child scopes of the global scope in the session.
...
Modules:
...
The privacy of a module behaves like a scope, but adding a module
to a session does not change the scope. And, the module does not have
its own scope, although the scripts in the module, like all Windows
PowerShell scripts, do have their own scope.
Now I understand the behavior, but it was the above and a few more experiments that led me to it:
If we change $message in the scriptblock to $local:message then all 3 tests have a blank space, because $message is not defined in the scriptblock's local scope.
If we use $global:message, all 3 tests print outside.
If we use $script:message, the first 2 tests print outside and the last prints inside.
Then I also read this in about_Scopes:
Numbered Scopes:
You can refer to scopes by name or by a number that
describes the relative position of one scope to another.
Scope 0 represents the current, or local, scope. Scope 1
indicates the immediate parent scope. Scope 2 indicates the
parent of the parent scope, and so on. Numbered scopes
are useful if you have created many recursive
scopes.
If we use $((get-variable -name message -scope 1).value) in order to attempt getting the value from the immediate parent scope, what happens? We still get outside rather than inside.
At this point it was clear enough to me that sessions and modules have their own declaration scope or context of sorts, at least for script blocks. The script blocks act like anonymous functions in the environment in which they're declared until you call GetNewClosure() on them, at which point they internalize copies of the variables they reference of the same name in the scope where GetNewClosure() was called (using locals first, up to globals). A quick demonstration:
$message = 'first message'
$sb = {write-host $message}
&$sb
#output: first message
$message = 'second message'
&$sb
#output: second message
$sb = $sb.getnewclosure()
$message = 'third message'
&$sb
#output: second message
I hope this helps.
Addendum: Regarding design.
JasonMArcher's comment made me think about a design issue with the scriptblock being passed into the module. In the code of your question, even if you use the GetNewClosure() workaround, you have to know the name of the variable(s) where the scriptblock will be executed in order for it to work.
On the other hand, if you used parameters to the scriptblock and passed $_ to it as an argument, the scriptblock does not need to know the variable name, it only needs to know that an argument of a particular type will be passed. So your module would use $props = & $Properties $_ instead of $props = & $Properties.GetNewClosure(), and your scriptblock would look more like this:
{ (param [System.IO.FileInfo]$fileinfo)
Write-Host Creating properties for $fileinfo.FullName
#{Name=$fileinfo.Name } # any other properties based on the file
}
See CosmosKey's answer for further clarification.
I believe you need to call getnewclosure() on that script block before you run it. Called from a script file or module, script blocks are evaluated at compile time. When you work from the console, there is no "compile time". It's evaluated at run time, so it behaves differenly there than when it's in the module.