I am new to Erlang trying to understand what's the relation of a module and a process in Erlang.
Dose each module spun up a new process in BEAM ?
When we call a function from another module is there message passing been done between two process ?
I tried to search go through Erlang docs, while working on a existing Erlang code
Functions execute inside processes, whether they are defined in a module or not.
Dose each module spun up a new process in BEAM ?
Nope. You can define 50 modules and the functions therein can all execute in a single process. Modules are more like namespaces.
When we call a function from another module is there message passing
been done between two process ?
There aren't two processes unless you start two processes.
Message passing between processes is done by calling send/2 or using the operator !. Messages are read inside a process using a receive clause.
Related
In SSIS package i have multiple scripts running within a job. At any given time i want to read how many scripts have been executed eg, 5/10 (50%) have been completed. Please tell how can i achieve that?
Currently there is no such functionality provided by SSIS to track progress of package execution.
It seems you need to write your own custom utility/application to implement same or use third party one.
There are few ways to do -
Using a switch called /Reporting or /Rep of DTEXEC at the command-line . For example:
DTEXEC /F ssisexample.dtsx /Rep P > progress.txt
2.Implement package logging or customize it.
3 . Implement Event handler on required executable. You can also use OnPipelineRowsSent log of Data Flow Task.
If you want to write your own application then below thread will provide nice starting point.
How do you code the Package Execution Progress window in C#
In my experience, we are using another software to monitor the jobs that are running. http://en.wikipedia.org/wiki/CA_Workload_Automation_AE
You can also try to create your own application that runs on the background that checks that status of your jobs, through checking the logs.
I am trying to detect / get notified whenever a new process is created in MAC. The easiest way is to poll all the processes and see if a new process has been launched but that is too time consuming and i wanted to know if i could somehow get some notification whenever a new process is launched using "forked" and "execve". Here is what i have already found :
On how a new process is launched in MAC :
OS X is a variety of Unix. New processes are created with the fork() system call. This creates an almost identical copy of the process that makes the call (the difference is that fork returns 0 in the child and the pid of the child in the parent). It's then normal to use one of the exec() syscalls in the child to transform the child into a process running a different executable.
How is new application launched on Mac?
On getting the list of all processes through polling
http://www.cocoabuilder.com/archive/cocoa/92971-bsd-processes-with-code.html
I have also gone through kAuth kext thing, but it seems beyond my level unless i have some example code for made simple so that i can understand on how to generate the kext and use it in a sample app.
https://developer.apple.com/library/mac/technotes/tn2127/_index.html
NSWorkspace has a notifier but that is only true for applications and not for all processes.
Any tutorial/ sample code with some basic understanding on how to go about this problem, will be greatly appreciated.
Is there any way to call mprirun inside FORTRAN program? I'm working on public linux cluster via ssh and the main idea is to automatically enqueue program after its execution is over.
I tried to write something like this at the end of the program:
CALL system('mpirun -np 16 -maxtime 100 TestNP')
But recieved this error:
sh: mpirun: command not found
Any ideas ?
The problem is the missing path prefix, so specifying an absolute path for mpirun should help. However there are several problems with your approach:
If every MPI process executes it, you would have too many instances running, so only one of the nodes (e.g. the master node) should execute it.
The original program won't be finished, until the one called via the system() call did not finish. So, if your queue is wall-clock limited, you don't gain anything at all.
Typically, tasks like this are done via shell-scripts. E.g. in Bash you would write something like:
while true; do
mpirun your_program
done
This would re-invoke mpirun continuously until not killed by you or the queuing system. (So be careful with it!)
How can a task be reused in SSIS without copy/paste?
For example, I'd like to use the tasks I've defined in an event handler for one executable in another executable, but not with all executables in the package. So far, I haven't found any solutions other than writing a complete custom component, which seems like overkill. Any suggestions?
Have you considered using an event at the package level, and filtering to only fire when your particular condition requires it?
E.g. you could use the OnPostExecute event just by putting a dummy task in your flow with a name that starts with a specific string like "RunMyTasks", and then check the System::SourceName to see if it starts with "RunMyTasks". If it does, then branch to run your tasks (and otherwise branch to handle the event as you normally would).
You could do a similar thing using OnVariableValueChanged - this might be better (although you'd need to test it). Create a variable with RaiseChangedEvent=TRUE. Create a script task / component to change the value of the variable; finally, put your task set into the event handler.
Check the scoping notes at the bottom of Jamie's post here.
If you can use third-party solutions, check the commercial CozyRoc SSIS+ library. It includes enhanced Script Task Plus, which allows export of script to external file and then link and reuse in other packages.
Our SSIS pacakges a structured as one Control package and many child packages (about 30) that are invoked from the control package. The child packages are invoked with Execute Package Task. There is one Execute Package Task per child package. Each Execute Package Task uses File Connection Manager to specify path to the child package dtsx file. There is one File Connection Manager per child package. Each File Connection Manager has an expression defined for ConnectionString property. This expression looks like this:
#[Template::FolderPackages]+"MyPackage.dtsx"
The file name is different for each package. The variable (FolderPackages) is specified in the SSIS package configuration file.
The error that is generated during run time is
Error 0x80070002 while loading package file "MyPackage.dtsx"
The system cannot find the file specified." The package that fails is different from run to run and sometimes no packages fail at all. This is when run on exactly the same environment/data etc.
I ran FileMon during this error and found out that when the error happens SSIS tries to read the dtsx file from a wrong place, namely from system32. I checked that this is identical to what would happen if #[Template::FolderPackages] variable were empty, but because the very same variable is used for every child package and works for some but doesn't work sometimes for others, I have no expalnation to this fact.
Anything obvious, or time to raise a support call with Microsoft?
Are you using Expressions on the SSIS variables directly? Variables with Expressions are calculated each time the variable is referenced by the consuming object which needs to use it. That is where the race condition bug exists, because sometimes the expression doesn't get evaluated if another thread is already evaluating a different variable, and the default value for the variable is provided to the consumer object.
If that matches your design, these two bugs on the connect site discuss the problem, and the workarounds:
https://connect.microsoft.com/SQLServer/feedback/details/332372/ssis-variable-expressions-dont-always-evaluate
A second one at
connect.microsoft.com/SQLServer/feedback/details/406534/ssis-2008-variable-expressions-dont-always-evaluate
A summary of workarounds is
{
- Note the parallel tasks that could run in you SSIS control flow and utilize these expression variables. If you have two tasks side-by-side if each relies upon the same variable, and that variable has an Expression to set its value, then you could hit this.
Manually sequentialize such tasks, so that they don't run in parallel. Ie. Add a green arrow on the control flow, so that the tasks occur in order Task1, Task2, Task3, rather than side-by-side on parallel paths and rather than inside the same container with no paths.
You could avoid variable expressions: Assigning local variables in the required order using a home-made script task that does the same kind of work, so that variables are not evaluated using expressions (ie. the thing which can hit this race condition). In other words, manually assign the variable values at a point in time in your control flow just before they are used. The point of using expressions on variables is to dynamically set a value based on another value whenever it is used, so this acheives a similar design goal but in a manual way.
Reduce threads to minimize potential: Setting the Dataflow task EngineThreads to 1 and MaxConcurrentExecutables to 1. This will help sequentalize execution of your package to one task at a time, but that has the side effect which may cause slower performance.
Create and set values on distinct copies of variables at different scope levels in the design, so that they evaluate in different parallel execution scopes and avoid the expression evaluation on parallel threads. Master::Var1, Child1::Var1, Child2::Var1
}
A bit of a stab in the dark but...
I've had a similar issue with variables where readonly=false and multiple components were reading the variable at the same time and causing locking issues.
I consistently recreated the problem by running a pair of dataflows that did nothing but reference the variable inside a for loop container and changed the variable to be read only and this resolved the problem.
If you temporarily hardcode the package name does this resolve the issue?
Turns out after sending trace info to Microsoft that we are encountering heap corruption. I'll update this question if we get to the bottom of it.
The current suggestion is to disable heap lookaside for dtexec.exe.
The official answer to this issue is that it is a bug in SQL 2005 and 2008. Many tasks accessing the same variable cause a race condition, and some tasks get the default value for the expression instead of the evaluated value.
The workaround is to ensure that the default value (the value defined in the property sheet for whatever property you are having trouble with) should be the value that will work in your production environment.
This way, when the race condition happens in prod, SSIS will fall back to the package value, which will still work.
In dev? Well you're just going to have to deal with that manually until we get a bug fix from Microsoft.
There is a KB article relating to this issue: http://support.microsoft.com/kb/2448991 which states when and where this was fixed.