I have written a task script using vb.net that have thread used in the code, the problem is how i can know when will be finished all the threads so i can return the success result.
Thanks alot.
i think you need to use a waitHandle object and the waitAll method
more info here: http://msdn.microsoft.com/en-us/library/system.threading.waithandle.aspx
That being said, I suspect you can refactor the design of your package to let the script task handle the execution, and let SSIS handle the execution scheduling. this gives you the parallelism you want without any of the hassle of multi threaded programming in .net.
a simple setup would be n foreach loops (which execute in serial) each running a partitioned chunk of the work load.
Another simpler option is have the package driven by variables and spawn multiple executions of the package. This could occur across 1-N servers to scale out.
Related
Greetings fellow software engineers,
As the title of this post states, I was wondering if there's any possibility to run two scripts at once (any kind of multithreading) inside of G1ANT.Studio?
Thanks a lot!
There is no possibility to run two scripts at once on the same computer for now, but you can own multiple G1ANT licenses that you can use on different machines, so that robots can communicate with each other and work in parallel.
By the way, if you want to increase the productivity, you can use triggers to launch scripts automatically at a given time (schedule trigger), after a change in a watched directory took place, for example when a new file appears (file trigger), or after you have received an email (mail trigger).
Hope this answers your question.
You can use macro functionality of G1ANT.Language and create there as many threads or tasks as you want. Downside is that you can only execute there C# code.
I'm looking for a process pool management library, maybe with an interface similar to Python's concurrent.futures.
My goal is to open N processes to execute a task, and when one finishes, create a new one in its stead. So at any single point in time, there are N running processes.
Is there something in existence?
Ended up writing my own class.
I hope it can be of use to someone: https://bitbucket.org/snippets/dorfire/8jdLn
I have a custom MSBuild task for xUnit.net. When the task is running, if I hit Ctrl+C, it 'tries' to cancel the task, but of course it fails (since my task doesn't support cancelation). No amount of MSDN doc searchs or Google-fu have landed on a solution. Since I can't find an obvious interface to implement, I'm guessing maybe cancelation is supported by way of some convention.
Has anybody done this before, and knows what's required to get cancelation to work?
Your task needs to implement ICancelableTask. It's a very simple interface added in 4.0.
Basically you just add a Cancel() method. It must be ready to be called on a different thread, at any time, and return promptly. Your task must then return from Execute promptly. Typically you'd set a boolean flag inside Cancel(). Then inside your task you'd typically have a loop processing each input in turn -- for example, copying one file after another -- and in each iteration, check the flag; if it's true, break out. It doesn't matter whether you return true or false from Execute in this context.
If you're deriving from ToolTask -- if your task spawns a tool, it's very strongly recommended that you do this, as it saves a great deal of code, handles async logging, and other things -- then it already handles Cancel automatically. When Cancel happens, it kills the tool it spawned and all its children. The C++ team's tasks in some cases override this default behavior, so that their compiler/linker has a few seconds to clean up their half-written outputs before returning.
(Trivia: when I first implemented this in MSBuild, I accidentally made VS bluescreen the box occasionally. This nearly shipped in VS10 beta but was discovered just in time. The bluescreen was because the logic for figuring out the process tree was wrong, and would sometimes kill a system process. Oops.)
Dan
I know you're well aware of the Task hierarchy, but on the offchance this is what you're looking for and it's just the fact that you're not implementing a ToolTask...
Inside MSBuild 2nd ed says (p118) of ToolTask.Cancel
This method is called to cancel the task execution. Once this method is called by MSBuild, if the task does not complete, it will be forcefully terminated
There are no other references to cancellation in it.
I'm working with .NET new TPL library and faced with some strange behavior for me I cannot explain. For some reason nested task is not started in my case. I've simplified the solution I have to the following:
bool flag = false;
for (int i = 0; i < 5; i++)
{
Task.Factory.StartNew(() =>
{
while (true) // a lot of newcoming tasks
{
Thread.Sleep(200); //do some work
Task.Factory.StartNew(() =>
{
flag = true;
});
}
});
}
Thread.Sleep(2000);
Assert.IsTrue(flag);
I have 5 Tasks that running concurrently. Each tasks retrieve some elements from pending queue, performs some operation and then try to run nested task for the results of this operation. The problem is that if there are too many elements (while(true) simulates this) and all 5 tasks are constantly running nested tasks are not started. The can only be started after most of tasks with while loop finish their execution.
It seems something wrong with while statements that blocks nested tasks to run, but I don't know what :)
Task.Factory.StartNew doesn't start a task, it adds the task to the list of tasks to be scheduled and the scheduler gets to decide when to run the task based on things like; number of available cores (size of thread pool), current CPU load and the throughput of existing work.
You should read the section about task scheduling here:
http://parallelpatterns.codeplex.com/releases/view/48562
Page 63 of the PDF onwards.
The LongRunning option "fixes" your problem by bypassing the thread pool entirely. This has some disadvantages in that it will allow you to create more threads than your system should really be using, this will degrade the performance by causing excessive context switching.
Experiments like the code you have above using thread sleep are misleading because they "fool" the scheduler. It sees that it added more work and yet the CPU load hasn't increased.63 You should replace the sleep with a tight loop which contains math (like calculating Sqrt() for example.
Why not simply have a single outer loop which reads items from a queue and executes them on Task. That way your application will make the most use of available parallelism of the system without overloading it.
The following answer might be worth a look:
Parallel Task Library WaitAny Design
I think you'll find that the library only starts parallel tasks roughly based on the number of cores you have available. i.e. it's not a good choice for tasks which are I/O bound where you might actually want to use significantly more threads than you have CPUs.
You're not really saying that the nested tasks don't start, are you? You're just saying that they don't get started at the point you'd like them to, but start later.
Whether this is possible I don't know, but it would mighty useful!
I have a process that fails periodically (running in Windows 2000). I then have just one chance to react to it before having to restart it and painfully wait for it to fail again. I didn't write the process so don't have the source to debug. The failure is seemingly random.
With a snapshot of the process I could repeatedly and quickly test reactions to the failure.
I had thought of running inside a VM but this isn't possible in this instance.
EDIT:
#Jon Cage asked:
When you say a snapshot, you mean capturing a process when it's about to fail (including memory, program state etc. etc.) ...and then replaying it's final few seconds repeatedly to see what effect it has on some other component?
This is exactly what I mean!
I think minidump is what you are looking for.
You can also used Userdump:
The User Mode Process Dumper
(userdump) dumps any running Win32
processes memory image (including
system processes such as csrss.exe,
winlogon.exe, services.exe, etc) on
the fly, without attaching a debugger,
or terminating target processes.
Generated dump file can be analyzed or
debugged by using the standard
debugging tools.
This article shows you how to use it.
My best bet is to start the process in a debugger (OllyDbg being my preferred tool).
The process will pause on an exception, and you can try to figure out what happened shortly before that.
This needs some understanding of assembler and does not allow to create a snapshot of the process for later analysis. You would need to write your own debugger for that - it should be theoretically possible.