I have a script running on one of my linux servers which handles batch file processing with a ProcessPoolExecutor and generally runs fine days or even weeks on end without any issue. Sometimes though it looks like a few of my child processes just die (I have no error message or exception at all and can't reproduce it even with killing cp's from the shell) and lead to the parent process just waiting for the return indefinitely...
Thats the call (the initializer doesn't have any effect in this case, it's just to handle the reverse scenario described in another very helpful thread on s.o.)
with ProcessPoolExecutor(max_workers=int(config['PERFORMANCE']['NumberOfProcesses']),
initializer=start_thread_to_terminate_when_parent_process_dies,
initargs=(os.getpid(),)
) as executor:
executor.map(process_main, file_list)
From what I've gathere the Pool should be able to recover in exactly the described scenario:
https://bugs.python.org/issue9205
Anyone got any idea? (thought about switching to the pebble library with it's timeout functionality or creating a separate watchdog script)
I'am planning to have a main OSX application, which the user can launch and a background process, which starts on OSX startup and runs prior to the main application
I need a CoreData database, to keep track of some changes... this database should be the same for the background task and foreground app...
What are the options?
Is it possible, that both access the same sqlite (which will be
located in app bundle?)? By setup with the same .sqlite file?
Should they have two identical databases, which they synchronize?
can this synchronisation be automated?
Should there be one database for the background process and the
main application should communicate with the background process?
Using a background process to update the datastore is fighting the frameworks. Also, your background process (and your main process, for that matter) won't be able to update the .sqlite file that lives in your bundle.
Instead of using a background process, use a background queue (via Grand Central Dispatch, and NSManagedObjectContext -performBlock:. Keeping the logic within one application will make your life easier. Your communication happens within the application, instead of having to use XPC.
Don't forget to handle the case of a partial, interrupted, update. The sequence I suggest is:
Application launches.
Background queue launches, pulls updated info from server, creates updated datastore using a temporary name.
If background update succeeds, main queue of application closes the old version of the datastore, then replaces it with the updated datastore. There is API to do this atomically.
Main thread reopens datastore and refreshes UI as needed.
If background update fails, reschedule an update attempt based on failure reasons (bad credentials, server unreachable, partial download, corrupt .sqlite).
If you're absolutely dead set on using two different processes, then you should still assume that the update might fail. So don't write to the live copy until you know you have a complete, valid, replacement.
Apple achieve this in the Notes app using what they call a "cross process change coordinator". This allows the accountsd daemon and the Notes app to access the same CoreData sqlite database. Their class ICNotesCrossProcessChangeCoordinator in the NotesShared framework works by using NSDistributedNotificationCenter to post the notification of changes between the processes and merge them into each others' context. There are many more implementation details of the technique but this should point you in the right direction.
I was reading about Processes. I wan't to know what really happens. My situation :
"I opened an Application. That creates a process say process1. I have other applications interfaced with this one and all these open up when i click a button inside my running application. I want to know Does my process1 create new processes and IPC happens OR processes for all the linked applications are created at once and then IPC happens?"
Obviously,a running application is a bunch of processes,or maybe a single process which has internally multiple threads acting within these processes.
So,your activity decides the creation and deletion of processes.say,if you are running an application such as media player and you suddenly start searching related info about the album---so here,totally a new process is created which helps interaction through web and after returning the output,it may die,may not,but the process was created on your request.Also,mostly ipc happens within processes,exactly as per your thinking,but shared memory communication is also one of the option,which is complicated and is less common.
One more thing to point out is that there are several 'daemon processes' which are running in the background and don't die before shutdown instruction!So,these processes are also sometimes related to the running application and serves its request.But,mostly,newer processes are created when we switch our task or perform certain action in the application.
I have a VB.NET desktop application that I'm using to monitor events in another windows application running on my system. I need to respond to certain events in a matter of seconds. One of the events I'm monitoring instantly changes the window title of a child window within the main process (I'm not changing it, the application I'm monitoring causes the change in it's own child window title). I have a function that uses windows API's to iterate through the title text of all the process's child windows, and I'm checking for certain values in the titles.
Is it a bad idea to be running this timer/title check once every second? Are there performance issues associated with running a timer in windows every second 24/7? Is it also bad for performance to be calling the API's which retrieve the titles of all the application's child windows? Could I eventually cause that application to crash by sending requests to it so often?
Thanks!
You will have to benchmark it to see, but if I recall correctly, iterating through every window has significant overhead.
Can't you simply monitor a single window? If you do that, you should be fine.
UPDATE 2.4.2010
Yeah, this is an old question but I thought I would give an update. So, I'm working with the ReportViewer again and it's still rendering slowly on the initial load. The only difference is that the SQL database is on the reporting server.
UPDATE 3.16.2009
I have done profiling and it's not the SQL that is making the ReportViewer render slowly on the first call. On the first call, the ReportViewer control locks up the UI thread and makes the program unresponsive. After about 5 seconds the ReportViewer will unlock the UI thread and display "Report is being generated" and then finally show the report. I know 5 seconds is not much but this shouldn't be happening. My coworker does the same thing in a program of his and the ReportViewer immediately displays the "Report is being generated" upon any request.
The only difference is that the reporting server is on one server and the data is on another server. However, when I am developing the reports within SSRS, there is no delay.
UPDATE
I have noticed that only the first load of the ReportViewer takes a long time; each subsequent load of the same or different reports loads fast.
I have a WinForms ReportViewer that I'm using in Remote processing mode that can take up to 30 seconds to render when the ReportViewer.RefreshReport() method is called. However, the report itself runs fast.
This is the code to setup my ReportViewer:
rvReport.ProcessingMode = ProcessingMode.Remote
rvReport.ShowParameterPrompts = False
rvReport.ServerReport.ReportServerUrl = New Uri(_reportServerURL)
rvReport.ServerReport.ReportPath = _reportPath
This is where the ReportViewer can take up to 30 seconds to render:
rvReport.RefreshReport()
I found the answer on other forums. MSDN explains that a DLL is searching for some Verisign web server and it takes forever... there are 2 ways to turn it off, one is a checkbox in internet explorer and another is adding some lines to the app.config file of the app.
You can pull a report in two modes, local and server. If you're running in local mode, it's going to pull both the data and the report definition onto your machine, then render them both. In server mode, it's going to just let SSRS do all the work, then pull back the information to render.
If you're using local mode, it could be a hardware issue. If you've got a huge dataset, that's a lot of data to store in memory.
Other than that, that's not a lot of info to go on...
Update: since you've noticed it's only the first call that takes a while, have you done any profiling to determine if the bulk of the work is done on the backend SQL calls or is spent in the actual report render?
If it's faster on subsequent calls, it's possible you're (incidentally) caching at one level or another. You can cache reports (http://www.sqlservercurry.com/2007/12/configure-report-to-be-cached-ssrs-2005.html) or it could be that the execution plan to return the data is being cached deep in SQL Server.
In summary of the various ideas already presented, it could be
startup time for the report viewer infrastructure on the client
cache loading time on the client
query execution time at the server
report rendering time at the server
Try running the report, closing down the client, restarting the client and running the report again. If the report is much faster the second time, repeat this experiment but load, run and unload another large application in between report runs.
If the second report run continues to be much quicker, then the difference you are seeing has more to do with the SQL Server's I/O cache than what's happening on the client. You can further test this by deliberately displacing the MSSQL cache by running a query that pulls a lot of data from tables that aren't used in the report.
All of the above is interesting but unimportant. If you want to ensure snappy report response Reporting Services provides extensive support for scheduled generation of reports, so that when the consumer requests the report, the only delay is network delivery.
If your users insist on reporting on up to the minute (live) data they'll either have to specify tighter constraint parameters or get used to waiting.
ReportServer always takes a while to wake up because it's running under IIS. There is a process time out on each AppPool. We have the same issue with our ASP.NET application's report viewer. You could try increasing the AppPool keep alive times in the IIS settings.
See here:
http://www.sqlreportingservices.net/Ask/5536.aspx
http://www.developmentnow.com/g/115_2005_9_0_0_597422/First-run-of-reports-is-SLOW.htm
I'm assuming you're running SQL2005 SSRS of course.
One option is to upgrade to 2008 where SSRS no longer depends on IIS.
Thinking way out of the box: Is the report server on different machine to the one running the application? The network could be taking a long time to resolve "reportServerURL". Once resolved the name would be cached and hence subsequent calls would be quicker..
I have had this problem before with badly configured DNS servers. Try replacing "reportServerUrl" with "reportServerIPAddress" and see if the initial call to ReportViewer is any faster.
I was having this same problem.
i find out that changing the default printer(slow network here) fix the problem.
The ReportViewer gets some information from the default printer,
and since the network here is very slow, i was having 10 seconds of delay
Hope it helps
UPDATE
I have noticed that only the first load of the ReportViewer takes a long time; each subsequent load of the same or different reports loads fast.
You are set to run on server which means the SRS server needs to do the rendering as such the first time there will be a delay for one or all of the following reasons (these are the slowest of the bunch, there are others but they are quicker):
DNS resolution: The URL needs to be resolved to an IP address. Once this is done it is cached locally which speeds it up.
ASP.NET/IIS needs time to warm up. There is all kinds of compilation and initial loading that must occur - after loaded it will remain in the servers memory until you restart IIS or the default clean up time occurs.
Reporting Services needs time to warm up in the same way as ASP.NET/IIS does.
To test for this use a network monitor such as Netmon (if you are a Microsoft fan) or Wireshark (my recommendation) and watch the traffic from your machine to the server. You'll see the DNS request, then the HTTP requests go and the delay will be in the returning data. On second call you will see the speed is vastly different in the return and DNS checks.
What you could do to prevent this is a warm up script - I don't know one for SRS but here is a link to a SharePoint one which would not be hard to change since it has the exact same issues.
It seems as though you are going after the SSRS report directly. You may want to hit the SSRS web service instead. That may improve your performance.
Here is a possible resolution for your problem:
Try to access the first report from web before accessing any report with the application.
If the problem doesn't appear, you could make an application that will "preload" the first report, in order to allow reporting services to do their start-up.
I've seen this kind of solution for some demo applications from Microsoft. The applications where using Analysis Services and Reporting Services.
Good luck otherwise
To my knowledge, I think it's a problem Microsoft is finding it tough resolve.
Initially, the report loader is only slow at firt time rendering of report and subsequent reports loads noramal (a bit faster).
To help counter this, place a Startup Form with a label (Label1) and Timer (Timer1) control. Set Label1.Text="Please, wait (about 15 secs)". Set Timer1.Interval=3.
At the form_Load event of the Startup Form, set Timer1.Start.
At Tick event of Timer1 place "frmMyReportForm.reportViewer1.SetDisplayMode(Microsoft.Reporting.WinForms.DisplayMode.Normal)"
"frmMyReportForm" any of the forms in your project containing a reportviewer control.
All the delays will be caught here so that when you generate the actual report, there will be no delays.
I hope this might be helpful to my fellow developers.