I thought this question would have been asked already, but I can't find it, so here goes: I've noticed that numpy.save commands only trigger, i.e. the file to-be-created is actually created, after the entire code has finished running. This is bad when the code takes days or weeks to run, and I want to pin down exactly which function, and what arguments into the function, are causing the bottleneck.
There is a similar issue with the print() command; it doesn't write to the output file immediately but rather waits until the entire code is finished before writing. I can force it to write immediately with this code:
def printnow(*messages):
w=open("output.log","a")
for message in messages:
w.write(str(message))
w.write(" ")
w.write("\n")
w.close()
I was wondering whether it's possible to do an analogous thing, i.e. force an immediate save, for numpy arrays. No need for appending; overwriting with the current value of the numpy array is fine.
If it makes a difference, I'm not running the code on my personal computer but a group server, which I issue commands to and check on using Putty and WinSCP.
Thanks
Edit: I tried another package, shelve, and it encounters the same problem. I create a global variable called function_calls and initialize it to 0. Then, at the start of the function that I suspect is causing the bottleneck, I put in the following code:
global function_calls
file='function_inputs'+str(function_calls)
function_shelf=shelve.open(file,'n')
for key in dir():
function_shelf[key]=locals()[key]
function_calls+=1
This code is intended to create a new file that saves the function inputs, each time the function is called. Unfortunately, 9 hours into starting the run, no files have been created. So I suspect Python is just waiting until the whole run is finished before creating the files I asked it to.
Related
I know the question seems a bit broad. I tried searching for answers, couldn't find much.If anyone could describe or point me to the right source.
Assuming a bytecode-based interpreter, the usual way to do this would be as follows:
You have a variable, the program counter, which tells you the index of the instruction to execute. Usually you increase that counter by 1, but when executing a branch, you instead set it to the target location of the jump.
For function calls you do the same thing, but you also push the old value of the counter plus one onto the call stack. Then when you execute the return instruction, you pop the value of the stack and set the counter to that.
I am currently reading the Programming Erlang Second Edition Writing Software for a concurrent world written by Joe Armstrong and I have the following assignment :
Write a function start(AnAtom, Fun) to register AnAtom as spawn(Fun). Make sure your program works correctly in the case when two parallel processes simultaneously evaluate start/2. In this case you must guarantee that one succeeds and the other fails.
I understand the first bit. I need to register the process of Fun to the AnAtom. However what does the second part want me to do?
If two processes call start/2 at the same time then one of them must fail? Why? Given that the AnAtom is different to any others (which will be done inside the body of start/2 why would I want to fail one of the processes?
From what I can understand so far we have:
a = spawn(process1).
b = spawn(process2).
a ! {self(), registerProcess} //which should call the start/2
b ! {self(), registerProcess} //which should call the start/2
What is the problem here? Two processes will evaluate start/2. Why fail one of them? I'm probably missing the logic here or what I understood so far is completely wrong. Can anybody explain this in easier terms so I can get my head around it?
I believe the exercise is asking you to think about what happens when two parallel process evaluate start/2 using the SAME atom as the first parameter. When start(a, MyFunction) completes, there should be a spawned function (running MyFunction) associated with the name (atom) a.... what happens if
start(cool, MyFun1) and
start(cool, MyFun2)
are both executed simultaneously? How do you guarantee that one succeeds and the other fails.... does this help?
EDIT: I think you are not understanding the register process part of the assignment. When you get done with start(name, MyFun), doing a whereis(name) from the repl should return the process identifier of the process that got created.
This is not about sending the process a message to give it a name, it is about registering the process your created under the name passed in as the first parameter to start/2
I'm new to psychopy and python. I'm trying to program a way to quit a script (that I didn't write), by pressing a key for example. I've added this to the while loop:
while n < total
start=time.clock()
if len(event.getKeys()) > 0:
break
# Another while loop here that ends when time is past a certain duration after 'start'.
And it's not working, it doesn't register any key presses. So I'm guessing key presses are only registered during specific times. What are those times? What is required to register key presses? That loop is extremely fast, sending signals every few milliseconds, so I can't just add wait commands in the loop.
If I could just have a parallel thread checking for a key press that would be good too, but that sounds complicated to learn.
Thanks!
Edits: The code runs as expected otherwise (in particular no errors). "core" and "event" are included. There aren't any other "event" command of any kind that would affect the "key press log".
Changing the rest of the loop content to something that includes core.wait statements makes it work. So for anybody else having this difficulty, my original guess was correct: key presses are not registered during busy times (i.e. in my case a while statement that constantly checks the time), or possibly only during specific busy times... Perhaps someone with more knowledge can clarify.
....So I'm guessing key presses are only registered during specific
times. What are those times? What is required to register key
presses?....
To try and answer your specific question, the psychopy api functions/methods that cause keyboard events to be registered are ( now updated to be literally every psychopy 1.81 API function to do this):
event.waitKeys()[1]
event.clearEvents()[1]
event.getKeys()[2]
event.Mouse.getPressed()
win.flip()
core.wait()
visual.Window.dispatchAllWindowEvents()
1: These functions also remove all existing keyboard events from the event list. This means that any future call to a function like getKeys() will only return a keyboard event if it occurred after the last time one of these functions was called.
2: If keyList=None, does the same as *, else removes keys from the key event list that are within the keyList kwarg.
Note that one of the times keyboard events are 'dispatched' is in the event.getKeys() call itself. By default, this function also removes any existing key events.
So, without being seeing the full source of the inner loop that you mention, it seems highly likely that the event.getKeys() is never returning a key event because key events are being consumed by some other call within the inner loop. So the chance that an event is in the key list when the outer getKeys() is called is very very low.
Update in response to OP's comment on Jonas' test script ( I do not have enough rep to add comments to answers yet):
... Strange that you say this ..[jonas example code].. works
and from Sol's answer it would seem it shouldn't. – zorgkang
Perhaps my answer is giving the wrong understanding, as it is intended to provide information that shows exactly why Jonas' example should, and does, work. Jonas' example code works because the only time key events are being removed from the event buffer is when getKeys() is called, and any events that are removed are also returned by the function, causing the loop to break.
This is not really an answer. Here's an attempt to minimally reproduce the error. If the window closes on keypress, it's a success. It works for me, so I failed to reproduce it. Does it work for you?
from psychopy import event, visual, core
win = visual.Window()
clock = core.Clock()
while True:
clock.reset()
if event.getKeys():
break
while clock.getTime() < 1:
pass
I don't have the time module installed, so I used psychopy.core.Clock() instead but it shouldn't make a difference, unless your time-code ends up in an infinite loop, thus only running event.getKeys() once after a few microseconds.
We can create a real-time monitor for a variable like this:
CreatePalette#Panel#Row[{"x = ", Dynamic[x]}]
(This is more interesting and useful if x happens to be something like $Assumptions. It's so easy to set a value and then forget about it.)
Unfortunately this stops working if the kernel is re-launched (Quit[], then evaluate something). The palette won't show changes in the value of x any more.
Is there a way to do this so it keeps working even across kernel sessions? I find myself restarting the kernel quite often. (If the resulting palette causes the kernel to be automatically started after Quit that's fine.)
Update: As mentioned in the comments, it turns out that the palette ceases working only if we quit by evaluating Quit[]. When using Evaluation -> Quit Kernel -> Local, it will keep working.
Link to same question on MathGroup.
I can only guess, because on my Ubuntu here the situations seems buggy. The trick with the Quit from the menu like Leonid suggested did not work here. Another one is: on a fresh Mathematica session with only one notebook open:
Dynamic[x]
x = 1
Dynamic[x]
x = 2
gives as expected
2
1
2
2
Typing in the next line Quit, evaluating and typing then x=3 updates only the first of the Dynamic[x].
Nevertheless, have you checked the command
Internal`GetTrackedSymbols[]
This gives not only the tracked symbols but additionally some kind of ID where the dynamic content belongs. If you can find out, what exactly these numbers are and investigate in the other functions you find in the Internal context, you may be able to add your palette Dynamic-content manually after restarting the kernel.
I thought I had something like that with
Internal`SetValueTrackExtra
but I'm currently not able to reproduce the behavior.
#halirutan's answer jarred my memory...
Have you ever come across: Experimental/ref/ValueFunction? (documentation address)
Although the documentation contains no examples, the 'more information' section provides the following tidbit:
The assignment ValueFunction[symb] = f specifies that whenever
symb gets a new value val, the expression f[symb,val] should be
evaluated.
i have used following code to repeat a process creation/close iteratively
dim vProcessInfo as new ProcessInfo
For i= 1 to 100
dim p as new Process
vProcessInfo.Arguments = "some"+i.toString()
p.StartInfo = vProcessInfo
p.Start()
p.WaitForExit()
p.Close()
Next i
the above code worked for me successfully. but it takes too much time for process creation and dispose. i had to change process argument dynamically in the iteration. is there any way to change the process argument dynamically. or is there any better method to reduce time. pls help me
"Is there any way to change the process argument dynamically" - do you mean you want to start one process, and change its command line arguments after it's started? No, you can't do that - but you could communicate with it in other ways, for example:
Using standard input/output (e.g. write lines of text to its standard input)
Using files (e.g. you write to a file, it monitors the directory, picks up the file and processes it)
Using named pipes or sockets
Creating a process is a relatively slow operation. You can't easily speed that up - but if you can change your process in some way like the above, and just launch it once, that should make it a lot faster.