I have a GtkTreeView object that uses a GtkListStore model that is constantly being updated as follows:
Get new transaction
Feed data into numpy array
Convert numbers to formatted strings, store in pandas dataframe
Add updated token info to GtkListStore via GtkListStore.set(titer, liststore_cols, liststore_data), where liststore_data is the updated info, liststore_cols is the name of the columns (both are lists).
Here's the function that updates the ListStore:
# update ListStore
titer = ls_full.get_iter(row)
liststore_data = []
[liststore_data.append(df.at[row, col])
for col in my_vars['ls_full'][3:]]
# check for NaN value, add a (space) placeholder is necessary
for i in range(3, len(liststore_data)):
if liststore_data[i] != liststore_data[i]:
liststore_data[i] = " "
liststore_cols = []
[liststore_cols.append(my_vars['ls_full'].index(col) + 1)
for col in my_vars['ls_full'][3:]]
ls_full.set(titer, liststore_cols, liststore_data)
Class that gets the messages from the websocket:
class MyWebsocketClient(cbpro.WebsocketClient):
# class exceptions to WebsocketClient
def on_open(self):
# sets up ticker Symbol, subscriptions for socket feed
self.url = "wss://ws-feed.pro.coinbase.com/"
self.channels = ['ticker']
self.products = list(cbp_symbols.keys())
def on_message(self, msg):
# gets latest message from socket, sends off to be processed
if "best_ask" and "time" in msg:
# checks to see if token price has changed before updating
update_needed = parse_data(msg)
if update_needed:
update_ListStore(msg)
else:
print(f'Bad message: {msg}')
When the program first starts, the updates are consistent. Each time a new transaction comes in, the screen reflects it, updating the proper token. However, after a random amount of time - seen it anywhere from 5 minutes to over an hour - the screen will stop updating, unless I change the focus of the window (either activate or inactive). This does not last long, though (only enough to update the screen once). No other errors are being reported, memory usage is not spiking (constant at 140 MB).
How can I troubleshoot this? I'm not even sure where to begin. The data back-ends seem to be OK (data is never corrupted nor lags behind).
As you've said in the comments that it is running in a separate thread then i'd suggest wrapping your "update liststore" function with GLib.idle_add.
from gi.repository import GLib
GLib.idle_add(update_liststore)
I've had similar issues in the past and this fixed things. Sometimes updating liststore is fine, sometimes it will randomly spew errors.
Basically only one thread should update the GUI at a time. So by wrapping in GLib.idle_add() you make sure your background thread does not intefer with the main thread updating the GUI.
Related
I'm currently working on an application that run in the background and sometime create an input dialog for the user to answer. If the user doesn't interact, I'd like to close the dialog after 30 seconds. I made a QThread that act like a timer and the "finished" signal should close the dialog. I unfortunately cannot find a way to close it.
At this point I'm pretty much lost. I completely new to QThread and a beginner in PyQt5
Here is a simplified version of the code (we are inside a class running a UI):
def Myfunction(self,q):
# q : [q1,q2,q3]
self.popup = counter_thread()
self.popup.start()
self.dial = QInputDialog
self.popup.finished.connect(self.dial.close)
text, ok = self.dial.getText(self, 'Time to compute !', '%s %s %s = ?'%(q[0], q[2], q[1]))
#[...]
I tried ".close()" and others but i got this error message:
TypeError: close(self): first argument of unbound method must have type 'QWidget'
I did it in a separated function but got the same problem...
You cannot close it because the self.dial you created is just an alias (another reference) to a class, not an instance.
Also, getText() is a static function that internally creates the dialog instance, and you have no access to it.
While it is possible to get that dialog through some tricks (installing an event filter on the QApplication), there's no point in complicating things: instead of using the static function, create a full instance of QInputDialog.
def Myfunction(self,q):
# q : [q1,q2,q3]
self.popup = counter_thread()
self.dial = QInputDialog(self) # <- this is an instance!
self.dial.setInputMode(QInputDialog.TextInput)
self.dial.setWindowTitle('Time to compute !')
self.dial.setLabelText('%s %s %s = ?'%(q[0], q[2], q[1]))
self.popup.finished.connect(self.dial.reject)
self.popup.start()
if self.dial.exec():
text = self.dial.textValue()
Note that I started the thread just before showing the dialog, in the rare case it may return immediately, and also because, for the same reason, the signal should be connected before starting it.
I would like to know how to disable the button while another method is running by that button, and then re enable it when that method is finished. I tried with the sample code, but that logic does not work, it still clickable and only becomes disabled when the loop finishes in another method. If I click it multiple times during the while loop, then it restart it again and again.
from PyQt5.QtWidgets import QApplication, QMainWindow, QPushButton
import sys
class MainWindow(QMainWindow):
def __init__(self):
super().__init__()
self.setWindowTitle("Demo")
self.button = QPushButton("Press me!")
self.button.clicked.connect(self.the_button_was_clicked)
self.setCentralWidget(self.button)
def the_button_was_clicked(self):
self.button.setText("You already pressed it.")
self.runner()
self.setWindowTitle("Changed title")
def runner(self):
self.button.setEnabled(False)
i = 0
while i < 5000000:
i = i+1
print(i, end=" ")
self.button.setEnabled(True)
app = QApplication(sys.argv)
window = MainWindow()
window.show()
app.exec()
You have 2 problems.
Almost any UI based toolkit uses an event loop, meaning that it has a while-like loop that waits for any event in order to react; as long as you run a blocking function in the same thread of that event loop, it means that that loop is not able to process its events, which will be queued instead and lately processed as soon as control returns to it; among these events there are paint events (those responsible of painting the UI) and input events (those coming from the user interaction); your while loop blocks both of them, meaning that the button will not be displayed as disabled, and all input events will be queued and processed only after the function returns;
the Python buffer (used by print()) is somehow limited, which can block furthermore anything else;
Since Python doesn't allow direct concurrence, you only have two choices: periodically ensure that the event loop process its events (see QCoreApplication.processEvents()), or use multiprocessing capabilities (which can be very tricky if you need UI interaction).
For this simple case, the suggested solution is the former one (including skipping the buffer at all until the string is completed):
def runner(self):
self.button.setEnabled(False)
QApplication.processEvents()
i = 0
print('started')
while i < 5000000:
i = i+1
if not i % 100000:
QApplication.processEvents()
print('finished')
self.button.setEnabled(True)
Remember that processEvents() is possible and tolerated, but you should not use it without awareness: it should only be used when you really know what you are doing and any other alternative is not a viable option. If the processing does not directly deal with CPU-bound aspects (eg. IO management, like file system operation or network retrieval), you should consider threading instead, and always use QThreads and custom signals whenever any result of the processing requires interaction with UI elements, since widgets are NOT thread-safe.
I am attempting to use clairvoyant's db-cleanup dag to clear metadata in our xcom table, but when I run it, I receive the following warning, printed thousands of times before I manually stop the job in order to not take down our mysql instance:
SAWarning: Loading context for <BaseXCom at 0x7f26f789b370> has changed within a load/refresh handler, suggesting a row refresh operation took place. If this event handler is expected to be emitting row refresh operations within an existing load or refresh operation, set restore_load_context=True when establishing the listener to ensure the context remains unchanged when the event handler completes.
The other cleanup tasks work fine, but it is the xcom table in particular I am having trouble with. We have hundreds/thousands of active dags and so the xcom table is constantly being written to nearly every second or two. I think that is what is causing this error, the fact that the data is continually changing while it is being queried.
I have been unable to find the cause of this or any examples of how this can be resolved. I tried adding a "restore_load_context":True line as per SQLAlchemy docs but it did not work.
Here are the snippets I attempted to add to the database object and the cleanup task:
{
"airflow_db_model": XCom,
"age_check_column": XCom.execution_date,
"keep_last": False,
"keep_last_filters": None,
"keep_last_group_by": None,
"restore_load_context":True
},
....
def cleanup_function(**context):
logging.info("Retrieving max_execution_date from XCom")
max_date = context["ti"].xcom_pull(
task_ids=print_configuration.task_id, key="max_date"
)
max_date = dateutil.parser.parse(max_date) # stored as iso8601 str in xcom
airflow_db_model = context["params"].get("airflow_db_model")
state = context["params"].get("state")
age_check_column = context["params"].get("age_check_column")
keep_last = context["params"].get("keep_last")
keep_last_filters = context["params"].get("keep_last_filters")
keep_last_group_by = context["params"].get("keep_last_group_by")
restore_load_context = context["params"].get("restore_load_context")
In order to not paste too much code here, I am using the same code in the db-cleanup dag. Has anyone encountered this and found a way to resolve?
I am very inexperienced with sqlalchemy and am entirely unsure where else to place this code or how to go about it.
Currently, I am working with a EV3 lego robot that is controlled by several neurons. Now I want to modify the code (running on
python3) in such a way that one can change certain parameter values on the run via the shell (Ubuntu) in order to manipulate the robot's dynamics at any time (and for multiple times). Here is a schema of what I have achieved so far based on a short example code:
from multiprocessing import Process
from multiprocessing import SimpleQueue
import ev3dev.ev3 as ev3
class Neuron:
(definitions of class variables and update functions)
def check_input(queue):
while (True):
try:
new_para = str(input("Type 'parameter=value': "))
float(new_para[2:0]) # checking for float in input
var = new_para[0:2]
if (var == "k="): # change parameter k
queue.put(new_para)
elif (var == "g="): # change parameter g
queue.put(new_para)
else:
print("Error". Type 'k=...' or 'g=...')
queue.put(0) # put anything in queue
except (ValueError, EOFError):
print("New value is not a number. Try again!")
(some neuron-specific initializations)
queue = SimpleQueue()
check = Process(target=check_input, args=(queue,))
check.start()
while (True):
if (not queue.empty()):
cmd = queue.get()
var = cmd[0]
val = float(cmd[2:])
if (var == "k"):
Neuron.K = val
elif (var == "g"):
Neuron.g = val
(updating procedure for neurons, writing data to file)
Since I am new to multiprocessing there are certainly some mistakes concerning taking care of locking, efficiency and so on but the robot moves and input fields occur in the shell. However, the current problem is that it's actually impossible to make an input:
> python3 controller_multiprocess.py
> Type 'parameter=value': New value is not a number. Try again!
> Type 'parameter=value': New value is not a number. Try again!
> Type 'parameter=value': New value is not a number. Try again!
> ... (and so on)
I know that this behaviour is caused by putting the exception of EOFError due to the fact that this error occurs when the exception is removed (and the process crashes). Hence, the program just rushes through the try-loop here and assumes that no input (-> empty string) was made over and over again. Why does this happen? - when not called as a threaded procedure the program patiently waits for an input as expected. And how can one fix or bypass this issue so that changing parameters gets possible as wanted?
Thanks in advance!
I've created an experiment in psychopy builder in which participants must vocally name pictures presented onscreen (for example, if a picture of a chair appears, the participant has to respond by saying "chair"). I've set up a code component to detect each vocal response, which ends the trial and initiates the next one. This part of the experiment works well, however I'm having trouble integrating EEG recording.
Some important information:
My trial loop reads images and triggerVal's out of a .csv file. I have an image component (called english_naming) that displays images for participants to name out-loud. The component's STOP field is defined as $vpvk.event_onset - this forces the trial to end and the next one to begin upon detection of a vocal response.
So, here is my (working) code component at present:
Begin Experiment:
from psychopy import parallel
port = parallel.port(address=61432)
Begin Routine
vpvk = vk.onsetVoiceKey(
sec=10) # creates the voice key
vpvk.start() #starts recording.
port.setData(triggerVal) # tells psychopy to read trigger values from the .csv file
End Routine
vpvk.stop() # ends the recording
port.setData(0) # resets the trigger value to 0 for the start of the next trial
My problem is this
At present, parallel port events are time-locked to the start of each trial, but I need them to be time-locked to participant's vocal responses. I tried inserting if vpvk.event_onset(): above port.setData(triggerVal), but this fails to generate any trigger codes at all. I've also tried if english_naming==FINISHED but the same problem occurred. I've tried a bunch of variants on these two lines of code, but nothing I can think of seems to work.
I would really really appreciate any advice on this problem. Thanks in advance!