In one place of my code, I use django_redis to update cache with a lock:
from django.core.cache import cache
with cache.lock('hello'):
# do stuff
In another place, I check whether cache is not locked using:
if not cache.get('hello'):
# do other stuff
However, when lock is set, get call fails with UnpicklingError: invalid load key, 'f'. Why is this happening? What am I doing wrong?
You can reproduce this behaviour with this snippet:
from django.core.cache import cache
with cache.lock('hello'):
cache.get('hello')
it's not very obviously, but as i understand your lock_key and cache_key must not be the same. for example this code:
cache_key = 'hello'
with cache.lock(cache_key):
cache.get(cache_key)
raise UnpicklingError: invalid load key....
In the same time this code:
cache_key = 'hello'
lock_key = cache_key + '_lock'
with cache.lock(lock_key):
cache.get(cache_key)
works as you want.
Related
from Bio import Entrez, __version__
print('Biopython version : ', __version__)
id_list = ["NC_045512.2", "ON248099.1", "ON248101.1", "ON248104.1", "ON248107.1", "ON248108.1", "ON248109.1", "ON248110.1", "ON248114.1", "ON247234.1", "ON247236.1", "ON247240.1", "ON247242.1", "ON247243.1", "ON247244.1", "ON247245.1", "ON247246.1", "ON247247.1", "ON247248.1"]
try:
print ("Downloading "+str(id_list[ax]))
net_handle = Entrez.efetch(db="nucleotide", id=id_list[ax], rettype="fasta", retmode="text")
net_handle_gb = Entrez.efetch(db="nucleotide", id=id_list[ax], rettype="gbwithparts", retmode="text")
except (IOError):
print ("Waiting")
sleep (30)
tell.app( 'Terminal', 'do script "' + command+ ' && ' + "python3 'All Covid SSR.py'" +'"')
I'm using this try block to download files from NCBI API. Up to this point, the try block was working and the exception was handled as given here. Now it isn't. The code gets stuck in the try section. Returns no errors or exceptions, so it doesn't trigger the exception handler.
I've set max_tries = 1, sleep_between_tries = 1 and got the personal API_Key too.
For context, I'm using the exceptions block to restart the downloading code on Mac Terminal.
I need the try and except section to automatically restart the script when Entrez.efetch returns an error. There isn't any error. The code just gets stuck after
print ("Downloading "+str(id_list[ax]))
Any suggestions are welcome.
P.S. I thought of changing the default timeout for requests.get because that's the module Entrez uses to access the NCBI API. Haven't found a way to do that. (There is no timeout variable that you can set in the efetch module.)
An explanation would be great!
I am failing to use readNetFromDarknet function for the reasons I do not understand. I am trying to use yolo3-spp with this configuration file https://github.com/pjreddie/darknet/blob/master/cfg/yolov3-spp.cfg.
import cv2
net = cv2.dnn.readNetFromDarknet("../models/yolov3-spp.weights", "../models/yolov3-spp.cfg")
However, this gives me the following error:
error: OpenCV(4.5.4) /tmp/pip-req-build-3129w7z7/opencv/modules/dnn/src/darknet/darknet_io.cpp:660: error: (-215:Assertion failed) separator_index < line.size() in function 'ReadDarknetFromCfgStream
Interestingly, if I use readNet instead of readNetFromDarknet, it seems to work just fine. Should I stick to using readNet instead and why readNetFromDarknet is not actually working?
Your problem in order function arguments. readNetFromDarknet waits first config file and second - weights. A readNet function can swap arguments if they have a wrong order.
So right code:
net = cv2.dnn.readNetFromDarknet("../models/yolov3-spp.cfg", "../models/yolov3-spp.weights")
I am attempting to use clairvoyant's db-cleanup dag to clear metadata in our xcom table, but when I run it, I receive the following warning, printed thousands of times before I manually stop the job in order to not take down our mysql instance:
SAWarning: Loading context for <BaseXCom at 0x7f26f789b370> has changed within a load/refresh handler, suggesting a row refresh operation took place. If this event handler is expected to be emitting row refresh operations within an existing load or refresh operation, set restore_load_context=True when establishing the listener to ensure the context remains unchanged when the event handler completes.
The other cleanup tasks work fine, but it is the xcom table in particular I am having trouble with. We have hundreds/thousands of active dags and so the xcom table is constantly being written to nearly every second or two. I think that is what is causing this error, the fact that the data is continually changing while it is being queried.
I have been unable to find the cause of this or any examples of how this can be resolved. I tried adding a "restore_load_context":True line as per SQLAlchemy docs but it did not work.
Here are the snippets I attempted to add to the database object and the cleanup task:
{
"airflow_db_model": XCom,
"age_check_column": XCom.execution_date,
"keep_last": False,
"keep_last_filters": None,
"keep_last_group_by": None,
"restore_load_context":True
},
....
def cleanup_function(**context):
logging.info("Retrieving max_execution_date from XCom")
max_date = context["ti"].xcom_pull(
task_ids=print_configuration.task_id, key="max_date"
)
max_date = dateutil.parser.parse(max_date) # stored as iso8601 str in xcom
airflow_db_model = context["params"].get("airflow_db_model")
state = context["params"].get("state")
age_check_column = context["params"].get("age_check_column")
keep_last = context["params"].get("keep_last")
keep_last_filters = context["params"].get("keep_last_filters")
keep_last_group_by = context["params"].get("keep_last_group_by")
restore_load_context = context["params"].get("restore_load_context")
In order to not paste too much code here, I am using the same code in the db-cleanup dag. Has anyone encountered this and found a way to resolve?
I am very inexperienced with sqlalchemy and am entirely unsure where else to place this code or how to go about it.
I have some questions when I build a JanusGraph Mixed index.
This is my code:
mgmt = graph.openManagement();
idx = mgmt.getGraphIndex('zhh1_index');
prop = mgmt.getPropertyKey('zhang');
mgmt.addIndexKey(idx, prop);
prop = mgmt.getPropertyKey('uri');
mgmt.addIndexKey(idx, prop);
prop = mgmt.getPropertyKey('age');
mgmt.addIndexKey(idx, prop);
mgmt.commit();
mgmt.awaitGraphIndexStatus(graph, 'zhh1_index').status(SchemaStatus.REGISTERED).call();
mgmt = graph.openManagement();
mgmt.updateIndex(mgmt.getGraphIndex('zhh1_index'),SchemaAction.ENABLE_INDEX).get();
mgmt.commit();
vertex2=graph.addVertex(label,'zhh1');
vertex2.property('zhang','male');
vertex2.property('uri','/zhh1/zhanghh');
vertex2.property('age','18');
vertex3=graph.addVertex(label,'zhh1');
vertex3.property('zhang','male');
vertex3.property('uri','/zhh1/zhangheng');
When the program executes this line:
mgmt.awaitGraphIndexStatus(graph, 'zhh1_index').status(SchemaStatus.REGISTERED).call();
the the log prints these information （and about 30s later, an exception like this: the sleep was interrupt）:
GraphIndexStatusReport[success=false, indexName='zhh1_index', targetStatus=ENABLED, notConverged={jiyq=INSTALLED, zhang=INSTALLED, uri=INSTALLED, age=INSTALLED}, converged={}, elapsed=PT1M0.096S]
I was so confused about this!
It keeps printing a lot for all indexes I have. Am I doing anything wrong? How to avoid such message?
When I execute the following statement separately, the following exception is reported：
exception：java.util.concurrent.ExecutionException:
mgmt.updateIndex(mgmt.getGraphIndex('zhh1_index'),SchemaAction.ENABLE_INDEX).get();
org.apache.tinkerpop.gremlin.driver.exception.ResponseException: Cannot invoke method get() on null object
Your index seems to be stuck in the INSTALLED state, which may happening due to a few reasons: please see this post and look at my answer-- specifically bullet numbers 2,3, and 5.
When did you buildMixedIndex() ?
REINDEX procedure may be required.
Currently, I am working with a EV3 lego robot that is controlled by several neurons. Now I want to modify the code (running on
python3) in such a way that one can change certain parameter values on the run via the shell (Ubuntu) in order to manipulate the robot's dynamics at any time (and for multiple times). Here is a schema of what I have achieved so far based on a short example code:
from multiprocessing import Process
from multiprocessing import SimpleQueue
import ev3dev.ev3 as ev3
class Neuron:
(definitions of class variables and update functions)
def check_input(queue):
while (True):
try:
new_para = str(input("Type 'parameter=value': "))
float(new_para[2:0]) # checking for float in input
var = new_para[0:2]
if (var == "k="): # change parameter k
queue.put(new_para)
elif (var == "g="): # change parameter g
queue.put(new_para)
else:
print("Error". Type 'k=...' or 'g=...')
queue.put(0) # put anything in queue
except (ValueError, EOFError):
print("New value is not a number. Try again!")
(some neuron-specific initializations)
queue = SimpleQueue()
check = Process(target=check_input, args=(queue,))
check.start()
while (True):
if (not queue.empty()):
cmd = queue.get()
var = cmd[0]
val = float(cmd[2:])
if (var == "k"):
Neuron.K = val
elif (var == "g"):
Neuron.g = val
(updating procedure for neurons, writing data to file)
Since I am new to multiprocessing there are certainly some mistakes concerning taking care of locking, efficiency and so on but the robot moves and input fields occur in the shell. However, the current problem is that it's actually impossible to make an input:
> python3 controller_multiprocess.py
> Type 'parameter=value': New value is not a number. Try again!
> Type 'parameter=value': New value is not a number. Try again!
> Type 'parameter=value': New value is not a number. Try again!
> ... (and so on)
I know that this behaviour is caused by putting the exception of EOFError due to the fact that this error occurs when the exception is removed (and the process crashes). Hence, the program just rushes through the try-loop here and assumes that no input (-> empty string) was made over and over again. Why does this happen? - when not called as a threaded procedure the program patiently waits for an input as expected. And how can one fix or bypass this issue so that changing parameters gets possible as wanted?
Thanks in advance!