Using python-rq with Zope - zope

I'm trying to use Python-RQ with Zope by calling an external method (for background tasks) from ZMI after a certain operation. The file called by external method resides in Extensions. It initialises connection to Redis and imports a module that runs the background tasks. The question is where should this to be imported file be placed ? Python-RQ does not seem to recognise if I put it inside Products directory. It throws no module named Products.xyz. Below is the code snippet
from redis import Redis
from rq import Queue
from Products.def_update_company_status import ae_update_company_status
q = Queue(connection=Redis())
def rq_worker(context):
q.enqueue(ae_update_company_status)
return 'DONE'
The rq_worker function is invoked by the external method.
Below is the error
18:12:40 default: Products.def_update_company_status.ae_update_company_status() (4b2b5c81-e329-4031-a3e7-b9b1bb198278)
18:12:40 ImportError: No module named Products.def_update_company_status
Traceback (most recent call last):
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/worker.py", line 588, in perform_job
rv = job.perform()
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/job.py", line 498, in perform
try:
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/job.py", line 206, in func
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/utils.py", line 150, in import_attribute
module = importlib.import_module(module_name)
File "build/bdist.linux-x86_64/egg/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named Products.def_update_company_status
Traceback (most recent call last):
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/worker.py", line 588, in perform_job
rv = job.perform()
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/job.py", line 498, in perform
try:
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/job.py", line 206, in func
File "/home/zope/ams/lib/python2.6/site-packages/rq-0.6.0-py2.6.egg/rq/utils.py", line 150, in import_attribute
module = importlib.import_module(module_name)
File "build/bdist.linux-x86_64/egg/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named Products.def_update_company_status
18:12:40 Moving job to u'failed' queue
18:12:40
18:12:40 *** Listening on default...

Related

Unable to load web page with seleniumwire

Unable to load the web page using seleniumwire, I am observing this error in the browser.
This page isn't working
xxx.xyz didn't send any data.
ERR_EMPTY_RESPONSE
When I replace seleniumwire with selenium while initializing the webdriver, the issue is no longer observed.
Seleniumwire was working fine and the below-mentioned error started occurring a couple of days ago.
Seleniumwire version: 4.4.0
Python 3.9
MacOS Big Sur
AttributeError: module 'lib' has no attribute 'SSL_CTX_get0_param'
ERROR:seleniumwire.server:127.0.0.1:61095: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/server.py",
line 113, in handle root_layer() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/modes/http_proxy.py",
line 9, in call layer() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/tls.py",
line 285, in call layer() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/http1.py",
line 100, in call layer() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/http.py",
line 206, in call if not self._process_flow(flow): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/http.py",
line 285, in _process_flow return self.handle_regular_connect(f) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/http.py",
line 224, in handle_regular_connect layer() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/tls.py",
line 278, in call self._establish_tls_with_client_and_server() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/tls.py",
line 358, in _establish_tls_with_client_and_server self._establish_tls_with_server() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/server/protocol/tls.py",
line 445, in _establish_tls_with_server self.server_conn.establish_tls( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/connections.py",
line 295, in establish_tls self.convert_to_tls(cert=client_cert, sni=sni, **kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/net/tcp.py",
line 382, in convert_to_tls context = tls.create_client_context( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/seleniumwire/thirdparty/mitmproxy/net/tls.py",
line 285, in create_client_context param = SSL._lib.SSL_CTX_get0_param(context._context)
AttributeError: module 'lib' has no attribute 'SSL_CTX_get0_param'
This looks like you are using an outdated version of the cryptography library.

Apache BEAM pipeline fails when writing TF Records - AttributeError: 'str' object has no attribute 'iteritems'

The issue started appearing over the weekend. For some reason, it feels to be a DataFlow issue.
Previously, I was able to execute the script and write TF records just fine. However, now, I am unable to initialize the computation graph to process the data.
The traceback is:
Traceback (most recent call last):
File "my_script.py", line 1492, in <module>
MyBeamClass()
File "my_script.py", line 402, in __init__
self.run()
File "my_script.py", line 514, in run
transform_fn_io.WriteTransformFn(path=self.JOB_DIR + '/transform/'))
File "/anaconda3/envs/ml27/lib/python2.7/site-packages/apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "/anaconda3/envs/ml27/lib/python2.7/site-packages/apache_beam/runners/dataflow/dataflow_runner.py", line 1238, in wait_until_finish
(self.state, getattr(self._runner, 'last_error_msg', None)), self)
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 176, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 531, in apache_beam.runners.worker.operations.DoOperation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 532, in apache_beam.runners.worker.operations.DoOperation.start
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 533, in apache_beam.runners.worker.operations.DoOperation.start
super(DoOperation, self).start()
File "apache_beam/runners/worker/operations.py", line 202, in apache_beam.runners.worker.operations.Operation.start
def start(self):
File "apache_beam/runners/worker/operations.py", line 206, in apache_beam.runners.worker.operations.Operation.start
self.setup()
File "apache_beam/runners/worker/operations.py", line 480, in apache_beam.runners.worker.operations.DoOperation.setup
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 485, in apache_beam.runners.worker.operations.DoOperation.setup
pickler.loads(self.spec.serialized_fn))
File "/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line 247, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 317, in loads
return load(file, ignore)
File "/usr/local/lib/python2.7/dist-packages/dill/_dill.py", line 305, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1232, in load_build
for k, v in state.iteritems():
AttributeError: 'str' object has no attribute 'iteritems'
I am using tensorflow==1.13.1 and tensorflow-transform==0.9.0 and apache_beam==2.7.0
with beam.Pipeline(options=self.pipe_opt) as p:
with beam_impl.Context(temp_dir=self.google_cloud_options.temp_location):
# rest of the script
_ = (
transform_fn
| 'WriteTransformFn' >>
transform_fn_io.WriteTransformFn(path=self.JOB_DIR + '/transform/'))
I was experiencing the same error.
It seems to be triggered by a mismatch in the tensorflow-transform versions of your local (or master) machine and the workers one (specified in the setup.py file).
In my case I was running tensorflow-transform==0.13 on my local machine whereas the workers were running 0.8.
Downgrading the local version to 0.8 fixed the issue.

sqlalchemy.orm.exc.DetachedInstanceError depending on Pyramid webapp serving method?

This error in thrown only on production Apache2 server, and not at all when using pserve method to test the project locally :
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <My_Table at 0x7fc82076d2d0> is not bound to a Session; lazy load operation of attribute 'my_table_relationship' cannot proceed
Crashing code in template is:
<tal:x repeat="oMot_cn oMot_mcnii_cni.r_mcnii_cp.r_newphonets">
This kind of chained relationships pattern had been working well so far, I am just trying to load the whole query object at launching time of the Pyramid app, instead of loading a new one each time a request is handled by the view/controller...
It keeps crashing on production while it does not crash on pserve dev testing, so it's hard to debug.
Whole traceback on production server:
mod_wsgi (pid=15170): Exception occurred processing WSGI script '/var/local/env_py3/env_py3_2an/MyProject/myproject.wsgi'.
Traceback (most recent call last):
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/chameleon/template.py", line 170, in render
self._render(stream, econtext, rcontext)
File "tp_lesson_T_ex_mots_from_audio_765c6cd23ec517b3893410ce969cf2d9.py", line 584, in render
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/chameleon/py26.py", line 5, in lookup_attr
return getattr(obj, key)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/sqlalchemy/orm/attributes.py", line 233, in __get__
return self.impl.get(instance_state(instance), dict_)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/sqlalchemy/orm/attributes.py", line 579, in get
value = self.callable_(state, passive)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/sqlalchemy/orm/strategies.py", line 479, in _load_for_state
(orm_util.state_str(state), self.key)
sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <Caract_phonet at 0x7fc82076d2d0> is not bound to a Session; lazy load operation of attribute 'r_newphonets' cannot proceed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/fanstatic/publisher.py", line 219, in __call__
return self.app(environ, start_response)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/fanstatic/injector.py", line 64, in __call__
response = request.get_response(self.app)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/webob/request.py", line 1296, in send
application, catch_exc_info=False)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/webob/request.py", line 1260, in call_application
app_iter = application(self.environ, start_response)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid/router.py", line 251, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid/router.py", line 227, in invoke_subrequest
response = handle_request(request)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid/tweens.py", line 21, in excview_tween
response = handler(request)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid_tm-0.7-py3.2.egg/pyramid_tm/__init__.py", line 82, in tm_tween
reraise(*exc_info)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid_tm-0.7-py3.2.egg/pyramid_tm/compat.py", line 13, in reraise
raise value
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid_tm-0.7-py3.2.egg/pyramid_tm/__init__.py", line 63, in tm_tween
response = handler(request)
File "/var/local/env_py3/env_py3_2an/lib/python3.2/site-packages/pyramid/router.py", line 161, in handle_request
response = view_callable(context, request)
Is there any missing PasteDeploy configuration to get it crashing the same way it does once it's pushed to production server ?

Kazoo package using Jython

Kazoo's fairly working under the Python, but the project which i'm working on requires to use it under the Jython.
Here is the issue:
>>> from kazoo.client import KazooClient
>>> zk = KazooClient('127.0.0.1')
>>> zk.start()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\jython2.7.0\Lib\site-packages\kazoo\client.py", line 541, in start
event = self.start_async()
File "C:\jython2.7.0\Lib\site-packages\kazoo\client.py", line 576, in start_async
self._connection.start()
File "C:\jython2.7.0\Lib\site-packages\kazoo\protocol\connection.py", line 170, in start
rw_sockets = self.handler.create_socket_pair()
File "C:\jython2.7.0\Lib\site-packages\kazoo\handlers\threading.py", line 165, in create_socket_pair
return utils.create_socket_pair(socket)
File "C:\jython2.7.0\Lib\site-packages\kazoo\handlers\utils.py", line 148, in create_socket_pair
temp_srv_sock.bind(('', port))
File "C:\jython2.7.0\Lib\_socket.py", line 1367, in meth
return getattr(self._sock,name)(*args)
File "C:\jython2.7.0\Lib\_socket.py", line 812, in bind
self.bind_addr = _get_jsockaddr(address, self.family, self.type, self.proto, AI_PASSIVE)
File "C:\jython2.7.0\Lib\_socket.py", line 1565, in _get_jsockaddr
addr = _get_jsockaddr2(address_object, family, sock_type, proto, flags)
File "C:\jython2.7.0\Lib\_socket.py", line 1594, in _get_jsockaddr2
hostname = {AF_INET: INADDR_ANY, AF_INET6: IN6ADDR_ANY_INIT}[family]
KeyError: 0
How i'd already said - there is no this kind issue using the python.
I'm pretty sure that it is connected with the Jython-version of the _socket.py file, but don't know the workaround.
What can you recommend?

gcloud auth login : Properties Parse Error

i installed google-cloud-sdk in ubuntu 14.04 and when tried to login,it is showing this error.
krish#jarvis:~$ gcloud auth login
Traceback (most recent call last):
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 87, in
from googlecloudsdk.calliope import base
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/calliope/base.py", line 8, in
from googlecloudsdk.core import log
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 413, in
_log_manager = _LogManager()
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 195, in init
self.console_formatter = _ConsoleFormatter(sys.stderr)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 172, in init
use_color = not properties.VALUES.core.disable_color.GetBool()
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 782, in GetBool
value = _GetBoolProperty(self, PropertiesFile.Load(), required)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1141, in Load
PropertiesFile._PROPERTIES = PropertiesFile(paths)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1160, in init
self.__Load(properties_path)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1174, in __Load
raise PropertiesParseError(e.message)
googlecloudsdk.core.properties.PropertiesParseError: File contains no section headers.
file: /home/krish/.config/gcloud/properties, line: 1
'h4\xaf\xe3\xda^\xa6\xe8\xb2\xdb`$?\x11\x7f\xce\xc1\x1f\x88\xcd"\x82c\x13Bj\x07\xc3\xe3\x9ds\xdd d\xe1\n'
This does look like you have a corruppted gcloud config file /home/krish/.config/gcloud/properties. Check what's the context, may be it's just one line and you can fix it, or just move/remove it (you will need to set all configurations once again if you did any).