I have an app that is distributing load on a bunch of workers. So far all workers are running on the same VM, have not needed to scale up yet.
My problem is that, like every 3-4 days, the worker crashes with the error message below - no contact between the client and the rabbitmq server in 1200 secs (I guess).
Traceback (most recent call last):
File "/var/www/vhosts/niklas/workers/builder.py", line 170, in <module>
BuildWorker().main()
File "/var/www/vhosts/niklas/lib/worker.py", line 29, in main
self.msgs.ch.start_consuming()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 722, in start_consuming
self.connection.process_data_events()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 93, in process_data_events
self.process_timeouts()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 99, in process_timeouts
self._call_timeout_method(self._timeouts.pop(timeout_id))
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 164, in _call_timeout_method
timeout_value['method']()
File "/usr/local/lib/python2.6/dist-packages/pika/heartbeat.py", line 85, in send_and_check
return self._close_connection()
File "/usr/local/lib/python2.6/dist-packages/pika/heartbeat.py", line 106, in _close_connection
HeartbeatChecker._STALE_CONNECTION % duration)
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 75, in close
self.process_data_events()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 91, in process_data_events
self._handle_timeout()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 198, in _handle_timeout
self._on_connection_closed(None, True)
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 235, in _on_connection_closed
raise exceptions.AMQPConnectionError(*self.closing)
pika.exceptions.AMQPConnectionError: (320, 'Too Many Missed Heartbeats, No reply in 1200 seconds')
My question is, what could possibly cause this?
This only happen to ~1 out of three workers, the others are running fine without any error message or warning (again, all workers and rabbitmq-server on the same VM).
I'm using the standard method in Python library pika, start_consuming(), to retrieve new requests. The code is way to big too attach here, and considering the error message, it seems to be out of my code or a system issue.
I'm using:
Python Pika 0.9.8
Rabbitmq 3.0.0
Debian 6.0
All workers are started inside screen
VM hosted at Linode, 512MB memory
We experienced a similar problem due to a bug (#236) in pika 0.9.8.
https://github.com/pika/pika/pull/236
This should be fixed in 0.9.9 or can be resolved by patching your pika library with the source code attached to the linked issue on github.
(Pika was closing a connection on 2 cumulative missed heartbeats rather than 2 consecutive ones).
Related
This question already has answers here:
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH error with Headless Chrome
(1 answer)
WebDriverException: Message: 'chromedriver' executable needs to be in PATH while setting UserAgent through Selenium Chromedriver python
(1 answer)
Closed 2 years ago.
I'm working on a website automation program for 1 month, all went well and worked perfectly fine until I had to reset my laptop to factory settings due to a black screen that kept on going after turning on the laptop.
I'm running a laptop with windows 10, Pycharm is the IDE and I code with Python (64-bit version). I use chromedrivers, sadly there's not a 64-bit version but no problem was ever given by that.
Now, when I try to start the code I saved from the finished project, it just keeps saying
"'chromedriver.exe' executable needs to be in PATH."
I looked for everything I could, I copied the chromedriver into the PyCharm folder, I unzipped it on my Desktop, I used the executable_path=r'...' option, I tried to install the webdriver manager... nothing helped and the webdriver manager couldn't even get downloaded.
I think it's important to say that I have 2 mass storages, one called "Windows (C:)" and the other "Data (D:)", which has 380 GB of space, so I installed Pycharm and Python into Data(D:).
I'd really appreciate it if anyone could help me with this problem, it just makes me crazy how I tried everything fixing this.. seems like no one had ever had such a dumb problem.
Anyways, have a great day dear StackOverflow Community!
This is the error code:
C:\Users\User\PycharmProjects\Ersters\venv\Scripts\python.exe C:/Users/User/PycharmProjects/Ersters/Ersters.py
Automatic generator started, please wait.
Traceback (most recent call last):
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\common\service.py", line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 854, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\User\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 1307, in _execute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the management file
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/User/PycharmProjects/Ersters/Ersters.py", line 15, in <module>
driver = webdriver.Chrome(options=options, executable_path=r'C:\Users\User\Desktop\Python\chromedriver.exe')
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "C:\Users\User\PycharmProjects\Ersters\venv\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
Process finished with exit code 1
I am trying to run gem5 in FS mode. It works fine. But If I switch to super user(using sudo su) and then try then I get below error-
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/test/gem5/src/python/m5/main.py", line 435, in main
exec filecode in scope
File "configs/example/fs.py", line 344, in <module>
test_sys = build_test_system(np)
File "configs/example/fs.py", line 99, in build_test_system
options.ruby, cmdline=cmdline)
File "/home/test/gem5/configs/common/FSConfig.py", line 637, in makeLinuxX86System
makeX86System(mem_mode, numCPUs, mdesc, self, Ruby)
File "/home/test/gem5/configs/common/FSConfig.py", line 562, in makeX86System
disk0.childImage(mdesc.disk())
File "/home/test/gem5/configs/common/Benchmarks.py", line 58, in disk
return disk(self.diskname)
File "/home/test/gem5/configs/common/SysPaths.py", line 58, in __call__
raise IOError, "Can't find a path to system files."
IOError: Can't find a path to system files.
Any lead??
First, avoid running gem5 as with sudo/as root if you can, for the same reasons as any other programs.
I would guess that the command through sudo is not seeing the M5_PATH environment variable: How to keep environment variables when using sudo Can you ensure that the variable is visible to the executable?
Finally, as mentioned at Gem 5 IOError: Can't find a path to system files. Full System X86 simulation setup M5_PATH is not needed anymore, so I would really just never use it, and just explicitly point to the files gem5 needs with the command line options.
I have a Windows 10 x64 based PC. I am trying to get this PDF-DIFF Python tool here: https://github.com/JoshData/pdf-diff
to run and it hits this error when it is running a line from Python Lib file called 'subprocess.py' file.
I posted this issue with full screenshots of the error and still waiting for response:
https://github.com/JoshData/pdf-diff/issues/30
Just to note that before installation of this github tool I :
1) installed Python 3.6 (tried both 32 bit and 64) and currently have 32 bit
2) installed Microsoft Visual Studio 14.0 Build tools (C++ build tools)
3) in command prompt ran command: pip install lxml so that I could have
the xml libraries downloaded and satisfy the tool's requirements
4) additionally have checked other stackoverflow threads suggesting that 'file not found' error is due to 'COMSPEC' variable being set differently in regedit vs. Python's subprocess.py.
In windows key registry, 'ComSpec' key set to: %SystemRoot%\system32\cmd.exe;
In python subprocess call is being made as such (last line is 997):
if shell:
startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
startupinfo.wShowWindow = _winapi.SW_HIDE
comspec = os.environ.get("COMSPEC", "cmd.exe")
args = '{} /c "{}"'.format (comspec, args)
#Start the process
try:
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
# no special security
None, None,
int(not close_fds),
creationflags,
env,
os.fspath(cwd) if cwd is not None else
None,
startupinfo)
I'm out of ideas, and feel the program requirements are satisfied.
So, when I run the command in the prompt to get my two PDF files compared and output comparison spit out:
C:\Python36\Scripts\pdf_diff>pdf-diff 683000.pdf 17368000.pdf > comparison_output.png
*I immediately get error shown below:
Traceback (most recent call last):
File "C:\Python36\Scripts\pdf_diff\pdf-diff-script.py", line 11, in
load_entry_point('pdf-diff==0.9.0', 'console_scripts', 'pdf-diff')()
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 496, in main
changes = compute_changes(args.files[0], args.files[1], top_margin=float(args.top_margin), bottom_margin=float(args.bottom_margin))
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 14, in compute_changes
docs = [serialize_pdf(0, pdf_fn_1, top_margin, bottom_margin), serialize_pdf(1, pdf_fn_2, top_margin, bottom_margin)]
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 29, in serialize_pdf
for run in box_generator:
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 102, in mark_eol_hyphens
for next_box in boxes:
File "C:\Python36\lib\site-packages\pdf_diff-0.9.0-py3.6.egg\pdf_diff\command_line.py", line 64, in pdf_to_bboxes
xml = subprocess.check_output(["pdftotext", "-bbox", fn, "/dev/stdout"])
File "C:\Python36\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\Python36\lib\subprocess.py", line 403, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Python36\lib\subprocess.py", line 709, in init
restore_signals, start_new_session)
File "C:\Python36\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I believe I found out the problem here. In the process of installing different versions of Python, namely 2.7 and 3.6 I broke something. I believe it could be the importing of different modules for the PDF-diff program. The PDF-DIFF program, according to manual is coded for Python 3. So, I believe the issue above was a versioning issue. I have for now removed Python 3.6 from my machine as I need 2.7 for other projects.
I saw this exception when I was testing pika
It works when I was using only defaults. However it threw an exception when I added credentials in the connection parameters.
import pika
credentials = pika.PlainCredentials('foo', 'bar')
connection = pika.BlockingConnection(pika.ConnectionParameters(host='localhost', credentials=credentials))
channel = connection.channel()
channel.queue_declare(queue='device_presence_info')
print ' [*] Waiting for messages. To exit press CTRL+C'
def callback(ch, method, properties, body):
print " [x] Received %s" % body
channel.basic_consume(callback,
queue='device_presence_info',
no_ack=True)
channel.start_consuming()
It gave me this error:
ubuntu#SDNS:~/lang/python/pika$ python receive.py
Traceback (most recent call last):
File "receive.py", line 12, in <module>
connection = pika.BlockingConnection(pika.ConnectionParameters(host='54.241.122.167', credentials=credentials))
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 107, in __init__
super(BlockingConnection, self).__init__(parameters, None, False)
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/base_connection.py", line 62, in __init__
on_close_callback)
File "/usr/local/lib/python2.6/dist-packages/pika/connection.py", line 590, in __init__
self.connect()
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 206, in connect
if not self._adapter_connect():
File "/usr/local/lib/python2.6/dist-packages/pika/adapters/blocking_connection.py", line 275, in _adapter_connect
raise exceptions.AMQPConnectionError(1)
pika.exceptions.AMQPConnectionError: 1
Can any help me ? Thanks.
That error normally means that either your credentials are wrong, or that the user you created doesn't have access to the exchange / queue you're trying to access. When creating the user, make sure to give access to the resources you're going to be using (or, if it's just for testing, to everything the same way the 'guest' account has). You can do this via the management plugin or the command line.
This post has a good sample on how to set the different permissions using the management plugin.
Install RabbitMQ server (for Debian/Ubuntu):
sudo apt-get install rabbitmq-server
We have an auto-deployment script that uses wsadmin and jython. The script appears to work as expected however after 6-7 redeployments the AdminTask object becomes unavilable, resulting in the following error when we attempt to use that object:
WASX7209I: Connected to process "server1" on node ukdlniqa41Node01 using SOAP connector; The type of process is: UnManagedProcess
WASX8011W: AdminTask object is not available.
...
Traceback (innermost last):
File "<string>", line 251, in ?
File "<string>", line 14, in main
File "<string>", line 38, in initialize
NameError: AdminTask
My question is, what would cause this AdminTask object to become unavailable? (it remains unavailable until we restart the server instance)
AdminTask may be available if one of the previous tasks does not finish properly. This happens often especially if your server is in development mode. I would suggest gathering Deployment Mustgather as per http://www-01.ibm.com/support/docview.wss?rs=180&context=SSCR4XA&q1=MustGatherDocument&uid=swg21199344&loc=en_US&cs=utf-8&lang=en and submitting the results to IBM.