pip-compile fails while parsing base.py - pip-tools

This is the first time I try to use pip-compile from pip-tools.
The base-level settings for a Django project are provided in main/settings/base.py. I added this section for pip-tools:
install_requires = [
'django-oscar>=3.0.2,<4.0.0',
'boto3==1.17.27',
'psycopg2-binary==2.8.6',
'pycountry==20.7.3',
]
pip-compile blows up when I try to use it. How might this be fixed?
$ pip-compile main/settings/base.py
Traceback (most recent call last):
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_vendor/packaging/requirements.py", line 113, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_vendor/pyparsing.py", line 1955, in parseString
raise exc
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_vendor/pyparsing.py", line 3250, in parseImpl
raise ParseException(instring, loc, self.errmsg, self)
pip._vendor.pyparsing.ParseException: Expected W:(abcd...), found '"' (at char 0), (line:1, col:1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_internal/req/constructors.py", line 342, in parse_req_from_line
req = Requirement(req_as_string)
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_vendor/packaging/requirements.py", line 115, in __init__
raise InvalidRequirement(
pip._vendor.packaging.requirements.InvalidRequirement: Parse error at "'"""'": Expected W:(abcd...)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/work/django/oscar/bin/pip-compile", line 8, in <module>
sys.exit(cli())
File "/var/work/django/oscar/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/var/work/django/oscar/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/var/work/django/oscar/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/var/work/django/oscar/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/var/work/django/oscar/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/var/work/django/oscar/lib/python3.8/site-packages/piptools/scripts/compile.py", line 369, in cli
constraints.extend(
File "/var/work/django/oscar/lib/python3.8/site-packages/piptools/_compat/pip_compat.py", line 26, in parse_requirements
yield install_req_from_parsed_requirement(parsed_req, isolated=isolated)
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_internal/req/constructors.py", line 458, in install_req_from_parsed_requirement
req = install_req_from_line(
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_internal/req/constructors.py", line 391, in install_req_from_line
parts = parse_req_from_line(name, line_source)
File "/var/work/django/oscar/lib/python3.8/site-packages/pip/_internal/req/constructors.py", line 357, in parse_req_from_line
raise InstallationError(msg)
pip._internal.exceptions.InstallationError: Invalid requirement: '"""' (from line 1 of main/settings/base.py)
Update 1: I get the same error for this section:
install_requires = [
]
Update 2: I made a file called requirements.in like this:
django-oscar>=3.0.2,<4.0.0
boto3==1.17.27
psycopg2-binary==2.8.6
pycountry==20.7.3
... and requirements.txt was generated properly:
$ pip-compile
# Long pause...
$ cat requirements.txt
#
# This file is autogenerated by pip-compile
# To update, run:
#
# pip-compile
#
asgiref==3.3.1
# via django
babel==2.9.0
# via
# django-oscar
# django-phonenumber-field
boto3==1.17.27
# via -r requirements.in
botocore==1.20.45
# via
# boto3
# s3transfer
django-extra-views==0.13.0
# via django-oscar
django-haystack==3.0
# via django-oscar
django-oscar==3.0.2
# via -r requirements.in
django-phonenumber-field==3.0.1
# via django-oscar
django-tables2==2.3.4
# via django-oscar
django-treebeard==4.4
# via django-oscar
django-widget-tweaks==1.4.8
# via django-oscar
django==3.1.7
# via
# django-extra-views
# django-haystack
# django-oscar
# django-phonenumber-field
# django-tables2
# django-treebeard
factory-boy==2.12.0
# via django-oscar
faker==6.4.1
# via factory-boy
jmespath==0.10.0
# via
# boto3
# botocore
phonenumbers==8.12.18
# via django-oscar
pillow==8.1.0
# via django-oscar
psycopg2-binary==2.8.6
# via -r requirements.in
purl==1.5
# via django-oscar
pycountry==20.7.3
# via -r requirements.in
python-dateutil==2.8.1
# via
# botocore
# faker
pytz==2021.1
# via
# babel
# django
s3transfer==0.3.6
# via boto3
six==1.15.0
# via
# django-extra-views
# purl
# python-dateutil
sqlparse==0.4.1
# via django
text-unidecode==1.3
# via faker
urllib3==1.26.4
# via botocore
Is pip-tools parsing of setup.py broken?

Related

How to set the settings of scrapy-redis to connect the remote redis-server with password?

I have tried two solutions:
1)`REDIS_HOST = '111.111.111.111'
REDIS_PORT = 12000
REDIS_PASSWORD = 'aaaaaaaa'`
but it will raise:
2017-11-23 15:03:13 [twisted] CRITICAL:
Traceback (most recent call last):
File "/home/yuyanggo/.local/lib/python3.6/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
result = g.send(result)
File "/home/yuyanggo/.local/lib/python3.6/site-packages/scrapy/crawler.py", line 79, in crawl
yield self.engine.open_spider(self.spider, start_requests)
redis.exceptions.ResponseError: NOAUTH Authentication required.
2)
REDIS_URL = 'redis://:aaaaaaaa#111.111.111.111:12000/0'
but I find that the data of redis is saved into localhost,not the remote redis-server.
try to put the following code in your setting.py file
REDIS_URL = 'redis://:{psw}#{host}:{port}'.format(
host='xx.xx.xx.xx', # your server ip
port='xxx',
psw='xxxx',
)

gcloud auth login : Properties Parse Error

i installed google-cloud-sdk in ubuntu 14.04 and when tried to login,it is showing this error.
krish#jarvis:~$ gcloud auth login
Traceback (most recent call last):
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/gcloud/gcloud.py", line 87, in
from googlecloudsdk.calliope import base
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/calliope/base.py", line 8, in
from googlecloudsdk.core import log
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 413, in
_log_manager = _LogManager()
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 195, in init
self.console_formatter = _ConsoleFormatter(sys.stderr)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/log.py", line 172, in init
use_color = not properties.VALUES.core.disable_color.GetBool()
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 782, in GetBool
value = _GetBoolProperty(self, PropertiesFile.Load(), required)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1141, in Load
PropertiesFile._PROPERTIES = PropertiesFile(paths)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1160, in init
self.__Load(properties_path)
File "/usr/bin/../lib/google-cloud-sdk/./lib/googlecloudsdk/core/properties.py", line 1174, in __Load
raise PropertiesParseError(e.message)
googlecloudsdk.core.properties.PropertiesParseError: File contains no section headers.
file: /home/krish/.config/gcloud/properties, line: 1
'h4\xaf\xe3\xda^\xa6\xe8\xb2\xdb`$?\x11\x7f\xce\xc1\x1f\x88\xcd"\x82c\x13Bj\x07\xc3\xe3\x9ds\xdd d\xe1\n'
This does look like you have a corruppted gcloud config file /home/krish/.config/gcloud/properties. Check what's the context, may be it's just one line and you can fix it, or just move/remove it (you will need to set all configurations once again if you did any).

GSUTIL traceback-Linux Mint

Im trying to install GSUTIL, after installation it gives the following output for every command,
Traceback (most recent call last):
File "/usr/local/bin/gsutil", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2749, in <module>
working_set = WorkingSet._build_master()
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 446, in _build_master
return cls._build_from_requirements(__requires__)
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 459, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 632, in resolve
raise VersionConflict(dist,req) # XXX put more info here
pkg_resources.VersionConflict: (httplib2 0.8 (/usr/lib/python2.7/dist-packages), Requirement.parse('httplib2>=0.9.1'))
That means you need to update the version of httplib2 installed on your system to at least v 0.9.1.

SchedulingError: Couldn't apply scheduled task downloader: argument must be an int, or have a fileno() method

i have added djcelery and kombu.transport.django to django apps.
settings.py:
import djcelery
djcelery.setup_loader()
from .celery import app as celery_app
BROKER_URL = "redis://abc:abc#localhost:6379/0"
CELERYBEAT_SCHEDULER = "djcelery.schedulers.DatabaseScheduler"
CELERY_TIMEZONE = "Europe/London"
CELERY_ENABLE_UTC = True
# store AsyncResult in redis
CELERY_RESULT_BACKEND = "redis"
REDIS_HOST = "localhost"
REDIS_PORT = 6379
REDIS_DB = 0
REDIS_VHOST = 0
REDIS_USER="xyz"
REDIS_PASSWORD="xyz"
REDIS_CONNECT_RETRY = True
celery.py
from __future__ import absolute_import
import os
from celery import Celery
from django.conf import settings
from . import celeryconfig
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
app = Celery('apps')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
#app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
celery supervisor configuration:
[program:celeryd]
command=/home/vagrant/.virtualenvs/project/bin/python /vagrant/project/manage.py celeryd -B -E -c 1 --settings=config.settings
directory=/vagrant/project
user=vagrant
autostart=true
autorestart=true
stdout_logfile=/var/log/supervisor/celeryd.log
stderr_logfile=/var/log/supervisor/celeryd_err.log
when a task is scheduled using django-celery interface which supposed to download some files, following errors are thrown:
Traceback (most recent call last):
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 203, in maybe_due
result = self.apply_async(entry, publisher=publisher)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 259, in apply_async
entry, exc=exc)), sys.exc_info()[2])
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/beat.py", line 251, in apply_async
**entry.options)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/task.py", line 555, in apply_async
**dict(self._get_exec_options(), **options)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/base.py", line 323, in send_task
reply_to=reply_to or self.oid, **options
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/celery/app/amqp.py", line 302, in publish_task
**kwargs
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 168, in publish
routing_key, mandatory, immediate, exchange, declare)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/connection.py", line 440, in _ensured
return fun(*args, **kwargs)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 180, in _publish
[maybe_declare(entity) for entity in declare]
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/messaging.py", line 111, in maybe_declare
return maybe_declare(entity, self.channel, retry, **retry_policy)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/common.py", line 99, in maybe_declare
return _maybe_declare(entity)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/common.py", line 110, in _maybe_declare
entity.declare()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/entity.py", line 505, in declare
self.queue_declare(nowait, passive=False)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/entity.py", line 531, in queue_declare
nowait=nowait)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 447, in queue_declare
return queue_declare_ok_t(queue, self._size(queue), 0)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 651, in _size
sizes = cmds.execute()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/redis/client.py", line 2157, in execute
conn.disconnect()
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 800, in disconnect
channel._on_connection_disconnect(self)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 461, in _on_connection_disconnect
self.connection.cycle._on_connection_disconnect(connection)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/transport/redis.py", line 259, in _on_connection_disconnect
self.poller.unregister(connection._sock)
File "/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/utils/eventio.py", line 85, in unregister
self._epoll.unregister(fd)
SchedulingError: Couldn't apply scheduled task downloader: argument must be an int, or have a fileno() method.
and i'm not getting any clue where from these errors are coming...
any suggestions will be greatly appreciated.
Update 1:
i use django-celery admin interface to schedule a task and task is defined inside an app registered in INSTALLED_APP as follows:
#shared_task
def downloader():
# execute static/shell_scripts/feed_downloader.sh
import subprocess
PROJECT_PATH = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
subprocess.call(PROJECT_PATH+"/static/shell_scripts/feed_downloader.sh", shell=True)
the feed_downloader.sh file calls a django management command which if i execute independetly works fine like:
python manage.py feed_downloader
if i add a print statement just before line no. 85 to check fd parameter in file
/home/vagrant/.virtualenvs/project/local/lib/python2.7/site-packages/kombu/utils/eventio.py
then following messages get printed:
see this <socket._socketobject object at 0x131b750>
see this None
see this <socket._socketobject object at 0x131bc90>
see this None

What dependencies do I need for USB programing in python with pyUSB?

I am trying to get the usb.find command to work properly in a python script I'm writing on Angstrom for the Beagleboard.
Here is my code:
#!/usr/bin/env python
import usb.core
import usb.util
import usb.backend.libusb01 as libusb
PYUSB_DEBUG_LEVEL = 'debug'
# find our device
# Bus 002 Device 006: ID 1208:0815
# idVendor 0x1208
# idProduct 0x0815
# dev = usb.core.find(idVendor=0xfffe, idProduct=0x0001)
# iManufacturer 1 TOROBOT.com
dev = usb.core.find(idVendor=0x1208, idProduct=0x0815,
backend=libusb.get_backend() )
I don't know what's missing, but here is what I do know.
When I don't specify the backend, no backend is found. When I do specify the backend usb.backend.libusb01 I get the following error:
root#beagleboard:~/servo# ./pyServo.py
Traceback (most recent call last):
File "./pyServo.py", line 17, in <module>
dev = usb.core.find(idVendor=0x1208, idProduct=0x0815, backend=libusb.get_backend() )
File "/usr/lib/python2.6/site-packages/usb/core.py", line 854, in find
return _interop._next(device_iter(k, v))
File "/usr/lib/python2.6/site-packages/usb/_interop.py", line 60, in _next
return next(iter)
File "/usr/lib/python2.6/site-packages/usb/core.py", line 821, in device_iter
for dev in backend.enumerate_devices():
File "/usr/lib/python2.6/site-packages/usb/backend/libusb01.py", line 390, in enumerate_devices
_check(_lib.usb_find_busses())
File "/usr/lib/python2.6/ctypes/__init__.py", line 366, in __getattr__
func = self.__getitem__(name)
File "/usr/lib/python2.6/ctypes/__init__.py", line 371, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: python: undefined symbol: usb_find_busses
What am I missing so that this will work properly?
Try installing python-pyusb through opkg, that should install the native dependencies.