I use Spacy 2.7:
Following the Set Membership example:
I tried the below with the IN logic - simple list of words...
doc = nlp(SOME_TEXT)
matcher = Matcher(nlp.vocab)
pattern = [{'LOWER': {'IN' : ["i","you","we","they"]}}]
matcher.add("myPattern",None, pattern)
matches = matcher(doc)
...
Get an exception
would:[{'LOWER': {'IN': ['i', 'you', 'we', 'they']}}] with index 0
Traceback (most recent call last):
File "test.py", line 85, in <module>
matcher.add(key,None, curr)
File "matcher.pyx", line 266, in spacy.matcher.Matcher.add
File "matcher.pyx", line 99, in spacy.matcher.init_pattern
TypeError: an integer is required
Went to the open source file (matcher.pyx) - in line 99, not sure what is the bug, or maybe I used that incorrect..
Sorry if this was confusing – but the GitHub thread you're referring to is still only the spec and proposal, i.e. the planned implementation. The changes will hopefully ship with spaCy v2.1.0 (since some of the changes to the Matcher internals are not fully backwards compatible).
Related
I am building an application backed by a Neptune database. Because I want the application to be scalable, I am using AWS Lambda + API gateway to build a REST API to interact with the database. This seems to be a reasonable idea based on the fact that this use case is documented in the Neptune docs.
The Neptune docs recommend reusing the websocket connection to the database across the entire execution context of the function, which is what I am doing at the moment. The docs also recommend resetting the connection and retrying upon errors (see here), which I am also using. However, I am seeing exceptions every now and then (perhaps every 20 requests on average). One of the exceptions I get is
ConnectionResetError: Cannot write to closing transport
which seems to be the same as this issue.
The other one is:
Traceback (most recent call last):
File "/var/task/chalice/app.py", line 1685, in _get_view_function_response
response = view_function(**function_args)
File "/var/task/app.py", line 57, in resource
return Resource(app.current_request, g).process()
File "/var/task/backoff/_sync.py", line 94, in retry
ret = target(*args, **kwargs)
File "/var/task/chalicelib/handlers/resource.py", line 106, in get
values = resources.valueMap().with_(WithOptions.tokens).toList()
File "/var/task/gremlin_python/process/traversal.py", line 57, in toList
return list(iter(self))
File "/var/task/gremlin_python/process/traversal.py", line 47, in __next__
self.traversal_strategies.apply_strategies(self)
File "/var/task/gremlin_python/process/traversal.py", line 548, in apply_strategies
traversal_strategy.apply(traversal)
File "/var/task/gremlin_python/driver/remote_connection.py", line 63, in apply
remote_traversal = self.remote_connection.submit(traversal.bytecode)
File "/var/task/gremlin_python/driver/driver_remote_connection.py", line 60, in submit
results = result_set.all().result()
File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 435, in result
return self.__get_result()
File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/var/task/gremlin_python/driver/resultset.py", line 90, in cb
f.result()
File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/var/lang/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/var/lang/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/var/task/gremlin_python/driver/connection.py", line 82, in _receive
data = self._transport.read()
File "/var/task/gremlin_python/driver/aiohttp/transport.py", line 104, in read
raise RuntimeError("Connection was already closed.")
RuntimeError: Connection was already closed.
In case it is relevant, I am using gremlingpython==3.5.1
It seems to me that these issues are all ultimately a consequence of using AWS Lambda, namely due to the mismatch between the longevity of websocket connections and the ephemeral nature of lambda execution contexts. The question then is: Am I doing the wrong thing by trying to use AWS lambda for my API? Would it be more appropriate to setup an EC2 instance and deal with the scalability in some other way?
P.S. Previously I did create and close a connection in every function execution (as previously recommended in the Neptune docs), which did work fine but was naturally slow.
The latest version of Neptune only supports Gremlin 3.4.11 (https://docs.aws.amazon.com/neptune/latest/userguide/engine-releases-1.0.5.1.html). I would start by using gremlin-python 3.4.11 and see if that resolves your issue. Gremlin-python 3.5 replaced Tornado with AIO HTTP (ref) for websocket connections and I suspect that change may be causing a slight change in behavior that a future release supporting Gremlin 3.5 will address.
I wonder whether the 'Connection was already closed' error message is not being treated as a retriable error by the retry logic?
What happens if you add this error message to the list of retriable_error_msgs in the Python example in the docs?
Problem Statement: Extract data stored in the .sdf file to python.
System config: Win 10 Pro 64-bit, python 3.5.2-64 bit, adodbapi library, SQL CE 3.5
I am fairly new to programming and I have picked up Python as my first language to learn. Currently, I have hit a wall in process of connecting a SQL CE 3.5 .sdf file.
I have used the adodbapi library. I have searched the web extensively over the past week to find a solution to this problem and to make sure that my connection string is correct. I have tried multiple options/solutions provided on stack overflow and https://www.connectionstrings.com/microsoft-sqlserver-ce-oledb-3-5/.
Code:
import adodbapi
cons_str = "Provider=Microsoft.SQLSERVER.MOBILE.OLEDB.3.5;" \
"Data Source=D:\Work\Programming\Python\SQL_DataTransfer\LF.sdf;"\
"Persist Security Info=False;" \
"SSCE:Max Database Size=4091"
connection = adodbapi.connect(cons_str)
print(connection)
Error Message:
Traceback (most recent call last):
File "D:\Work\Programs\Python35.virtualenvs\sql_output\lib\site-packages\adodbapi\adodbapi.py", line 93, in make_COM_connecter
c = Dispatch('ADODB.Connection') #connect after CoIninialize v2.1.1 adamvan
NameError: name 'Dispatch' is not defined
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Work\Programs\Python35.virtualenvs\sql_output\lib\site-packages\adodbapi\adodbapi.py", line 112, in connect
co.connect(kwargs)
File "D:\Work\Programs\Python35.virtualenvs\sql_output\lib\site-packages\adodbapi\adodbapi.py", line 269, in connect
self.connector = connection_maker()
File "D:\Work\Programs\Python35.virtualenvs\sql_output\lib\site-packages\adodbapi\adodbapi.py", line 95, in make_COM_connecter
raise api.InterfaceError ("Windows COM Error: Dispatch('ADODB.Connection') failed.")
adodbapi.apibase.InterfaceError: Windows COM Error: Dispatch('ADODB.Connection') failed.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:/Work/Programming/Python/SQL_DataTransfer/SQL_CE_reportDB.py", line 8, in
connection = adodbapi.connect(cons_str)
File "D:\Work\Programs\Python35.virtualenvs\sql_output\lib\site-packages\adodbapi\adodbapi.py", line 116, in connect
raise api.OperationalError(e, message)
adodbapi.apibase.OperationalError: (InterfaceError("Windows COM Error: Dispatch('ADODB.Connection') failed.",), 'Error opening connection to "Provoider=Microsoft.SQLSERVER.MOBILE.OLEDB.3.5;Data Source=D:\Work\Programming\Python\SQL_DataTransfer\LF.sdf;Persist Security Info=False;SSCE:Max Database Size=4091"')
At this point any help is much appreciated.
Thank you,
Sincerely,
JD.
Looks like you have a typo:
Provoider => Provider
adodbapi version = '2.6.0.6' depends on pypiwin32 to be installed in your Python environment.
For adodbapi.py, from line 51:
if api.onIronPython:
from System import Activator, Type, DBNull, DateTime, Array, Byte
from System import Decimal as SystemDecimal
from clr import Reference
def Dispatch(dispatch):
type = Type.GetTypeFromProgID(dispatch)
return Activator.CreateInstance(type)
def getIndexedValue(obj,index):
return obj.Item[index]
else: # try pywin32
try:
import win32com.client
import pythoncom
import pywintypes
onWin32 = True
def Dispatch(dispatch):
return win32com.client.Dispatch(dispatch)
except ImportError:
import warnings
warnings.warn("pywin32 package (or IronPython) required for adodbapi.",ImportWarning)
def getIndexedValue(obj,index):
return obj(index)
In my situation, I traced the fact that the Dispatch function was not defined because an ImportError exception is generated at line 62, (import win32com.client) caught in the except block but for some reason the warning message was not displayed in my console.
Try:
pip install pypiwin32
and the ImportError exception described above should not be raised anymore.
I'm trying to use twisted to handle data generated by a binary (which indefinitely dumps lines onto stdout). Since by data is inherently line-delimited, I was trying to used the LineReciever instead of trying to parse data. The following is the relevant bit of the code which seems to be causing trouble :
class ProtocolBareQDAL41xB(ProcessProtocol, LineReceiver):
...
def outReceived(self, data):
print "Got Data:" + repr(data)
self.dataReceived(data)
def lineReceived(self, line):
print "Got Line: " + line
self._process_line(line)
...
This 'works' for the first of two lines in the output. I don't know yet if it works for only one line, or if it works for all but the last line. The resulting output looks something like :
$ python BareQDAL41xB.py
Made Connection
<Process pid=16486 status=-1>
Got Data:'No device found!\nMultiple devices found! Please connect only one.\n'
Got Line: No device found!
Got Serial Number : found!
Unhandled Error
Traceback (most recent call last):
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/python/log.py", line 101, in callWithLogger
return callWithContext({"system": lp}, func, *args, **kw)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/python/log.py", line 84, in callWithContext
return context.call({ILogContext: newCtx}, func, *args, **kw)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext
return func(*args,**kw)
--- <exception caught here> ---
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 597, in _doReadOrWrite
why = selectable.doRead()
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/process.py", line 274, in doRead
return fdesc.readFromFD(self.fd, self.dataReceived)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/fdesc.py", line 94, in readFromFD
callback(output)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/process.py", line 277, in dataReceived
self.proc.childDataReceived(self.name, data)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/process.py", line 931, in childDataReceived
self.proto.childDataReceived(name, data)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/internet/protocol.py", line 604, in childDataReceived
self.outReceived(data)
File "BareQDAL41xB.py", line 104, in outReceived
self.dataReceived(data)
File "/media/ldata/code/virtualenvs/tendril/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 573, in dataReceived
self.transport.disconnecting):
exceptions.AttributeError: 'Process' object has no attribute 'disconnecting'
processExited, status 0
processEnded, status 0
LineReciever seems to be expecting the transport to implement disconnecting.
Is it possible to use twisted's LineReciever with twisted's ProcessProtocol, or should I implement the line parser in my protocol instead?
LineReceiver is already a Protocol, which implements different interfaces than IProcessProtocol.
Luckily, recent versions of Twisted already contain an adapter that does what you want - which is to treat a subprocess as a stream of bytes. Rather than calling spawnProcess directly, use ProcessEndpoint, and you can pass a regular ProtocolFactory, no ProcessProtocol involved.
However, as a commenter has already pointed out, there's a bug here, where the disconnecting attribute is not formally part of ITransport, but LineReceiver (and LineOnlyReceiver) depend on it anyway, and since it's not part of the interface, ProcessEndpoint doesn't implement it. That should definitely be fixed, but in the meanwhile, we'll need to work around it.
As a happy accident, Twisted's built-in support for wrapping protocols, WrappingFactory, already has support for the disconnecting attribute, specifically because of this ugly disparity between the theory of the interface specifications and the reality of the most popular ITransport implementations. So even a do-nothing wrapper will work around the problem. You can implement this like so:
from zope.interface import implementer
from twisted.internet.interfaces import IStreamClientEndpoint
from twisted.protocols.policies import WrappingFactory
#implementer(IStreamClientEndpoint)
class DisconnectingWorkaroundEndpoint(object):
def __init__(self, endpoint):
self._endpoint = endpoint
def connect(self, protocolFactory):
return self._endpoint.connect(WrappingFactory(protocolFactory))
and then when you construct your ProcessEndpoint, do:
endpoint = DisconnectingWorkaroundEndpoint(ProcessEndpoint(...))
Sorry for the delay on answering; while you've probably worked out your own workaround, I hope this will be useful to others with the same question!
The issue I'm having is that I'm trying to build an Angstrom image from scratch using bitbake (since Angstrom is now Yocto Compatible) but I've run into an error the moment I run the bitbake systemd-image
Traceback (most recent call last):
File "/usr/bin/bitbake", line 234, in <module>
ret = main()
File "/usr/bin/bitbake", line 197, in main
server = ProcessServer(server_channel, event_queue, configuration)
File "/usr/lib/pymodules/python2.7/bb/server/process.py", line 78, in __init__
self.cooker = BBCooker(configuration, self.register_idle_function)
File "/usr/lib/pymodules/python2.7/bb/cooker.py", line 76, in __init__
self.parseConfigurationFiles(self.configuration.file)
File "/usr/lib/pymodules/python2.7/bb/cooker.py", line 510, in parseConfigurationFiles
data = _parse(os.path.join("conf", "bitbake.conf"), data)
TypeError: getVar() takes exactly 3 arguments (2 given)
ERROR: Error evaluating '${TARGET_OS}:${TRANSLATED_TARGET_ARCH}:build-${BUILD_OS}:pn-${PN}:${MACHINEOVERRIDES}:${DISTROOVERRIDES}:${CLASSOVERRIDE}:forcevariable${#bb.utils.contains("TUNE_FEATURES", "thumb", ":thumb", "", d)}${#bb.utils.contains("TUNE_FEATURES", "no-thumb-interwork", ":thumb-interwork", "", d)}'
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 116, in expandWithRefs
s = __expand_var_regexp__.sub(varparse.var_sub, s)
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 60, in var_sub
var = self.d.getVar(key, 1)
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 260, in getVar
return self.expand(value, var)
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 132, in expand
return self.expandWithRefs(s, varname).value
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 117, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
TypeError: getVar() takes exactly 3 arguments (2 given)
ERROR: Error evaluating '${#bb.parse.BBHandler.vars_from_file(d.getVar('FILE'),d)[0] or 'defaultpkgname'}'
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 117, in expandWithRefs
s = __expand_python_regexp__.sub(varparse.python_sub, s)
File "/usr/lib/pymodules/python2.7/bb/data_smart.py", line 76, in python_sub
value = utils.better_eval(codeobj, DataContext(self.d))
File "/usr/lib/pymodules/python2.7/bb/utils.py", line 387, in better_eval
return eval(source, _context, locals)
File "PN", line 1, in <module>
TypeError: getVar() takes exactly 3 arguments (2 given)
I've been at this for a while now, searching on different sites. Originally I tried following the guide at the developer section on the Angstrom site, but once I got some errors (prior to this one I'm putting here), I found Derek Molloy's site http://derekmolloy.ie/building-angstrom-for-beaglebone-from-source/ which solved those errors and gave a little more detail into the process.
Eventually I stumbled onto another forum post which decribed my problem, but unfortunately the answers weren't really clear (for me anyway) http://comments.gmane.org/gmane.linux.distributions.angstrom.devel/7431. I'm at a loss on what could be wrong, and I'm pretty much new to Yocto project so I'm unsure if there's any steps missing or something that's implicit that I have overlooked, so I would deeply appreciate anyone who could point me on the right direction on this.
As side note, I've been thinking that it could be something having to do with the environment-angstrom-... file that I have, since mine is environment-angstrom-v2013.12 and all the other examples use previous versions, I'm wondering if there's a new step involved when working with this.
Is there a reason why you are using a system-wide bitbake instead of the one that is compatible with that release of Angstrom?
Don't use a system-wide bitbake, as the bitbake API can and does change over time. Use the corresponding bitbake for that release of angstrom.
(This is breaking because your bitbake requires getVar to take three arguments but your angstrom layers are only passing two)
I'm doing a migration from sqlite to oracle backend. The oracle database already exists and is maintained by other people. Its version is Oracle9i Enterprise Edition Release 9.2.0.1.0.
I have a simple model:
class AliasType(models.Model):
id = models.AutoField(primary_key=True, db_column="F_ALIAS_ID")
name = models.CharField(u"Type name", max_length=255, unique=True, db_column="F_ALIAS_NAME")
class Meta:
db_table = "ALIAS"
./manage.py syncdb does not return any errors. But when I try to create a new instance and save it to the database, I get the following error:
>>> AliasType.objects.create(name="test")
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/manager.py", line 138, in create
return self.get_query_set().create(**kwargs)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/query.py", line 360, in create
obj.save(force_insert=True, using=self.db)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/base.py", line 460, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/base.py", line 553, in save_base
result = manager._insert(values, return_id=update_pk, using=using)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/manager.py", line 195, in _insert
return insert_query(self.model, values, **kwargs)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/query.py", line 1435, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/sql/compiler.py", line 791, in execute_sql
cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/backends/util.py", line 18, in execute
return self.cursor.execute(sql, params)
File "/mnt/Data/private/projects/envs/termary-oracle/src/django/django/db/backends/oracle/base.py", line 630, in execute
return self.cursor.execute(query, self._param_generator(params))
IntegrityError: ORA-01400: cannot insert NULL into ("SINCE"."ALIAS"."F_ALIAS_ID")
If I specify id, e.g. AliasType.objects.create(id=5, name="test"), it works. I thought django should be able to retrieve id value automatically. I've learnt that Oracle does not support autoincrement, and I should use triggers and sequences. I was told that there is an existing sequence in the database that returns ids for all new rows, and I know its name, say SEQ_GET_NEW_ID.
So the question is how to implement that in the most elegant way, i.e. how to tell Django to get id values for all new objects from the sequence named SEQ_GET_NEW_ID without hacking it too much (e.g. overriding save() methods for all models)?
There is a ticket open (#1946) to allow exactly that, overriding the default sequence name. But as it's not closed yet, I don't think there is a way without hacking.
I haven't used Oracle before, but a quick search suggests that it is possible to create aliases/synonyms for sequences. manage.py sqlall <app> should show you the sequence name Django is expecting. So you probably could just make this an alias for SEQ_GET_NEW_ID.