How to check the type of a document element (sub document vs list)? "ReqlServerCompileError: Variable name not found in: var_1" - rethinkdb-python

I extend the RethinkDb API by providing some extra functions.
For example I simplify the expression
site_ids = r.table('periods')\
['regions']\
.concat_map(lambda row: row['sites'])\
['id']
to
site_ids = f['periods']\
.unwind('regions.sites.id')
using a custom unwind method that is able to resolve a path of nested document elements. If an item in the given path is a list, its entries are concatenated with concat_map. Otherwise the item is accessed with bracket notation:
def unwind(self, path):
items = path.split('.')
cursor = self._cursor
for item in items:
is_list = isinstance(cursor[item].run().next(), list)
if is_list:
cursor = cursor.concat_map(lambda row: row[item])
else:
cursor = cursor[item]
return self.wrap(self._f, cursor)
=> How can I improve the type check to find out if an element is a list? The check should not require an extra .run() and it should work in main queries as well as in sub queries.
My current implementation with the expression
is_list = isinstance(cursor[item].run().next(), list)
works fine in "main queries" like
result = f['periods'] \
.unwind('regions.sites.plants.product.process.technologies')\
.populate_with('periods', 'technologies')\
.sum('specific_cost_per_year') \
.run()
It does not work in sub queries, e.g. inside a mapping function:
def period_mapper(period):
return {
'year': period['start'],
'site_ids': f.wrap(period).unwind('regions.sites.id')
}
f.table('periods')\
.map(period_mapper)\
.run()
I get the error
rethinkdb.errors.ReqlServerCompileError: Variable name not found in:
var_1['regions']
^^^^^
because I am not able to .run() a query on the passed variable argument "period".
I tried to replace the if-then-else condition with r.branch but that did not help.
=> How can I choose an operator based on the type of the current cursor content in a better way?
Code of my selection class that wraps a RethinkDb cursor:
from rethinkdb.ast import RqlQuery
# needs to inherit from RqlQuery for the json serialization to work
class AbstractSelection(RqlQuery):
def __init__(self, f, cursor):
self._f = f
self._cursor = cursor
def __getitem__(self, identifier):
cursor = self._cursor[identifier]
return self.wrap(self._f, cursor)
def __repr__(self):
return self._cursor.__repr__()
def __str__(self):
return self._cursor.__str__()
def build(self):
return self._cursor.build()
#property
def _args(self): # required for json serialization
return self._cursor._args
#property
def optargs(self): # required for json serialization
return self._cursor.optargs
def wrap(self, r, cursor):
raise NotImplemented('Needs to be implemented by inheriting class')
def unwind(self, path):
items = path.split('.')
cursor = self._cursor
for item in items:
is_list = isinstance(cursor[item].run().next(), list)
if is_list:
cursor = cursor.concat_map(lambda row: row[item])
else:
cursor = cursor[item]
return self.wrap(self._f, cursor)
def pick(self, path, query):
return self.unwind(path).get(query)
def populate(self, collection_name, path):
return self.map(lambda identifier:
self._f[collection_name]
.pick(path, {'id': identifier})
)
def get(self, query):
cursor = self._cursor.filter(query)[0]
return self.wrap(self._f, cursor)
def to_array(self):
return [item for item in self._cursor]

I managed to use type_of in combination with branch. Accessing the item with bracket notation returns a STREAM and I had to get the first item with [0] before using type_of to check for the 'ARRAY' type. This also works if the property is not an array:
def unwind(self, path):
items = path.split('.')
cursor = self._cursor
r = self._f._r
for item in items:
cursor = r.branch(
cursor[item][0].type_of() == 'ARRAY',
cursor.concat_map(lambda row: row[item]),
cursor[item]
)
return self.wrap(self._f, cursor)

Related

Groovy : Class.forName().newInstance() error

I have this following method in which I return a List<ImField> object using the List<GPathResult> filteredList. I perform filteredList.each closure where I generate
class at runtime and assign it a static type of ImField.
static List<ImField> getFields(GPathResult root,String fieldClass, String fieldType){
List<GPathResult> filteredList = root.children().findAll{
XMLSlurperUtil.name(it as GPathResult) == fieldType
} as List<GPathResult>
List<ImField> fields = []
filteredList.each{GPathResult it, int index ->
fields.add(Class.forName(fieldClass).newInstance() as ImField)
fields[index].set(it)
}
fields
}
The function call would look like so :
ImStageUtil.getFields(root, ImFieldFactory.SOURCE_FIELD, ImParserConstants.SOURCE_FIELD)
where ImFieldFactory.SOURCE_FIELD = "com.dto.fields.SourceField"
and ImParserContants.SOURCE_FIELD = "SOURCEFIELD"
the error occurs at the .each closure line:
No signature of method: com.extractor.ImStageUtil$_getFields_closure11.doCall() is applicable for argument types: (groovy.util.slurpersupport.NodeChild) values: []
Possible solutions: doCall(groovy.util.slurpersupport.GPathResult, int), findAll(), findAll(), isCase(java.lang.Object), isCase(java.lang.Object)
groovy.lang.MissingMethodException: No signature of method: com.extractor.ImStageUtil$_getFields_closure11.doCall() is applicable for argument types: (groovy.util.slurpersupport.NodeChild) values: []
Possible solutions: doCall(groovy.util.slurpersupport.GPathResult, int), findAll(), findAll(), isCase(java.lang.Object), isCase(java.lang.Object)
I've tried to create a similar script to your example, there are two things I had to modify (if your filteredList is not empty, which you need to check first):
1- You need to use collect() after the findAll{} closure, this allows you to collect all entries and add them to your filteredList.
2- You're using .each{} and you're providing a List along with the index, this should be replaced by .eachWithIndex{} because the first one doesn't expect an index.
Here is a simplified version of your code:
import groovy.util.slurpersupport.GPathResult
def text = '''
<list>
<technology>
<name>Groovy</name>
</technology>
</list>
'''
def list = new XmlSlurper().parseText(text)
def List getFields(GPathResult root,String fieldClass, String fieldType){
List<GPathResult> filteredList = root.children().findAll{
//println(it)
it != null
}.collect() as List<GPathResult>
println('list: ' + filteredList.getClass() + ', ' + filteredList.size())
filteredList.eachWithIndex{GPathResult it, int index ->
println('it: ' + it)
}
}
getFields(list, '', '')
This last example doesn't raise any exception for me.
Hope this helps.

Airflow: Best way to pass BigQuery result as XCom [duplicate]

I'm using Airflow 1.8.1 and I want to push the result of a sql request from PostgreOperator.
Here's my tasks:
check_task = PostgresOperator(
task_id='check_task',
postgres_conn_id='conx',
sql="check_task.sql",
xcom_push=True,
dag=dag)
def py_is_first_execution(**kwargs):
value = kwargs['ti'].xcom_pull(task_ids='check_task')
print 'count ----> ', value
if value == 0:
return 'next_task'
else:
return 'end-flow'
check_branch = BranchPythonOperator(
task_id='is-first-execution',
python_callable=py_is_first_execution,
provide_context=True,
dag=dag)
and here is my sql script:
select count(1) from table
when i check the xcom value from check_task it retrieves none value.
If i'm correct, airflow automatically pushes to xcom when a query returns a value. However, when you look at the code of the postgresoperator you see that it has an execute method that calls the run method of the PostgresHook (extension of dbapi_hook). Both methods do not return anything, as such it pushes nothing to xcom.
What we did to fix this is create a CustomPostgresSelectOperator, a copy of the PostgresOperator, but instead of 'hook.run(..)' do 'return hook.get_records(..)'.
Hope that helps you.
Finally, I created a new Sensor ExecuteSqlOperator in the plugin manager under $AIRFLOW_HOME/plugins.
I used CheckOperator as an example and I modified the returned value: the basic running of this operator was exactly the reverse of what I needed.
Here's the of the default ExecuteSqlOperator:
CheckOperator
and here is my customized SqlSensor: ReverseSqlSensor
class SqlExecuteOperator(BaseOperator):
"""
Performs checks against a db. The ``CheckOperator`` expects
a sql query that will return a single row.
Note that this is an abstract class and get_db_hook
needs to be defined. Whereas a get_db_hook is hook that gets a
single record from an external source.
:param sql: the sql to be executed
:type sql: string
"""
template_fields = ('sql',)
template_ext = ('.hql', '.sql',)
ui_color = '#fff7e6'
#apply_defaults
def __init__(
self, sql,
conn_id=None,
*args, **kwargs):
super(SqlExecuteOperator, self).__init__(*args, **kwargs)
self.conn_id = conn_id
self.sql = sql
def execute(self, context=None):
logging.info('Executing SQL statement: ' + self.sql)
records = self.get_db_hook().get_first(self.sql)
logging.info("Record: " + str(records))
records_int = int(records[0])
print (records_int)
return records_int
def get_db_hook(self):
return BaseHook.get_hook(conn_id=self.conn_id)

Odoo 10 selection fields value

How can i get selection fields value in odoo 10?
def compute_default_value(self):
return self.get_value("field")
I tried this,
def compute_default_value(self):
return dict(self._fields['field'].selection).get(self.type)
Also tried this,but it is not working.
Please help me, i could not find the solution.
Thank you.
You can do this in a following manner:
self._fields['your_field']._desription_selection(self.env)
This will return the selection list of pairs (value, label).
If you just need possible values, you can use get_values method.
self._fields['your_field'].get_values(self.env)
But it's not a common way. Most of the time people define selections differently and then use those definitions. For example, I commonly use classes for those.
class BaseSelectionType(object):
""" Base abstract class """
values = None
#classmethod
def get_selection(cls):
return [(x, cls.values[x]) for x in sorted(cls.values)]
#classmethod
def get_value(cls, _id):
return cls.values.get(_id, False)
class StateType(BaseSelectionType):
""" Your selection """
NEW = 1
IN_PROGRESS = 2
FINISHED = 3
values = {
NEW: 'New',
IN_PROGRESS: 'In Progress',
FINISHED: 'Finished'
}
You can use this class wherever you want, just import it.
state = fields.Selection(StateType.get_selection(), 'State')
And it's really handy to use those in the code. For example, if you want to do something on a specific state:
if self.state == StateType.NEW:
# do your code ...
I don't get the question fully, but let me try to answer. Why not just define the selection as method and use it for both situations:
from datetime import datetime
from odoo import models, fields
class MyModel(models.Model):
_name = 'my.model'
def month_selection(self):
return [(1, 'Month1'), (2, 'Month2')]
def compute_default_value(self):
selection = self.month_selection()
# do whatever you want here
month = fields.Selection(
selection=month_selection, string='Month',
default=datetime.now().month, required=True)

Odoo: Access field by it's name (given as string)

I have a model, where I want to access a field, given by a string. Example:
def test(self):
field = 'name'
name = getattr(self, field)
This works fine - name is set to self.name. But then I want to access a related field:
def test2(self):
field = 'partner_id.name'
name = getattr(self, field)
That doesn't work (because 'partner_id.name' does not exist on self). Any idea how to do it right?
getattr doesn't support the dot notation, only simple attribute names. You can however create a simple function that does:
def getfield(model, field_name):
value = model
for part in field_name.split('.'):
value = getattr(value, part)
return value
You would use it like this:
def test2(self):
field = 'partner_id.name'
name = getfield(self, field)
You need to use the object that contain partner_id.name
def test2(self):
field = 'name'
object = self.pool.get('res.partner').browse(cr, uid, self.partner_id.id)#v7
#object = self.env['res.partner'].browse(self.partner_id.id)#v8
name = getattr(object, field)
I also came across another solution, inspired by the mail template system:
from openerp.tools.safe_eval import safe_eval as eval
def test2(self):
field = 'partner_id.name'
field = 'object.' + field
name = eval(field, {'object': self})

What do I return for function field value?

I have a function field, but I don't know what should the function return.
Here's my code:
the function:
def _property_expense_preset_expenses(self, cr, uid, ids, expenses, arg, context):
spus = self.browse(cr, uid, ids)
_spu = False
for spu in spus:
_spu = spu
if(_spu):
expenses_acc = {}
property_expense_presets = _spu.property_expense_presets
for property_expense_preset in property_expense_presets:
expenses = property_expense_preset.expense_preset.expenses
for expense in expenses:
expenses_acc[expense.id] = expense
return expenses_acc
else:
return {}
The field definition:
'expenses' : fields.function(
_property_expense_preset_expenses,
type='one2many',
obj="property.expense",
method=True,
string='Expenses'
),
The code above doesn't work, it raises an error : KeyError: 788
Like all function fields, it must return a dictionary with an entry and value for every ID you get passed in ids, although your value can be False, None, []
In your case your functional field is declared as a one2many type which means your functional field must return a dictionary with an entry per id and the value, a list of integers that represent the ids of the related table, in your case, property.expense.
A very common pattern is:
def _property_expense_preset_expenses(self, cr, uid, ids, field, arg, context = None):
res = {}
for spu in self.browse(cr, uid, ids, context = context):
res[spu.id] = []
for preset in spu.property_expense_presets:
res[spu.id].extend([x.id for x in preset.expense_preset.expenses])
return res
Assuming ids contains 1,2,3 you will get a result of
{1: [...], 2: [...], 3: []}
Where each list contains the integer ids of the expenses or an empty list if there are none.
As a general comment, I note your code doesn't default the context argument to None or pass the context as a named argument to the browse method - it is important to do both.
What is Function Field? What return this functional field function?
In OpenERP functional field is a field that return the calculated/logical value that value store into table. That value you can not get directly. that's why we want to use function field and return some value.
When you inserting data into object model functional field call every time defined function and that function logic code whatever you made and return that functional field value.
Example
class me.branch(osv.osv):
_name = "me.branch"
_order = 'name'
def _get_branch_name(self, cr, uid, ids, field_name, arg, context=None):
r = {}
for branch in self.browse(cr, uid, ids, context=context):
r[branch.id] = branch.name.split('/')[-1]
return r
_columns = {
'name': fields.char('Name', required=True),
'branch_name': fields.function(_get_branch_name, type='char', string='Branch', readonly=1, store=True),
}
Above example code branch name (eg. saas/demo/rarone ) already exist in another table.
But I want to get that branch name only last slash (/) after string (rarone) and store into this table only.