Inheriting "exclude" meta parameter from super schema (marshmallow) - marshmallow

I have an hierarchy of objects and an hierarchy of schemas corresponding to them. A schema at an intermediate level of this hierarchy excludes a specific inherited field. I expect that schemas inheriting from it will "inherit" this exclusion, but this does not seem to be the case if they add their own excluded fields in their Meta classes:
from marshmallow import fields
from marshmallow.schema import Schema
class AncestorSchema(Schema):
a = fields.Str()
b = fields.Str()
class IntermediateSchema(AncestorSchema):
c = fields.Str()
class Meta:
exclude = ('b',)
class FinalSchema(IntermediateSchema):
d = fields.Str()
class Meta:
exclude = ('c',)
value = dict(
a="Field A",
b="Field B",
c="Field C",
d="Field D"
)
print(IntermediateSchema().dump(value).data)
>>> {'c': 'Field C', 'a': 'Field A'}
print(FinalSchema().dump(value).data)
>>> {'d': 'Field D', 'a': 'Field A', 'b': 'Field B'}
In the example above, FinalSchema inherits from IntermediateSchema (which excludes field b) and excludes field c in its own Meta class. An expected behavior would be that the resulting schema will exclude both b and c, but actually it excludes only c.
Of course it's possible to manually include superschema's excluded fields the inheriting schema's excluded fields, but that's not the point of inheritance, and besides, it's cumbersome.
I wonder whether the desired behavior can be achieved in an elegant way, or whether the current behavior of schema inheritance is in fact a bug.
Inspecting the source code of marshmallow shows that inheritance of data from superschemas' meta classes is at least partially supported (namely, ordered Meta option value is inherited from superschemas).

The other answer doesn't work, because self is not defined. I've found a solution that does work.
from marshmallow import fields
from marshmallow.schema import Schema
class AncestorSchema(Schema):
a = fields.Str()
b = fields.Str()
class IntermediateSchema(AncestorSchema):
c = fields.Str()
class Meta:
exclude = ('b',)
class FinalSchema(IntermediateSchema):
d = fields.Str()
def __init__(self, *args, **kwargs):
self.opts.exclude += ('c',)
super().__init__(*args, **kwargs)

You also need to specify the base class for the Meta class. You also need to use some sort of reflection to get the value from the base class and append to it.
from marshmallow import fields
from marshmallow.schema import Schema
class AncestorSchema(Schema):
a = fields.Str()
b = fields.Str()
class IntermediateSchema(AncestorSchema):
c = fields.Str()
class Meta:
exclude = ('b',)
class FinalSchema(IntermediateSchema):
d = fields.Str()
def __init__(self, *args, **kwargs):
self.opts.exclude += ('c',)
super().__init__(*args, **kwargs)

Related

Marshmallow - Sort field values by declared order

I've read the docs and searched this site but cannot seem to find a solution to sorting field values by the order in which they are declared. The docs state that adding ordered = True to the class Meta will solve this problem -
class MySchema(Schema):
class Meta:
ordered = True
However, I am not using class Meta in my schema. My schema simply looks like -
class MySchema(Schema):
id = fields.Integer()
name = fields.Str()
category = fields.Str()
So in this situation, how and where would I set ordered = True? Thanks!
I solved the issue by changing my schema class to -
class MySchema(Schema):
class Meta:
ordered = True
id = fields.Integer()
name = fields.Str()
category = fields.Str()
and then also adding JSON_SORT_KEYS=False to my app's config.py file.

SQL Query logic to Django ORM Query logic

I have tried to think about how the following SQL query would be structured as a Django ORM query but I have had no luck in my multiple attempts. Can anyone help?
SELECT targets_genetarget.gene, count(targets_targetprediction.gene) as total
FROM targets_genetarget
LEFT OUTER JOIN targets_targetprediction on targets_targetprediction.gene =
targets_genetarget.gene
WHERE list_name LIKE %s
GROUP BY targets_genetarget.gene
class GeneTarget(models.Model):
list_name = models.CharField(max_length=100)
gene = models.CharField(max_length=50)
date_added = models.DateField(auto_now=True)
class Meta:
unique_together = (('list_name', 'gene'),)
def __str__(self):
return self.list_name
class TargetPrediction(models.Model):
specimen_id = models.CharField(max_length=100)
patient_peptide = models.ForeignKey(Peptide, on_delete=models.CASCADE, verbose_name="Peptide", related_name="predictions")
allele = models.ForeignKey(Allele, on_delete=models.CASCADE, verbose_name="Allele", related_name="predictions")
gene = models.CharField(max_length=50)
class Meta:
unique_together = (('specimen_id', 'patient_peptide', 'allele', 'gene'),)
def get_absolute_url(self):
return f'/samples/specid-{self.specimen_id}'
def __str__(self):
return (f'Specimen: {self.specimen_id} Peptide: {self.patient_peptide} Allele: {self.allele} Gene: {self.gene} ')
There's nothing stopping you declaring the TargetPrediction.gene field as a foreign key using the to_field attribute, so you wouldn't need to change the data at all:
class TargetPrediction(models.Model):
...
gene = models.ForeignKey("GeneTarget", to_field="gene")
Now your query simply becomes:
GeneTarget.objects.filter(list_name="whatever").values("gene").annotate(total=Count("targetprediction"))

Writeable Nested Serializers with PrimaryKeyRelatedField

I'm having a problem creating (nested) serializers for the following models:
class Chapter(Model):
uuid = UUIDField(primary_key=True, editable=False)
description = TextField(blank=True)
class Section(Model):
uuid = UUIDField(primary_key=True, editable=False)
description = TextField(blank=True)
chapter = ForeignKey(Chapter, related_name='sections', on_delete=CASCADE)
Here is the serializer for the Chapter model with the create method:
class ChapterSerializer(ModelSerializer):
uuid = UUIDField(read_only=False)
sections = SectionSerializer(many=True, read_only=False)
class Meta:
model = Chapter
fields = ('uuid', 'description', 'sections')
def create(self, validated_data):
sections = validated_data.pop('sections', [])
chapter = Chapter.objects.create(**validated_data)
for section in sections:
section.update({'chapter': chapter})
section = Section.objects.create(**section)
return chapter
And here are two slightly different variants of the Section serializer:
class SectionSerializer(ModelSerializer):
uuid = UUIDField(read_only=False)
chapter = PrimaryKeyRelatedField(read_only=true)
class Meta:
model = Section
fields = ('uuid', 'description', 'chapter')
class SectionSerializer(ModelSerializer):
uuid = UUIDField(read_only=False)
chapter = PrimaryKeyRelatedField(queryset=Chapter.objects.all())
class Meta:
model = Section
fields = ('uuid', 'description', 'chapter')
Now if I try to create a new Chapter with nested Sections and the PrimaryKeyRelatedField in the Section serializer has the queryset parameter set, I get the following error:
'sections': [{'chapter': ['Invalid pk "x" - object does not exist.']}]
If I use the variant with the read_only=true parameter, creating a new Chapter with nested Sections works, but now I can no longer create a new Section (Chapter field is set to an existing Chapter UUID) because the chapter field is removed while validating the input data.
django.db.utils.IntegrityError: null value in column "chapter_id" violates not-null constraint
I think I could solve this by duplicating the SectionSerializer, but this seems to be a very crude solution ...
Anyone knowing of an better approach?
OK, answering my own question:
I finally found a way to get a "nestable" and a "writeable" version of the SectionSerializer without code duplication:
class NestedSectionSerializer(ModelSerializer):
uuid = UUIDField(read_only=False)
class Meta:
model = Section
exclude = ('chapter', )
class SectionSerializer(NestedSectionSerializer):
url = serializers.HyperlinkedIdentityField(view_name='section-detail')
class Meta:
model = Section
fields = '__all__'
class ChapterSerializer(ModelSerializer):
url = serializers.HyperlinkedIdentityField(view_name='chapter-detail')
uuid = UUIDField(read_only=False)
sections = NestedSectionSerializer(many=True, read_only=False)
class Meta:
model = Chapter
fields = '__all__'
def create(self, validated_data):
sections = validated_data.pop('sections', [])
chapter = Chapter.objects.create(**validated_data)
for section in sections:
section.update({'chapter': chapter})
section = Section.objects.create(**section)
return chapter
The key here ist to explicitly exclude the reverence to chapter in the "nested" serializer but add it again in the "writeable" serializer. This even scales to deeply nested structures!
And by using inheritance I can avoid most code duplication.
I just pulled my hair about this very same question.
The answer is actually in the DRF documentation: you need to explicitely define the queryset argument for the PrimaryKeyRelatedField if you want them to be writable.
In your case, only the second variant of SectionSerializer is necessary.

Serialization of item's fields by different name

Is there a way to have scrapy fields to be serialized under a different name? For example field 'product_name' would become 'product name'.
class PropertyItem(Item):
product_name = Field()
then
l.add_xpath('product_name','//[#id="some_id"]/text()')
will be serialized as 'product_name': "some value", while i want it to be 'some other name': "some value"
Thanks
I'm not sure if I completely understand you but you can always use item pipelines to edit and change the items your spider returns.
For example, you can do something like this:
class FooPipeline(object):
def process_item(self, item, spider):
new_value = item['product_name'] + ' new name'
del item['product_name']
item['some other name'] = new_value
return item
By default scrapy.Item fields are static and only defined fields can be set. You can avoid this by overriding __setitem__() magic method:
class TestItem(scrapy.Item):
name = scrapy.Field()
def __setitem__(self, key, value):
self._values[key] = value
And result:
t = TestItem()
t['name2'] = 'one'
print(t)
>>> {'name2': 'one'}
# even though name2 is not defined
If you define additional fields in the __init__ method or your Item class, you are less restricted in choosing their names:
class TestItem(scrapy.Item):
name = scrapy.Field()
def __init__(self):
super().__init__()
self.fields["product name"] = scrapy.Field()

Django Rest Framework - Could not resolve URL for hyperlinked relationship using view name "field-detail"

I know that this inconvenient is very presented, may be that I need learn more about of serializer relationships
I have the following model:
class Field(models.Model):
FIELD_TYPE_NATURE = 'Grama natural'
FIELD_TYPE_ARTIFICIAL = 'Grama sintetica'
FIELD_TYPE_CHOICES = (
(FIELD_TYPE_NATURE, u'Grama natural'),
(FIELD_TYPE_ARTIFICIAL, u'Grama sintetica'),
)
MODALITY_11 = 'Fútbol 11'
MODALITY_8 = 'Fútbol 8'
MODALITY_CHOICES = (
(MODALITY_11, u'Fútbol 11'),
(MODALITY_8, u'Fútbol 8'),
)
name = models.CharField(
max_length=150,
unique=True,
db_index=True,
primary_key=True,
)
field_type = models.CharField(
choices=FIELD_TYPE_CHOICES,
default=False,
blank=False,
max_length=20,
verbose_name=('Tipo de material/grama de la cancha')
)
modality = models.CharField(
max_length=40,
blank=False,
verbose_name='Modalidad'
)
photo = models.ImageField(upload_to='fields', blank=True, null=True)
location = models.CharField(max_length=150, blank=False)
def __str__(self):
return '%s %s %s' % (self.name, self.field_type, self.location)
My serializer is the following:
class FieldSerializer(serializers.HyperlinkedModelSerializer):
#url = serializers.HyperlinkedIdentityField(view_name='field-detail',)
class Meta:
model = Field
fields = ('url', 'name','field_type','modality','photo','location')
My viewset is:
class FieldViewSet(viewsets.ModelViewSet):
queryset = Field.objects.all()
serializer_class = FieldSerializer
This is my router:
router = routers.DefaultRouter()
router.register(r'fields', FieldViewSet)
And my url:
...
url(r'^api/', include(router.urls)),
...
When I go to the http://localhost:8000/api/fields/ url I get the following message:
File "/home/bgarcial/.virtualenvs/fuupbol2/lib/python3.5/site-packages/rest_framework/relations.py", line 386, in to_representation
raise ImproperlyConfigured(msg % self.view_name)
django.core.exceptions.ImproperlyConfigured: Could not resolve URL for **hyperlinked relationship using view name "field-detail". You may have failed to include the related model in your API, or incorrectly configured the `lookup_field` attribute on this field.**
[11/Nov/2016 16:39:53] "GET /api/fields/ HTTP/1.1" 500 187477
When I use HyperlinkedIdentityField in my FieldSerializer class:
class FieldSerializer(serializers.HyperlinkedModelSerializer):
url = serializers.HyperlinkedIdentityField(view_name='field-detail',)
class Meta:
model = Field
fields = ('url', 'name','field_type','modality','photo','location')
I follow getting the same error.
Althought when I go to the url http://localhost:8000/api/fields/ I want get is a list of my objects, then is possible that I should put:
url = serializers.HyperlinkedIdentityField(view_name='field-list',)
?
I use HyperlinkedIdentityField according to:
This field can be applied as an identity relationship, such as the
'url' field on a HyperlinkedModelSerializer. It can also be used for
an attribute on the object.
I put the field-list in my view_name attribute and I get the error related
Could not resolve URL for hyperlinked relationship using view name "field-list". You may have failed to include the related model in your API, or incorrectly configured the `lookup_field` attribute on this field.
I don't understand the situations when I should use view_name attribute in relation to if I wnt get a list objects, a object detail and so ... although here explain something about it.
When I should use HyperlinkedModelSerializer and ModelSerializer