I am trying to pull down selected columns from joining two tables and returning the results using api.marshal_list_with. My model definition is as follows:
project_model = api.model('Project', {
'id': fields.String(readOnly=True, description='Unique project id'),
'name': fields.String(required=True, description='Project name'),
'created_user_id': fields.String(description='Id for user that created the project'),
'created_time': fields.String(readOnly=True, description='Time of first creation'),
'updated_time': fields.String(description='Time of last modification'),
'access_type': fields.String(description='Level of access')})
The actual request code is:
#api.marshal_list_with(project_model)
def get(self):
"""
Get details of all projects associated with user
"""
user_id = get_jwt_identity()
projects_l = db.session.query(Project, ProjectUserJunction).join(
ProjectUserJunction).filter(ProjectUserJunction.user_id == user_id).values(
Project.id, Project.name, Project.created_user_id,
Project.created_time, Project.updated_time,
ProjectUserJunction.access_type)
return projects_l
When I make the get call though, the response I get is a bunch of nulls:
{
"id": null,
"name": null,
"created_user_id": null,
"created_time": null,
"updated_time": null,
"access_type": null
}
If I print out the contents of project_l on the server, it's not blanks, so I know that the query is not broken. It may just be something with marshaling that I am missing. How can I resolve this issue?
I guess casting qs to list, should do the trick.
Related
I have the below information from a temp table where one of the columns (i think formatted as json) is having the below information I'm trying to extract the information from that column such as path, brand, color, or activity_group on different columns. split this information.
{
"fragment": null,
"host": null,
"parameters": null,
"path": "\"Fashion\",\"activity_group\":\"Fleece - Hoods / Pants\",\"brand\":\"MICHAEL Michael Kors\",\"budget_curve\":\"Crew Sweaters\",\"category\":\"Clothing\",\"category_id\":\"Clothing\",\"color\":\"Pearl Hethr 036\",\"image_url\":\"https://images.sportsdirect.com/images/products/66003602_l.jpg\",\"is_full_price\":true,\"name\":\"Logo Tape Sweatshirt\",\"price\":58,\"price_excl\":48.33,\"price_excl_gbp\":48.33,\"price_gbp\":58,\"product_discount\":0,\"product_discount_gbp\":0,\"product_id\":\"660036\",\"quantity\":1,\"sku\":\"66003602390\",\"sub_category\":\"Crew Sweaters\",\"subtotal\":48.33,\"subtotal_gbp\":48.33,\"tax\":9.67,\"tax_gbp\":9.67,\"total\":58,\"total_gbp\":58,\"variant\":\"12 (M)\"},{\"activity\":\"Fashion\",\"activity_group\":\"Leggings\",\"brand\":\"MICHAEL Michael Kors\",\"budget_curve\":\"Leggings\",\"category\":\"Clothing\",\"category_id\":\"Clothing\",\"color\":\"Pearl Hthr 036\",\"image_url\":\"https://images.sportsdirect.com/images/products/67601302_l.jpg\",\"is_full_price\":false,\"name\":\"Logo Tape Leggings\",\"price\":50,\"price_excl\":41.67,\"price_excl_gbp\":41.67,\"price_gbp\":50,\"product_discount\":35,\"product_discount_gbp\":35,\"product_id\":\"676013\",\"quantity\":1,\"sku\":\"67601302390\",\"sub_category\":\"Leggings\",\"subtotal\":41.67,\"subtotal_gbp\":41.67,\"tax\":8.33,\"tax_gbp\":8.33,\"total\":50,\"total_gbp\":50,\"variant\":\"12 (M)\"}]",
"port": null,
"query": null,
"scheme": "[{\"activity\""
}
I tried to use parse_url and parse_json however I am not sure I am using this correctly, can someone advise what code instead parse I can use?
table name: order_dupe_check_cleaned
column name: PRODUCTS_NEED_CHECK
This looks like a JSON string. Use json python library to extract the information from the column.
import json
data = '{"fragment": null, "host": null, "parameters": null, "path": "", "Fashion","activity_group":"Fleece - Hoods / Pants","brand":"The North Face","budget_curve":"Full Zip Fleece Tops","category":"Clothing","category_id":"Clothing","color":"JK3 Black","image_url":"https://images.sportsdirect.com/images/products/55435003_l.jpg","is_full_price":true,"name":"Men’s 100 Glacier Full-Zip "scheme": "[{"activity"}'
try:
data_dict = json.loads(data)
brand = data_dict['brand']
color = data_dict['color']
activity_group = data_dict['activity_group']
print(brand)
print(color)
print(activity_group)
except json.decoder.JSONDecodeError as e:
print("There is an error in the JSON string: ", e)
This will print the brand, color, and activity_group values. You can then store the data in separate columns.
I started my adventure with Karate a month ago. I have a simple GET test called getAllCars.feature showing a list of cars currently available:
[
{
"brandName": "BMW",
"id": 1,
"winterTires": false,
"modelName": "X5"
},
{
"brandName": "Opel",
"id": 34,
"winterTires": true,
"modelName": "Insignia"
},
{
"brandName": "Mercedes-Benz",
"id": 36,
"winterTires": true,
"modelName": "GLE Coupe"
},
{
"brandName": "Huydai",
"id": 251,
"winterTires": false,
"modelName": "i30"
}
]
I have to use each id as a parameter for the next feature file, the problem is, the list of cars is dynamic, ids don't repeat and I will have to use this list of ids for several other feature files. I managed to create a helper getCarIds.feature, which creates an array of objects "carId": "#number":
Feature: Get car IDs
Scenario: Get car IDs
* call read('classpath:x/automation/cars/getAllCars.feature')
* def carIds = $response[*].id
* def carFeeder = karate.mapWithKey(carIds, 'carId')
The following getCarParameters.feature has to iterate over the array from getCarIds.feature and pass each id as a parameter to get a response with performance parameters of each car and I don't know how to use each id separately as a parameter (keeping in mind that the list of ids is changing):
Feature: Get parameters of each car
Scenario: Get parameters for each car
* call read('classpath:x/automation/cars/getCarIds.feature')
Given url carUrl + '/carparameters'
And param carId =
When method GET
Then status 200
I managed to do it when passing the values from getCarIds.feature to getCarParameters.feature like described here by adding following line to getCarIds.feature:
* call read('classpath:x/automation/cars/getCarParameters.feature') carFeeder
but several other tests require car ids. I need getCarIds.feature to be reusable, so I would have to retrieve data from feature file, which creates the array with ids, instead of passing it to the GET feature and apparently it isn't so easy. Maybe my approach is completely wrong.
I think this is a valid case for karate.callSingle(): https://github.com/karatelabs/karate#karatecallsingle
So you can actually stick this in any feature and it is guaranteed to execute only once across your test suite. If the data is indeed something used by a majority of your test suite, you could even do this initialization in karate-config.js.
So this should work. First the reusable feature common.feature. Instead of the hard-coded response, you know how to make an actual HTTP request.
#ignore
Feature:
Scenario:
* def response =
"""
[
{
"brandName": "BMW",
"id": 1,
"winterTires": false,
"modelName": "X5"
},
{
"brandName": "Opel",
"id": 34,
"winterTires": true,
"modelName": "Insignia"
}
]
"""
* print 'getting car ids'
* def carIds = response.map(x => ({ id: x.id }))
Note the use of the JS map() function above, which I have started to recommend instead of JsonPath.
And here is a feature that uses the above. This uses the new #setup annotation that makes it easy to "loop": https://github.com/karatelabs/karate#setup
You can try this example quickly, and watch it make 2 requests using a param id from the loop.
Feature:
#setup
Scenario:
* def data = karate.callSingle('call-single-common.feature').carIds
Scenario Outline:
* url 'https://httpbin.org/get'
* param id = id
* method get
Examples:
| karate.setup().data |
There are other ways to loop, refer: https://github.com/karatelabs/karate#data-driven-features
Hello great people of SO!
I hope you all have a good day and have a good health
Note: I'm not good at SQL
Sorry for bad english, but I will try my best to explain my issue
I'm using Laravel v8.x for my app, and after setting up model relationships, events, queues, etc, now I'm working for SQL
ATM, I have 2 Models,
User
Post
Relationships:
User hasMany Post
User belongsToMany User (Block)
User belongsToMany User (Follow)
Post belongsTo User
Database:
5 record for User
2 record for Block
3 records for Post
Table: (Using faker)
users
[
{ id: 1, name: 'Jonathan Beatrice', username: 'kiana.fay', ... },
{ id: 2, name: 'Lacey Kirlin', username: 'kenna.turner', ... },
{ id: 3, name: 'Alexander Schiller', username: 'cassandra95', ... },
{ id: 4, name: 'Daniel Wickozky', username: 'nkoepp', ... },
{ id: 5, name: 'Maymie Lehner', username: 'frami.felton', ... }
]
block
[
{ id: 1, by_id: 1, to_id: 2 }, // User #1 block user #2
{ id: 2, by_id: 4, to_id: 1 } // User #4 block user #1
]
posts
[
{ id: 1, user_id: 2, body: 'Test post', ... },
{ id: 2, user_id: 5, body: 'Lorem ipsum dolor sit amet ...', ... },
{ id: 3, user_id: 4, body: 'ABCD festival soon! ...', ... },
]
Everything works fine and smooth
Now that I want to implement search system, I have a problem, since I'm not good with SQL
Here's my code
SearchController.php
use ...;
use ...;
...
public function posts(Request $request)
{
// For testing purpose
$user = User::with(['userBlocks', 'blocksUser'])->find(1);
// Get all id of user that $user block
// return [2]
$user_blocks = $user->userBlocks->pluck('pivot')->pluck('to_id')->toArray();
// Get all id of user that block $user
// return [4]
$blocks_user = $user->blocksUser->pluck('pivot')->pluck('by_id')->toArray();
// Merge all ids above (must be unique())
// return [2, 4]
$blocks = array_merge($user_blocks, $blocks_user);
// .../search?q=xxx
$query = $request->query('q');
$sql = Post::query();
// Search for posts that has `posts`.`body` LIKE ? ($query)
$sql->where('body', 'LIKE', "%$query%");
// This is where I got confused
$sql->orWhereHas('user', function ($post_user) use ($blocks, $query) {
$post_user
->whereNotIn('id', $blocks) // Exclude posts that has user and their id not in (x, x, x, x, ... ; $block variable above)
->where('name', 'LIKE', "%$query%") // Find user that has name LIKE ? ($query)
->orWhere('username', 'LIKE', "%$query%"); // or Find user that has username LIKE ? ($query)
});
$sql->orderBy('created_at', 'DESC');
$sql->with(['user']);
$posts = $sql->simplePaginate(10, ['*'], 'p');
return $posts;
}
I run the code, .../search?q=e
Note:
All users has alphabet E in their names
And also all posts has alphabet E in their body
We (as User #1), block User #2, and User #4, block us (User #1)
Result: Controller returned all posts
This is the query when I use DB::enableQueryLog() and DB::getQueryLog()
SELECT
*
FROM
`posts`
WHERE `body` LIKE ?
AND EXISTS
(SELECT
*
FROM
`users`
WHERE `posts`.`user_id` = `users`.`id`
AND (
`id` NOT IN (?)
AND `username` LIKE ?
OR `name` LIKE ?
))
ORDER BY `created_at` ASC
LIMIT 11 OFFSET 0
Goal: Search all posts that has body LIKE ?, OR posts that has user; username LIKE ? or name LIKE ? (But also exclude the user we block and the user that block us
Thanks in advance
If there's any unclear explanation, I will edit it A.S.A.P
If I run on my recent laravel install, with my proposed change for one of your issues, version 7.19.1, I get this query:
SELECT
*
FROM
`posts`
WHERE `body` LIKE ?
OR EXISTS <- line of interest
(SELECT
*
FROM
`users`
WHERE `posts`.`user_id` = `users`.`id`
AND (
`id` NOT IN (?)
AND (`username` LIKE ?
OR `name` LIKE ?) <- extra brackets ive added
))
ORDER BY `created_at` ASC
LIMIT 11 OFFSET 0
Have a look at the line of interest, and compare it with the query your version of laravel is running. The AND EXISTS line is being incorrectly generated by laravel. OrWhereHas isnt behaving correctly in your version, I can't find the release number to see where it was fixed.
Id recommend upgrading to latest if possible, but thats not always an option. I've had a dig around, and it looks like the user in this question here encountered a similar problem:
WhereHas() / orWhereHas not constraining the query as expected
You can try moving your $sql->with(['user']); to before you OrWhereHas clause. I'm not sure if that will change it to OR, but its worth a try.
Second thing, I've added whereNested to your OR clause to ensure the precedence is correct, which adds the extra brackets in the query above, as in you dont want:
(`id` NOT IN (1, 2, 3)
AND `name` LIKE % test %)
OR `username` LIKE % test %
Since then it would include your blocked posts in the exists clause.
So final changes look like this, which I think fufills your description:
$sql->with(['user']); //deleted from original position and move here
$sql->where('body', 'LIKE', "%$query%")->whereNotIn('id', $blocks); //additional line
$sql->orWhereHas('ambience', function ($post_user) use ($blocks, $query) {
$post_user
->whereNotIn('id', $blocks);
$post_user->whereNested(function($post_user) use ($query) { //new bit
$post_user->where('name', 'LIKE', "%$query%")
->orWhere('username', 'LIKE', "%$query%");
});
});
I've got an API endpoint POST /data.
The received data is formatted in a certain way which is different from the way I store it in the db.
I'll use geometry type from postgis as an example.
class MyPostgisModel(models.Model):
...
position = models.PointField(null=True)
my_charfield = models.CharField(max_length=10)
...
errors = JSONField() # Used to save the cleaning and validation errors
class MyPostgisSerializer(serializers.ModelSerializer):
class Meta:
model = MyPostgisModel
fields = [
...
"position",
...
"my_charfield",
"errors",
]
def to_internal_value(self, data):
...
# Here the data is coming in the field geometry but in the db, it's called
# position. Moreover I need to apply the `GEOSGeometry(json.dumps(...))`
# method as well.
data["position"] = GEOSGeometry(json.dumps(data["geometry"]))
return data
The problem is that there is not only one field like position but many. And I would like (maybe wrongly) to do like the validate_*field_name* scheme but for cleaning (clean_*field_name*).
There is another problem. In this scheme, I would like to still save the rest of the data in the database even if some fields have raised ValidationError (eg: a CharField that is too long) but are not part of the primary_key/a unique_together constraint. And save the related errors into a JSONField like this:
{
"cleaning_errors": {
...
"position": 'Invalid format: {
"type": "NotAValidType", # Should be "Point"
"coordinates": [
4.22,
50.67
]
}'
...
},
"validating_errors": {
...
"my_charfield": "data was too long: 'this data is way too long for 10 characters'",
...
}
}
For the first problem, I thought of doing something like this:
class BaseSerializerCleanerMixin:
"""Abstract Mixin that clean fields."""
def __init__(self, *args, **kwargs):
"""Initialize the cleaner strategy."""
# This is the error_dict to be filled by the `clean_*field_name*`
self.cleaning_error_dict = {}
super().__init__(*args, **kwargs)
def clean_fields(self, data):
"""Clean the fields listed in self.fields_to_clean before validating them."""
cleaned_data = {}
for field_name in getattr(self.Meta, "fields", []):
cleaned_field = (
getattr(self, "clean_" + field_name)(data)
if hasattr(self, "clean_" + field_name)
else data.get(field_name)
)
if cleaned_field is not None:
cleaned_data[field_name] = cleaned_field
return cleaned_data
def to_internal_value(self, data):
"""Reformat data to put it in the database."""
cleaned_data = self.clean_fields(data)
return super().to_internal_value(cleaned_data)
I'm not sure that's a good idea and maybe there is an easy way to deal with such things.
For the second problem ; catching the errors of the validation without specifying with is_valid() returning True when no primary_key being wrongly formatted, I'm not sure how to proceed.
sorry I have a problem with raw, I'm trying to add up the total sales for each month, but I have an error.
this is my sight.
class TotalSale(ListAPIView):
serializer_class = TotalSaleSerealizer
def get_queryset(self):
queryset = Sale.objects.raw("SELECT 1 id, SUM(totalprice),
to_char(datesale,'yyyy-MM') FROM sales_sale group by
to_char(datesale,'yyyy-MM')")
return queryset
the to_char I am using to change the format of my date and so I can calculate the sales of each month, this query works well when I do it directly in Postgresql, but when I do it in django I do not vote for the correct data.
1,'1197','2018-10'
1,'612','2018-09'
1,'1956','2018-08'
and it's fine I calculate the sum of the sales of each month
But when I do that in Django, this comes to me.
{
"id": 1,
"totalprice": 144,
"datesale": "2018-08-06"
},
{
"id": 1,
"totalprice": 144,
"datesale": "2018-08-06"
},
{
"id": 1,
"totalprice": 144,
"datesale": "2018-08-06"
}
I think the error is for the 1 id, just filter the data of the data that has id 1, my question is why that happens, how can I solve it try to remove the 1 id but another error comes out, how can I fix that problem.
I think that the rest is irrelevant, because the view is the most important, but here is the remaining code.
class TotalSaleSerealizer(ModelSerializer):
class Meta:
model = Sale
fields = ('id','totalprice', 'datesale')
class Sale(models.Model):
id_customer = models.ForeignKey(Customer, null =False, blank=False,
on_delete=models.CASCADE)
id_user = models.ForeignKey(User, null =False, blank=False,
on_delete=models.CASCADE)
datesale = models.DateField(auto_now_add=True)
totalprice = models.IntegerField()
these are the serealizer and the model