After I run an async task
tasks.add.apply_async( (10, 10))
I checked the result backends database table celery_taskmeta and noticed the result containing something like gAJLBC4=
I couldn't find in the docs what that result implies and whether I can store the actual result of the function call ( ie, the return value ) in the table as is.
For this instance where I am executing a task which adds two numbers : 10 and 10 , the result column in celery_taskmeta should have 20 as per my understanding ( which is probably wrong ) .
How should I achieve that ?
I'm assuming that result is also serialized? I'm using a redis broker and not clear which configuration I need to set to be able to retrieve the actual return value.
the best way to get the result is not to query the database directly and instead to use the result api
result = tasks.add.apply_async( (10, 10))
result.ready
> True
result.result
> 20
Related
I am trying to call an AzureML UDF from Stream Analytics query and that UDF expects an array of 5 rows and 2 columns. The input data is streamed from an IoT hub and we have two fields in the incoming messages: temperature & humidity.
This would be the 'passthrough query' :
SELECT GetMetadataPropertyValue([room-telemetry], 'IoTHub.ConnectionDeviceId') AS RoomId,
Temperature, Humidity
INTO
[maintenance-alerts]
FROM
[room-telemetry]
I have an AzureML UDF (successfully created) that should be called with the last 5 records per RoomId and that will return one value from the ML Model. Obviously, there are multiple rooms in my stream, so I need to find a way to get some kind of windowing of 5 records Grouped per RoomId. I don't seem to find a way to call the UDF with the right arrays selected from the input stream. I know I can create a Javascript UDF that would return an array from the specific fields, but that would be record/by record, where here I would need this with multiple records that are grouped by the RoomId.
Someone has any insights?
Best regards
After the good suggestion of #jean-sébastien and an answer to an isolated question for the array-parsing, I finally was able to stitch everything together in a solution that builds. (still have to get it to run at runtime, though).
So, the solution exists in using CollectTop to aggregate the latest rows of the entity you want to group by, including the specification of a Time Window.
And the next step was to create the javascript UDF to take that data structure and parse it into a multi-dimensional array.
This is the query I have right now:
-- Taking relevant fields from the input stream
WITH RelevantTelemetry AS
(
SELECT engineid, tmp, hum, eventtime
FROM [engine-telemetry]
WHERE engineid IS NOT NULL
),
-- Grouping by engineid in TimeWindows
TimeWindows AS
(
SELECT engineid,
CollectTop(2) OVER (ORDER BY eventtime DESC) as TimeWindow
FROM
[RelevantTelemetry]
WHERE engineid IS NOT NULL
GROUP BY TumblingWindow(hour, 24), engineid
)
--Output timewindows for verification purposes
SELECT engineid, Udf.Predict(Udf.getTimeWindows(TimeWindow)) as Prediction
INTO debug
FROM TimeWindows
And this is the Javascript UDF:
function getTimeWindows(input){
var output = [];
for(var x in input){
var array = [];
array.push(input[x].value.tmp);
array.push(input[x].value.hum);
output.push(array);
}
return output;
}
I have an REST-ish endpoint that creates a day object and sets its order property to whatever the maximum order is +1. I'm having an issue where calling that endpoint in rapid succession results in some of the days having the same order. How do I solve this?
SQL Query is like so.
insert into "days" ("order", "program_id") values (
(select max(days.order)+1
from "days"
where "days"."program_id" = '5'), '5')
returning *
And it results in something like
{"program_id":5,"id":147,"order":38}
{"program_id":5,"id":150,"order":38}
{"program_id":5,"id":148,"order":38}
{"program_id":5,"id":149,"order":38}
{"program_id":5,"id":151,"order":39}
{"program_id":5,"id":152,"order":40}
{"program_id":5,"id":153,"order":41}
If it helps, I'm on Node (Express) and using Knex and Objection to build my queries for a Postgres database. The JavaScript code is as follows.
json.order = knex.select(knex.raw('max(days.order)+1'))
.from('days')
.where('days.program_id', json.program_id);
return await Days
.query(trx)
.returning('*')
.insert(json);
I'm also using max+1 as I want the order values to increment on a per program basis. So days of a program will have unique orders, but it is possible to have days of different programs with the same order.
Thanks!
You could probably add select ... for update locking to the subquery where you are calculating max+1:
json.order = knex.select(knex.raw('max(days.order)+1'))
.forUpdate() // <-------- lock rows
.from('days')
.where('days.program_id', json.program_id);
So after that no other concurrent connection can read those rows that are used for calculating max until this transaction ends.
I have a TraitQuestion model that have a number of traits or answers - each of which has a value such as 1, 2, 3, or -3105 (for none of the above). I can't change these values because they are required for an external API.
I'd like to order them by the value, but when I do that, the answer with -3105 shows up at the top even though it says "None of the above":
Answer 4 = -3105
Answer 1 = 1
Answer 2 = 2
Answer 3 = 3
Any idea on how we could order this so that it's
1
2
3
-3105?
I'm pretty new to SQL but it seems like I should be able to do something like this:
#trait_question.traits.order('CASE WHEN value AS int > 0 then 1 ELSE 0 END')
But that doesn't work. Any ideas how I can adjust the order call SQL such that it would order them in the correct order?
EDIT:
This is Postgresql 9.4.1 via Amazon AWS
I don't know if this will work for you or not but try this:
#trait_question.traits.order('ABS(value)')
The ABS will change the negative value to positive hence taking the absolute value each time. If your database field is string then you can do it like this as suggested by Tom which worked for him:
#trait_question.traits.order('ABS(CAST(value AS int))')
Hey you can try this way
#trait_question.traits.order("CASE WHEN (value AS integer > 0) THEN value ELSE (value As integer)*(-1) END")
Other wise use
#trait_question.traits.order("abs(value AS integer)")
I'm not sure what exactly you try to do. and I can't make a normal comment, but I'll give it a try.
can you go to your projekt folder run rails dbconsole
and execute
SELECT * FROM traitquestions ORDER BY value
This schould give you all entries ordered by the value, if you only want all possible values, run
SELECT value FROM traitquestions ORDER BY value GROUP BY value
if it works you can just create a method
def get_values
find_by_sql "SELECT value FROM traitquestions ORDER BY value GROUP BY value"
end
and call this from the controller (or where ever you want)
First, I do this select:
SELECT TOP 1 ID FROM Person where Name=${NAME} AND YOB=${YOB};
I need to do SQL request in JMeter:
select * from Person where ID=${ID}, where ${ID} is global variable returned in first select. First select returns collection, but there is only 1 value. How to extract this value and assign it to variable?
As per JDBC Request Sampler Documentation you can refer the value as $A1
A_#=2 (number of rows)
A_1=column 1, row 1
A_2=column 1, row 2
C_#=2 (number of rows)
C_1=column 3, row 1
C_2=column 3, row 2
See The Real Secret to Building a Database Test Plan With JMeter for more tips and tricks on database testing with Apache JMeter.
You can use one of the many post-processor components to extract values from your response. In your case the Regular Expression extractor is likely most suitable.
Ok so I'm having a bit of a learning moment here and after figuring out A way to get this to work, I'm curious if anyone with a bit more postgres experience could help me figure out a way to do this without doing a whole lotta behind the scene rails stuff (or doing a single query for each item i'm trying to get)... now for an explaination:
Say I have 1000 records, we'll call them "Instances", in the database that have these fields:
id
user_id
other_id
I want to create a method that I can call that pulls in 10 instances that all have a unique other_id field, in plain english (I realize this won't work :) ):
Select * from instances where user_id = 3 and other_id is unique limit 10
So instead of pulling in an array of 10 instances where user_id is 3 and you can get multiple instances with the other_id is 5, I want to be able to run a map function on those 10 instances and get back something like [1,2,3,4,5,6,7,8,9,10].
In theory, I can probably do one of two things currently, though I'm trying to avoid them:
Store an array of id's and do individual calls making sure the next call says "not in this array". The problem here is I'm doing 10 individual db queries.
Pull in a large chunk of say, 50 instances and sorting through them in ruby-land to find 10 unique ones. This wouldn't allow me to take advantage of any optimizations already done in the database and I'd also run the risk of doing a query for 50 items that don't have 10 unique other_id's and I'd be stuck with those unless I did another query.
Anyways, hoping someone may be able to tell me I'm overlooking an easy option :) I know this is kind of optimizing before it's really needed but this function is going to be run over and over and over again so I figure it's not a waste of time right now.
For the record, I'm using Ruby 1.9.3, Rails 3.2.13, and Postgresql (Heroku)
Thanks!
EDIT: Just wanted to give an example of a function that technically DOES work (and is number 1 above)
def getInstances(limit, user)
out_of_instances = false
available = []
other_ids = [-1] # added -1 to avoid submitting a NULL query
until other_ids.length == limit || out_of_instances == true
instance = Instance.where("user_id IS ? AND other_id <> ALL (ARRAY[?])", user.id, other_ids).limit(1)
if instance != []
available << instance.first
other_ids << instance.first.other_id
else
out_of_instances = true
end
end
end
And you would run:
getInstances(10, current_user)
While this works, it's not ideal because it's leading to 10 separate queries every time it's called :(
In a single SQL query, it can be achieved easily with SELECT DISTINCT ON... which is a PostgreSQL-specific feature.
See http://www.postgresql.org/docs/current/static/sql-select.html
SELECT DISTINCT ON ( expression [, ...] ) keeps only the first row of
each set of rows where the given expressions evaluate to equal. The
DISTINCT ON expressions are interpreted using the same rules as for
ORDER BY (see above). Note that the "first row" of each set is
unpredictable unless ORDER BY is used to ensure that the desired row
appears first
With your example:
SELECT DISTINCT ON (other_id) *
FROM instances
WHERE user_id = 3
ORDER BY other_id LIMIT 10