I have a collection of resources. An incoming process would require a set of resources. For example, let's say I have resources A, B, C, and D in the collection and a process comes along asking for resource A, B and D; I would like the process to stop until A, B and D become available.
This of course is a trivial example. I would like to know if one can generate a list of Resource.request() events and then use yield and '&' or some other method to wait until all of them are granted (i.e. all events are triggered).
Here's what I'm doing....
req1 = A.request()
req2 = B.request()
req3 = D.request()
yield req1 & req2 & req3
Here's what I would like to do....
list_of_resources_needed = [A,B,D]
req_list = []
for r in list_of_resources_needed:
req_list.append(r.request())
yield * All events in req_list *
You can use the AllOf or AnyOf events, for which the Environment provides shortcuts: http://simpy.readthedocs.org/en/latest/api_reference/simpy.core.html#simpy.core.Environment.all_of
Related
I have "a parent" A type which contains "a child" B type.
This is a simplified version of the main data structures i have in my app.
A and B and A_id and B_id are all separate elm modules.
If i can make this simplification work, then maybe is easier to solve my actual problem.
Basically my problem is how to create a fuzzer for A.
with the condition that both A_id and B_id .. need to share the same A_id.
type A
= A { id : A_id
, b : B -- A contains B.
}
type A_id
= A_id String
type B
= B { id : B_id } -- B contains B_id
type B_id
= B_id ( A_id, String ) -- B_id contains A_id. The exact same A_id as its parent A
a_idFuzzer : Fuzzer A_id
a_idFuzzer =
Fuzz.string
|> Fuzz.map A_id
aFuzzer : Fuzzer A
aFuzzer =
a_idFuzzer
|> Fuzz.map
(\a_id ->
bFuzzer a_id
|> Fuzz.map
(\b ->
-- here i just need the b,
-- but in reality i need more children
-- for assembling the A data structure.
-- i need a C and D with a cFuzzer and a dFuzzer...
-- and both C and D depend on having the same A_id value.
-- like B does.
A
{ id = a_id
, b = b
}
)
)
-- im passing A_id as an argument since is generated only once on the parent ( A )
-- and is shared with this B child.
bFuzzer : A_id -> Fuzzer B
bFuzzer a_id =
Fuzz.string
|> Fuzz.map (\s -> B_id ( a_id, s ))
|> Fuzz.map (\id -> B { id = id })
So how to create this Fuzzer A?
For the code above i get the Fuzzer (Fuzzer A) error as opposed to Fuzzer A.
In my actual app i get the more complicated error:
Fuzzer ( Fuzzer ( Fuzzer ( Fuzzer Exchange ))) vs Fuzzer Exchange.
I basically need to flatten it with andThen - but no such function exists in the fuzz elm test package - for some not so obvious reason.
What i tried:
I'm battling this problem for 3 days - somebody in slack suggested that andthen was removed on purpose and i'm supposed to use the custom fuzzer - i learned deeper how shrinkers work (i didn't knew them before) and how to use Fuzz.custom just to test if they are right.
Fuzz.custom needs a generator and a shrinker.
I can build the generator and generate everything i need, but i can't build shrinkers - since the B and A and C and D.. so on are all opaque data structures - in their own module - so i need to get all their properties with getters - in order to shrink them.
So for the example above - to shrink B i need to extract the b_id and run it trough a shrinker.. and then put that back into B by creating a new B - using the public api for B.. and I don't have public getter api for all the properties I keep on B, C , D etc.. and it just seems wrong to do it this way ( to add getters that i don't need in the app - just for testing purposes.. )
All this mess because andThen on the fuzz module was removed... but maybe there is a way, maybe they were right - and i'm not seeing the solution.
Link to the fuzzer module: here
So how to build a fuzzer for A datatype?
Any ideas how to deal with this nested fuzzers? How to flatten them back to one level?
Or to phrase it differently, How to build fuzzes that depend on each other like above? (an example i have on my mind would be - like running an http request that depends on another http request completing before it can start - since it needs the data form the previous request .. this model is seen thought functional programming and is usually done with andThen or bind or stuff.)
Any insight is appreciated. Thanks :)
I can build the generator and generate everything i need, but i can't build shrinkers
Then don't bother. Pass Shrink.noShrink to Fuzz.custom. The only disadvantage will come when you have a test that fails, and you will be given several large values of type A rather than (ideally) one small one.
As you work with your complex type, you'll get a better sense for how to shrink its values that cause test failures into "smaller" values that still cause test failures. For that matter, you'll get better at generating "interesting" values that find test failures.
In the next major release of elm-test (timeline not set), there will considerable improvements to Shrinkers, including better docs, the removal of lazy lists in favor of regular Elm lists, and renaming to "Simplifier".
Considering I have the following relationships:
class House(Model):
name = ...
class User(Model):
"""The standard auth model"""
pass
class Alert(Model):
user = ForeignKey(User)
house = ForeignKey(House)
somevalue = IntegerField()
Meta:
unique_together = (('user', 'property'),)
In one query, I would like to get the list of houses, and whether the current user has any alert for any of them.
In SQL I would do it like this:
SELECT *
FROM house h
LEFT JOIN alert a
ON h.id = a.house_id
WHERE a.user_id = ?
OR a.user_id IS NULL
And I've found that I could use prefetch_related to achieve something like this:
p = Prefetch('alert_set', queryset=Alert.objects.filter(user=self.request.user), to_attr='user_alert')
houses = House.objects.order_by('name').prefetch_related(p)
The above example works, but houses.user_alert is a list, not an Alert object. I only have one alert per user per house, so what is the best way for me to get this information?
select_related didn't seem to work. Oh, and surely I know I can manage this in multiple queries, but I'd really want to have it done in one, and the 'Django way'.
Thanks in advance!
The solution is clearer if you start with the multiple query approach, and then try to optimise it. To get the user_alerts for every house, you could do the following:
houses = House.objects.order_by('name')
for house in houses:
user_alerts = house.alert_set.filter(user=self.request.user)
The user_alerts queryset will cause an extra query for every house in the queryset. You can avoid this with prefetch_related.
alerts_queryset = Alert.objects.filter(user=self.request.user)
houses = House.objects.order_by('name').prefetch_related(
Prefetch('alert_set', queryset=alerts_queryset, to_attrs='user_alerts'),
)
for house in houses:
user_alerts = house.user_alerts
This will take two queries, one for houses and one for the alerts. I don't think you require select related here to fetch the user, since you already have access to the user with self.request.user. If you want you could add select_related to the alerts_queryset:
alerts_queryset = Alert.objects.filter(user=self.request.user).select_related('user')
In your case, user_alerts will be an empty list or a list with one item, because of your unique_together constraint. If you can't handle the list, you could loop through the queryset once, and set house.user_alert:
for house in houses:
house.user_alert = house.user_alerts[0] if house.user_alerts else None
The system I'm building has smart groups. By smart groups, I mean groups that update automatically based on these rules:
Include all people that are associated with a given client.
Include all people that are associated with a given client and have these occupations.
Include a specific person (i.e., by ID)
Each smart groups can combine any number of these rules. So, for example, a specific smart list might have these specific rules:
Include all people that are associated with client 1
Include all people that are associated with client 5
Include person 6
Include all people associated with client 10, and who have occupations 2, 6, and 9
These rules are OR'ed together to form the group. I'm trying to think about how to best store this in the database given that, in addition to supporting these rules, I'd like to be able to add other rules in the future without too much pain.
The solution I have in mind is to have a separate model for each rule type. The model would have a method on it that returns a queryset that can be combined with other rules' querysets to, ultimately, come up with a list of people. The one downside of this that I can see is that each rule would have its own database table. Should I be concerned about this? Is there, perhaps, a better way to store this information?
Why not use Q objects?
rule1 = Q(client = 1)
rule2 = Q(client = 5)
rule3 = Q(id = 6)
rule4 = Q(client = 10) & (Q(occupation = 2) | Q(occupation = 6) | Q(occupation = 9))
people = Person.objects.filter(rule1 | rule2 | rule3 | rule4)
and then store their pickled strings into the database.
rule = rule1 | rule2 | rule3 | rule4
pickled_rule_string = pickle.dumps(rule)
Rule.objects.create(pickled_rule_string=pickled_rule_string)
Here are the models we implemented to deal with this scenario.
class ConsortiumRule(OrganizationModel):
BY_EMPLOYEE = 1
BY_CLIENT = 2
BY_OCCUPATION = 3
BY_CLASSIFICATION = 4
TYPES = (
(BY_EMPLOYEE, 'Include a specific employee'),
(BY_CLIENT, 'Include all employees of a specific client'),
(BY_OCCUPATION, 'Include all employees of a speciified client ' + \
'that have the specified occupation'),
(BY_CLASSIFICATION, 'Include all employees of a specified client ' + \
'that have the specified classifications'))
consortium = models.ForeignKey(Consortium, related_name='rules')
type = models.PositiveIntegerField(choices=TYPES, default=BY_CLIENT)
negate_rule = models.BooleanField(default=False,
help_text='Exclude people who match this rule')
class ConsortiumRuleParameter(OrganizationModel):
""" example usage: two of these objects one with "occupation=5" one
with "occupation=6" - both FK linked to a single Rule
"""
rule = models.ForeignKey(ConsortiumRule, related_name='parameters')
key = models.CharField(max_length=100, blank=False)
value = models.CharField(max_length=100, blank=False)
At first I was resistant to this solution as I didn't like the idea of storing references to other objects in a CharField (CharField was selected, because it is the most versatile. Later on, we might have a rule that matches any person whose first name starts with 'Jo'). However, I think this is the best solution for storing this kind of mapping in a relational database. One reason this is a good approach is that it's relatively easy to clean hanging references. For example, if a company is deleted, we only have to do:
ConsortiumRuleParameter.objects.filter(key='company', value=str(pk)).delete()
If the parameters were stored as serialized objects (e.g., Q objects as suggested in a comment), this would be a lot more difficult and time consuming.
Is there a programming language or package that supports table based reactive declarative programming in memory very similar to the SQL language and trigger facility?
For example, I could define PERSON and JOB tables as functions
name: PERSON -> STRING
female: PERSON -> BOOLEAN
mother: PERSON -> PEOPLE
father: PERSON -> PEOPLE
title: JOB -> STRING
company: JOB -> STRING
salary: JOB -> INTEGER
empoyee: JOB -> PERSON
Then I would like to calculate functions like:
childcount: PERSON -> INTEGER
childcount(P) = |{ Q in PERSON : father(Q) = P or mather(Q) = P }|
income: PERSON -> INTEGER
income(P) = SUM { salary(J) : J in JOB and empoyee(J) = P }
incomeperchild: PERSON -> INTEGER
incomeperchild(P) = income(P) / childcount(P)
parent: PERSON x PERSON -> BOOLEAN
person(P,Q) = (P = father(Q)) or (P = mother(Q))
error: PERSON -> BOOLEAN
error(P) = (female(P) and (exists Q in PERSON)(father(Q) = P))
or (not female(P) and (exists Q in PERSON)(mother(Q) = P))
or (exists Q in PERSON)(parent(P,Q) and error(Q))
So essentially I would like to have calculated columns in tables that are automatically updated whenever values in the tables change. Similar things could be expressed with SQL triggers, but I would like to have such functionality built into a language and executed in memory. The propagation of changes need to be optimized. Are there frameworks to do this?
The observer patter and reactive programming focuses on individual objects. But I do not want to maintain pointers and extra structure for each row in my tables as there could be million of rows. All the rules are generic (although they can refer to different rows via parent/children relations, etc), so some form of recursion is required.
One way to approach this is via attribute grammars: http://www.haskell.org/haskellwiki/Attribute_grammar
I have this SQL expression:
SELECT Musclegroups.Name, COUNT(DISTINCT Workouts.WorkoutID) AS Expr1
FROM Workouts INNER JOIN
Series ON Workouts.WorkoutID = Series.WorkoutID INNER JOIN
Exercises ON Series.ExerciseID = Exercises.ExerciseID INNER JOIN
Musclegroups ON Musclegroups.MusclegroupID = Exercises.MusclegroupID
GROUP BY Musclegroups.Name
Since Im working on a project which uses EF in a WCF Ria LinqToEntitiesDomainService, I have to query this with LINQ (If this isn't a must then pls inform me).
I made this expression:
var WorkoutCountPerMusclegroup = (from s in ObjectContext.Series1
join w in ObjectContext.Workouts on s.WorkoutID equals w.WorkoutID
where w.UserID.Equals(userid) && w.Type.Equals("WeightLifting")
group s by s.Exercise.Musclegroup into g
select new StringKeyIntValuePair
{
TestID = g.Select(n => n.Exercise.MusclegroupID).FirstOrDefault(),
Key = g.Select(n => n.Exercise.Musclegroup.Name).FirstOrDefault(),
Value = g.Select(n => n.WorkoutID).Distinct().Count()
});
The StringKeyIntValuePair is just a custom Entity type I made so I can send down the info to the Silverlight client. Also this is why I need to set an "TestID" for it, as it is an entity and it needs one.
And the problem is, that this linq query produces this horrible SQL statement:
http://pastebay.com/144532
I suppose there is a better way to query this information, a better linq expression maybe. Or is it possible to just query with plain SQL somehow?
EDIT:
I realized that the TestID is unnecessary because the other property named "Key" (the one on which Im grouping) becomes the key of the group, so it will be a key also. And after this, my query looks like this:
var WorkoutCountPerMusclegroup = (from s in ObjectContext.Series1
join w in ObjectContext.Workouts on s.WorkoutID equals w.WorkoutID
where w.UserID.Equals(userid) && w.Type.Equals("WeightLifting")
group w.WorkoutID by s.Exercise.Musclegroup.Name into g
select new StringKeyIntValuePair
{
Key = g.Key,
Value = g.Select(n => n).Distinct().Count()
});
This produces the following SQL: http://pastebay.com/144545
This seems far better then the previous sql statement of the half-baked linq query.
But is this good enough? Or this is the boundary of LinqToEntities capabilities, and if I want even more clear sql, I should make another DomainService which operates with LinqToSQL or something else?
Or the best way would be using a stored procedure, that returns Rowsets? If so, is there a best practice to do this asynchronously, like a simple WCF Ria DomainService query?
I would like to know best practices as well.
Compiling of lambda expression linq can take a lot of time (3–30s), especially using group by and then FirstOrDefault (for left inner joins meaning only taking values from the first row in the group).
The generated sql excecution might not be that bad but the compilation using DbContext which cannot be precompiled with .NET 4.0.
As an example 1 something like:
var q = from a in DbContext.A
join b ... into bb from b in bb.DefaultIfEmtpy()
group new { a, b } by new { ... } into g
select new
{
g.Key.Name1
g.Sum(p => p.b.Salary)
g.FirstOrDefault().b.SomeDate
};
Each FirstOrDefault we added in one case caused +2s compile time which added up 3 times = 6s only to compile not load data (which takes less than 500ms). This basically destroys your application's usability. The user will be waiting many times for no reason.
The only way we found so far to speed up the compilation is to mix lambda expression with object expression (might not be the correct notation).
Example 2: refactoring of previous example 1.
var q = (from a in DbContext.A
join b ... into bb from b in bb.DefaultIfEmtpy()
select new { a, b })
.GroupBy(p => new { ... })
.Select(g => new
{
g.Key.Name1
g.Sum(p => p.b.Salary)
g.FirstOrDefault().b.SomeDate
});
The above example did compile a lot faster than example 1 in our case but still not fast enough so the only solution for us in response-critical areas is to revert to native SQL (to Entities) or using views or stored procedures (in our case Oracle PL/SQL).
Once we have time we are going to test if precompilation works in .NET 4.5 and/or .NET 5.0 for DbContext.
Hope this helps and we can get other solutions.