Using forall() predicate in minizinc as assignment statement without 'constraint' - optimization

I have a Minizinc program for generating the optimal charge/discharge schedule for a grid-connected battery, given a set of prices by time-interval.
My program works (sort of; subject to some caveats), but my question is about two 'constraint' statements which are really just assignment statements:
constraint forall(t in 2..T)(MW_SETPOINT[t-1] - SALE[t] = MW_SETPOINT[t]);
constraint forall(t in 1..T)(PROFIT[t] = SALE[t] * PRICE[t]);
These just mean Energy SALES is the delta in MW_SETPOINT from t-1 to 1, and PROFIT is SALE * PRICE for each interval. So it seems counterintuitive to me to declare them as 'constraints'. But I've been unable to formulate them as assignment statements without throwing syntax errors.
Question:
Is there a more idiomatic way to declare such assignment statements for an array which is a function of other params/variables? Or is making assignments for arrays in constraints the recommended/idiomatic way to do it in Minizinc?
Full program for context:
% PARAMS
int: MW_CAPACITY = 10;
array[int] of float: PRICE;
% DERIVED PARAMS
int: STARTING_MW = MW_CAPACITY div 2; % integer division
int: T = length(PRICE);
% DECISION VARIABLE - MW SETPOINT EACH INTERVAL
array[1..T] of var 0..MW_CAPACITY: MW_SETPOINT;
% DERIVED/INTERMEDIATE VARIABLES
array[1..T] of var -1*MW_CAPACITY..MW_CAPACITY: SALE;
array[1..T] of var float: PROFIT;
var float: NET_PROFIT = sum(PROFIT);
% CONSTRAINTS
%% If start at 5MW, and sell 5 first interval, setpoint for first interval is 0
constraint MW_SETPOINT[1] = STARTING_MW - SALE[1];
%% End where you started; opt schedule from arbitrage means no net MW over time
constraint MW_SETPOINT[T] = STARTING_MW;
%% these are really justassignment statements for SALE & PROFIT
constraint forall(t in 2..T)(MW_SETPOINT[t-1] - SALE[t] = MW_SETPOINT[t]);
constraint forall(t in 1..T)(PROFIT[t] = SALE[t] * PRICE[t]);
% OBJECTIVE: MAXIMIZE REVENUE
solve maximize NET_PROFIT;
output["DAILY_PROFIT: " ++ show(NET_PROFIT) ++
"\nMW SETPOINTS: " ++ show(MW_SETPOINT) ++
"\nMW SALES: " ++ show(SALE) ++
"\n$/MW PRICES: " ++ show(PRICE)++
"\nPROFITS: " ++ show(PROFIT)
];
It can be run with
minizinc opt_sched_hindsight.mzn --solver org.minizinc.mip.coin-bc -D "PRICE = [29.835, 29.310470000000002, 28.575059999999997, 28.02416, 28.800690000000003, 32.41052, 34.38542, 29.512390000000003, 25.66587, 25.0499, 26.555529999999997, 28.149440000000002, 30.216509999999996, 32.32415, 31.406609999999997, 36.77642, 41.94735, 51.235209999999995, 50.68137, 64.54481, 48.235170000000004, 40.27663, 34.93675, 31.10404];"```

You can play with Array Comprehensions: (quote from the docs)
Array comprehensions have this syntax:
<array-comp> ::= "[" <expr> "|" <comp-tail> "]"
For example (with the literal equivalents on the right):
[2*i | i in 1..5] % [2, 4, 6, 8, 10]
Array comprehensions have more flexible type and inst requirements than set comprehensions (see Set Comprehensions).
Array comprehensions are allowed over a variable set with finite type,
the result is an array of optional type, with length equal to the
cardinality of the upper bound of the variable set. For example:
var set of 1..5: x;
array[int] of var opt int: y = [ i * i | i in x ];
The length of array will be 5.
Array comprehensions are allowed where the where-expression
is a var bool. Again the resulting array is of optional
type, and of length equal to that given by the generator expressions. For example:
var int x;
array[int] of var opt int: y = [ i | i in 1..10 where i != x ];
The length of the array will be 10.
The indices of an evaluated simple array comprehension are
implicitly 1..n, where n is the length of the evaluated
comprehension.
Example:
int: MW_CAPACITY = 10;
int: STARTING_MW = MW_CAPACITY div 2;
array [int] of float: PRICE = [1.0, 2.0, 3.0, 4.0];
int: T = length(PRICE);
array [1..T] of var -1*MW_CAPACITY..MW_CAPACITY: SALE;
array [1..T] of var 0..MW_CAPACITY: MW_SETPOINT = let {
int: min_i = min(index_set(PRICE));
} in
[STARTING_MW - sum([SALE[j] | j in min_i..i])
| i in index_set(PRICE)];
array [1..T] of var float: PROFIT =
[SALE[i] * PRICE[i]
| i in index_set(PRICE)];
solve satisfy;
Output:
~$ minizinc test.mzn
SALE = array1d(1..4, [-10, -5, 0, 0]);
----------
Notice that index_set(PRICE) is nothing else but 1..T and that min(index_set(PRICE)) is nothing else but 1, so one could write the above array comprehensions also as
array [1..T] of var 0..MW_CAPACITY: MW_SETPOINT =
[STARTING_MW - sum([SALE[j] | j in 1..i])
| i in 1..T];
array [1..T] of var float: PROFIT =
[SALE[i] * PRICE[i]
| i in 1..T];

Related

z3py: Symbolic expressions cannot be cast to concrete Boolean values

I'm having troubles to define the objective fucntion in a SMT problem with z3py.
Long story, short, I have to optimize the placing of smaller blocks inside a board that has fixed width but variable heigth.
I have an array of coordinates (represented by an array of integers of length 2) and a list of integers (representing the heigth of the block to place).
# [x,y] list of integer variables
P = [[Int("x_%s" % (i + 1)), Int("y_%s" % (i + 1))]
for i in range(blocks)]
y = [int(b) for a, b in data[2:]]
I defined the objective function like this:
obj= Int(max([P[i][1] + y[i] for i in range(blocks)]))
It calculates the max height of the board given the starting coordinate of the blocks and their heights.
I know it could be better, but I think the problem would be the same even with a different definition.
Anyway, if I run my code, the following error occurs on the line of the objective function:
" raise Z3Exception("Symbolic expressions cannot be cast to concrete Boolean values.") "
While debugging I've seen that is P[i][1] that gives an error and I think it's because the program reads "y_i + 3" (for example) and they can't be added togheter.
Point is: it's obvious that the objective function depends on the variables of the problem, so how can I get rid of this error? Is there another place where I should define the objective function so it waits to have the P array instantiated before doing anything?
Full code:
from z3 import *
from math import ceil
width = 8
blocks = 4
x = [3,3,5,5]
y = [3,5,3,5]
height = ceil(sum([x[i] * y[i] for i in range(blocks)]) / width) + 1
# [blocks x 2] list of integer variables
P = [[Int("x_%s" % (i + 1)), Int("y_%s" % (i + 1))]
for i in range(blocks)]
# value/ domain constraint
values = [And(0 <= P[i][0], P[i][0] <= width - 1, 0 <= P[i][1], P[i][1] <= height - 1)
for i in range(blocks)]
obj = Int(max([P[i][1] + y[i] for i in range(blocks)]))
board_problem = values # other constraints I've not included for brevity
o = Optimize()
o.add(board_problem)
o.minimize(obj)
if (o.check == 'unsat'):
print("The problem is unsatisfiable")
else:
print("Solved")
The problem here is that you're calling Python's max on symbolic values, which is not designed to work for symbolic expressions. Instead, define a symbolic version of max and use that:
# Return maximum of a vector; error if empty
def symMax(vs):
m = vs[0]
for v in vs[1:]:
m = If(v > m, v, m)
return m
obj = symMax([P[i][1] + y[i] for i in range(blocks)])
With this change your program will go through and print Solved when run.

How to sample from a sum of two distributions: binomial and poisson

Is there a way to predict a value from a sum of two distributions? I am getting a syntax error on rstan when I try to estimate y here: y ~ binomial(,) + poisson()
library(rstan)
BH_model_block <- "
data{
int y;
int a;
}
parameters{
real <lower = 0, upper = 1> c;
real <lower = 0, upper = 1> b;
}
model{
y ~ binomial(a,b)+ poisson(c);
}
"
BH_model <- stan_model(model_code = BH_model_block)
BH_fit <- sampling(BH_model,
data = list(y = 5,
a = 2),
iter= 1000)
Produces this error:
SYNTAX ERROR, MESSAGE(S) FROM PARSER:
error in 'model2c6022623d56_457bd7ab767c318c1db686d1edf0b8f6' at line 13, column 20
-------------------------------------------------
11:
12: model{
13: y ~ binomial(a,b)+ poisson(c);
^
14: }
-------------------------------------------------
PARSER EXPECTED: ";"
Error in stanc(file = file, model_code = model_code, model_name = model_name, :
failed to parse Stan model '457bd7ab767c318c1db686d1edf0b8f6' due to the above error.
Stan doesn't support integer parameters, so you can't technically do that. For two real variables, it'd look like this:
parameters {
real x;
real y;
}
transformed parameters {
real z = x + y;
}
model {
x ~ normal(0, 1);
y ~ gamma(0.1, 2);
}
Then you get the sum distribution for z. If the variables are discrete, it won't compile.
If you don't need z in the model, then you can do this in the generated quantities block,
generated quantities {
int x = binomial_rng(a, b);
int y = poisson_rng(c);
int z = x + y;
}
The drawback of doing this is that none of the variables are available in the model block. If you need discrete parameters, they need to be marginalized as described in the user's guide chapter on latent discrete parameters (also in the chapter on mixtures and HMMs). This is not so easy with a Poisson, because support isn't bounded. If the expectations of the two discrete distributions is small, then you can do it approximately with a loop over plausible values.
It looked from the example in the original post that z is data. That's a slightly different marginalization over x and y, but you only sum over the x and y such that x + y = z, so the combinatorics are greatly reduced.
An alternative is to substitute the Binomial with a Poisson, and use Poisson additivity:
BH_model_block <- "
data{
int y;
int a;
}
parameters{
real <lower = 0, upper = 1> c;
real <lower = 0, upper = 1> b;
}
model{
y ~ poisson(a * b + c);
}
"
This differs in that if b is not small, the Binomial has a lower variance than the Poisson, but maybe there is overdispersion anyhow?

Integer casting within arithmetic expressions in Kotlin

In the following Kotlin code example I expected the value of parameter i to be equal to 0, such as is the case for the parameter k. The IDE reports all i, j and k as Int. Is it a bug or do I need to readjust my understanding of Kotlin casting inside expressions? For example, is there a rule to always promote/cast to Double inside expressions involving division, but not multiplication?
fun main() {
//Kotlin 1.3.61
val x = 100 * 1.0/100 //Double
val i = 100 * 1/100 //Int
val j = 1/100 //Int
val k = 100 * j //Int
println(x) //1.0
println(i) //1
println(j) //0
println(k) //0
}
I expected the value of parameter i to be equal to 0
The output is arithmetically right: 100 * 1 / 100 = (100 * 1) / 100 = 100 / 100 = 1 / 1 = 1
, such as is the case for the parameter k.
The value j is 0, so anything multiplied by it will be zero, as in case of k.
is there a rule to always promote/cast to Double inside expressions
involving division, but not multiplication?
If you divide integers, you will get an integer back. If one of the numbers is a Double, the result will be a Double:
val x = 100 * 1.0/100 //Double because 1.0 is a Double
--
There is actually already a discussion on kotlin forum to your problem here:
Mathematically speaking the current behaviour is correct.
This is called integer devision and results in the quotient as an
answer

Find nth int with 10 set bits

Find the nth int with 10 set bits
n is an int in the range 0<= n <= 30 045 014
The 0th int = 1023, the 1st = 1535 and so on
snob() same number of bits,
returns the lowest integer bigger than n with the same number of set bits as n
int snob(int n) {
int a=n&-n, b=a+n;
return b|(n^b)/a>>2;
}
calling snob n times will work
int nth(int n){
int o =1023;
for(int i=0;i<n;i++)o=snob(o);
return o;
}
example
https://ideone.com/ikGNo7
Is there some way to find it faster?
I found one pattern but not sure if it's useful.
using factorial you can find the "indexes" where all 10 set bits are consecutive
1023 << x = the (x+10)! / (x! * 10!) - 1 th integer
1023<<1 is the 10th
1023<<2 is the 65th
1023<<3 the 285th
...
Btw I'm not a student and this is not homework.
EDIT:
Found an alternative to snob()
https://graphics.stanford.edu/~seander/bithacks.html#NextBitPermutation
int lnbp(int v){
int t = (v | (v - 1)) + 1;
return t | ((((t & -t) / (v & -v)) >> 1) - 1);
}
I have built an implementation that should satisfy your needs.
/** A lookup table to see how many combinations preceeded this one */
private static int[][] LOOKUP_TABLE_COMBINATION_POS;
/** The number of possible combinations with i bits */
private static int[] NBR_COMBINATIONS;
static {
LOOKUP_TABLE_COMBINATION_POS = new int[Integer.SIZE][Integer.SIZE];
for (int bit = 0; bit < Integer.SIZE; bit++) {
// Ignore less significant bits, compute how many combinations have to be
// visited to set this bit, i.e.
// (bit = 4, pos = 5), before came 0b1XXX and 0b1XXXX, that's C(3, 3) + C(4, 3)
int nbrBefore = 0;
// The nth-bit can be only encountered after pos n
for (int pos = bit; pos < Integer.SIZE; pos++) {
LOOKUP_TABLE_COMBINATION_POS[bit][pos] = nbrBefore;
nbrBefore += nChooseK(pos, bit);
}
}
NBR_COMBINATIONS = new int[Integer.SIZE + 1];
for (int bits = 0; bits < NBR_COMBINATIONS.length; bits++) {
NBR_COMBINATIONS[bits] = nChooseK(Integer.SIZE, bits);
assert NBR_COMBINATIONS[bits] > 0; // Important for modulo check. Otherwise we must use unsigned arithmetic
}
}
private static int nChooseK(int n, int k) {
assert k >= 0 && k <= n;
if (k > n / 2) {
k = n - k;
}
long nCk = 1; // (N choose 0)
for (int i = 0; i < k; i++) {
// (N choose K+1) = (N choose K) * (n-k) / (k+1);
nCk *= (n - i);
nCk /= (i + 1);
}
return (int) nCk;
}
public static int nextCombination(int w, int n) {
// TODO: maybe for small n just advance naively
// Get the position of the current pattern w
int nbrBits = 0;
int position = 0;
while (w != 0) {
final int currentBit = Integer.lowestOneBit(w); // w & -w;
final int bitPos = Integer.numberOfTrailingZeros(currentBit);
position += LOOKUP_TABLE_COMBINATION_POS[nbrBits][bitPos];
// toggle off bit
w ^= currentBit;
nbrBits++;
}
position += n;
// Wrapping, optional
position %= NBR_COMBINATIONS[nbrBits];
// And reverse lookup
int v = 0;
int m = Integer.SIZE - 1;
while (nbrBits-- > 0) {
final int[] bitPositions = LOOKUP_TABLE_COMBINATION_POS[nbrBits];
// Search for largest bitPos such that position >= bitPositions[bitPos]
while (Integer.compareUnsigned(position, bitPositions[m]) < 0)
m--;
position -= bitPositions[m];
v ^= (0b1 << m--);
}
return v;
}
Now for some explanation. LOOKUP_TABLE_COMBINATION_POS[bit][pos] is the core of the algorithm that makes it as fast as it is. The table is designed so that a bit pattern with k bits at positions p_0 < p_1 < ... < p_{k - 1} has a position of `\sum_{i = 0}^{k - 1}{ LOOKUP_TABLE_COMBINATION_POS[i][p_i] }.
The intuition is that we try to move back the bits one by one until we reach the pattern where are all bits are at the lowest possible positions. Moving the i-th bit from position to k + 1 to k moves back by C(k-1, i-1) positions, provided that all lower bits are at the right-most position (no moving bits into or through each other) since we skip over all possible combinations with the i-1 bits in k-1 slots.
We can thus "decode" a bit pattern to a position, keeping track of the bits encountered. We then advance by n positions (rolling over in case we enumerated all possible positions for k bits) and encode this position again.
To encode a pattern, we reverse the process. For this, we move bits from their starting position forward, as long as the position is smaller than what we're aiming for. We could, instead of a linear search through LOOKUP_TABLE_COMBINATION_POS, employ a binary search for our target index m but it's hardly needed, the size of an int is not big. Nevertheless, we reuse our variant that a smaller bit must also come at a less significant position so that our algorithm is effectively O(n) where n = Integer.SIZE.
I remain with the following assertions to show the resulting algorithm:
nextCombination(0b1111111111, 1) == 0b10111111111;
nextCombination(0b1111111111, 10) == 0b11111111110;
nextCombination(0x00FF , 4) == 0x01EF;
nextCombination(0x7FFFFFFF , 4) == 0xF7FFFFFF;
nextCombination(0x03FF , 10) == 0x07FE;
// Correct wrapping
nextCombination(0b1 , 32) == 0b1;
nextCombination(0x7FFFFFFF , 32) == 0x7FFFFFFF;
nextCombination(0xFFFFFFEF , 5) == 0x7FFFFFFF;
Let us consider the numbers with k=10 bits set.
The trick is to determine the rank of the most significant one, for a given n.
There is a single number of length k: C(k, k)=1. There are k+1 = C(k+1, k) numbers of length k + 1. ... There are C(m, k) numbers of length m.
For k=10, the limit n are 1 + 10 + 55 + 220 + 715 + 2002 + 5005 + 11440 + ...
For a given n, you easily find the corresponding m. Then the problem is reduced to finding the n - C(m, k)-th number with k - 1 bits set. And so on recursively.
With precomputed tables, this can be very fast. 30045015 takes 30 lookups, so that I guess that the worst case is 29 x 30 / 2 = 435 lookups.
(This is based on linear lookups, to favor small values. By means of dichotomic search, you reduce this to less than 29 x lg(30) = 145 lookups at worse.)
Update:
My previous estimates were pessimistic. Indeed, as we are looking for k bits, there are only 10 determinations of m. In the linear case, at worse 245 lookups, in the dichotomic case, less than 50.
(I don't exclude off-by-one errors in the estimates, but clearly this method is very efficient and requires no snob.)

How to declare constraints with variable as array index in Z3Py?

Suppose x,y,z are int variables and A is a matrix, I want to express a constraint like:
z == A[x][y]
However this leads to an error:
TypeError: object cannot be interpreted as an index
What would be the correct way to do this?
=======================
A specific example:
I want to select 2 items with the best combination score,
where the score is given by the value of each item and a bonus on the selection pair.
For example,
for 3 items: a, b, c with related value [1,2,1], and the bonus on pairs (a,b) = 2, (a,c)=5, (b,c) = 3, the best selection is (a,c), because it has the highest score: 1 + 1 + 5 = 7.
My question is how to represent the constraint of selection bonus.
Suppose CHOICE[0] and CHOICE[1] are the selection variables and B is the bonus variable.
The ideal constraint should be:
B = bonus[CHOICE[0]][CHOICE[1]]
but it results in TypeError: object cannot be interpreted as an index
I know another way is to use a nested for to instantiate first the CHOICE, then represent B, but this is really inefficient for large quantity of data.
Could any expert suggest me a better solution please?
If someone wants to play a toy example, here's the code:
from z3 import *
items = [0,1,2]
value = [1,2,1]
bonus = [[1,2,5],
[2,1,3],
[5,3,1]]
choices = [0,1]
# selection score
SCORE = [ Int('SCORE_%s' % i) for i in choices ]
# bonus
B = Int('B')
# final score
metric = Int('metric')
# selection variable
CHOICE = [ Int('CHOICE_%s' % i) for i in choices ]
# variable domain
domain_choice = [ And(0 <= CHOICE[i], CHOICE[i] < len(items)) for i in choices ]
# selection implication
constraint_sel = []
for c in choices:
for i in items:
constraint_sel += [Implies(CHOICE[c] == i, SCORE[c] == value[i])]
# choice not the same
constraint_neq = [CHOICE[0] != CHOICE[1]]
# bonus constraint. uncomment it to see the issue
# constraint_b = [B == bonus[val(CHOICE[0])][val(CHOICE[1])]]
# metric definition
constraint_sumscore = [metric == sum([SCORE[i] for i in choices ]) + B]
constraints = constraint_sumscore + constraint_sel + domain_choice + constraint_neq + constraint_b
opt = Optimize()
opt.add(constraints)
opt.maximize(metric)
s = []
if opt.check() == sat:
m = opt.model()
print [ m.evaluate(CHOICE[i]) for i in choices ]
print m.evaluate(metric)
else:
print "failed to solve"
Turns out the best way to deal with this problem is to actually not use arrays at all, but simply create integer variables. With this method, the 317x317 item problem originally posted actually gets solved in about 40 seconds on my relatively old computer:
[ 0.01s] Data loaded
[ 2.06s] Variables defined
[37.90s] Constraints added
[38.95s] Solved:
c0 = 19
c1 = 99
maxVal = 27
Note that the actual "solution" is found in about a second! But adding all the required constraints takes the bulk of the 40 seconds spent. Here's the encoding:
from z3 import *
import sys
import json
import sys
import time
start = time.time()
def tprint(s):
global start
now = time.time()
etime = now - start
print "[%ss] %s" % ('{0:5.2f}'.format(etime), s)
# load data
with open('data.json') as data_file:
dic = json.load(data_file)
tprint("Data loaded")
items = dic['items']
valueVals = dic['value']
bonusVals = dic['bonusVals']
vals = [[Int("val_%d_%d" % (i, j)) for j in items if j > i] for i in items]
tprint("Variables defined")
opt = Optimize()
for i in items:
for j in items:
if j > i:
opt.add(vals[i][j-i-1] == valueVals[i] + valueVals[j] + bonusVals[i][j])
c0, c1 = Ints('c0 c1')
maxVal = Int('maxVal')
opt.add(Or([Or([And(c0 == i, c1 == j, maxVal == vals[i][j-i-1]) for j in items if j > i]) for i in items]))
tprint("Constraints added")
opt.maximize(maxVal)
r = opt.check ()
if r == unsat or r == unknown:
raise Z3Exception("Failed")
tprint("Solved:")
m = opt.model()
print " c0 = %s" % m[c0]
print " c1 = %s" % m[c1]
print " maxVal = %s" % m[maxVal]
I think this is as fast as it'll get with Z3 for this problem. Of course, if you want to maximize multiple metrics, then you can probably structure the code so that you can reuse most of the constraints, thus amortizing the cost of constructing the model just once, and incrementally optimizing afterwards for optimal performance.