I'm trying to set up a ampl model which clusters given points in a 2-dimensional space according to the model of Saglam et al(2005). For testing purposes I want to generate randomly some datapoints and then calculate the euclidian distance matrix for them (since I need this one). I'm aware that I could only make the distance matrix without the data points but in a later step the data points will be given and then I need to calculate the distances between each the points.
Below you'll find the code I've written so far. While loading the model I keep getting the error message "i is not defined". Since i is a subscript that should run over x1 and x1 is a parameter which is defined over the set D and have one subscript, I cannot figure out why this code should be invalid. As far as I understand, I don't have to define variables if I use them only as subscripts?
reset;
# parameters to define clustered
param m; # numbers of data points
param n; # numbers of clusters
# sets
set D := 1..m; #points to be clustered
set L := 1..n; #clusters
# randomly generate datapoints
param x1 {D} = Uniform(1,m);
param x2 {D} = Uniform(1,m);
param d {D,D} = sqrt((x1[i]-x1[j])^2 + (x2[i]-x2[j])^2);
# variables
var x {D, L} binary;
var D_l {L} >=0;
var D_max >= 0;
#minimization funcion
minimize max_clus_dis: D_max;
# constraints
subject to C1 {i in D, j in D, l in L}: D_l[l] >= d[i,j] * (x[i,l] + x[j,l] - 1);
subject to C2 {i in D}: sum{l in L} x[i,l] = 1;
subject to C3 {l in L}: D_max >= D_l[l];
So far I tried to change the line form param x1 to
param x1 {i in D, j in D} = ...
as well as
param d {x1, x2} = ...
Alas, nothing of this helped. So, any help someone can offer is deeply appreciated. I searched the web but I found nothing useful for my task.
I found eventually what was missing. The line in which I calculated the parameter d should be
param d {i in D, j in D} = sqrt((x1[i]-x1[j])^2 + (x2[i]-x2[j])^2);
Retrospectively it's clear that the subscripts i and j should have been mentioned on the line, I don't know how I could miss that.
Recently I'm planning to manipulate a stack of images and the goal is to extract a specific subset of slices from there, for example only even or odd or arbitrary indexes, and then save them into another dataset.
In DM, there are a number of helpful functions in the Volume menu but unfortunately, they cannot really fullfill what I want to do.
I am just wondering whether this idea can be realized via scripting.
Many thanks for your help in advance.
There are two ways you can go about it, one of them only suitable for data up to 3D and generally slower than the other, but more flexible.
As you have been asking for arbitrary subsampling, I'm starting with that option, but it is more likely that the second option gives you what you want: orthogonal, regular subsampling.
If you are in a hurry, the short answer is: Use the SliceN command.
1) Using expressions (arbitrary subsampling)
Individual pixel positions in an Image data (img) can be addressed
using the notations
img[ X, 0 ] ... for 1D data at position X
img[ X, Y ] ... for 2D data at position X/Y
img[ X, Y, Z ] ... for 3D data at position X/Y/Z
Note that even if this addresses a single number, the result is an expression of size 1x1 or 1x1x1 and not a scalar number, therefore you can not do: number num = img[10,4]
However, you can use a little trick to use any of the functions that convert an expression to a single number like f.e. summation. So you can do: number num = sum(img[10,4])
So how does this relate to your question? Well, in the expressions above, we used scalar values as X, Y and Z, and the resulting expressions were expressions of size 1x1 and 1x1x1, but
You can use expressions of any size as X, Y, Z in this notations, as long as all of them are expressions of same size. The resulting addressed data is of this size with values references by the according coordinates.
This will become clearer with the examples below. Starting out with a simple 1D example:
image img1D := RealImage( "TestData", 4, 100 )
image coord := RealImage( "Coordinates", 4, 10 )
img1D = 1000 + icol // Just sum test data
coord = trunc(100*Random()) // random integer 0-99
image subImg := img1D[coord,0]
img1D.ShowImage()
coord.ShowImage()
subImg.ShowImage()
Our testdata (img1D) here is just a linear graph from 1000 to 1099 using the icol expression which, at each pixel, represents that pixels X coordinate.
The coordinate image (coord) is containing random integer values between 0 and 99.
The 'magic' happens in the subImg. We use an expression with the coord image as X coordinates. That images is of size 10(x1), so the outcoming expression is of size 10(x1) which we assign to the image subImg before showing it.
Note, that the expression we have built is really just pointing to that data of the image. Instead of showing it as a new image, we could have use that expression to change these points in the data instead, using:
img1D[coord,0] = 0
Taking it from here, it is straight forward to extend the example to 2D:
image img2D := RealImage( "TestData", 4, 30, 30 )
image coordX := RealImage( "Coordinates X", 4, 10 )
image coordY := RealImage( "Coordinates Y", 4, 10 )
img2D = 10000 + icol + irow * 100
coordX = trunc(30*Random())
coordY = trunc(30*Random())
img2D[coordX,coordY] = 0
coordX.ShowImage()
coordY.ShowImage()
img2D.ShowImage()
...and 3D:
image img3D := RealImage( "TestData", 4, 30, 30, 30 )
image coordX := RealImage( "Coordinates X", 4, 10 )
image coordY := RealImage( "Coordinates Y", 4, 10 )
image coordZ := RealImage( "Coordinates Y", 4, 10 )
img3D = 10000 + icol + irow * 100 + iplane * 1000
coordX = trunc(30*Random())
coordY = trunc(30*Random())
coordZ = trunc(30*Random())
img3D[coordX,coordY,coordZ] = 0
coordX.ShowImage()
coordY.ShowImage()
coordZ.ShowImage()
img3D.ShowImage()
Unfortunately, it ends here.
You can no longer do this type of addressing in 4D or 5D data, because expression with 4 parameters are already defined to address a rectangle region in 2D data as img[T,L,B,R]
2) Using SliceN (orthogonal subsampling)
Data subsets along the dimension directions of data can be addressed using the command SliceN and its simplified variants Slice1, Slice2 and Slice3.
The SliceN command is maybe one of my favourite commands in the language when dealing with data. It looks intimidating at first, but it is straight forward.
Lets start with its simplified version for 1D extraction, Slice1.
To extract 1D data from any data up to 3D with the Slice1 command, you need the following (-and these are exactly the 7 parameters used by the command-):
data source
start point in the source
sampling direction
sampling length
sampling step-size
The only thing you need to know on top of that is:
The start point is always defined as a X,Y,Z triplet, even if the data source is only 2D or 1D. 0 is used for the not needed
dimensions.
Directions are given as dimension index: 0 = X, 1 = Y, 2 = Z
Step-size can be negative to indicate opposite directions
The specified sampling must be contained within the source data.(You can not 'extrapolate')
So a very simple example of extracting a 1D data of a 3D dataset would be:
number sx = 20
number sy = 20
number sz = 20
image img3D := RealImage( "Spectrum Image", 4, sx, sy, sz )
img3D = 5000000 + icol + irow * 100 + iplane * 10000
number px = 5
number py = 7
image spec1D := Slice1( img3D, px,py,0, 2,sz,1 )
ShowImage( img3D )
ShowImage( spec1D )
This example showed a quite typical situation in analytical microscopy when dealing with "3D Spectrum Image" data: Extracting a "1D Spectrum" at a specific spatial position.
The example did that for the spatial point px,py. Starting at the point at that position (px,py,0), it samples along the Z direction (2) for all pixels of the data (sz) with a step-size of 1.
Note, that the command again returns an expression within the source data, and that you can use this to set values as well, just using f.e.:
Slice1( img3D, px,py,0, 2,sz,1 ) = 0
The extension for 2D and 3D data using the commands Slice2 and Slice3 is straight forward. Instead of defining one output direction, you define two or three, respectively. Each with a triplet of numbers: direction, length, step-size.
The following example extracts an "image plane" of a "3D Spectrum image":
number sx = 20
number sy = 20
number sz = 20
image img3D := RealImage( "Spectrum Image", 4, sx, sy, sz )
img3D = 5000000 + icol + irow * 100 + iplane * 10000
number pz = 3
image plane2D := Slice2( img3D, 0,0,pz, 0,sx,1, 1,sy,1 )
ShowImage( img3D )
ShowImage( plane2D )
And the following example "rotates" a 3D image:
number sx = 6
number sy = 4
number sz = 3
image img3D := RealImage( "Spectrum Image", 4, sx, sy, sz )
img3D = 1000 + icol + irow * 10 + iplane * 100
image rotated := Slice3( img3D, 0,0,0, 0,sx,1, 2,sz,1, 1,sy,1 )
ShowImage( img3D )
ShowImage( rotated )
You can get all sorts of rotations, mirroring, binning with these
commands. If you want the full flexibility to get any expression up to
5D from any source data up to 5D, then you need the most versatile
SliceN command.
It works exactly the same, but you need to specify both the dimensionality of the source data, and the dimensionality of the output expression. Then, the 'starting' point needs to be defined with as many coordinates as the source data dimension suggests, and you need one triplet of specification for each output dimension.
For a source data of N dimensions and want an output of M dimensions you need: 2 + N + 3*M parameters.
As an example, lets extract the "plane" at specific spatial position from a "4D Diffraction image" data, which stores a 2D image at each spatial location of a 2D scan:
number sx = 9
number sy = 9
number kx = 9
number ky = 9
image img4D := RealImage( "Diffraction Image", 4, sx, sy, kx, ky )
img4D = 50000 + icol + irow * 10 + idimindex(2)*100 + idimindex(3)*1000
number px = 3
number py = 4
image img2D := SliceN( img4D, 4, 2, px,py,0,0, 2,kx,1, 3,ky,1 )
ShowImage( img4D )
ShowImage( img2D )
Sets
i / 1, 2 /;
Parameters
j(i) / 2, 1 /;
Variables
x(i);
So, here I have an index i, a parameter that depends on i which gives the same values as i, and a variable that depends on x.
If I want to get x(2), I could of course write x(2), but what if I wanted to write x(j(1)). Since j(1) = 2, this ought to be the same, but GAMS doesn't like it, and says it expected a set.
How can I do this?
You could use a mapping:
see example below which maps x(1) to P(2) and x(2) to P(1), using the mapping j which maps 1 to 2 and 2 to 1.
Set i / 1, 2 /;
alias(k,i);
set j(i,k) /
1.2
2.1
/;
parameter P(i);
P("1") = 10;
P("2") = 20;
Variables
x(i);
loop(j(i,k), x.l(k) =P(i));
execute_unload "test.gdx";
Not sure if I understand the question correctly, but maybe you just mean x(i) = j(i) ? Then for all set elements of i x will take the same value of j. If you only want the first set element: x("1") = j("1"). j("1") = 2, so x("1") will equal 2 too.
I this example is used
# random locations for the cities
param cx{i in N} := Uniform01();
param cy{i in N} := Uniform01();
for generate random locations for the cities
How to read data file with coordinates in GLPK and how format?
1 2 3 4
1,2,3,4
(1,2),(3,4)
{1,2},{3,4}
Glpk uses GNU MathProg, a subset of AMPL, so given the following parameter and set declarations:
set N := 1..2;
param cx{i in N};
param cy{i in N};
you can read the data as follows
data;
param:
cx cy :=
1 1 2
2 3 4;
Note that in this case parameters cx and cy shouldn't be defined in the model, so you should either remove the := Uniform01() part or change it to default Uniform01().
I have a data set that looks like this (SAS 9.4):
data opt_test;
input ID GRP $ x1 MIN MAX y z;
cards;
2 F 10 9 11 1.5 100
3 F 10 9 11 1.2 50
4 F 11 9 11 .9 20
8 G 5 4 6 1.2 300
9 G 6 4 6 .9 200
;
run;
I want to create a new variable x2 that maximizes a function based on x1, x2, y, and z.
I am having two main problems:
The syntax on my proc optmodel has some errors that I have not been able to fix "Subscript 1 may not be a set" and constraint has incomplete declaration". UPDATE: I figured this part out.
I need for the value of x2 to be the same for all members of the same GRP. So, id 2,3,4 would have same x2. ID 8 and 9 would have same x2.
Below is my attempt. This will ultimately be able to run with sevarl different GRP of varying numbers of ID.
Thanks in advance for any assistance.
proc optmodel;
set<num> ID;
var x2{ID} >= 0;
string GRP{ID};
number x1{ID};
number MIN{ID};
number MAX{ID};
number y{ID};
number z{ID};
max sales=sum{i in ID}(x2[i])*(1-(x2[i]-x1[i])*y[i]/x1[i])*z[i];
con floor_METRIC1{i in ID}: x2[i]>=MIN[i];
con ceiling_METRIC1{i in ID}: x2[i]<=MAX[i];
read data opt_test into
ID=[ID]
GRP
x1
MIN
MAX
y
z
;
solve;
print x2;
quit;
If you want the value of x2 to be the same for all ids in the same group, then you only need one variable x2 per group. To keep track of which ids are in which group you could use an array of sets indexed by group:
set<num> ID;
string GRP{ID};
set GRPS = setof{i in ID} GRP[i];
set IDperGRP{gi in GRPS} = {i in ID: GRP[i] = gi};
When you use = (as opposed to init), you provide OPTMODEL with a function you don't need to update later. If you change any of the GRP or ID data, optmodel will recompute GRPS and IDperGRP as needed.
Now you can use the GRPS set and the IDperGRP array of sets to rewrite your objective to more closely match your business rule:
max sales = sum{gi in GRPS} sum{i in IDperGRP[gi]}
(x2[gi]) * (1-(x2[gi]-x1[i])*y[i]/x1[i]) * z[i];
Writing the expression this way makes it clearer (to me at least) that it can be simplified further as long as x1, y, and z are constants.
Adding the new sets also makes it clearer (to me at least) that the bounds from floor_METRIC1 and ceiling_METRIC1 can be combined to tighten the domain of x2. Since MIN and MAX are constants, you can move the constraints into direct variable bounds by adding >= and <= clauses to the declaration of x2. Since x2 will now depend on MIN and MAX, you will have to declare those before x2. IMHO that makes your intent clearer:
number MIN{ID};
number MAX{ID};
var x2{gi in GRPS} >= max{i in IDperGRP[gi]} MIN[i]
<= min{i in IDperGRP[gi]} MAX[i]
;