I have a netcdf data file with the following variables shown in the ncdump snippet below:
netcdf elev.0.5-deg {
dimensions:
lat = 360 ;
lon = 720 ;
time = UNLIMITED ; // (1 currently)
variables:
float lat(lat) ;
lat:title = "Latitude" ;
lat:units = "degrees_north" ;
lat:scale_factor = 1.f ;
lat:add_offset = 0.f ;
float lon(lon) ;
lon:title = "Longitude" ;
lon:units = "degrees_east" ;
lon:scale_factor = 1.f ;
lon:add_offset = 0.f ;
double time(time) ;
time:title = "Time" ;
time:units = "hours since 1-1-1 0:0:0" ;
time:scale_factor = 1.f ;
time:add_offset = 0.f ;
short data(time, lat, lon) ;
data:long_name = "meters, from 5-min data" ;
data:add_offset = 0.f ;
data:scale_factor = 1.f ;
data:missing_value = 32767s ;
data:units = "meters, from 5-min data" ;
I am trying to eliminate the time coordinate variable from this file in order to get a 3 dimensional netcdf data file. Basically having data the data values for the first time in the netcdf files and that is all. I have tried to accomplish this with ncks which is what they showed I should do when doing a google search. I tried the following command with ncks:
ncks -x -v time elev.0.5-deg.nc test.nc
but I cannot get the time dimension removed. What should I be doing to tweak this command to get the desired results of showing the data for just the initial time step and only having dimensions lat and lon in test.nc?
Use ncwa
ncwa -a time in.nc out.nc
You should be able to do this using CDO:
cdo --reduce_dim -copy elev.0.5-deg.nc test.nc
Related
I have a Minizinc program for generating the optimal charge/discharge schedule for a grid-connected battery, given a set of prices by time-interval.
My program works (sort of; subject to some caveats), but my question is about two 'constraint' statements which are really just assignment statements:
constraint forall(t in 2..T)(MW_SETPOINT[t-1] - SALE[t] = MW_SETPOINT[t]);
constraint forall(t in 1..T)(PROFIT[t] = SALE[t] * PRICE[t]);
These just mean Energy SALES is the delta in MW_SETPOINT from t-1 to 1, and PROFIT is SALE * PRICE for each interval. So it seems counterintuitive to me to declare them as 'constraints'. But I've been unable to formulate them as assignment statements without throwing syntax errors.
Question:
Is there a more idiomatic way to declare such assignment statements for an array which is a function of other params/variables? Or is making assignments for arrays in constraints the recommended/idiomatic way to do it in Minizinc?
Full program for context:
% PARAMS
int: MW_CAPACITY = 10;
array[int] of float: PRICE;
% DERIVED PARAMS
int: STARTING_MW = MW_CAPACITY div 2; % integer division
int: T = length(PRICE);
% DECISION VARIABLE - MW SETPOINT EACH INTERVAL
array[1..T] of var 0..MW_CAPACITY: MW_SETPOINT;
% DERIVED/INTERMEDIATE VARIABLES
array[1..T] of var -1*MW_CAPACITY..MW_CAPACITY: SALE;
array[1..T] of var float: PROFIT;
var float: NET_PROFIT = sum(PROFIT);
% CONSTRAINTS
%% If start at 5MW, and sell 5 first interval, setpoint for first interval is 0
constraint MW_SETPOINT[1] = STARTING_MW - SALE[1];
%% End where you started; opt schedule from arbitrage means no net MW over time
constraint MW_SETPOINT[T] = STARTING_MW;
%% these are really justassignment statements for SALE & PROFIT
constraint forall(t in 2..T)(MW_SETPOINT[t-1] - SALE[t] = MW_SETPOINT[t]);
constraint forall(t in 1..T)(PROFIT[t] = SALE[t] * PRICE[t]);
% OBJECTIVE: MAXIMIZE REVENUE
solve maximize NET_PROFIT;
output["DAILY_PROFIT: " ++ show(NET_PROFIT) ++
"\nMW SETPOINTS: " ++ show(MW_SETPOINT) ++
"\nMW SALES: " ++ show(SALE) ++
"\n$/MW PRICES: " ++ show(PRICE)++
"\nPROFITS: " ++ show(PROFIT)
];
It can be run with
minizinc opt_sched_hindsight.mzn --solver org.minizinc.mip.coin-bc -D "PRICE = [29.835, 29.310470000000002, 28.575059999999997, 28.02416, 28.800690000000003, 32.41052, 34.38542, 29.512390000000003, 25.66587, 25.0499, 26.555529999999997, 28.149440000000002, 30.216509999999996, 32.32415, 31.406609999999997, 36.77642, 41.94735, 51.235209999999995, 50.68137, 64.54481, 48.235170000000004, 40.27663, 34.93675, 31.10404];"```
You can play with Array Comprehensions: (quote from the docs)
Array comprehensions have this syntax:
<array-comp> ::= "[" <expr> "|" <comp-tail> "]"
For example (with the literal equivalents on the right):
[2*i | i in 1..5] % [2, 4, 6, 8, 10]
Array comprehensions have more flexible type and inst requirements than set comprehensions (see Set Comprehensions).
Array comprehensions are allowed over a variable set with finite type,
the result is an array of optional type, with length equal to the
cardinality of the upper bound of the variable set. For example:
var set of 1..5: x;
array[int] of var opt int: y = [ i * i | i in x ];
The length of array will be 5.
Array comprehensions are allowed where the where-expression
is a var bool. Again the resulting array is of optional
type, and of length equal to that given by the generator expressions. For example:
var int x;
array[int] of var opt int: y = [ i | i in 1..10 where i != x ];
The length of the array will be 10.
The indices of an evaluated simple array comprehension are
implicitly 1..n, where n is the length of the evaluated
comprehension.
Example:
int: MW_CAPACITY = 10;
int: STARTING_MW = MW_CAPACITY div 2;
array [int] of float: PRICE = [1.0, 2.0, 3.0, 4.0];
int: T = length(PRICE);
array [1..T] of var -1*MW_CAPACITY..MW_CAPACITY: SALE;
array [1..T] of var 0..MW_CAPACITY: MW_SETPOINT = let {
int: min_i = min(index_set(PRICE));
} in
[STARTING_MW - sum([SALE[j] | j in min_i..i])
| i in index_set(PRICE)];
array [1..T] of var float: PROFIT =
[SALE[i] * PRICE[i]
| i in index_set(PRICE)];
solve satisfy;
Output:
~$ minizinc test.mzn
SALE = array1d(1..4, [-10, -5, 0, 0]);
----------
Notice that index_set(PRICE) is nothing else but 1..T and that min(index_set(PRICE)) is nothing else but 1, so one could write the above array comprehensions also as
array [1..T] of var 0..MW_CAPACITY: MW_SETPOINT =
[STARTING_MW - sum([SALE[j] | j in 1..i])
| i in 1..T];
array [1..T] of var float: PROFIT =
[SALE[i] * PRICE[i]
| i in 1..T];
I would like to describe a very simple image (really a vector) of length 2, like (1,2) for the purpose of some linear algebra.
The following creates a two dimensional image with a y axis of length 1:
image a := [2,1]: {
{1, 2}
}
MatrixPrint(a)
This outputs
{
{1, 2}
}
How would I in a similar fashion output this instead?
{123,45}
Additionally, if I had image of arbitrary shape (a, b), how can I slice it to extract a one dimensional image at a value n, either along the x or y axes? (Extracting a line profile along one of the image axes)
In your example you do define a 2D image, so you get a 2D output. If the image really would be 1D, your output would be 1D, i.e.
image a := [2]: {123, 45}
MatrixPrint(a)
So your second question actually is the answer to your first: You need to do a 1D slice of the data, which you can do with the command slice1() as follows:
image a := [2,1]: {
{123, 45}
}
MatrixPrint( a.slice1(0,0,0,0,2,1) )
Note some peculiarities of the command:
The command always assume the input is 3D, so the first 3 parameters are the start-index triplet x/y/z even if it is just 2D or 1D data.
the 2nd triplet specifies the sampling of the slice. First the dimensions index (0=x) then the number of sampling steps (2) and then the stepsize (1)
Similar slice commands exist for 2D slices, 3D slices and nD Slices from nD data.
The matrixPrint command only outputs to the results window. There is no way to reroute this to some string. However, you can easily make yourself a method that would do that (albeit not very fast for big data):
string VectorPrint( image img, string FormatStr, number maxNum )
{
if ( !img.ImageIsValid() ) return "{invalid}"
if ( 1 != img.ImageGetNumDimensions() ) return "{not 1D}"
string out = "{ "
number nx = img.ImageGetDimensionSize(0)
if (( nx <= maxNum ) || ( maxNum <= 2) )
{
for( number i=0; i<min(nx,maxNum); i++)
out += Format( sum(img[0,i]), FormatStr ) + ", "
out = out.left( out.len() - 2 )
}
else
{
for( number i=0; i<maxNum-1; i++)
out += Format( sum(img[0,i]), FormatStr ) + ", "
out = out.left( out.len() - 2 ) + ", ... , "
out += Format( sum(img[0,nx-1]), FormatStr )
}
out += " }"
return out
}
image a := [10,4]: {
{1,2,3,4,5,6,7,8,9,10},
{123, 45, 12.3, -12, 55, 1.2, 9999, 89.100, 1e-10, 0},
{0,0,0,0,0,0,0,0,0,0},
{1,2,3,4,5,6,7,8,9,10}
}
// Slice 2D image to 1D image at n'th line
number n = 1
image line := a.slice1(0,n,0,0,a.ImageGetDimensionSize(0),1)
// Printout with given number format and a maximum number of entries
string fStr = "%3.1f"
number maxN = 3
Result( "\n "+VectorPrint( line, fStr, maxN ) )
I am trying to find three parameters (a, b, c) to fit my experimental data using ODE solver and optimization by least squares using Scilab in-built functions.
However, I keep having the message "submatrix incorrectly defined" at line "y_exp(:,1) = [0.135 ..."
When I try another series of data (t, yexp) such as the one used in the original template I get no error messages. The template I use was found here: https://wiki.scilab.org/Non%20linear%20optimization%20for%20parameter%20fitting%20example
function dy = myModel ( t , y , a , b, c )
// The right-hand side of the Ordinary Differential Equation.
dy(1) = -a*y(1) - b*y(1)*y(2)
dy(2) = a*y(1) - b*y(1)*y(2) - c*y(2)
endfunction
function f = myDifferences ( k )
// Returns the difference between the simulated differential
// equation and the experimental data.
global MYDATA
t = MYDATA.t
y_exp = MYDATA.y_exp
a = k(1)
b = k(2)
c = k(3)
y0 = y_exp(1,:)
t0 = 0
y_calc=ode(y0',t0,t,list(myModel,a,b,c))
diffmat = y_calc' - y_exp
// Make a column vector
f = diffmat(:)
MYDATA.funeval = MYDATA.funeval+ 1
endfunction
// Experimental data
t = [0,20,30,45,75,105,135,180,240]';
y_exp(:,1) =
[0.135,0.0924,0.067,0.0527,0.0363,0.02445,0.01668,0.012,0.009]';
y_exp(:,2) =
[0,0.00918,0.0132,0.01835,0.0261,0.03215,0.0366,0.0393,0.0401]';
// Store data for future use
global MYDATA;
MYDATA.t = t;
MYDATA.y_exp = y_exp;
MYDATA.funeval = 0;
function val = L_Squares ( k )
// Computes the sum of squares of the differences.
f = myDifferences ( k )
val = sum(f.^2)
endfunction
// Initial guess
a = 0;
b = 0;
c = 0;
x0 = [a;b;c];
[fopt ,xopt]=leastsq(myDifferences, x0)
Does anyone know how to approach this problem?
Just rewrite lines 28,29 as
y_exp = [0.135,0.0924,0.067,0.0527,0.0363,0.02445,0.01668,0.012,0.009
0,0.00918,0.0132,0.01835,0.0261,0.03215,0.0366,0.0393,0.0401]';
or insert a clear at line 1 (you may have defined y_exp before with a different size).
The Fit Image Palette is quite nice and powerful. Is there a script interface that we can access it directly?
There is a script interface, and the example script below will get you started. However, the script interface is not officially supported. It might therefore be buggy or likely to change in future GMS versions.
For GMS 2.3 the following script works:
// create the input image:
Image input := NewImage("formula test", 2, 100)
input = 500.5 - icol*11.1 + icol*icol*0.11
// add some random noise:
input += (random()-0.5)*sqrt(abs(input))
// create image with error data (not required)
Image errors := input.ImageClone()
errors = tert(input > 1, sqrt(input), 1)
// setup fit:
Image pars := NewImage("pars", 2, 3)
Image parsToFit := NewImage("pars to fit", 2, 3)
pars = 10; // starting values
parsToFit = 1;
Number chiSqr = 1e6
Number conv_cond = 0.00001
Result("\n starting pars = {")
Number xSize = pars.ImageGetDimensionSize(0)
Number i = 0
for (i = 0; i < xSize; i++)
{
Result(GetPixel(pars, i, 0))
if (i < (xSize-1)) Result(", ")
}
Result("}")
// fit:
String formulaStr = "p0 + p1*x + p2*x**2"
Number ok = FitFormula(formulaStr, input, errors, pars, parsToFit, chiSqr, conv_cond)
Result("\n results pars = {")
for (i = 0; i < xSize; i++)
{
Result(GetPixel(pars, i, 0))
if (i < (xSize-1)) Result(", ")
}
Result("}")
Result(", chiSqr ="+ chiSqr)
// plot results of fit:
Image plot := PlotFormula(formulaStr, input, pars)
// compare the plot and original data:
Image compare := NewImage("Compare Fit", 2, 100, 3)
compare[icol, 0] = input // original data
compare[icol, 1] = plot // fit function
compare[icol, 2] = input - plot // residuals
ImageDocument linePlotDoc = CreateImageDocument("Test Fitting")
ImageDisplay linePlotDsp = linePlotDoc.ImageDocumentAddImageDisplay(compare, 3)
linePlotDoc.ImageDocumentShow()
I'm quite new to CUDA and GPU programming. I'm trying to write a Kernel for an application in physics. The parallelization is made over a quadrature of directions, each direction resulting in a sweep of a 2D cartesian domain. Here is the kernel. it actually works well, giving good results.
However, a very high number of registers per blocks leads to a spill to local memory that harshly slow down the code performance.
__global__ void KERNEL (int imax, int jmax, int mmax, int lg, int lgmax,
double *x, double *y, double *qd, double *kappa,
double *S, double *G, double *qw, double *SkG,
double *Ska,double *a, double *Ljm, int *data)
{
int m = 1+blockIdx.x*blockDim.x + threadIdx.x ;
int tid = threadIdx.x ;
//Var needed for thread execution
...
extern __shared__ double shared[] ;
//Read some data from Global mem
mu = qd[ (m-1)];
eta = qd[ MSIZE+(m-1)];
wm = qd[3*MSIZE+(m-1)];
amu = fabs(mu);
aeta= fabs(eta);
ista = data[ (m-1)] ;
iend = data[1*MSIZE+(m-1)] ;
istp = data[2*MSIZE+(m-1)] ;
jsta = data[3*MSIZE+(m-1)] ;
jend = data[4*MSIZE+(m-1)] ;
jstp = data[5*MSIZE+(m-1)] ;
j1 = (1-jstp) ;
j2 = (1+jstp)/2 ;
i1 = (1-istp) ;
i2 = (1+istp)/2 ;
isw = ista-istp ;
jsw = jsta-jstp ;
dy = dx = 1.0e-2 ;
for(i=1 ; i<=imax; i++) Ljm[MSIZE*(i-1)+m] = S[jsw*(imax+2)+i] ;
//Beginning of the vertical Sweep, can be from left to right,
// or opposite depending on the thread
for(j=jsta ; j1*jend + j2*j<=j2*jend + j1*j ; j=j+jstp) {
Lw = S[j*(imax+2)+isw] ;
//Beginning of the horizontal Sweep, can be from left to right,
// or opposite depending on the thread
for(i=ista ; i1*iend + i2*i<=i2*iend + i1*i ; i=i+istp) {
ax = dy ;
Lx = ax*amu/ex ;
ay = dx ;
Ly = ay*aeta/ey ;
dv = ax*ay ;
L0 = dv*kappaij ;
Sp = S[j*(imax+2)+i]*dv ;
Ls = Ljm[MSIZE*(i-1)+m] ;
Lp = (Lx*Lw+Ly*Ls+Sp)/(Lx+Ly+L0) ;
Lw = Lw+(Lp-Lw)/ex ;
Ls = Ls+(Lp-Ls)/ey ;
Ljm[MSIZE*(i-1)+m] = Ls ;
shared[tid] = wm*Lp ;
__syncthreads();
for (s=16; s>0; s>>=1) {
if (tid < s) {
shared[tid] += shared[tid + s] ;
}
}
if(tid==0) atomicAdd(&SkG[imax*(j-1)+(i-1)],shared[tid]*kappaij);
}
// End of horizontal sweep
}
// End of vertical sweep
}
How can i optimize the execution of this code ? I run it over 8 blocks of 32 threads.
The occupancy for this kernel is really low, limited by the registers according to the Visual profiler.
I have no idea on how to improve it.
Thanks !
First of all, you are using blocks of 32 threads, because of that, occupancy kernel is too low. Your gpu is running only 256 threads in parallel but it can run up to 1536 threads per multiprocessor (compute capability 2.x)
How many registers are you using?
You also can try to declare your variables into their local scope, helping to the device to reuse better the registers.