I am trying to connect to read a large table (5,000,000 rows) from the OrientDB database with select and native API driver. When I connect to the local database, the applications loads for some time and then such a message is displayed:
java.lang.OutOfMemoryError: Java heap space
Here are the parameters I defined for JVM in order to improve performance. Without these parameters I get GC overhead limit error:
-Xmx800m -Dstorage.diskCache.bufferSize=7200 -server -XX:+PerfDisableSharedMem
What should I change in this settings or in the cluster settings in order to fix thsi error and to run the application?
OrientDB is set up in a clustered mode using Docker:
version: '3'
services:
node1:
image: orientdb:latest
command: /orientdb/bin/server.sh -Ddistributed=true
volumes:
- /orientdb/config1:/opt/orientdb/config
- /orientdb/databases1:/orientdb/databases
- /orientdb/backup1:/orientdb/backup
- ./data:/orientdb/bin/data
environment:
ORIENTDB_ROOT_PASSWORD: 'pwd1'
ORIENTDB_NODE_NAME: node1
ports:
- "2424:2424"
- "2480:2480"
node2:
image: orientdb:latest
command: /orientdb/bin/server.sh -Ddistributed=true
volumes:
- /orientdb/config2:/opt/orientdb/config
- /orientdb/databases2:/orientdb/databases
- /orientdb/backup2:/orientdb/backup
- ./data2:/orientdb/bin/data
environment:
ORIENTDB_ROOT_PASSWORD: 'pwd1'
ORIENTDB_NODE_NAME: node2
depends_on:
- node1
OrientDB configurations setup:
val orientGraph = new OrientGraph("url","user","password")
How I run the query:
val orientTableRows:OrientDynaElementIterable = orientGraph.command(new OCommandSQL("select count(*) as count from Table1"))
.execute()
build.sbt file with OrientDB dependencies:
val orientDbVersion = "3.0.4"
libraryDependencies ++= Seq(
"com.orientechnologies" % "orientdb-graphdb" % orientDbVersion,
"com.orientechnologies" % "orientdb-distributed" % orientDbVersion,
"com.orientechnologies" % "orientdb-core" % orientDbVersion,
"com.orientechnologies" % "orientdb-client" % orientDbVersion,
"com.orientechnologies" % "orientdb-server" % orientDbVersion,
"com.orientechnologies" % "orientdb-test-commons" % orientDbVersion,
"com.orientechnologies" % "orientdb-tools" % orientDbVersion,
"com.orientechnologies" % "orientdb-etl" % orientDbVersion,
"com.tinkerpop.blueprints" % "blueprints-core" % "2.6.0"
)
Related
How can the following formula be simplified in Maxima:
diff(h((x-1)^2),x,1)
Mathematically it should be : 2*(x-1)*h'((x-1)^2)
But maxima gives : d/dx h((x-1)^2)
Maxima doesn't apply the chain rule by default, but there is an add-on package named pdiff (which is bundled with the Maxima installation) which can handle it.
pdiff means "positional derivative" and it uses a different, more precise, notation to indicate derivatives. I'll try it on the expression you gave.
(%i1) load ("pdiff") $
(%i2) diff (h((x - 1)^2), x);
2
(%o2) 2 h ((x - 1) ) (x - 1)
(1)
The subscript (1) indicates a first derivative with respect to the argument of h. You can convert the positional derivative to the notation which Maxima usually uses.
(%i3) convert_to_diff (%);
!
d !
(%o3) 2 (x - 1) (----- (h(g485))! )
dg485 ! 2
!g485 = (x - 1)
The made-up variable name g485 is just a place-holder; the name of the variable could be anything (and if you run this again, chances are you'll get a different variable name).
At this point you can substitute for h or x to get some specific values. Note that ev(something, nouns) means to call any quoted (evaluation postponed) functions in something; in this case, the quoted function is diff.
(%i4) ev (%, h(u) := sin(u));
!
d !
(%o4) 2 (x - 1) (----- (sin(g485))! )
dg485 ! 2
!g485 = (x - 1)
(%i5) ev (%, nouns);
2
(%o5) 2 cos((x - 1) ) (x - 1)
c2: (x-0.5)^2+y^2=1;
solve(c2,y);
cs2: map(rhs, %);
at(cs2,[x=0.5]);
I expect to see [-1,1], but I get this instead
What should I do?
Incidentally, maxima cannot plot this equation for a reason unknown to me.
plot2d(cs2, [x,-2.5,2.5],[same_xy, true]);
plot2d: expression evaluates to non-numeric value everywhere in plotting range.
plot2d: expression evaluates to non-numeric value everywhere in plotting range.
plot2d: nothing to plot.
Build info:
wxbuild_info();
wxMaxima version: 19.07.0
using wxWidgets version: wxWidgets 3.0.4
Maxima version: 5.43.2
Maxima build date: 2020-02-21 05:22:38
Host type: x86_64-pc-linux-gnu
System type: BSD BSD NIL
Lisp implementation type: GNU Common Lisp (GCL)
Lisp implementation version: GCL 2.6.12
Whole output:
I can't reproduce the behavior you're seeing. I compiled Maxima 5.43.2 (version number slightly mixed up in build_info output below) and I don't get the results you're seeing. Here's what I see with the command line interface (I get the same result with wxMaxima).
(%i1) c2: (x - 0.5)^2 + y^2 = 1;
2 2
(%o1) y + (x - 0.5) = 1
(%i2) solve (c2, y);
rat: replaced -0.5 by -1/2 = -0.5
2 2
sqrt((- 4 x ) + 4 x + 3) sqrt((- 4 x ) + 4 x + 3)
(%o2) [y = - ------------------------, y = ------------------------]
2 2
(%i3) cs2: map (rhs, %);
2 2
sqrt((- 4 x ) + 4 x + 3) sqrt((- 4 x ) + 4 x + 3)
(%o3) [- ------------------------, ------------------------]
2 2
(%i4) at (cs2, x = 0.5);
(%o4) [- 1.0, 1.0]
(%i5) plot2d(cs2, [x,-2.5,2.5],[same_xy, true]);
plot2d: expression evaluates to non-numeric value somewhere in plotting range.
plot2d: expression evaluates to non-numeric value somewhere in plotting range.
(%o5) [/tmp/maxout9196.gnuplot, /tmp/maxplot.x11]
The plot2d displays a circle.
For the record, build_info reports:
Maxima version: "5.43.1_10_g5476356eb"
Maxima build date: "2022-01-31 12:13:20"
Host type: "i686-pc-linux-gnu"
Lisp implementation type: "CLISP"
Lisp implementation version: "2.49 (2010-07-07) (built 3605610186) (memory 3852648802)"
User dir: "/home/robert/.maxima"
Temp dir: "/tmp"
Object dir: "/home/robert/tmp/maxima-read-only-git/maxima-code/binary/5_43_1_10_g5476356eb/clisp/2_49__2010_07_07___built_3605610186___memory_3852648802_"
Frontend: false
It's puzzling that you are seeing a different result for solve. Do you get the same result in command line Maxima, or something else?
I am solving an optimization problem using MATLAB with YALMIP which is similar to a unit commitment problem.
This is a Mixed Integer Linear Programming problem.
I am considering two decision variables --> P and T_room
I formulated the problem in MATLAB with YALMIP,
When I use CPLEX and MOSEK solvers, I am getting different results.
Can anyone explain to me why that is happening?
CPLEX solver results
MOSEK solver results
clc;
clear all;
rng(0,'twister');
%% time step of the demand response --> 15 mins
deltaT = 15/60 ;
%% number of air conditioners
Nunits = 3;
%% demand required for each interval
Pdemand = [2 3 4 2 5 5 8 9]';
% Pdemand = randi([2,10],8,1);
%% time interval considered for the study --> 15 mins intervals for 2
%%% hours
Horizon = length(Pdemand) ;
%% cost for each appliance ( need to think little about this approach )
Q = diag([rand(1,Nunits)]);
%% defining the possible reduction levels for the appliances
% ratedPower = [1 6 4]';
ratedPower = randi([5,7],Nunits,1);
%% defining minimum and maximum power limits for the appliances
Pmax = randi([6,8],Nunits,1);
Pmin = zeros(Nunits,1);
% possible power levels for appliances (discrete levels)
LevelsOutput = [0 0.5 0.75 1.0] ;
% matrix containing all the possible levels of output power for all the
%% appliances
PowerLevelMatrix = ratedPower * LevelsOutput ;
%% parameters based on the temperature
%%% outside temperature (we assume that the outdoor temperature remains the same throughout the optimisation process)
T_outdoor = randi([34, 38], Nunits,1);
%%% preferred lower level of temperature for the appliances
T_lower = randi([20,21], Nunits,1) ;
%%% preferred upper level of temperature for the appliances
T_upper = randi([28, 29], Nunits,1) ;
%%% initial temperature for the appliances
T_initial = randi([22, 28], Nunits,1);
%% parameters of the house
%% equivalent thermal resistance
R = (2.5-1.5)*rand(Nunits,1) + 1.5 ;
%%% equivalent thermal capacitance
Ca = (2.5-1.5)*rand(Nunits,1) + 1.5 ;
%% for the air condtioner
c = repmat(0.03,Nunits,1);
d = repmat(-0.4,Nunits,1);
a = repmat(0.06,Nunits,1);
b = repmat(-0.3,Nunits,1);
%% defining the variables of the demand response problem
%%% power levels for the appliances
P = sdpvar(Nunits, Horizon, 'full');
%% slack variable for exceeding the demand
% Demand_Penalty = 50;
% DemandSlack = sdpvar(1,Horizon);
%% decision variable for the room temperature
T_room = sdpvar(Nunits,Horizon,'full') ;
%% Constraints for the optimisation problem
Constraints = [];
for k = 1: Horizon
% Constraints = [Constraints, sum(P(:,k)) + DemandSlack(k) >= Pdemand(k)];
% Constraints = [Constraints,DemandSlack(k) >= 0];
Constraints = [Constraints, sum(P(:,k))>= Pdemand(k)];
end
%%% constraint on min amd max power levels
for k = 1: Horizon
Constraints = [Constraints, Pmin <= P(:,k) <= Pmax];
end
%%% adding the constraint on discrete power levels for each appliance
for nApp = 1 : Nunits
for k = 1 : Horizon
Constraints = [Constraints, ismember(P(nApp,k),PowerLevelMatrix(nApp,:))];
end
end
%% the temperature should be within the desired limits --> cannot go beyond those limits
for k = 2: Horizon
Constraints = [Constraints, T_lower <= T_room(:,k) <= T_upper ];
end
%% adding the initial constraint of temperature
Constraints = [Constraints, T_room(:,1) == T_initial];
%%% adding the temperature model of the constraint for the inverter type
%%% air condtioner
for k = 2: Horizon
Constraints = [Constraints, T_room(:,k) == T_outdoor - ((a./c).*P(:,k) + ((b.*c - a.*d)./c)).*R.*(1- exp((-deltaT)./(R.*Ca))) - T_outdoor.* exp((-deltaT)./(R.*Ca)) + T_room(:,k-1).* exp((-deltaT)./(R.*Ca)) ];
end
%% adding a sample constraints to the temperature update
% for k =2: Horizon
%
% Constraints = [Constraints, T_room(:,k) == T_room(:,k-1) ];
% end
%%% defining the objective function
% weighting factors
alpha = 1 ;
beta = 1 ;
Objective = 0 ;
for k = 2 : Horizon
Objective = Objective + alpha * Q * P(:,k) + beta * ((2* T_room(:,k) - T_lower - T_upper)./(T_upper - T_lower));
% Objective = Objective + P(:,k)' * Q * P(:,k);
% Objective = Objective + Demand_Penalty * DemandSlack(k) ;
end
%%% solving the optimisation problem
%
ops = sdpsettings('verbose',2, 'debug', 1, 'solver', 'mosek');
% ops = sdpsettings('verbose',1, 'debug', 1);
solution = optimize (Constraints, Objective, ops) ;
Pout = value(P);
T_room_out = value(T_room);
Your code does not make sense to begin with, as your objective is a vector
>> Objective
Linear matrix variable 3x1 (full, real, 42 variables)
With that, YALMIP will solve three optimization problems, and then values you obtain when applying value command will be the values from the last solution.
Having said that, concentrating on the last objective you have defined (although I think you intended to minimize the sum or something), I get the same objective with both solvers (not the same solution, the solution simply isn't unique)
>> solution = optimize (Constraints, Objective(3), sdpsettings('solver','cplex'))
>> value(Objective(3))
ans =
1.8418
>> value(P)
ans =
0 5.2500 3.5000 5.2500 3.5000 5.2500 5.2500 5.2500
0 3.0000 3.0000 6.0000 6.0000 6.0000 3.0000 4.5000
5.0000 5.0000 5.0000 5.0000 5.0000 5.0000 0 0
>> solution = optimize (Constraints, Objective(3), sdpsettings('solver','mosek'))
>> value(Objective(3))
ans =
1.8418
ans =
3.5000 3.5000 3.5000 3.5000 3.5000 3.5000 5.2500 5.2500
3.0000 3.0000 3.0000 3.0000 6.0000 0 3.0000 6.0000
3.7500 5.0000 5.0000 5.0000 5.0000 5.0000 0 0
I have to increment a number in velocity template engine.
#set ($newIndex = $oldIndex+ 1)
$newIndex
but i am getting incorrect result e.g if value of $oldIndex is 1 then i am getting 11 instead of 2
Assuming oldIndex defined as a number, it seems similar to substraction not working
Your example is not copied correctly. If a dash is next to an identifier, it becomes part of the identifier. So this example would fail, since the "-" sign is actually part of the Total2 identifier.
Add space after variable name:
#set ($newIndex = $oldIndex + 1)
It has nothing to do with spaces in this case, but rather with the initial content of $oldIndex.
The following template:
#set ($i = '1')
#set ($i = $i + 1)
$i.class.name / $i
#set ($j = 1)
#set ($j = $j + 1)
$j.class.name / $j
displays:
11 / java.lang.String
2 / java.lang.Integer
So you need to convert $oldIndex to a number, but the way to do it depends of the tools you have available in your context to do so (if you can't put $oldIndex as a number in the context in the first place). You would typically do something like #set ($oldIndex = $number.toNumber($oldIndex)) if you have the NumberTool present in your context, for instance.
There's a method that will probably work in all environments:
#set ($dummy = 1) ## creates an Integer variable
#set ($oldIndex = $dummy.valueOf($oldIndex)) ## uses Integer.valueOf() method
And that's it, $oldIndex contains an integer! Please note that the Integer.valueOf() method may throw a NumberFormatException if given something else than a number representation.
I ran julia --track-allocation prof.jl resulting in the following output:
- using FixedSizeArrays
-
- immutable KernelVals{T}
- wavenumber::T
- vect::Vec{3,T}
- dist::T
- green::Complex{T}
- gradgreen::Vec{3,Complex{T}}
- end
-
- function kernelvals(k, x, y)
- r = x - y
0 R2 = r[1]*r[1]
0 R2 += r[2]*r[2]
0 R2 += r[3]*r[3]
0 R = sqrt(R2)
-
0 γ = im*k
0 expn = exp(-γ * R)
0 fctr = 1.0 / (4.0*pi*R)
0 green = fctr * expn
64 gradgreen = -(γ + 1/R) * green / R * r
-
0 KernelVals(k, r, R, green, gradgreen)
- end
-
- function payload()
- x = Vec{3,Float64}(0.47046262275611883,0.8745228524771103,-0.049820876498487966)
0 y = Vec{3,Float64}(-0.08977259509004082,0.543199687600189,0.8291184043296924)
0 k = 1.0
0 kv = kernelvals(k,x,y)
- return kv
- end
-
- function driver()
- println("Flush result: ", payload())
0 Profile.clear_malloc_data()
0 payload()
- end
-
- driver()
I cannot get rid of the final memory allocation on the line starting gradgreen.... I ran #code_warntype kernelsvals(...), revealing no type instability or uncertainty.
The allocation pattern is identical on julia-0.4.6 and julia-0.5.0-pre.
This function will be the inner kernel in a boundary element method I am implementing. It will be called literally millions of times, resulting in a gross memory allocation that can grow to be a multiple of the physical memory available
to me.
The reason I am using FixedSizeArrays is to avoid allocations related to the creation of small Arrays.
The precise location where the allocation is reported depends in a very sensitive manner on the code. At some point the memory profiler was blaming 1/(4*pi*R) as the line triggering allocation.
Any help or general tips on how to write code resulting in predictable allocation patterns is highly appreciated.
After some experiments I finally managed to get rid of all allocations. The culprit turned out to be the promotion architecture as extended in FixedSizeArrays. Apparently multiplying a complex scalar and a real vector creates a temporary along the way.
Replacing the definition of gradgreen with
c = -(γ + 1/R) * green / R
gradgreen = Vec(c*r[1], c*r[2], c*r[3])
results in allocation-free runs. In my benchmark example execution time came down from 6.5 seconds to 4.15 seconds. Total allocation size from 4.5 GB to 1.4 GB.
EDT: Reported this issue to FixedSizeArrays developers, who fixed it immediately (thank you!). Allocations disappeared completely.