Go agouti testing fill textarea with PhantomJS perfomance issue - testing

I am using agouti with gomega and ginkgo in Go to test an upload form of our application consisting of an textarea which we fill.
This Code works fine for 1500 rows:
upload_externalData := page.Find("#upload_externalData")
buf := bytes.NewBuffer(nil)
f, err := os.Open("./files/external.log")
io.Copy(buf, f)
externalData := string(buf.Bytes())
Expect(upload_externalData.Fill(string(externalData))).Should(Succeed())
When increasing the imported data to the normal 25000 rows, PhantomJS blocks one CPU Core at 100% and nothing else happens.
Is there a way to achieve this?

Related

Why won't the BitmapImage for WizardStyle=modern resize in Inno Setup? [duplicate]

Bitmap for Inno Setup WizardImageFile (and WizardSmallImageFile) looks terrible because when Windows 7 has the large system fonts enabled, the Wizard is bigger than usual, but the images are scaled terrible wrong.
Is there a fix?
There is no similar issue if I add own picture somewhere like this:
BitmapImage1.AutoSize := True;
BitmapImage1.Align := alClient;
BitmapImage1.Left := 0;
BitmapImage1.Top := 0;
BitmapImage1.stretch := True;
BitmapImage1.Parent := Splash;
These are bitmap images, they naturally scale badly. You are just lucky that your own images do not look that bad when scaled.
You have to prepare your own set of images for common scaling factors.
Common scaling factors used nowadays are 100%, 125%, 150% and 200%. So you should have four sizes for the images, like:
WizardImage 100.bmp
WizardImage 125.bmp
WizardImage 150.bmp
WizardImage 200.bmp
WizardSmallImage 100.bmp
WizardSmallImage 125.bmp
WizardSmallImage 150.bmp
WizardSmallImage 200.bmp
Inno Setup can automatically select the best version of the image since 5.6.
Just list your versions of the images in the WizardImageFile and WizardSmallImageFile. You can use wildcards:
[Setup]
WizardImageFile=WizardImage *.bmp
WizardImageFile=WizardSmallImage *.bmp
On older versions of Inno Setup (or if your need to customize the selection algorithm or when you have additional custom images in the wizard), you would have to select the images programatically.
The following example does more or less the same what Inno Setup 5.6:
[Setup]
; Use 100% images by default
WizardImageFile=WizardImage 100.bmp
WizardSmallImageFile=WizardSmallImage 100.bmp
[Files]
; Embed all other sizes to the installer
Source: "WizardImage *.bmp"; Excludes: "* 100.bmp"; Flags: dontcopy
Source: "WizardSmallImage *.bmp"; Excludes: "* 100.bmp"; Flags: dontcopy
[Code]
function GetScalingFactor: Integer;
begin
if WizardForm.Font.PixelsPerInch >= 192 then Result := 200
else
if WizardForm.Font.PixelsPerInch >= 144 then Result := 150
else
if WizardForm.Font.PixelsPerInch >= 120 then Result := 125
else Result := 100;
end;
procedure LoadEmbededScaledImage(Image: TBitmapImage; NameBase: string);
var
Name: String;
FileName: String;
begin
Name := Format('%s %d.bmp', [NameBase, GetScalingFactor]);
ExtractTemporaryFile(Name);
FileName := ExpandConstant('{tmp}\' + Name);
Image.Bitmap.LoadFromFile(FileName);
DeleteFile(FileName);
end;
procedure InitializeWizard;
begin
{ If using larger scaling, load the correct size of images }
if GetScalingFactor > 100 then
begin
LoadEmbededScaledImage(WizardForm.WizardBitmapImage, 'WizardImage');
LoadEmbededScaledImage(WizardForm.WizardBitmapImage2, 'WizardImage');
LoadEmbededScaledImage(WizardForm.WizardSmallBitmapImage, 'WizardSmallImage');
end;
end;
You might want to do the same for the SelectDirBitmapImage, the SelectGroupBitmapImage and the PreparingErrorBitmapImage.
See also:
How to detect and "fix" DPI settings with Inno Setup?
Inno Setup Placing image/control on custom page

Delphi 10.2: Using Local SQL with Firedac Memory Tables

How can I use Firedac LocalSQL with FDMemtable? Is there any working example available?
According to the Embarcadero DocWiki I set up a local connection (using SQLite driver), a LocalSQL component and connected some Firedac memory tables to it. Then I connected a FDQuery and try to query the memory tables. But the query always returns "table xyz not known" even if I set an explicit dataset name for the memory table in the localSQL dataset collection.
I suspect that I miss something fundamental that is not contained in the Embarcadero docs. If anyone has ever got this up and running I would be grateful for some tips.
Here is some code I wrote for an answer here a while ago, which is a self-contained example of using LocalSQL, tested in D10.2 (Seattle). It should suffice to get you going. Istr that the key to getting it working was a comment somewhere in the EMBA docs that FD's LocalSQL is based on Sqlite, as you've noted.
procedure TForm3.CopyData2;
begin
DataSource2.DataSet := FDQuery1;
FDConnection1.DriverName := 'SQLite';
FDConnection1.Connected := True;
FDLocalSQL1.Connection := FDConnection1;
FDLocalSQL1.DataSets.Add(FDMemTable1);
FDLocalSQL1.Active := True;
FDQuery1.SQL.Text := 'select * from FDMemTable1 order by ID limit 5';
FDQuery1.Active := True;
FDMemTable1.Close;
FDMemTable1.Data := FDQuery1.Data;
end;
procedure TForm3.FormCreate(Sender: TObject);
var
i : integer;
MS : TMemoryStream;
begin
FDMemTable1.CreateDataSet;
for i := 1 to 10 do
FDMemTable1.InsertRecord([i, 'Row:' + IntToStr(i), 10000 - i]);
FDMemTable1.First;
// Following is to try to reproduce problem loading from stream
// noted by the OP, but works fine
MS := TMemoryStream.Create;
try
FDMemTable1.SaveToStream(MS, sfBinary);
MS.Position := 0;
FDMemTable1.LoadFromStream(MS, sfBinary);
finally
MS.Free;
end;
end;
As you can see, you can refer in the SQL to an existing FireDAC dataset simply by using its component name.

Is it possible to use adodb stream for large files using pre-allocated chunks?

I'm currently using comobject adodb.stream to grab very large files (multiple gb) from archive.org as an authenticated user and consequently receiving an out of memory error at allocation time. Is there a way to use adodb.stream to set the chunk size and append/concatenate to the output file?
Alternative solutions are welcome.
Here I am using AHKscript:
file_save_location := save
Overwrite := True
get_site := "https://archive.org/download/file.zip" ;;multi-gb file
post_site :="https://archive.org/account/login.php"
post_data :="username=" username_str "&password=" password_str "&remember=CHECKED&referer=https://archive.org&action=login&submit=Log in"
WebRequest := ComObjCreate( "WinHttp.WinHttpRequest.5.1" )
WebRequest.Open("POST", post_site)
WebRequest.SetRequestHeader("Content-Type", "application / zip, application / octet - stream""application / zip, application / octet - stream")
WebRequest.SetRequestHeader("Cookie", "test-cookie=1")
WebRequest.SetRequestHeader("Accept-Encoding","gzip,deflate,sdch")
WebRequest.Send(post_data)
WebRequest.Open("HEAD",get_site)
WebRequest.Send()
WebRequest.Open("GET",get_site)
WebRequest.Send()
ADODBObj := ComObjCreate( "ADODB.Stream" )
ADODBObj.Type := 1
ADODBObj.Open()
;;ADODBObj.Position := 0 ;;getting warm
ADODBObj.Write( WebRequest.ResponseBody )
ADODBObj.SaveToFile(file_save_location, Overwrite ? 2:1)
ADODBObj.Close()
ADODBObj:=""
WebRequest:=""

Memory error on raspberry pi in for loop taking dark frame images with Picamera

ok, so I'm trying to take 100 dark frames with the picamera. exposure_mode is set to "off" and the shutter speed is set at 5 milseconds I'm doing this in a fairly straightforward for loop:
for i in range(NUM_DARK_FRAMES):
print ('loop %s' % (i+1))
camera.capture(output, 'jpeg', bayer=True )
arr = output.array
print ('saving array')
numpy.save('%sDarkFrame_%s' % (dark_frames_path, i+1), arr)
gc.collect()
from the print statements I'm seeing that it's getting to loop 38, and returns a memoryerror. I added the explicit garbage collect because it helped someone else with a similar issue, to no avail. any ideas?
It seems that this is due to a limitation of the Raspberry Pi, in the advanced recipes pasted below, they suggest using streaming to capture a series of images, but this doesn't work with jpeg format, which is necessary for raw bayer data captures:
https://picamera.readthedocs.io/en/release-1.10/recipes2.html
I solved this by making an outer for loop that controls the camera instantiation and an inner loop that runs for 25 images. this clears the cache and prevents the problem from arising (it usually occurred between loop 37-39).

Setting up diagnostic error messages in large Mathematica projects

Whenever I create a large Mathematica project I run into this problem: Preventing avalanche of runtime errors in Mathematica, i.e., Mathematica's error message are opaque, archaic, and legion.
The idea then is to disable all of Mathematica's own error messages and implement type checking and error messages of your own in every Function and Module. However I have not found a simple and efficient way of doing this and end up with, e.g., some function generating an error 20 function calls deep and then get a whole cascade of error messages all the way back up to the main routine.
How would you set up a simple mechanism for this that only generates one error message at the function that experiences the error and a simple list of the chain of function calls?
EDIT: Since it has come up in a couple of answers; I am specifically looking for something lightweight regarding the output it produces (otherwise I could just stick with Mathematica's error messages) and obviously also lightweight in computational overhead. So while Stack and Trace are definitely light on the overhead, their output in complex projects is not quick to parse and some work needs to be done simplifying it.
YAsI - Yet Another (silly?) Idea ...
Re-reading your question ...
The idea then is to disable all of Mathematica's own error messages and implement type checking and error messages of your own in every Function and Module.
Found this:
$MessagePrePrint = ( #; Print[Stack[_][[;; -5]]]; Abort[]) &
v[x_, y_] := w[x, y];
w[x_, y_] := x/y;
StackComplete#v[1, 0];
During evaluation of In[267]:= {StackComplete[v[1,0]];,
StackComplete[v[1,0]], v[1,0], w[1,0], 1/0, 1/0, Message[Power::infy,1/0]}
Out[267]= $Aborted
conclusion ... Aborts at first message and leaves a "reasonable" stack trace. "Reasonable" means "Should be improved".
But it is completely non-intrusive!
To get the ball rolling here is one idea that I've been toying with; the creation of a pseudo stack.
First make a global variable theStack={} and then in every Function or Module start with AppendTo[theStack,"thisFuncName"] and end with theStack=Most#theStack. Assuming moderate (~a few tens) depth of function calls, this should not add any significant overhead.
Then implement your own typing/error checking and use Print#theStack;Abort[]; on errors.
Refinements of this method could include:
Figuring out a way to dynamically get "thisFuncionName" so that the AppendTo[] can be made into an identical function call for all Functions and Module.
Using Message[] Instead of Print[].
Pushing other important variables / stateful information on theStack.
One attempt to implement #Timo's idea (theStack)
Incomplete and perhaps flawed, but just to keep thinking about it:
Clear["Global`*"];
funcDef = t_[args___] \[CircleMinus] a_ :>
{t["nude", args] := a,
ReleaseHold[Hold[t[args] :=
(If[! ValueQ[theStack], theStack = {}];
AppendTo[theStack, ToString[t]];
Check[ss = a, Print[{"-TheStack->", Evaluate#theStack}];
Print#Hold[a]; Abort[]];
theStack = Most#theStack;
Return[ss])
]]};
v[x_, y_]\[CircleMinus] (Sin# g[x, y]) /. funcDef;
g[x_, y_]\[CircleMinus] x/y /. funcDef;
v[2, 3]
v[2, 0]
Output:
Out[299]= Sin[2/3]
During evaluation of In[295]:= Power::infy: Infinite expression 1/0 encountered. >>
During evaluation of In[295]:= {-TheStack->,{v,g}}
During evaluation of In[295]:= Hold[2/0]
Out[300]= $Aborted
A suggestion for extracting stack, maybe something that relies on Trace?
An example of using Trace below, from Chris Chiasson. This code saves evaluation tree of 1 + Sin[x + y] + Tan[x + y] into ~/temp/msgStream.m
Developer`ClearCache[];
SetAttributes[recordSteps, HoldAll];
recordSteps[expr_] :=
Block[{$Output = List#OpenWrite["~/temp/msgStream.m"]},
TracePrint[Unevaluated[expr], _?(FreeQ[#, Off] &),
TraceInternal -> True];
Close /# $Output;
Thread[
Union#Cases[
ReadList["~/temp/msgStream.m", HoldComplete[Expression]],
symb_Symbol /;
AtomQ#Unevaluated#symb &&
Context#Unevaluated#symb === "System`" :>
HoldComplete#symb, {0, Infinity}, Heads -> True],
HoldComplete]
];
recordSteps[1 + Tan[x + y] + Sin[x + y]]
To answer Samsdram's question, the code below (also from Chris) gives evaluation tree of a Mathematica expression. Here is the post from MathGroup with source code and examples.
(Attributes## = {HoldAllComplete}) & /# {traceToTreeAux, toVertex,
HoldFormComplete, getAtoms, getAtomsAux}
MakeBoxes[HoldFormComplete[args___], form_] :=
MakeBoxes[HoldForm[args], form]
edge[{head1_, pos1_, xpr1_}, {head2_, pos2_, xpr2_}] :=
Quiet[Rule[{head1, vertexNumberFunction#pos1, xpr1}, {head2,
vertexNumberFunction#pos2, xpr2}], {Rule::"rhs"}]
getAtomsAux[atom_ /; AtomQ#Unevaluated#atom] :=
Sow[HoldFormComplete#atom, getAtomsAux]
getAtomsAux[xpr_] := Map[getAtomsAux, Unevaluated#xpr, Heads -> True]
getAtoms[xpr_] := Flatten#Reap[getAtomsAux#xpr][[2]]
toVertex[traceToTreeAux[HoldForm[heldXpr_], pos_]] := toVertex[heldXpr]
toVertex[traceToTreeAux[HoldForm[heldXprs___], pos_]] :=
toVertex#traceToTreeAux[Sequence[], pos]
(*this code is strong enough to not need the ToString commands,but \
some of the resulting graph vertices give trouble to the graphing \
routines*)
toVertex[
traceToTreeAux[xpr_, pos_]] := {ToString[
Short#Extract[Unevaluated#xpr, 0, HoldFormComplete], StandardForm],
pos, ToString[Short#First#originalTraceExtract#{pos}, StandardForm]}
traceToTreeAux[xpr_ /; AtomQ#Unevaluated#xpr, ___] := Sequence[]
traceToTreeAux[_HoldForm, ___] := Sequence[]
traceToTreeAux[xpr_, pos_] :=
With[{lhs = toVertex#traceToTreeAux[xpr, pos],
args = HoldComplete ## Unevaluated#xpr},
Identity[Sequence][
ReleaseHold[
Function[Null, edge[lhs, toVertex##], HoldAllComplete] /# args],
ReleaseHold#args]]
traceToTree[xpr_] :=
Block[{vertexNumber = -1, vertexNumberFunction,
originalTraceExtract},
vertexNumberFunction[arg_] :=
vertexNumberFunction[arg] = ++vertexNumber;
originalTraceExtract[pos_] :=
Extract[Unevaluated#xpr, pos, HoldFormComplete]; {MapIndexed[
traceToTreeAux, Unevaluated#xpr, {0, Infinity}]}]
TraceTreeFormPlot[trace_, opts___] :=
Block[{$traceExpressionToTree = True},
Through#{Unprotect, Update}#SparseArray`ExpressionToTree;
SparseArray`ExpressionToTree[trace, Infinity] = traceToTree#trace;
With[{result = ToExpression#ToBoxes#TreeForm[trace, opts]},
Through#{Unprotect, Update}#SparseArray`ExpressionToTree;
SparseArray`ExpressionToTree[trace, Infinity] =.;
Through#{Update, Protect, Update}#SparseArray`ExpressionToTree;
result]];
TraceTreeFormPlot[Trace[Tan[x] + Sin[x] - 2*3 - 55]]
Perhaps we have been over thinking this. What if we just tweaked the pattern matching on the arguments a little. For instance, if we modified the function to check for a numeric quantity and added some code to print an error if it fails. For instance,
TypeNumeric[x_] := If[! NumericQ[Evaluate[x]],
Print["error at "]; Print[Stack[]]; Print["Expression "]; Print[x]; Print["Did
not return a numeric value"];Return[False],
(*Else*)
Return[True];]
SetAttributes[TypeNumeric, HoldAll];
Step 2: If you have a function, f[x_] that requires a numeric quantity, just write it with the standard pattern test and all should be well
Input:
f[x_?TypeNumeric] := Sqrt[x]
f[Log[y]]
f[Log[5]]
Output:
error at
{f}
Expression
Log[y]
Did not return a numeric value
f[Log[y]]
Sqrt[Log[5]]
I believe this will work and, it makes robust type checking as simple as a writing a function or two. The problem is that this could be hugely inefficient because this code evaluates the expression x twice, once for the type checking and once for real. This could be bad if an expensive function call is involved.
I haven't figured out the way around this second problem and would welcome suggestions on that front. Are continuations the way out of this problem?
Hope this helps.