I am using artillery and I am exporting the results as a json file with -o result.json
The problem is that it seems that the json files structure are not consistent depending on the arrivalRate.
When arrivalRate=5, the latencies branch for the intermediate reslts contains 4 fields (see image below).
When I set arrivalRate to a higher value, for example, arrivalRate=20 the json response just includes the latency without the other 3 fields. (see image below).
I would like to get all the information as in the first image.
Related
I am trying to build a financial dashboard with Flask and pymongo. The starting point is a flask form which saves data in a MongoDB database. One of the fields in the form is a FileField (wtforms) which allows the upload of a PDF, which is then stored in MongoDB with GridFS.
Now I manage to save the pdf and I can see the resulting entries within the .files and .chunks collections. Now I would like to build a function that retrieves the PDFs and analyses them with some basic NLP, however I struggle with the getting meaningful data.
When I do:
storage = gridfs.GridFS(db, collection)
data = storage.get('some id')
a = data.read()
The result is a binary file. If I continue with:
with open(data, 'rb') as f:
b = f.read()
The result is "ValueError: embedded null byte or sometimes an empty "byte string".
Any help on this?
To follow up on the above, I found a solution for myself that consists in 2 separate functions:
(1) Upon upload of the form and before uploading the files to MongoDB, I apply a function based on pdfminer that extracts the string content of the PDF and tranform it into a list of sentences using NLTK. I will then store this list in the .files via the storage.put(file, sent_list = sent_list) #sent_list being the variable name of the list of sentences.
Whenever I wish to run NLP operations on the file, I will just call the "sent_list" variable from mongodb.
(2) If I wish to display the stored pdf in its original content however, I included the following function as a separate route.
storage = GridFS(db, collection)
data = storage.get_last_version(filename)
response = make_response(data.read())
extension = data.filename.split('.')[-1]
response.headers['Content-Type'] = f'application/{extension}'
response.headers['Content-Disposition'] = f'inline; filename={data.filename}'
return response
(2) will open a new tab in my flask app showing the .pdf file in its original format.
I hope this helps anyone coming across a similar problem in the future.
Is there a way to script a pair of dual-EELS SIs so that DM recognises them as siblings (say, for use in the "SI->Align SI by Peak" menu option)?
I have a script which performs a transformation on a pair of dualEELS SIs where the results are given in new images with all the tags copied from the original. The new SIs do not seem to be acknowledged by the SI options as a pair, however. A MWE of how this occurs is below:
image a, b
GetTwoLabeledImagesWithPrompt("Get SI", \
"Get DualEELS SIs", \
"Low-Loss", a, \
"High-Loss", b)
image LL, HL
LL := a.ImageClone()
HL := b.ImageClone()
LL.ShowImage()
HL.ShowImage()
Assuming that the two inputs are real dualEELS SIs. Attempting afterward to run a method like "SI->Align SI by Peak" on the outputs does not recognise the second SI as a sibling.
I suspect that my issue is in properly assigning the four EELS:Dual acquire sibling:UID tags highlighted in the image provided, however I have no idea how (or if) these are accessible from the scripting language.
Thanks in advance for any help you are able to render.
Yes, "siblings" in DigitalMicrograph are recognized by tag-checks and tagging of the Unique Image Id (UID).
Depending on the exact application/plugin, there might be additional tag-checks before a sibling can be accepted (i.e. "is it EELS data?", "is it spatially compatible?", etc.) but he prime-mechanism is using the UID.
A bit of background info
The UID is a set of four long-numbers generated at random whenever a new image data is created, and then subsequently stored with the data. It is "unique" by the assumption that a set of four randomly generated 8-byte longs are "unique".
If you create an image, save it to disc, and open it, the UID will be the same. (It is stored with the data.)
If you ImageClone() an image, it gets a new UID.
If you copy a image file on the hard drive and rename it, it will retain the UID.
Creating and using UIDs
The commands to get an image's UID are described in the F1 help documentation here:
And the example section even has a script showing how one uses UID to "link" data together:
I am working on a project that integrates Imebra inside an android application. The application is supposed to extract all the data from a given DICOM file and put them into a .xml file. I need a little bit of help with it. For example, I don't know how to get all the VR tags that the given DICOM has, instead of getting them one by one using tag ids.
Thank you for your help.
Load the file using CodecFactory.load(filename).
Then you can use DataSet.getTags() to retrieve a list of tags stored into the DICOM structure.
The returned class TagsIds is a list containing all the TagId: scan each tag ID and retrieve it via DataSet.getString() (to retrieve the value as string) and DataSet.getDataType() to retrieve its VR.
When DataSet.getString() fails then you are dealing with a sequence (an embedded DICOM structure) which can be retrieved with DataSet.getSequenceItem().
You can use the static method DicomDictionary.getTagName() to get a description of a particular tag.
For example, if my text data come from a database, how can I get one line/doc(as a database record) using the same mechanism (subclassing Dataset such that the pipeline described here still works) as TextLineDataset ?
By looking at the source code of TextLineDataset, I find that make_dataset_resource() seems an import method to be implemented. But I can't find where the actual code of yielding a line from a file as the docstring of TextLineDataset says: A Dataset comprising lines from one or more text files.
I have two documents:
Document 1 (input)
Document 2 (output)
Document 2 is the result of passing Document 1 through a transformation process which leaves any content and formatting intact (verified by side-by-side compare in Word).
However, the process removes many id numbers from the .docx files.
For example,
<w:p w:rsidP="00B600D6" w:rsidR="00F55D78" w:rsidRDefault="00B600D6">
becomes
<w:p>
according to a dump of each document via the following code:
Body body = ((Document)newerPackage.getMainDocumentPart().getJaxbElement()).getBody();
Node node = org.docx4j.XmlUtils.marshaltoW3CDomDocument(body).getDocumentElement();
TransformerFactory tf = TransformerFactory.newInstance();
Transformer transformer = tf.newTransformer();
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "no");
transformer.setOutputProperty(OutputKeys.METHOD, "xml");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
transformer.transform(new DOMSource(node),
new StreamResult(new OutputStreamWriter(System.out, "UTF-8")));
Using the docx4j Differencer comparison method recommended here, everything (except the first line which has no formatting applied) is shown as a modification.
Question is: Are the diffs a result of the missing id's, the formatting or something else?
In case it's important, we're using docx4j in this context to perform automated sanity/regression tests on our round-tripping proceess (i.e. apply the "loss-less" process and expect no differences)
Disclosure: I work on docx4j
If the only difference between paragraphs is the rsid attributes, they will still be detected as different.
You could "clean" the documents before performing the comparison, so that neither docx has rsid attributes. See the Filter sample.
By the way, an easier way to see the XML for an object (eg a single paragraph, or the entire body) is to use XmlUtils.marshaltoString