I'm running a simple penguin pipeline in interactive mode with a split train/eval, the transform step run but i can't get post_transform_statistics artifacts.
Inside the dedicated artifacts folder /tmp/tfx-penguin_custom_INTERACTIVE-nq5dn56x/Transform/post_transform_stats/5, i have just one FeaturesStats.pb inside, but not subfolders Split-train and Split-eval with a FeaturesStats.pb inside each.
However, I have the subfolders inside artifacts dedicated to transformed examples (/tmp/tfx-penguin_custom_INTERACTIVE-nq5dn56x/Transform/transformed_examples/5/).
Here is how i define the transform components by explicitly providing splits and also disable_statistics=False:
transform = tfx.components.Transform(
examples=example_gen.outputs['examples'],
schema=schema_gen.outputs['schema'],
disable_statistics=False,
splits_config= transform_pb2.SplitsConfig(
analyze=['train'], transform=['train', 'eval']),
module_file=_transformer_module_file)
I went to the docstring and even the __init__ of the component https://github.com/tensorflow/tfx/blob/master/tfx/components/transform/component.py, it seems there is nothing i would have forgotten or mistaken but i was very disturbed to read following comment with an untraceable location for stats....
disable_statistics: If True, do not invoke TFDV to compute pre-transform
and post-transform statistics. When statistics are computed, they will
will be stored in the `pre_transform_feature_stats/` and
`post_transform_feature_stats/` subfolders of the `transform_graph`
export.
For now, the workaround is to explicitly disable stats in the transform component and define next to it, a dedicated statistics components to work on transformed features splits but it would have been great to have the splits statistics inside transform component directly.
Thanks for any help
This is expected as StatisticsGen in Transform is currently working on the entire transform dataset regardless of split/span.
To generate separate statistics for different splits, please use StatisticsGen component.
Thank you!
I'm trying to make a line chart on Data Studio but I'm going crazy.
I need to do a control/filter for METR_1, METR_2, METR_3, to hide/show them or just one of them.
DIMEN
METR_1
METR_2
METR_3
DIMEN1
Metr_1a
Metr_2a
Metr_3a
DIMEN2
Metr_1b
Metr_2b
Metr_3b
DIMEN3
Metr_1c
Metr_2c
Metr_3c
In all my solutions, I have to replace the dimension but I need it.
Do you have any advice?
Thanks anyway
Not sure if this solves your problem (could you share a screenshot of your line chart?) but you might be looking for the optional metric feature. More instructions at the link, but you can enable this by flipping a switch in the 'Metrics' panel.
Your report users will be able to select any combination of metrics (METR_1, METR_2, METR_3) for display, from the chart header.
I'm trying to use "videojs-thumbnails" plugin from https://github.com/brightcove/videojs-thumbnails and noticed that thumbnail's time, specified in the plugin configuration is not matching the timestamp in the seek bar. Coming across different comments regarding this issue I found suggestion at https://github.com/brightcove/videojs-thumbnails/issues/43 to replace line
mouseTime = Math.floor((left - progressControl.el().offsetLeft) / progressControl.width() * duration);
to
mouseTime = Math.floor((left) / progressControl.width() * duration);
By removing
- progressControl.el().offsetLeft
However, that produces still not exact time match.
Finally I came with redefined value for the
var left
Getting it from the current value of
.vjs-mouse-display
So, my final codes are:
left=parseInt((document.querySelector('.vjs-mouse-display').style.left),10);
mouseTime = Math.floor((left) / progressControl.width() * duration);
Now everything works correctly.
Greatly appreciate for comments/suggestions.
This is NOT an answer to your specific question / issue, but rather, an alternative implentation approach.
I, too, wanted thumbnails for my video(s), so I proto-typed a page
that used videoJS's plugin. I don't recall all the details of the
issues that I ran into trying to use that plugin, but I finally decided
to abandon the plugin, and design an alternative, which has its own
separate 'slider' just above the viewer. [ One 'drags' my slider,
(rather than hover along it, as you do on YouTube's videos), so that it
can work straight-forwardly on touch-screens...i.e. on Android, etc. ]
And, rather than try to extract images from the video in real-time,
(See: How to generate video preview thumbnails for use in VideoJS? ), I chose to prepare the images, ahead of time, using 'ffmpeg' and the cmd-line interface to 'ImageMagick'.
Details of that part are here:
http://weasel.firmfriends.us/GeeksHomePages/subj-video-and-audio.html#implementing-video-thumbnails
My 'proof-of-concept' webpage based on that approach is here:
https://weasel.firmfriends.us/Private3-BB/
I hope this is helpful.
Is there a way to script a pair of dual-EELS SIs so that DM recognises them as siblings (say, for use in the "SI->Align SI by Peak" menu option)?
I have a script which performs a transformation on a pair of dualEELS SIs where the results are given in new images with all the tags copied from the original. The new SIs do not seem to be acknowledged by the SI options as a pair, however. A MWE of how this occurs is below:
image a, b
GetTwoLabeledImagesWithPrompt("Get SI", \
"Get DualEELS SIs", \
"Low-Loss", a, \
"High-Loss", b)
image LL, HL
LL := a.ImageClone()
HL := b.ImageClone()
LL.ShowImage()
HL.ShowImage()
Assuming that the two inputs are real dualEELS SIs. Attempting afterward to run a method like "SI->Align SI by Peak" on the outputs does not recognise the second SI as a sibling.
I suspect that my issue is in properly assigning the four EELS:Dual acquire sibling:UID tags highlighted in the image provided, however I have no idea how (or if) these are accessible from the scripting language.
Thanks in advance for any help you are able to render.
Yes, "siblings" in DigitalMicrograph are recognized by tag-checks and tagging of the Unique Image Id (UID).
Depending on the exact application/plugin, there might be additional tag-checks before a sibling can be accepted (i.e. "is it EELS data?", "is it spatially compatible?", etc.) but he prime-mechanism is using the UID.
A bit of background info
The UID is a set of four long-numbers generated at random whenever a new image data is created, and then subsequently stored with the data. It is "unique" by the assumption that a set of four randomly generated 8-byte longs are "unique".
If you create an image, save it to disc, and open it, the UID will be the same. (It is stored with the data.)
If you ImageClone() an image, it gets a new UID.
If you copy a image file on the hard drive and rename it, it will retain the UID.
Creating and using UIDs
The commands to get an image's UID are described in the F1 help documentation here:
And the example section even has a script showing how one uses UID to "link" data together:
I can't get the two series of numbers to scale together.
Here is sample code that you can paste into...
http://code.google.com/intl/en/apis/chart/docs/chart_playground.html
cht=lxy
chs=400x300
chd=t:20,30,40|1,4,2|24,34,44|3,7,1
chds=20,40,1,4,24,44,1,7
chxr=0,20,54,2|1,0,7,1
chxt=x,y
chxs=0,ff0000,12,0,lt
1,0000ff,10,1,lt
chco=FF0000,00FF00
chdl=Apples
Oranges
chtt=Some+Values
chts=0000ff,24
Translated:
chd=t:s,e,r,i,e,s,1|s,e,r,i,e,s,2|...ors:series1,series2,...ore:series1,series2,...
chds=<series_1_min>,<series_1_max>,...
chxr=<axis_index>,<start_val>,<end_val>,<step>|...
The three varying parameters in question are:
chd=t:20,30,40|1,4,2|24,34,44|3,7,1
chds=20,40,1,4,24,44,1,7
chxr=0,20,54,2|1,0,7,1
Can anyone get this simple example working?
The chart supports multiple series but for some reason I can't scale it so that the values are displayed within scale.
Any help appreciated,
Chris
After some practice I think I needed to make the chds values the same for both sets of data. I need to play with it more but I thought I would post a working example for posterity.
cht=lxy
chs=400x300
chd=t:20,30,40|1,4,2|24,34,44,48|3,7,1,2
chds=10,50,0,10,10,50,0,10
chxr=0,10,50|1,0,10,1
chxt=x,y
chxs=0,ff0000,12,0,lt
1,0000ff,10,1,lt
chco=FF0000,00FF00
chdl=Apples
Oranges
chtt=Some+Values
chts=0000ff,24
chm=o,0000FF,0,-1,5,0|o,ff0000,1,-1,5,0
chg=10,10
Scaling can be tricky to get right.
Try using chds='a' for automatic scaling at first.