Using C++ I was able to write an event file containing a graphdef without a problem. I used EventsWriter::WriterEvent() API. I looked great on TensorBoard.
After a deep dive, I found code in tensorflow.core.util, tensorflow.core.platform, and tensorflow.core.lib.io that wraps the tensorflow::Event in a record in this format: length, masked CRC of length, data, masked CRC of data. (github source here)
But the problem is that I do not want to statically link to the contrib TensorFlow library with my app. Instead, I'd like to make my app lightweight and decoupled from the library by using my local protoc-compiled headers (.pb.h) and sources (.pb.c).
I am able to create an event file using protobufs, but they are not visualized on TensorBoard. While using the debugger on Tensorboard source, I see an DataLossError exception when launching Tensorboard here: tensorboard/backend/event_processing/event_file_loader.py. The DataLossError exception is likely due to the fact that the tensorflow::Event is not wrapped as described above.
If you or anyone knows a strategy to write TB-compatible event files in C++ without using the contrib tensorflow library, please let me know.
So the solution is to wrap each Event record with these fours fields:
uint64 (length)
uint32 (masked crc of length)
byte (data[length])
uint32 (masked crc of data)
See WriteRecord() here
Related
I'm new in embedded electronic/programming but I have a project. For this project, I want to write raw data provided by sensor in 512b buffers on a SD card (SPI mode) with no filesystem.
I'm trying to do it by using the low level functions of the FATFS library. I've heard that I have to create my own format and rewrite functions to do it, but I'm lost in all those lines code... I suppose that I have first to initialize SD by using a command sequence (CMD0, CMD8, CMD55, ACMD41...).
I'm not sure for the next steps, if I have to open a file with the fopen function and then use the fwrite function...
If somebody can explain me how it work for a non filesystem SD card and guide me in the steps to follow I would be very grateful.
Keep in mind that cards with a memory capacity > 2Tb are not supported in SPI mode
(Physical Layer Simplified Specification.
If you don't want to use a file system, then fopen, and fwrite are irrelevant. You simply use the the low-level block driver. If you are referring to http://www.elm-chan.org/fsw/ff/00index_e.html, then the API subset you need is :
disk_status - Get device status
disk_initialize - Initialize device
disk_read - Read data
disk_write - Write data
disk_ioctl - Control device dependent functions
However, since you then have to manage the blocks somehow so you know which blocks are available and where to write next (like one large file), you will essentially be writing your own filesystem (albeit a very simple and limited one). So it begs the question of why you would not just use an existing filesystem?
There are many reasons for not using FAT in an embedded system but few of them will be resolved by implementing your own "raw" filesystem without you doing a lot of work reinventing the filesystem! Consider something more robust and designed for resource constrained embedded systems with potentially unreliable power source, such as littlefs.
I am going to transmit the complex data produced from matlab by using usrp b210 and gnuradio on windows.
gnuradio flowgraph is following:
Here, data type is complex.
Problem:
I want sample-rate is 20MHz or 30.72MHz, but gnuradio is stoped, logging out 'UUUUUUU...' or 'UUOUUUUU...'.
why?, and how to fix it? please help me. thanks.
GRC will even warn you that you must not use a throttle block in your flow graph if you have hardware. Read the console log!
Other than that, you're expecting your storage to read and write simultaneously 16 MS/s ยท (32 bit I + 32 bit Q) = 1.024 Gb/s, reliably. It's also not very likely it's fast enough to do that.
My program launchs an arbitrary number of sounds streamed from FreeSound.org and plays them using javafx.scene.media AudioClip instances.
I was trying to figure out whether it could be possible to capture the generated output to disk, from within the same program? Any pointers?
Instead of using AudioClip you could use SourceDataLine for playback. This class allows you to progressively read the audio data, exposing it for handling. You would have to decode from bytes to PCM for each incoming line, then add the PCM from all the lines to be merged, and recode that back to bytes for your output.
I suspect with a little minor tweaking you could get the library I wrote AudioCue to work for you. It has an optional mixer that will handle multiple cue inputs. The inputs make use of SourceDataLine and the mixer uses the logic I described. You would have to tweak the code to output to disk. I could probably help with that if this project is still live for you.
When we evaluate the tensor returned by tensorflow.contrib.ffmpeg.decode_audio(), the ffmpeg log shows up in the terminal, leading to a flood of messages when decoding a large number of files. Is it possible to disable the verbosity for ffmpeg ?
Looks like it's hard-coded in the contrib op here: https://github.com/tensorflow/tensorflow/blob/0b437ada0654820e21f81bd415f691251408346a/tensorflow/contrib/ffmpeg/default/ffmpeg_lib.cc#L52
You could modify that and compile TensorFlow from source, send a PR to make it a flag (or an op attribute which gets passed in), or file a feature request.
I recorded a signal with GNU Radio using a file sink block which outputs a raw binary file that can be analyzed or used as a source of input into GNU Radio.
I want to edit this raw file so that when I use it as a source inside GNU Radio it transmits my changed file instead of the original. For example: The signal is very long and repeats a pattern, I want to edit the file to reduce the number of repeated signals and save it back to the raw format to transmit using gnuradio later.
I tried importing the file into Audacity as a raw file (selecting 32bit float with 1 channel and 48k as the sample rate). This works for me to see the signal as audio data and I can even edit it but I'm not sure if it's saving it correctly when I export it as raw data. Also, the time indices in audacity seem to be way off; the signal should only be microseconds but audacity is showing it as a total of several seconds!
Anyone have any luck with editing the raw file sink output from GNU Radio?
I was able to consistently make this work. There seemed to be 3 things preventing this from working properly.
1) I was doing it wrong! I needed to output both the Real and the Imaginary numbers to a 2 channel wav file.
2) Using a spectrum analyzer, I was able to see that audacity was doing something really weird with the wav file when you delete a section of audio, so to combat this I "silenced" the section of audio I wanted to delete.
3) There seems to be a bug with Gnuradio and the Osmocom Sink (yes, I have the latest version of both, from source). If you run your flow graph, start transmitting then stop the flow graph by clicking the red X in Gnuradio (Kill the flow graph) it keeps my device (HackRF) transmitting! If you try to transmit a new file or the same file again, it will not transmit that signal because it's already trying to transmit something. In order to stop the device from transmitting, just close the block popup window that appears when you run the flow graph.
The 3rd item might not be a bug because I might have been stopping my flow graphs incorrectly to begin with, but following Michael Ossmann's tutorial on using the HackRF with Gnuradio, he says to click the red X to properly shutdown the flow graph and clean everything up; this appears to NOT be the case.
In the gr-utils/octave folder of the GNU Radio source code there are several functions for Octave and Matlab. Some of them allow to retrieve and store raw binary files of the corresponding data type.
For example, if your signal is constructed from float samples you can use the read_float_binary function to import the samples stored by the file sink block into Octave/Matlab. Then make your modifications to the signal and store it back again using the write_float_binary function. The stored file can be the imported to your flowgraph using a file source block.