I need to detect whether a particular file is binary file saved by boost::serialization. Looking at a few files, I saw they have those 16 bytes at the beginning:
0000000 0016 0000 0000 0000 6573 6972 6c61 7a69 | ........serializ
0000010 7461 6f69 3a6e 613a 6372 6968 6576 0009 | ation::archive..
Is that a reliable sequence to detect such an archive?
Reliable? Probably yes, as long as you keep using the same implementation of boost::serialization. Possibly also across updates of the library. But the reference documentation doesn't mention any support for file magic, so officially, the effect of checking the file header is undefined, and might silently break in future versions. If you need to be robust, it is better to attempt to deserialize and catch whatever is thrown on failure. So, for a quick file browser that prints informative icons, sure, go ahead and use the header detection. In a enterprise system for archiving backups, not so much.
Related
I encountered a problem while working with the TextIO structure,
because every input waites for a newline chacter and for the buffer to be full...
How can i work with BinIO and stdIn to solve that problem?
Any helpfull input is appreciated.
BTW: I am using MLTton so there is nothing more than the usual standard libs.
As a last resort, you could write it yourself in C, and then call it from SML using the foreign function interface. You can find out more info about MLton's FFI here: http://mlton.org/ForeignFunctionInterface
I encountered a problem while working with the TextIO structure, because every input waites for a newline chacter and for the buffer to be full... How can i work with BinIO and stdIn to solve that problem?
BinIO, like TextIO, implements buffered I/O. (They both implement the IMPERATIVE_IO signature.) For unbuffered I/O, you need to go "down" a level in abstraction, and use an implementation of PRIMITIVE_IO or POSIX_IO.
Specifically, Posix.IO.readVec lets you read unbufferedly from a file descriptor. (In the case of standard input, the file descriptor is Posix.FileSys.stdin.)
However, if your standard input is from the console (as opposed to being redirected from a file, or taken from a pipe, or whatnot), then there's a very good chance that the console only provides input to MLton after the user hits Enter. Using Posix.IO will bypass the line-buffering functionality that MLton provides, but if you also need to bypass your console's line buffering, then you'll likely need to use special C libraries (specific to your operating system), with the foreign function interface that Matt mentions in his answer.
I am trying to flash the very first u-boot binary file (uboot.bin) into blank NOR flash of a brand new blank board which has marvel 370 soc(ARM) using Teraterm(xmodem/ymodem/zmodem)
When I compile the uboot, I get two binaries like uboot-uart.bin and uboot.bin.
What is the difference between two binaries?
I have been instructed to make some dip switch changes and then load uboot-uart.bin first into the prototype board.
From manual I understand that the dip switch setting is to set "Boot from Uart" to Boot source list.
I am new to embedded and want to learn more about this from u-boot perspective. Where can I learn about this?
Would also like to know what these xmodem,ymodem,zmodem things are?
And would also like to learn how to customize u-boot for a custom board using marvel 370 soc(ARM)?
I would be happy if someone can point to good resources.
XModem itself is a quite simple protocol which is meant to send files over a serial link it is explained in detail here.
Most Marvell ARM-Chips in the last couple of years have the possibility to upload a binary via UART using the XModem protocol. There are two ways to do that.
By sending a special sequence to the chip during bootup (which can be done without any changes to the bootstrap options).
By setting up the bootstrap options accordingly (via DIP-Switches in your case)
In both cases the chip will then initiate an Xmodem-download. TeraTerm should have an option to upload files via the xmodem protocol. IIRC it is available under File/Transfer/XModem/Send.
If you know just send your "uboot-uart.bin" file to the Armada 370 (which will take some time). The SoC will now boot the file just like if it was loaded from NAND or any other source.
The only difference between your uboot-uart.bin and uboot.bin is most probably the special header which has to be put in front of the actual uboot-binary, it contains the bootdevice type the image was meant for, the address in memory where the image should be loaded to and a lot of board specific settings. The exact structure and content is usually explained in the very excellent datasheets from Marvell.
For customizing uboot I can only suggest to dig into the code provided by Marvell and change it according to your own board. You'll find the board specific files under boards/Marvell.
I have got a PE executable file *.exe (32-bit), which is an small application (2.6Mb) to update firmware software of TV device. However, the update mechanism was only available up to 2013-03-12. I want to hack this executable just for pleasure. I'm trying to find this expiration date in file hexdump using PE Explorer, and replace it by some date in future to make this program work.
I found this article about binary date format:
binary date format
I am trying to find something like this value:
2013-03-xx: 0x713xxxxx
Is this a good approach to solve my task? Any suggestions? Do you know any others tools for hexdump that may be useful?
Best regard,
WP
There are likely a lot of values of the form 0x713xxxxx -- 2.6 MB might be larger than you've thought when you start looking through it more or less at random (you don't actually know that the application uses this date format internally).
The conventional approach to deal with this sort of problem is to use a tool to step through the program, examining the code that is executing, until you find the point where the check occurs. Then simply disable the check so that it always fails -- by altering the date, or simply by altering the code.
A popular tool for stepping through code that you do not control is the Interactive Dissassembler, IDA. You can download a freeware version of it here: https://www.hex-rays.com/products/ida/support/download_freeware.shtml
It might be harder than you think to do what you want, but you'll almost certainly learn a lot by trying.
Be aware of the legal issues you may be getting yourself into by making modifications to someone else's binaries, particularly if you distribute them afterwards.
dumpbin is a good PE parser (but if I were you, I won't do such kind of time stamp hacks :))
I wrote a JSON encoder/decoder for Rebol 2. I'm rewriting it for Rebol 3 and would prefer to make it available in Rebol 3 as a codec:
load %data.json
save %data.json object
decode 'json to-binary {["some","json"]}
How should I go about this?
At the time of asking, documentation on this subject is scarce.
The simple answer is that you can't. As for why, there are several answers to that.
The current codec model is undocumented. Only Carl knows how to write codecs at the moment. Though someone might be able to look at the source and figure it out, noone has yet.
The current codec functions aren't really flexible enough to make what you want here. You could make a decoder, but not an encoder because encode only takes images, binaries or strings. Maybe just having a decoder is enough for you though.
Only native codecs are supported by the current model. You can't write codecs in Rebol code, and for a parser that means that you can't use parse. Do you have a native code version of your parser?
The current codec model is a placeholder for the codec model we hope to eventually have. This is why it hasn't been that well documented. We eventually hope to support incremental en/decoding, codecs that can read directly from files or other ports, and Rebol codecs. However, we haven't really decided on the final model yet, or even started the discussion of how it will be designed, beyond initial criticisms.
I wish there was a better answer to your question right now.
Codecs were imo in highly non-finished state. I proposed some kind of streamed mechanism was needed, IIRC Carl understood the need, but the solution was never imo outlined. Ditto for tasks - current "implementation" is far from what Carl envisioned for R3 tasking ...
There is a tool, read this: http://rebol2.blogspot.it/2012/12/json.html
and this:
http://www.rebol.com/article/0522.html
However json is a complex way to handle data, rebol block is perfect in my humble opinion.
I just noticed that system/catalog/codecs is removed from Rebol 3; so I suppose that the feature was removed after 31-Mar-2009.
>> ? system/catalog
SYSTEM/CATALOG is an object of value:
datatypes block! length: 56
actions block! length: 56
natives block! length: 159
errors object! [Throw Note Syntax Script Math Access Command...
reflectors block! length: 6
boot-flags block! length: 18
Would it be possible to take the source code from a SNES emulator (or any other game system emulator for that matter) and a game ROM for the system, and somehow create a single self-contained executable that lets you play that particular ROM without needing either the individual rom or the emulator itself to play? Would it be difficult, assuming you've already got the rom and the emulator source code to work with?
It shouldn't be too difficult if you have the emulator source code. You can use a method that is often used to store images in c source files.
Basically, what you need to do is create a char * variable in a header file, and store the contents of the rom file in that variable. You may want to write a script to automate this for you.
Then, you will need to alter the source code so that instead of reading the rom in from a file, it uses the in memory version of the rom, stored in your variable and included from your header file.
It may require a little bit of work if you need to emulate file pointers and such, or you may be lucky and find that the rom loading function just loads the whole file in at once. In this case it would probably be as simple as replacing the file load function with a function to return your pointer.
However, be careful for licensing issues. If the emulator is licensed under the GPL, you may not be legally allowed to store a proprietary file in the executable, so it would be worth checking that, especially before you release / distribute it (if you plan to do so).
Yes, more than possible, been done many times. Google: static binary translation. Graham Toal has a good howto paper on the subject, should show up early in the hits. There may be some code out there I may have left some code out there.
Completely removing the rom may be a bit more work than you think, but not using an emulator, definitely possible. Actually, both requirements are possible and you may be surprised how many of the handheld console games or set top box games are translated and not emulated. Esp platforms like those from Nintendo where there isnt enough processing power to emulate in real time.
You need a good emulator as a reference and/or write your own emulator as a reference. Then you need to write a disassembler, then you have that disassembler generate C code (please dont try to translate directly to another target, I made that mistake once, C is portable and the compilers will take care of a lot of dead code elimination for you). So an instruction of a make believe instruction set might be:
add r0,r0,#2
And that may translate into:
//add r0,r0,#2
r0=r0+2;
do_zflag(r0);
do_nflag(r0);
It looks like the SNES is related to the 6502 which is what Asteroids used, which is the translation I have been working on off and on for a while now as a hobby. The emulator you are using is probably written and tuned for runtime performance and may be difficult at best to use as a reference and to check in lock step with the translated code. The 6502 is nice because compared to say the z80 there really are not that many instructions. As with any variable word length instruction set the disassembler is your first big hurdle. Do not think linearly, think execution order, think like an emulator, you cannot linearly translate instructions from zero to N or N down to zero. You have to follow all the possible execution paths, marking bytes in the rom as being the first byte of an instruction, and not the first byte of an instruction. Some bytes you can decode as data and if you choose mark those, otherwise assume all other bytes are data or fill. Figuring out what to do with this data to get rid of the rom is the problem with getting rid of the rom. Some code addresses data directly others use register indirect meaning at translation time you have no idea where that data is or how much of it there is. Once you have marked all the starting bytes for instructions then it is a trivial task to walk the rom from zero to N disassembling and or translating.
Good luck, enjoy, it is well worth the experience.