I have problem with Binary open file in c++/cli. How to open the whole file and save the contest of the file to the array ^.
A typical method for reading the contest [sic] of a Binarny [sic] file is to:
1. Determine the length of the file.
2. Allocate dynamic memory for the file.
3. Block read the file, in binary mode, into memory.
Some operating systems may have a memory map capability. This allows a file to be treated as an array. The OS is in charge of reading the file into memory. It may read the entire file, or it may read pages as necessary (on demand).
See std::ifstream::read, std::ifstream::seekg and std::ifstream::tellg.
Related
I am working on understanding some ground concepts in embedded Systems. My question is similar to understand hexedit of an elf .
In order to burn compiler output to ROM, the .out file is converted to HEX (say intel-hex). I wonder how the following informations are preserved in HEX format:
Section header
Symbol tables, debug symbols, linker symbols etc.
Elf header.
If these are preserved in HEX file, how they can be read from hex file?
A bit out question but how the microcontroller on boot knows where .data .bss etc. exists in HEX and to be coppied to RAM?
None of that is preserved. A HEX file only contains the raw program and data. https://en.wikipedia.org/wiki/Intel_HEX
The microcontroller does not know where .data and .bss are located - it doesn't even know that they exist. The start-up code which is executed before main() is called contains the start addresses of those sections - the addresses are hard-coded into the program. This start-up code will be in the HEX file like everything else.
The elements in points 1 to 3 are not included in the raw binary since they serve no purpose in the application; rather they are used by the linker and the debugger on the development host, and are unnecessary for program execution where all you need is the byte values and the address to write them to, which is more or less all the hex file contains (may also contain a start address record).
Systems that have dynamic linking or self-hosted debug capabilities (such as VxWorks for example) use the object file file.
With respect to point 5, the microcontroller does not need to know; the linker uses that information when resolving absolute and relative addresses in the object code. Once filly resolved (linked), the addresses are embedded in the code directly. Again where dynamic loading/linking is used the object file meta-data is required and such systems do not normally load a raw hex file or binary.
MS Word's .docx files contain a bunch of .xml files.
Setup.exe files spit out hundreds of files that a program uses.
Zips, rars etc also hold lots of compressed stuff.
So how are they made? What does MS Word or another program that produces these files have to do to put files inside files?
When I looked this up I just got a bunch of results about compression, but let's say I wanted to make a program that 'wraps' files inside a file without making the final result any smaller. What would I even have to write?
I'm not asking/expecting any source code that does this, I just need a pointer. Is there something you think I'm misunderstanding based on what I've asked here?
Even a simple link to an article or some documentation would be greatly appreciated.
Ok, I'll just come up with some headers for ordinary files and write them along with the bytes of the actual files into one custom-defined file. You guys were very helpful, thank you!
Historically, Windows had a number of technologies to support solutions like this. These were often called Compound Files or Structured storage. However, I don't think the newer Office documents use these technologies. I think the Office file formats are similar to ZIP files with a different extensions. If you change a file with .docx extension to .zip and open it with your favorite compression tool, you'll see a bunch of folders and XML files.
Here are some links to descriptions of different file formats that create "files within files"
Zip file format
Compound File Binary Format (CFBF)
Structured Storage
Compound Document File Format
Office Open XML I: Exploring the Office Open XML Formats
At least on POSIX systems (e.g. Linux), a file is only a stream (i.e. a sequence) of bytes. And you can only grow (or shrink, i.e. truncate) it at the end - there is no way to insert bytes in the middle (without copying the rest).
You need some conventions, and some additional software, to handle it otherwise.
You might be interested in Sqlite, which gives you a library to handle some (e.g.) *.sqlite file as an SQL database
You could also use GDBM - a library giving you some indexed file abstraction.
libtar is a library to manipulate tar archives. See also tardy, a tar file postprocessor.
We have tools like objdump, readelf, and dumbin to show executable file contents.
But when an executable file is loaded into memory (a process is created), the segments in memory is usually different from the segments in the executable file. For example, when loaded, two extra segments namely stack and heap are allocated (we overlook details of page mapping here).
Is there a tool that help show the in-time memory segment/status of a process?
Windows executables use the Portable Executable format. This format describes sections of memory that are allocated when the process is loaded, and optionally raw data (.text, .data sections) to be loaded into those sections.
Each section will typically have a file offset specifying where in the raw file the data is located, and a Virtual Address at which the data will be loaded. These may or may not resemble each other.
PE Explorer can give you details on the sections (and everything else about a PE file) of an executable.
Immunity Debugger will allow you to attach to a running process and see its memory map.
This is a bit of a two part question, for working with 40mb xml files.
• What’s a reasonable size to store in memory for a program running continually in the background?
• How to find what has changed in an XML file.
So on the first read the XML is loaded into NSData, then uploaded to the server.
Now instead of uploading a 40mb XML every time it changes, I would prefer to upload a “delta” file containing only what has changed. The program would monitor the file for change, and activate when it’s been modified. From what I can see, I would need to parse an old version of the xml file and parse the modified xml file, then compare them? Is it unreasonable to store 80mb in memory like this every time the file is modified?. Now I’m assuming that this has to be done with a DOM parser because I can’t see how you could compare two files like that with a SAX parser since it only has part of the file stored?
I'm a newbie at this so any help would be appreciated!
To compare two files:
There are many ways to do, (As file is to be considered, I may not be correct):
sdiff file1.xml file2.xml A unix command
You can use this command with apple script.
-[NSFileManager contentsEqualAtPath:andPath:]
This method checks to see if two files at given path are the same file, then compares their size, and finally compares their contents.
For other part:
What size is considered for background process, I dont think so, for an application it matters. You can save these into temporary files. Even safari uses 130+ MB as you can easily check through Activity monitor.
NSXMLParser ended up being the most useful for this
If I understand correctly, most programming language which provide a library function to retrieve the size of a file use a system call. But then, what does the system do under the hood? Does it depend on the file system? Is the size information stored in some kind of a file header?
Yes, this is filesystem dependent, but many filesystems do it in roughly the same way: for each file, there is a block on the hard drive that stores metadata about the file, including its size.
For many of the filesystems used in Linux/UNIX, for example, this block is called an inode. Note that the inode is not actually part of the file, so it's not really a header; it exists in a region of the disc that is reserved for storing metadata, not file data.
On NTFS, the filesystem used by Windows, file size data is stored in the master file table. This is roughly equivalent to the inode table on a Linux filesystem.
It's stored in a file's metadata, which you can retrieve with stat on POSIX systems. The metadata also includes, for example, when the file was last modified or accessed.
They're stored in a structure called an Inode: http://en.wikipedia.org/wiki/Inode
This contains all of your file's metadata; when you modify the contents of a file (or really do anything with it), your Inode gets updated.