i found the following method to create swap, from here
dd if=/dev/zero of=/root/myswapfile bs=1M count=1024
Is there any thumb rule that to decide block size ? What is the best block size for Swap in any machine ? Here it is 1M..
When creating a regular file for swap using dd the blocksize is a convenience to allow the count parameter to create a file of the specified size.
The underlying disk structure is unchanged by the bs= in the dd command.
Related
I haven't used openvms for 20+ years. It was my 1st OS. I've been asked if it possible to copy the data from RMS files from openvms server to windows as a text file - so that it's readable.
No-one has experience or knowledge of the record structures etc.
The files are xyz.DAT and are relative files. I'm hoping the dat files are fixed length.
My 1st attempt would be to try and use Datatrieve (DTR) but get an error that the image isn't loaded.
Thought it might be as easy using CONVERT/FDL = nnnn.FDL - by changing the Relative to Sequential. The file seems still to be unreadable.
Is there an easy way to stream an RMS index file to a flat ASCII file?
I use to use COBOL and C to access the data in the past but had lots of libraries to help....
I've notice some solution may use odbc to connect but not sure what I can or cannot install on the server.
I can FTP using Filezilla to the server....
Another plan writing C application to read a file and output out as string.....or DCL too.....doesn't have to be quick...
Any ideas
Has mentioned before
The simple solution MIGHT be to to just use: $ TYPE/OUT=test.TXT test.DAT.
This will handle Relatie and Indexed files alike.
It is much the same as $ CONVERT / FDL=NL: test.DAT test.TXT
Both will just read records from the source and transfer the bytes, byte for byte, to the records in a sequential file.
FTP in ASCII mode will transfer that nicely to windows.
You can also use an 'inline' FDL file to generate a 'unix' LF file like:
$ conv /fdl="record; format stream_lf" test.DAT test.TXT
Or CR-LF file using:
$ conv /fdl="record; format stream" test.DAT test.TXT
Both can be transferring in Binary or Ascii with FTP.
MOSTLY - because this really only works well for TEXT ONLY source .DAT file.
There should be no CR, LF, FF or NUL characters in the source or things will break.
As 'habo' points out, use DUMP /RECORD=COUNT=3 to see how 'readable' the source data is.
If you spot 'binary' data using DUMP then you will need to find a record defintion somewhere which maps byte to Integers or Floating points or Dates as needed.
These defintions can be COBOL LIB files, or BASIC MAPS and are often stores IN the CDD (Common Data Dictionary) or indeed in DATATRIEVE .DIC DICTIONARIES
To use such definition you likely need a program to just read following the 'map' and write/print as text. Normally that's not too hard - notably not when you can find an example program on the server to tweak.
If it is just one or two 'suspect' byte ranges, then you can create a DCL loop to read and write and use F$EXTRACT to select the chunks you like.
If you want further help, kindly describe in words what kind of data is expected and perhaps provide the output from DUMP for 3 or 5 rows.
Good luck!
Hein.
My project is likely to have instances where input datasets are overwritten but the contents are not changed. Is there a way in Snakemake to check for build changes using checksums instead of timestamps?
For example, Scons checks for build changes in both code and data using md5 hashes (hashes computed only where timestamps have changed). But I'd so much prefer to use Snakemake because of its other killer features.
The desired behavior is similar to the between workflow caching functionality described in the docs. In the docs it says:
There is no need to use this feature to avoid redundant computations within a workflow. Snakemake does this already out of the box.
But all references to this issue point to Snakemake only using timestamps within a normal workflow.
Using the ancient marker or using touch to adjust timestamps won't work for me as that will require too much manual intervention.
I eventually found an old SO post indicating that I could do this by writing my own script to compare checksums and then feeding that into Snakemake, but I'm not sure if this is still the only option.
I'm not aware of a built-in solution in snakemake. Maybe here's how I would go about it.
Say your input data is data.txt. This is the file(s) that is overwritten possibly without changing. Instead of using this file directly in the snakemake rules, you use a cached copy that is overwritten only if the md5 has changed between original and cache. The checking can be done before the rule all using standard python code.
Here's a pseudo-code example:
input_md5 = get_md5('data.txt')
cache_md5 = get_md5('cache/data.txt')
if input_md5 != cache_md5:
# This will trigger the pipeline because cache/data.txt is newer than output
copy('data.txt', 'cache/data.txt')
rule all:
input:
'stuff.txt'
rule one:
input:
'cache/data.txt',
output:
'stuff.txt',
EDIT: This pseudo-code saves the md5 of the cached input files so they don't need to be recomputed each time. It also saves to file the timestamp of the input data so that the md5 of the input is recomputed only if such timestamp is newer than the cached one:
for each input datafile do:
current_input_timestamp = get_timestamp('data.txt')
cache_input_timestamp = read('data.timestamp.txt')
if current_input_timestamp > cache_input_timestamp:
input_md5 = get_md5('data.txt')
cache_md5 = read('cache/data.md5')
if input_md5 != cache_md5:
copy('data.txt', 'cache/data.txt')
write(input_md5, 'cache/data.md5')
write(current_input_timestamp, 'data.timestamp.txt')
# If any input datafile is newer than the cache, the pipeline will be triggered
However, this adds complexity to the pipeline so I would check whether it is worth it.
I have a binary file (capture.bin) from the rtl_sdr tool. I convert it to a .cfile with this manual http://sdr.osmocom.org/trac/wiki/rtl-sdr#Usingthedata
Where can I get the data in this file? The goal is to get a numerical format output from the the source. Is this possible?
That actually is covered by a GNU Radio FAQ entry.
What is the file format of a file_sink? How can I read files produced by a file sink?
All files are in pure binary format. Just bits. That’s it. A floating point data stream is saved as 32 bits in the file, one after the other. A complex signal has 32 bits for the real part and 32 bits for the imaginary part. Reading back a complex number means reading in 32 bits, saving that to the real part of a complex data structure, and then reading in the next 32 bits as the imaginary part of the data structure. And just keep reading the data.
Take a look at the Octave and Python files in gr-utils for reading in data using Octave and Python’s Scipy module.
The exception to the format is when using the metadata file format. These files are produced by the File Meta Sink: http://gnuradio.org/doc/doxygen/classgr_1_1blocks_1_1file__meta__sink.html block and read by the File Meta Source block. >See the manual page on the metadata file format for more information about how to deal with these files.
A one-line Python command to read the entire file into a numpy array is:
f = scipy.fromfile(open("filename"), dtype=scipy.uint8)
Replace the dtype with scipy.int16, scipy.int32, scipy.float32, scipy.complex64 or >whatever type you were using.
Update
scipy.fromfile will be deprecated in v2.0 so instead use numpy library
f = numpy.fromfile(open("filename"), dtype=numpy.uint8)
Using a hex-editor to mount a NTFS volume, I've found an offset within the volume containing data I'm interested in. How can I figure out the full path/name of the file containing this volume offset?
Perhaps there are still some people searching for the solution. There is a tool for this problem: SleuthKit Tools.
Given an byte offset from the beginning of the partition table you have to divide it by the block size of your NTFS-Partition (usually 4096).
ifind /dev/... -d block_offset => inode_number
ffind /dev/... inode_number => Location of file
You need to read the MFT and parse the Data attributes for each file to find the one that includes the particular offset.
Note that you might need to look at every files stream, not only the default, so you have to parse all the Data attributes.
Unfortunately, I couldn't find a quick link to the binary structure of the NTFS Data attribute. you're on your own for this one.
I'm using this method by Dave DeLong to calculate the size of a folder using the Carbon File Manager API:
http://github.com/davedelong/BuildCleaner/blob/b2712242b4eea1fff0e78a08b393a417e3019c8a/NSFileManager+FileSize.m
(Its the first method there)
The issue I'm having is that some folders (.app bundles in my case) are not reporting the correct size. For example, DiskWarrior is 8.2MB, and the method reports 6.6MB
Any ideas on why this is happening?
THANKS
I've improved on that source code of mine that you've linked to. Here's the newer version:
http://github.com/davedelong/BetterInfo/blob/aa1cfe079dad6207a7ddac84b108a768c2cc7156/NSFileManager+BetterInfo.m (You'll also need the corresponding .h file and this support file)
Now instead of returning and NSUInteger, it returns a struct of type "BIItemSyze", which has six members: dataLogicalSize, dataPhysicalSize, resourceLogicalSize, resourcePhysicalSize, logicalSize, and physicalSize.
The Finder will report file sizes rounded to the nearest multiple of the block size (usually 4 KB) followed by the real size in bytes, and many (most) applications are bundles of files, so the true size of the application may be far smaller than the size shown as the first ("on disk") value.
You can test this out by doing something (in the Terminal) like:
echo -n 'foo' > foo.txt
If you Get Info on this file in the Finder, it will report the size as "4 KB on disk (3 bytes)".
If you know how to use applescript's in your code, here's a method o return the size that you'd see in the Finder Get Info window. Note that the returned value is in bytes.
on getSizeInBytesFromPosixPath(posixPath)
try
set b to (POSIX file posixPath) as string
tell application "Finder" to return size of (b as alias)
on error
return 0
end try
end getSizeInBytesFromPosixPath