I am new in this field, and working on payment gateway, please tell me what is the difference between packed and unpacked binary data used in iso8583 message...!
The schema definition files for ISO8583 are available at http://dfdlschemas.github.io/ISO8583. In ISO8583_1993.xsd it says:
* This DFDL schema provides a DFDL model for ISO8583 1993 binary data
* where each bitmap in the message is encoded as 8 bytes of binary data
* (8 bits per byte). The bitmaps are said to be 'packed'.
So, the term "packed" refers to the bitmaps, which can be either packed or unpacked.
In en.wikipedia.org/wiki/ISO_8583#Bitmaps, it says
The bitmap may be transmitted as 8 bytes of binary data, or as 16 hexadecimal > characters 0-9, A-F in the ASCII or EBCDIC character sets.
In data structures, packed binary data usually means that more (if not all available) bit combinations are used to encode some values, while unpacked means that some bit combinations remain unused, either to improve readability or to make certain calculations easier (but unpacked data takes more space).
For example, one unsigned byte (8 bits) can encode numbers from 0 to 255. If the numbers are BCD encoded, only numbers from 0 to 99 can be represented, and some bit combinations remain unused. However, it is in some cases easier to base calculations on a BCD encoded number than on a binary encoded number.
In summary, ISO 8583 defines two different encodings:
packed which is 8 bytes of binary data
unpacked which is 16 bytes as hexadecimal characters (in two different encodings, but that is another aspect).
One obvious difference is, that when you dump this data to a console, you can immediately read the unpacked data as hexadecimal numbers, while the binary encoding will only print some garbage characters, depending on your console, your locale and the font which you have installed.
Related
I'm dealing with a device that has both options to send data through UDP connection. As I couldn't find any comparison or something, could someone explain the difference in processing both?
Hex data transfers a byte as two hex characters, using only 4 bits of the available 8 bits. Ascii data transfers either 7bits or 8bits at a time, thus using the full range of 0..255 while a hex character only transfers 0..15.
For example, the number 18 is transferred as 12 hex coded (taking up two bytes) but as 18 ascii-encoded(taking up one byte 00010002).
In google Bigquery ....I have loaded a 1.5GB csv file from googlstorage after successfully loading,.... my table size is 250MB why this is so?
Likely because the binary encoding of numbers is more efficient than encoding them as strings. For example, the string "1234567890" takes 10 bytes (at least, or 20 bytes if it is UTF-16 encoded), but it can be represented by a 4 byte integer which only takes, ehm, 4 bytes.
Furthermore, the table in bigquery can also leave out the separators, because it knows how many bytes each field is wide. Thats another byte saved for every ,.
Just created an application in Adobe Air.
Customer now says he wants security on the DVD (stop DVD from being copied or serial key)
Is there any way that I can provide some form of Serial Key Protection in Adobe AIR.
I was thinking something like writing a small script with say 50 or so 'serial numbers' in some for of xml or database.
When the script is run it allows for the execution of the program if correct serial number is given else aborts proces.
Urgent request, if someone can provide an answer there is a few £'s in it for them.
I would encode some info into serials for application to test validness (some kind of checksum.) This requires several tasks:
Encode bytes into readable symbols of serial numbers. Using limited alphabet of 10 digits and 22 latin chars, we get 32 variants per symbol giving 5 bit. So, 20 symbols are worth 100 bits, that's 16 bytes and a half. The hard part is to slice original 17 bytes into 5-bit chunks (nothing really hard, really, can be solved with shifts and masks, just requires careful coding.)
Decode symbols of serial into original bytes. Using our custom alphabet tables, convert each symbol into 5 bit and glue them together in 17 bytes (again shifts, masks and careful coding :)
Define serial number contents. While possibilities are countless, it can be done simple: first few bytes are 'magic' fixed ones. If application decodes them from serial, it is valid. The rest bytes are randomly varying from serial to serial.
If we leave it this way, all our valid serials will start from the same symbols - encoded 'magic' bytes. To mix things up, I suggest using symmetrical encryption. There is library as3 crypto which provides RC4 algorithm. You can test it on demo page - choose 'Secret key' section, set encryption to RC4, and formats of key, text and cipher text to 'hex'.
Now generation of serial looks like this: take 'magic' bytes, add random ones to get 16 bytes total. Encrypt this message and convert into serial.
Program will check serial so: convert symbols into 16 bytes, decrypt them with same key, and check for 'magic' bytes to be present.
If you leave four bytes for 'magic' ones, this means one valid serial to 4 billions. For common folks, this should be enough of protection (uncommon ones will just decompile your program and shortcut the checks, so look into software like SecureSWF to prevent that.)
I'm currently trying to decipher WAV files. From headers to the PCM data.
I've found a PDF (http://www.tdt.com/T2Support/technical_notes/tn0132.pdf) detailing the anatomy of a WAV file, and I've been able to extract and make sense of the appropriate header data using Ghex2. But my questions are:
Why are the integers bytes stored backwards? I.e. dec. 20 is stored as 0x14000000 instead of 0x00000014.
Are the integers of the PCM data also stored backwards?
WAV files are little-endian (least significant bytes first) because the format originated for operating systems running on intel processor based machines which use the little endian format to store numbers.
If you think about it kind of makes sense because if you want to cast a long integer to a short one or even a character the starting address remains the same you just look at less bytes.
Consequently, for 16 bit encoding upwards, little-endian format will be used for the PCM as well. This is quite handy since you will be able to pull them in as integers. don't forget they will be stored as two's complement signed integers if they are 16 bit, but not if they are 8 bit. (see http://www-mmsp.ece.mcgill.ca/Documents/AudioFormats/WAVE/WAVE.html for more detail)
"Backwards" is subjective. Some machines are big-endian, others are little-endian. In byte-oriented contexts like file formats and network protocols, the order is arbitrary. Some formats like to specify big- or little-endian, others like to be flexible and accept either form, with a flag indicating which is in use.
Looks like WAV files just like little-endian.
Why does sql database use UTF-8 Encoding? do they both use 8-bit to store a character?
UTF-8 is used to support a large range of characters. In UTF-8, up to 4 bytes can be used to represent a single character.
Joel has written an article on this subject that you may want to refer to
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
For "normal" characters, only 8 bits are used. For characters that do not fit in 8 bits more bits can be used. This makes UTF-8 is a variable length encoding.
Wikipedia has a good article on UTF-8.
ASCII only defines 128 character. So only 7 bits. But is normally stored with 8 bits/character. RS232 (old serial communication) can be used with bytes of 7 bits.
ASCII can only represent a limited number of characters at one time. It isn't very useful to represent any language that isn't based on a Latin character set. However, UTF-8 which is an encoding standard for UCS-4 (Unicode) can represent almost any language. It does this by chaining multiple bytes together to represent one character (or glyph to be more correct).
A more sophisticated encoding increases the index access time drastically. It's something to think about, when encountering performance problems in writing or reading from an database.