I'm dealing with a device that has both options to send data through UDP connection. As I couldn't find any comparison or something, could someone explain the difference in processing both?
Hex data transfers a byte as two hex characters, using only 4 bits of the available 8 bits. Ascii data transfers either 7bits or 8bits at a time, thus using the full range of 0..255 while a hex character only transfers 0..15.
For example, the number 18 is transferred as 12 hex coded (taking up two bytes) but as 18 ascii-encoded(taking up one byte 00010002).
Related
doing this in byte is easy with filestream, but I cant get it to work with a bitarray.
I want to develop file compression algorithms, just as a hobby.
The method in question checks if the combined occurence of three combination of bytes occur enough times for the filesize to benefit from adding two bits at the beginning of every byte, as to whether the next byte is one of the three pre-stored bytes, if both bits are zero it assumes no, and just continues reading the file. now writing another byte for each byte would make all of this redundant.
If someone could tell me how to do this, I would much appreciate it.
As I want to do it, it is not possible. Extra bits are inevitable as the smallest unit which by the hard drive can be written to is a byte. the extra bits have to be dealt with in the logic of decompression. a bulletproof solution is to add some extra bits at the beginning of the file (or wherever the metadata is stored for the filecompression) which represent the cutoff in the last byte. 3 bits is enough as you can represent the number 7 with it. (obviously there isn't a whole extra unnecessery byte, therefor if the compressed file is still divisible by eight these three bits should be 000 or numerically a zero, 8 does not need to be written) this way once the last byte is being read the program should ignore as many bits as the three bits equal to.
I am new in this field, and working on payment gateway, please tell me what is the difference between packed and unpacked binary data used in iso8583 message...!
The schema definition files for ISO8583 are available at http://dfdlschemas.github.io/ISO8583. In ISO8583_1993.xsd it says:
* This DFDL schema provides a DFDL model for ISO8583 1993 binary data
* where each bitmap in the message is encoded as 8 bytes of binary data
* (8 bits per byte). The bitmaps are said to be 'packed'.
So, the term "packed" refers to the bitmaps, which can be either packed or unpacked.
In en.wikipedia.org/wiki/ISO_8583#Bitmaps, it says
The bitmap may be transmitted as 8 bytes of binary data, or as 16 hexadecimal > characters 0-9, A-F in the ASCII or EBCDIC character sets.
In data structures, packed binary data usually means that more (if not all available) bit combinations are used to encode some values, while unpacked means that some bit combinations remain unused, either to improve readability or to make certain calculations easier (but unpacked data takes more space).
For example, one unsigned byte (8 bits) can encode numbers from 0 to 255. If the numbers are BCD encoded, only numbers from 0 to 99 can be represented, and some bit combinations remain unused. However, it is in some cases easier to base calculations on a BCD encoded number than on a binary encoded number.
In summary, ISO 8583 defines two different encodings:
packed which is 8 bytes of binary data
unpacked which is 16 bytes as hexadecimal characters (in two different encodings, but that is another aspect).
One obvious difference is, that when you dump this data to a console, you can immediately read the unpacked data as hexadecimal numbers, while the binary encoding will only print some garbage characters, depending on your console, your locale and the font which you have installed.
Just created an application in Adobe Air.
Customer now says he wants security on the DVD (stop DVD from being copied or serial key)
Is there any way that I can provide some form of Serial Key Protection in Adobe AIR.
I was thinking something like writing a small script with say 50 or so 'serial numbers' in some for of xml or database.
When the script is run it allows for the execution of the program if correct serial number is given else aborts proces.
Urgent request, if someone can provide an answer there is a few £'s in it for them.
I would encode some info into serials for application to test validness (some kind of checksum.) This requires several tasks:
Encode bytes into readable symbols of serial numbers. Using limited alphabet of 10 digits and 22 latin chars, we get 32 variants per symbol giving 5 bit. So, 20 symbols are worth 100 bits, that's 16 bytes and a half. The hard part is to slice original 17 bytes into 5-bit chunks (nothing really hard, really, can be solved with shifts and masks, just requires careful coding.)
Decode symbols of serial into original bytes. Using our custom alphabet tables, convert each symbol into 5 bit and glue them together in 17 bytes (again shifts, masks and careful coding :)
Define serial number contents. While possibilities are countless, it can be done simple: first few bytes are 'magic' fixed ones. If application decodes them from serial, it is valid. The rest bytes are randomly varying from serial to serial.
If we leave it this way, all our valid serials will start from the same symbols - encoded 'magic' bytes. To mix things up, I suggest using symmetrical encryption. There is library as3 crypto which provides RC4 algorithm. You can test it on demo page - choose 'Secret key' section, set encryption to RC4, and formats of key, text and cipher text to 'hex'.
Now generation of serial looks like this: take 'magic' bytes, add random ones to get 16 bytes total. Encrypt this message and convert into serial.
Program will check serial so: convert symbols into 16 bytes, decrypt them with same key, and check for 'magic' bytes to be present.
If you leave four bytes for 'magic' ones, this means one valid serial to 4 billions. For common folks, this should be enough of protection (uncommon ones will just decompile your program and shortcut the checks, so look into software like SecureSWF to prevent that.)
You often see database fields set to have a magnitude of 255 characters, what is the traditional / historic reason why? I assume it's something to do with paging / memory limits, and performance but the distinction between 255 and 256 has always confused me.
varchar(255)
Considering this is a capacity or magnitude, not an indexer, why is 255 preferred over 256? Is a byte reserved for some purpose (terminator or null or something)?
Presumably varchar(0) is a nonsense (has zero capacity)? In which case 2^8 of space should be 256 surely?
Are there other magnitudes that provide performance benefits? For example is varchar(512) less performant than varchar(511) or varchar(510)?
Is this value the same for all relations databases, old and new?
disclaimer - I'm a developer not a DBA, I use field sizes and types that suit my business logic where that is known, but I'd like to know the historic reason for this preference, even if it's no longer relevant (but even more if it still is relevant).
Edit:
Thanks for the answers, there seems to be some concensus that a byte is used to store size, but this doesn't settle the matter definitively in my mind.
If the meta data (string length) is stored in the same contiguous memory/disk, it makes some sense. 1 byte of metadata and 255 bytes of string data, would suit each other very nicely, and fit into 256 contiguous bytes of storage, which presumably is neat and tidy.
But...If the metadata (string length) is stored separately from the actual string data (in a master table perhaps), then to constrain the length of string's data by one byte, just because it's easier to store only a 1 byte integer of metadata seems a bit odd.
In both cases, it would seem to be a subtlety that probably depends on the DB implementation. The practice of using 255 seems pretty widespread, so someone somewhere must have argued a good case for it in the beginning, can anyone remember what that case was/is? Programmers won't adopt any new practice without a reason, and this must have been new once.
With a maximum length of 255 characters, the DBMS can choose to use a single byte to indicate the length of the data in the field. If the limit were 256 or greater, two bytes would be needed.
A value of length zero is certainly valid for varchar data (unless constrained otherwise). Most systems treat such an empty string as distinct from NULL, but some systems (notably Oracle) treat an empty string identically to NULL. For systems where an empty string is not NULL, an additional bit somewhere in the row would be needed to indicate whether the value should be considered NULL or not.
As you note, this is a historical optimisation and is probably not relevant to most systems today.
255 was the varchar limit in mySQL4 and earlier.
Also 255 chars + Null terminator = 256
Or 1 byte length descriptor gives a possible range 0-255 chars
255 is the largest numerical value that can be stored in a single-byte unsigned integer (assuming 8-bit bytes) - hence, applications which store the length of a string for some purpose would prefer 255 over 256 because it means they only have to allocate 1 byte for the "size" variable.
From MySQL Manual:
Data Type:
VARCHAR(M), VARBINARY(M)
Storage Required:
L + 1 bytes if column values require 0 – 255 bytes, L + 2 bytes if values may require more than 255 bytes
Understand and make choice.
255 is the maximum value of a 8 bit integer : 11111111 = 255.
Are there other magnitudes that provide performance benefits? For example is varchar(512) less performant than varchar(511) or varchar(510)?
Recollected the fundamentals of the bits/bytes storage, it requires one byte to store integers below 256 and two bytes for any integer between 256 and 65536.
Hence, it requires same space (two bytes) to store 511 or 512 or for that matter 65535....
Thus it is clear that the this argument mentioned in the discussion above is N/A for varchar(512) or varchar(511).
A maximum length of 255 allows the database engine to use only 1 byte to store the length of each field. You are correct that 1 byte of space allows you to store 2^8=256 distinct values for the length of the string.
But if you allow the field to store zero-length text strings, you need to be able to store zero in the length. So you can allow 256 distinct length values, starting at zero: 0-255.
It used to be that all strings required a NUL terminator, or "backslash-zero". Updated databases don't have that. It was "255 characters of text" with a "\0" added automatically at the end so the system knew where the string ended. If you said VARCHAR(256), it would end up being 257 and then you'd be in the next register for one character. Wasteful. That's why everything was VARCHAR(255) and VARCHAR(31). Out of habit the 255 seems to have stuck around but the 31's became 32's and the 511's became 512's. That part is weird. It's hard to make myself write VARCHAR(256).
Often varchars are implemented as pascal strings: holding the actual length in the byte #0. The length was therefore bound to 255. (Value of a byte varies from 0 to 255.)
8 bits unsigned = 256 bytes
255 characters + byte 0 for length
I think this might answer your question. Looks like it was the max limit of varchar in earlier systems. I took it off another stackoverflow question.
It's hard to know what the longest postal address is, of course, which is why many people choose a long VARCHAR that is certainly longer than any address. And 255 is customary because it may have been the maximum length of a VARCHAR in some databases in the dawn of time (as well as PostgreSQL until more recently).
Are there disadvantages to using a generic varchar(255) for all text-based fields?
Data is saved in memory in binary system and 0 and 1 are binary digits. Largest binary number that can fit in 1 byte (8-bits) is 11111111 which converts to decimal 255.
Disclaimer: I'm very new to SQL and databases in general.
I need to create a field that will store a maximum of 32 characters of text data. Does "VARCHAR(32)" mean that I have exactly 32 characters for my data? Do I need to reserve an extra character for null-termination?
I conducted a simple test and it seems that this is a WYSIWYG buffer. However, I wanted to get a concrete answer from people who actually know what they're doing.
I have a C[++] background, so this question is raising alarm bells in my head.
Yes, you have 32 characters at your disposal. SQL does not concern itself with nul terminated strings like some programming languages do.
Your VARCHAR specification size is the max size of your data, so in this case, 32 characters. However, VARCHARS are a dynamic field, so the actual physical storage used is only the size of your data, plus one or two bytes.
If you put a 10-character string into a VARCHAR(32), the physical storage will be 11 or 12 bytes (the manual will tell you the exact formula).
However, when MySQL is dealing with result sets (ie. after a SELECT), 32 bytes will be allocated in memory for that field for every record.