My data is 30KB on disk (Serialized object) was size should the binary field in t-sql be?
Is the brackets bit bytes ?
... so is binary(30000) .... 30KB?
Thanks
You need to use the varbinary(max) data type; the maximum allowed size for binary is 8,000 bytes. Per the MSDN page on binary and varbinary:
varbinary [ ( n | max) ]
Variable-length binary data. n can be a value from 1 through 8,000. max indicates that the maximum storage size is 2^31-1 bytes. The storage size is the actual length of the data entered + 2 bytes. The data that is entered can be 0 bytes in length.
The number after binary() is the number of bytes, see MSDN:
binary [ ( n ) ]
Fixed-length binary data of n bytes. n
must be a value from 1 through 8,000.
Storage size is n+4 bytes.
Whether 30kb is 30000 or 30720 bytes depends on which binary prefix system your file system is using.
Related
for example
> SETBIT bitmapsarestrings 2 1
> SETBIT bitmapsarestrings 3 1
> SETBIT bitmapsarestrings 5 1
> SETBIT bitmapsarestrings 10 1
> SETBIT bitmapsarestrings 11 1
> SETBIT bitmapsarestrings 14 1
> GET bitmapsarestrings
"42"
Binary storage should not like this: 0010 0110 0001 1100 ?
stored in this way, why value is 42?
These SETBIT operations will make the value as a binary string, whose length is 2 bytes or 16 bits. After the settings, the value will be 0b 00110100 00110010 in binary format.
The first byte (0b 00110100) is 52, which is the ascii code of '4', and the second byte (0b 00110010) is 50, which is the ascii code of '2'. So when you get the value of the string, it returns "42".
What #for_stack said, or just refer to the lines immediately above that example (https://redis.io/commands/setbit):
Bitmaps are not an actual data type, but a set of bit-oriented operations defined on the String type (for more information refer to the Bitmaps section of the Data Types Introduction page). This means that bitmaps can be used with string commands, and most importantly with SET and GET.
Because Redis' strings are binary-safe, a bitmap is trivially encoded as a bytes stream. The first byte of the string corresponds to offsets 0..7 of the bitmap, the second byte to the 8..15 range, and so forth.
How many bytes do the following types take up in BigQuery:
Timestamp
Datetime
Date
My guess was that date could be stored in 2 bytes, and a timestamp perhaps 8, but I wasn't sure about that and it is not mentioned on the https://cloud.google.com/bigquery/docs/reference/standard-sql/data-types page.
The size of BigQuery's data types is as follows:
Data type Size
INT64/INTEGER 8 bytes
FLOAT64/FLOAT 8 bytes
NUMERIC 16 bytes
BOOL/BOOLEAN 1 byte
STRING 2 bytes + the UTF-8 encoded string size
BYTES 2 bytes + the number of bytes in the value
DATE 8 bytes
DATETIME 8 bytes
TIME 8 bytes
TIMESTAMP 8 bytes
STRUCT/RECORD 0 bytes + the size of the contained fields
GEOGRAPHY 16 bytes + 24 bytes * the number of vertices in the geography type (you can verify the number of vertices using the ST_NumPoints function)
Null values for any data type are calculated as 0 bytes.
A repeated column is stored as an array, and the size is calculated
based on the number of values. For example, an integer column (INT64)
that is repeated (ARRAY) and contains 4 entries is calculated
as 32 bytes (4 entries x 8 bytes).
See more details in Data size calculation section of Pricing documentation
I was checking out the difference between char vs varchar2 from google. I came across a word LEADING LENGTH in this link . THERE it was written that
Suppose you store the string ‘ORATABLE’ in a CHAR(20) field and a VARCHAR2(20) field. The CHAR field will use 22 bytes (2 bytes for leading length). The VARCHAR2 field will use 10 bytes only (8 for the string, 2 bytes for leading length).
Q1:How does the char field will use 22 bytes if the string is of 8 characters if (1 byte = 1 char)?
Q2 What is the LEADING LENGTH ? why it does occupy 2 bytes?
The CHAR() datatype pads the string with characters. So, for 'ORATABLE', it looks like:
'ORATABLE '
12345678901234567890
The "leading length" are two bytes at the beginning that specify the length of the string. Two bytes are needed because one byte is not enough. Two bytes allow lengths up to 65,535 units; one byte would only allow lengths up to 255.
The important point both CHAR() and VARCHAR2() use the same internal format, so there is little reason to sue CHAR(). Personally, I would only use it for fixed-length codes, such as ISO country codes or US social security numbers.
MSDN has this to say on the subject:
nvarchar [ ( n | max ) ]
Variable-length Unicode character data. ncan be a value from 1 through 4,000. max indicates that the maximum storage size is 2^31-1 bytes. The storage size, in bytes, is two times the number of characters entered + 2 bytes. The data entered can be 0 characters in length. The ISO synonyms for nvarchar are national char varying and national character varying.
This leaves me confused. I can define a column as being 1 - 4000 long, or 2147483647 long but nothing inbetween? Is my understanding correct? Why can't I be explicit about values inbetween?
NVARCHAR(MAX) covers everything else (not just 2 billion characters). If you need more than 4,000 characters the data is most certainly going to be off-page, so as far as behavior is concerned it doesn't matter if you've used 4,001 characters, 10,000 characters, or 10,000,000 characters. It only occupies the space you need, so don't think that you are wasting (2 billion characters - the length of your actual string).
Max will accept values between 4001 and 1073741823 (bear in mind storage size is approx 2x the length of the actual string).
The restriction is basically that anything over 4000 characters must be a MAX.
Because 4000 characters or less has one behavior in terms of storage and MAX has another behavior in terms of storage. And you really don't want to start forcing string length calculations on things that are 1M characters long do you? My current understanding is that up to 4000 characters is stored in-table and MAX is stored out-of-table.
Also NVARCHAR(MAX) and VARCHAR(MAX) are replacements for text and ntext.
I have a table with a varbinary(max) column, i am trying to assign to that column a zero-lengh binary buffer, but instead of getting a zero-length value in the table, i am getting an 8000 bytes long value filled with zeros:
* the dataSize column in the shown query was added using DATALENGHT(data) ("SELECT _index, dataSize=DATALENGHT(data), data FROM....") and shows the actual size on the table of the value
Where does the 8000 bytes long empty buffer come from? is this some kind of default behavior?
If your source column is binary(8000), then DATALENGTH(data) will return 8000 (it is fully padded) and data will contain the full 8000 bytes.
But since you are using
SELECT _index, dataSize=DATALENGTH(data), data FROM
It cannot be a binary(8000) column - because a fixed size column will report the same datalength for all rows. It is likely some data was copied there from a BINARY(8000) variable or other means some time in the past.