I'm using python and cryptography.io to sign and verify messages. I can get a DER-encoded bytes representation of a signature with:
cryptography_priv_key.sign(message, hash_function)
...per this document: https://cryptography.io/en/latest/hazmat/primitives/asymmetric/ec/
A DER-encoded ECDSA Signature from a 256-bit curve is, at most, 72 bytes; see: ECDSA signature length
However, depending on the values of r and s, it can also be 70 or 71 bytes. Indeed, if I examine length of the output of this function, it varies from 70-72. Do I have that right so far?
I can decode the signature to ints r and s. These are both apparently 32 bytes, but it's not clear to me whether that will always be so.
Is it safe to cast these two ints to bytes and send them over the wire, with the intention of encoding them again on the other side?
The simple answer is, yes, they will always be 32 bytes.
The more complete answer is that it depends on the curve. For example, a 256-bit curve has an order of 256-bits. Similarly, a 128-bit curve only has an order of 128-bits.
You can divide this number by eight to find the size of r and s.
It gets more complicated when curves aren't divisible by eight, like secp521r1 where the order is a 521-bit number.
In this case, we round up. 521 / 8 is 65.125, thus it requires that we free 66 bytes of memory to fit this number.
It is safe to send them over the wire and encode them again as long as you keep track of which is r and s.
Related
To all homomorphic encryption experts out there:
I'm using the PALISADE library:
int plaintextModulus = 65537;
float sigma = 3.2;
SecurityLevel securityLevel = HEStd_128_classic;
uint32_t depth = 2;
//Instantiate the crypto context
CryptoContext<DCRTPoly> cc = CryptoContextFactory<DCRTPoly>::genCryptoContextBFVrns(
plaintextModulus, securityLevel, sigma, 0, depth, 0, OPTIMIZED);
could you please explain (all) the parameters especially intrested in ptm, depth and sigma.
Secondly I am trying to make a Packed Plaintext with the cc above.
cc->MakePackedPlaintext(array);
What is the maximum size of the array? On my local machine (8GB RAM) when the array is larger than ~8000 int64 I get an free(): invalid next size (normal) error
Thank you for asking the question.
Plaintext modulus t (denoted as t here) is a critical parameter for BFV as all operations are performed mod t. In other words, when you choose t, you have to make sure that all computations do not wrap around, i.e., do not exceed t. Otherwise you will get an incorrect answer unless your goal is to compute something mod t.
sigma is the distribution parameter (used for the underlying Learning with Errors problem). You can just set to 3.2. No need to change it.
Depth is the multiplicative depth of the circuit you are trying to compute. It has nothing to with the size of vectors. Basically, if you have AxBxCxD, you have a depth 3 with a naive approach. BFV also supports more efficient binary tree evaluation, i.e., (AxB)x(CxD) - this option will reduce the depth to 2.
BFV is a scheme that supports packing. By default, the size of packed ciphertext is equal to the ring dimension (something like 8192 for the example you mentioned). This means you can pack up to 8192 integers in your case. To support larger arrays/vectors, you would need to break them into batches of 8192 each and encrypt each one separately.
Regarding your application, the CKKS scheme would probably be a much better option (I will respond on the application in more detail in the other thread).
I have some experience with the SEAL library which also uses the BFV encryption scheme. The BFV scheme uses modular arithmetic and is able to encrypt integers (not real numbers).
For the parameters you're asking about:
The Plaintext Modulus is an upper bound for the input integers. If this parameter is too low, it might cause your integers to overflow (depending on how large they are of course)
The Sigma is the distribution parameter for Gaussian noise generation
The Depth is the circuit depth which is the maximum number of multiplications on a path
Also for the Packed Plaintext, you should use vectors not arrays. Maybe that will fix your problem. If not, try lowering the size and make several vectors if necessary.
You can determine the ring dimension (generated by the crypto context based on your parameter settings) by using cc->GetRingDimension() as shown in line 113 of https://gitlab.com/palisade/palisade-development/blob/master/src/pke/examples/simple-real-numbers.cpp
I want to verify a message signed by my trezor hardware wallet.
Basically I have these information.
.venv/bin/trezorctl btc get-public-node -n 0
Passphrase required:
Confirm your passphrase:
node.depth: 1
node.fingerprint: ea66f037
node.child_num: 0
node.chain_code: e02030f2a7dfb474d53a96cb26febbbe3bd3b9756f4e0a820146ff1fb4e0bd99
node.public_key: 026b4cc594c849a0d9a124725997604bc6a0ec8f100b621b1eaed4c6094619fc46
xpub: xpub69cRfCiJ5BVzesfFdsTgEb29SskY74wYfjTRw5kdctGN2xp1HF4udTP21t68PAQ4CBq1Rn3wAsWr84wiDiRmmSZLwkEkv4qK5T5Y7EXebyQ
$ .venv/bin/trezorctl btc sign-message 'aaa' -n 0
Please confirm action on your Trezor device
Passphrase required:
Confirm your passphrase:
message: aaa
address: 17DB2Q3oZVkQAffkpFvF4cwsXggu39iKdQ
signature: IHQ7FDJy6zjwMImIsFcHGdhVxAH7ozoEoelN2EfgKZZ0JVAbvnGN/w8zxiMivqkO8ijw8fXeCMDt0K2OW7q2GF0=
I wanted to use python3-ecdsa. When I want to verify the signature with any valid public key, I get an AssertionError: (65, 64), because the base64.b64decode of the signature is 65 bytes, but should be 64.
When I want to load the node.public_key into a ecdsa.VerifyingKey, I get an AssertionError: (32, 64), because the bytes.fromhex return 32 bytes, but every example I found uses 64 bytes for the public key.
Probably I need to convert the bip32 xpub to a public key, but I really dont know how.
Solutiion
python-ecdsa needs to be at version 0.14 or greater to handle compressed format of the public key.
import ecdsa
import base64
import hashlib
class DoubleSha256:
def __init__(self, *args, **kwargs):
self._m = hashlib.sha256(*args, **kwargs)
def __getattr__(self, attr):
if attr == 'digest':
return self.double_digest
return getattr(self._m, attr)
def double_digest(self):
m = hashlib.sha256()
m.update(self._m.digest())
return m.digest()
def pad_message(message):
return "\x18Bitcoin Signed Message:\n".encode('UTF-8') + bytes([len(message)]) + message.encode('UTF-8')
public_key_hex = '026b4cc594c849a0d9a124725997604bc6a0ec8f100b621b1eaed4c6094619fc46'
public_key = bytes.fromhex(public_key_hex)
message = pad_message('aaa')
sig = base64.b64decode('IHQ7FDJy6zjwMImIsFcHGdhVxAH7ozoEoelN2EfgKZZ0JVAbvnGN/w8zxiMivqkO8ijw8fXeCMDt0K2OW7q2GF0=')
vk = ecdsa.VerifyingKey.from_string(public_key, curve=ecdsa.SECP256k1)
print(vk.verify(sig[1:], message, hashfunc=DoubleSha256))
Public-key. Mathematically an elliptic curve public key is a point on the curve. For the elliptic curve used by Bitcoin, secp256k1, as well as other X9-style (Weierstrass form) curves, there are (in practice) two standard representations originally established by X9.62 and reused by many others:
uncompressed format: consists of one octet with value 0x04, followed by two blocks of size equal to the curve order size containing the (affine) X and Y coordinates. For secp256k1 this is 1+32x2 = 65 octets
compressed format: consists of one octet with value 0x02 or 0x03 indicating the parity of the Y coordinate, followed by a block of size equal tot he curve order containing the X coordinate. For secp256k1 this is 1+32 = 33 octets
The public key output by your trezor is the second form, 0x02 + 32 octets = 33 octets. Not 32.
I've never seen an X9EC library (ECDSA and/or ECDH) that doesn't accept at least the standard uncompressed form, and usually both. It is conceivable your python library expects only the uncompressed form without the leading 0x04, but if so this gratuitous and rather risky nonstandardness, unless a very good explanation is provided in the doc or code, would make me suspicious of its quality. If you do need to convert the compressed form to uncompressed you must implement the curve equation, which for secp256k1 can be found in standard references, not to mention many implementations. Compute x^3 + a*x + b, take the square root in F_p, and choose either the positive or negative value that has the correct parity (agreeing with the leading byte here 0x02).
The 'xpub' is a base58check encoding of a hierarchical deterministic key, which is not just an EC(DSA) key but adds metadata for the key derivation process. If you base58 decode it and remove the check, you get (in hex):
0488B21E01EA66F03700000000E02030F2A7DFB474D53A96CB26FEBBBE3BD3B9756F4E0A820146FF1FB4E0BD99026B4CC594C849A0D9A124725997604BC6A0EC8F100B621B1EAED4C6094619FC46good
which breaks down exactly as your display showed:
0488B21E fixed prefix
01 .depth
EA66F037 .fingerprint
00000000 .child_num
E02030F2A7DFB474D53A96CB26FEBBBE3BD3B9756F4E0A820146FF1FB4E0BD99 .chain_code
026B4CC594C849A0D9A124725997604BC6A0EC8F100B621B1EAED4C6094619FC46 .public_key
Confirming this, the ripemd160 of sha256 of (the bytes that are shown in hex as) 026B4CC594C849A0D9A124725997604BC6A0EC8F100B621B1EAED4C6094619FC46 is (the bytes shown in hex as) 441e1d2adf9ff2a6075d71d0d8782228e0df47f8, and prefixing the version byte 00 for mainnet to that and base58check encoding gives the address 17DB2Q3oZVkQAffkpFvF4cwsXggu39iKdQ as shown.
Signature. Mathematically an X9.62-type ECDSA signature is two integers, called r and s. There are two different standards for representing them, and Bitcoin uses both with variations:
ASN.1 DER format. DER is a general purpose encoding that contains 'tag' and 'length' metadata and variable length data depending on the numeric values, here r and s; for secp256k1 in general this encoding is usually 70 to 72 octets but occasionally less. However, to avoid certain 'malleability' attacks current Bitcoin requires use of 's' values less than half the curve order, commonly called 'low-s', which reduces the maximum length of the ASN.1 DER encoding to 71 octets. Bitcoin uses this for transaction signatures, and adds a 'sighash' byte immediately following it (in the 'scriptsig' aka redeem script) indicating certain options on how the signature was computed (and thus should be verified).
'plain' or P1363 format. This is fixed length and consists simply of the r and s values as fixed-length blocks; for secp256k1 this is 64 octets. Bitcoin uses this for message signatures but it adds a 'recovery' byte' to the beginning that allows determining the publickey from the message and signature if necessary, making the total 65 octets.
See https://bitcoin.stackexchange.com/questions/38351/ecdsa-v-r-s-what-is-v/38909 and https://bitcoin.stackexchange.com/questions/12554/why-the-signature-is-always-65-13232-bytes-long .
If your python library is designed for general purpose ECDSA, not Bitcoin, and wants a 64-byte signature, that almost certainly is the 'plain' format which corresponds to the Bitcoin message signature (here decoded from base64) with the first byte removed.
I'm a bit confused on the difference between SHA-2 and SHA-256 and often hear them used interchangeably. I think SHA-2 a "family" of hash algorithms and SHA-256 a specific algorithm in that family. Can anyone clear up the confusion.
The SHA-2 family consists of multiple closely related hash functions. It is essentially a single algorithm in which a few minor parameters are different among the variants.
The initial spec only covered 224, 256, 384 and 512 bit variants.
The most significant difference between the variants is that some are 32 bit variants and some are 64 bit variants. In terms of performance this is the only difference that matters.
On a 32 bit CPU SHA-224 and SHA-256 will be a lot faster than the other variants because they are the only 32 bit variants in the SHA-2 family. Executing the 64 bit variants on a 32 bit CPU will be slow due to the added complexity of performing 64 bit operations on a 32 bit CPU.
On a 64 bit CPU SHA-224 and SHA-256 will be a little slower than the other variants. This is because due to only processing 32 bits at a time, they will have to perform more operations in order to make it through the same number of bytes. You do not get quite a doubling in speed from switching to a 64 bit variant because the 64 bit variants do have a larger number of rounds than the 32 bit variants.
The internal state is 256 bits in size for the two 32 bit variants and 512 bits in size for all four 64 bit variants. So the number of possible sizes for the internal state is less than the number of possible sizes for the final output. Going from a large internal state to a smaller output can be good or bad depending on your point of view.
If you keep the output size fixed it can in general be expected that increasing the size of the internal state will improve security. If you keep the size of the internal state fixed and decrease the size of the output, collisions become more likely, but length extension attacks may become easier. Making the output size larger than the internal state would be pointless.
Due to the 64 bit variants being both faster (on 64 bit CPUs) and likely to be more secure (due to larger internal state), two new variants were introduced using 64 bit words but shorter outputs. Those are the ones known as 512/224 and 512/256.
The reasons for wanting variants with output that much shorter than the internal state is usually either that for some usages it is impractical to use such a long output or that the output need to be used as key for some algorithm that takes an input of a certain size.
Simply truncating the final output to your desired length is also possible. For example a HMAC construction specify truncating the final hash output to the desired MAC length. Due to HMAC feeding the output of one invocation of the hash as input to another invocation it means that using a hash with shorter output results in a HMAC with less internal state. For this reason it is likely to be slightly more secure to use HMAC-SHA-512 and truncate the output to 384 bits than to use HMAC-SHA-384.
The final output of SHA-2 is simply the internal state (after processing length extended input) truncated to the desired number of output bits. The reason SHA-384 and SHA-512 on the same input look so different is that a different IV is specified for each of the variants.
Wikipedia:
The SHA-2 family consists of six hash functions with digests (hash
values) that are 224, 256, 384 or 512 bits: SHA-224, SHA-256, SHA-384,
SHA-512, SHA-512/224, SHA-512/256.
Assume that I know 80% of SHA1 input. Whether cracking remaining 20% from the SHA1 hash value is easier than cracking the whole input? If it is so, by what percentage?
Eg: I know the x's in the input SHA1(xxxxxxxxyy)=hash value
Assume there are 10 bytes in the input. To crack the whole input, we'd have to try 2^(10*8) inputs. With 80% given, we only have to try 2^(2*8) inputs. That's about a quintillion times fewer. If the input size goes up, the ratio gets even larger.
SHA1 is irreversible today with about 100 unknown bits (12 bytes) in the input. With only 20% of the input unknown, that means the input size would need to be about 500 bits to be secure, or about 62 bytes.
It actually matters whether the unknown part is at the beginning or the end. Each 32 bits of known data at the beginning reduces the number of needed operations by a bit more than you might expect because some of the calculations can be re-used.
Intel's official optimization guide has a chapter on converting from MMX commands to SSE where they state the fallowing statment:
Computation instructions which use a memory operand that may not be aligned to a 16-byte boundary must be replaced with an unaligned 128-bit load (MOVDQU) followed by the same computation operation that uses instead register operands.
(chapter 5.8 Converting from 64-bit to 128-bit SIMD Integers, pg. 5-43)
I can't understand what they mean by "may not be aligned to a 16-byte boundary", could you please clarify it and give some examples?
Certain SIMD instructions, which perform the same instruction on multiple data, require that the memory address of this data is aligned to a certain byte boundary. This effectively means that the address of the memory your data resides in needs to be divisible by the number of bytes required by the instruction.
So in your case the alignment is 16 bytes (128 bits), which means the memory address of your data needs to be a multiple of 16. E.g. 0x00010 would be 16 byte aligned, while 0x00011 would not be.
How to get your data to be aligned depends on the programming language (and sometimes compiler) you are using. Most languages that have the notion of a memory address will also provide you with means to specify the alignment.
I'm guessing here, but could it be that "may not be aligned to a 16-byte boundary" means that this memory location has been aligned to a smaller value (4 or 8 bytes) before for some other purposes and now to execute SSE instructions on this memory you need to load it into a register explicitly?
Data that's aligned on a 16 byte boundary will have a memory address that's an even number — strictly speaking, a multiple of two. Each byte is 8 bits, so to align on a 16 byte boundary, you need to align to each set of two bytes.
Similarly, memory aligned on a 32 bit (4 byte) boundary would have a memory address that's a multiple of four, because you group four bytes together to form a 32 bit word.