I am looking to convert EC key generated using cryptography module to their respective OpenSSH strings. like
ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAhANiNlmyHtBUgaPXG+CtCVK8mQxBUtDjX3/nqqPZAHhduAAAAIE/JNDqLTeq9WVa5XWyU2Y7NJXfV54wakHmsP5gRNeh2
This is the code I use for EC key generation
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import ec
key=ec.generate_private_key(ec.SECP256R1(), default_backend())
I tried the following.
numbers = key.private_numbers()
opensshpublic = 'ecdsa-sha2-nistp256' + base64.b64encode('nistp256' + numbers.public_numbers.x, numbers.public_numbers.y)
but that didn't seem to work.
I suppose there should be a easy way to do this but I am missing something.
Cryptography added support for doing this in June 2016, it is possible as follows
from cryptography.hazmat.primitives import serialization
key.public_bytes(serialization.Encoding.OpenSSH, serialization.PublicFormat.OpenSSH)
this gives those neat OpenSSH public keys
Related
I have Apache Arrow data on the server (Python) and need to use it in the browser. It appears that Arrow Flight isn't implemented in JS. What are the best options for sending the data to the browser and using it there?
I don't even need it necessarily in Arrow format in the browser. This question hasn't received any responses, so I'm adding some additional criteria for what I'm looking for:
Self-describing: don't want to maintain separate schema definitions
Minimal overhead: For example, an array of float32s should transfer as something compact like a data type indicator, length value and sequence of 4-byte float values
Cross-platform: Able to be easily sent from Python and received and used in the browser in a straightforward way
Surely this is a solved problem? If it is I've been unable to find a solution. Please help!
Building off of the comments on your original post by David Li, you can implement a non-streaming version what you want without too much code using PyArrow on the server side and the Apache Arrow JS bindings on the client. The Arrow IPC format satisfies your requirements because it ships the schema with the data, is space-efficient and zero-copy, and is cross-platform.
Here's a toy example showing generating a record batch on server and receiving it on the client:
Server:
from io import BytesIO
from flask import Flask, send_file
from flask_cors import CORS
import pyarrow as pa
app = Flask(__name__)
CORS(app)
#app.get("/data")
def data():
data = [
pa.array([1, 2, 3, 4]),
pa.array(['foo', 'bar', 'baz', None]),
pa.array([True, None, False, True])
]
batch = pa.record_batch(data, names=['f0', 'f1', 'f2'])
sink = pa.BufferOutputStream()
with pa.ipc.new_stream(sink, batch.schema) as writer:
writer.write_batch(batch)
return send_file(BytesIO(sink.getvalue().to_pybytes()), "data.arrow")
Client
const table = await tableFromIPC(fetch(URL));
// Do what you like with your data
Edit: I added a runnable example at https://github.com/amoeba/arrow-python-js-ipc-example.
Keplr just added the feature to import an account using a private key (which is awesome!). This was primarily requested because a number of lunie users lost their mnemonic phrases but maintained their access to lunie itself. That meant lunie could export a private key but not the mnemonic itself. It may also be userful for future Ethermint users who want to migrate from metamask (which also provides the ability to export and import private keys).
The keys command in the Cosmos SDK CLI also allows a user to export a private key, however I'm unsure how to convert a key from that format into one that could be imported in keplr. The CLI command is as follows:
gaiacli keys export [name]
It requests the passphrase to decrypt the key and then a new passphrase to encrypt the exported key. The results are in the following format:
-----BEGIN TENDERMINT PRIVATE KEY-----
type: secp256k1
kdf: bcrypt
salt: C49BCB6A8358745812F5770A63BD09AD
NmqXg+dPDvVKawZwyER6l3V41tKxWaiIU/or2G7t9SBKRJ0oRREchssK4NpRp+Di
5KNHxHz2QXHLhbPQweo9iVkPPrNQ1uiSGH7maoY=
=GHXH
-----END TENDERMINT PRIVATE KEY-----
(this is an example created for the purpose of this question)
How can I convert this key into something I can use for keplr or metamask?
This feature just got merged!
https://github.com/cosmos/cosmos-sdk/pull/8043
The command is:
packaged export <name> --unarmored-hex and --unsafe
Where packaged is the name of your daemon binary CLI and name is the name of the key you want to export.
I want to encrypt values in one column of my Pandas (or PySpark) dataframe, e.g. to take the the column mobno in the following dataframe, encrypt it and put the result in the encrypted_value column:
I want to use AWS KMS encryption key. My question is: what is the most elegant way how to achieve this?
I am thinking about using UDF, which will call the boto3's KMS client. Something like:
#udf
def encrypt(plaintext):
response = kms_client.encrypt(
KeyId=aws_kms_key_id,
Plaintext=plaintext
)
ciphertext = response['CiphertextBlob']
return ciphertext
and then applying this udf on the whole column.
But I am not quite confident this is the right way. This stems from the fact that I am an encryption-rookie - first, I don't even know this kms_client_encrypt function is meant for encrypting values (from the columns) or it is meant for manipulate the keys. Maybe the better way is to obtain the key and then use some python encryption library (such as hashlib).
I would like to have some clarification on the encryption process and also recommendation what the best approach to column encryption is.
Since Spark 3.3 you can do AES encryption (and decryption) without UDF.
aes_encrypt(expr, key[, mode[, padding]]) - Returns an encrypted value of expr using AES in given mode with the specified padding. Key lengths of 16, 24 and 32 bits are supported. Supported combinations of (mode, padding) are ('ECB', 'PKCS') and ('GCM', 'NONE'). The default mode is GCM.
Arguments:
expr - The binary value to encrypt.
key - The passphrase to use to encrypt the data.
mode - Specifies which block cipher mode should be used to encrypt messages. Valid modes: ECB, GCM.
padding - Specifies how to pad messages whose length is not a multiple of the block size. Valid values: PKCS, NONE, DEFAULT. The DEFAULT padding means PKCS for ECB and NONE for GCM.
from pyspark.sql import functions as F
df = spark.createDataFrame([('8223344556',)], ['mobno'])
df = df.withColumn('encrypted_value', F.expr("aes_encrypt(mobno, 'your_secret_keyy')"))
df.show()
# +----------+--------------------+
# | mobno| encrypted_value|
# +----------+--------------------+
# |8223344556|[9B 33 DB 9B 5D C...|
# +----------+--------------------+
df.printSchema()
# root
# |-- mobno: string (nullable = true)
# |-- encrypted_value: binary (nullable = true)
To avoid many calls to the KMS service in a UDF, use AWS Secrets Manager instead to retrieve your encryption key and pycrypto to encrypt the column. The following works:
from pyspark.sql.functions import udf, col
from Crypto.Cipher import AES
region_name = "eu-west-1"
session = boto3.session.Session()
client = session.client(service_name='secretsmanager', region_name=region_name)
get_secret_value_response = client.get_secret_value(SecretId=secret_name)
secret_key = json.loads(get_secret_value_response['SecretString'])
clear_text_column = 'mobo'
def encrypt(key, text):
obj = AES.new(key, AES.MODE_CFB, 'This is an IV456')
return obj.encrypt(text)
def udf_encrypt(key):
return udf(lambda text: encrypt(key, text))
df.withColumn("encrypted", udf_encrypt(secret_key)(col(clear_text_column))).show()
Or alternatively, using more efficient Pandas UDF as suggested by #Vektor88 (PySpark 3 syntax):
from functools import partial
encrypt_with_key = partial(encrypt, secret_key)
#pandas_udf(BinaryType())
def pandas_udf_encrypt(clear_strings: pd.Series) -> pd.Series:
return clear_strings.apply(encrypt_with_key)
df.withColumn('encrypted', pandas_udf_encrypt(clear_text_column)).show()
Here is my code:
error_reporting(E_ALL);
require __DIR__ . '/vendor/autoload.php';
use phpseclib\Net\SSH2;
use phpseclib\Crypt\RSA;
$ssh = new SSH2('stg.net');
$key = new RSA();
$key->loadKey(file_get_contents('/Users/me/.ssh/my_private_key'));
if (!$ssh->login('username', $key)) {
print_r($ssh->getLastError());
print_r($ssh->getErrors());
exit('Login Failed');
}
echo $ssh->exec('pwd');
echo $ssh->exec('ls -la');
Output:
Array
(
)
In vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php there is function _privatekey_login($username, $privatekey)
$publickey = $privatekey->getPublicKey(RSA::PUBLIC_FORMAT_RAW);
if ($publickey === false) {
return false;
}
I'm getting false, maybe I have to set public key too? How can it be done?
How to debug it?
+++ UPDATE +++
I tried advices/hints from these tickets too:
phpseclib always gives login failed with no log
Net/SSH2 - phpseclib login failing - error: "failedArray"
The problem is that the key in question is an ECDSA key. Quoting https://github.com/phpseclib/phpseclib/issues/1082#issuecomment-396122366 :
My library supports EdDSA keys in the OpenSSH format (ie. the kind
that ssh-keygen would generate), the PuTTY format (ie. the kind
puttygen would generate), in libsodium format and in the format
specified in this IETF Draft:
https://datatracker.ietf.org/doc/html/draft-ietf-curdle-pkix-07
If libsodium / sodium_compat are being used the keys are converted
from whatever format they were in to the libsodium format to
facilitate libsodium's use.
Encrypted OpenSSH private keys are not supported for the same reason
sodium_compat does not support Argon2i - it's too slow. OpenSSH uses a
custom form of bcrypt that does bcrypt 128 times as I recall and
encrypts a different string etc so PHP's bcrypt implementation cannot
be used and since bcrypt uses a custom key expansion OpenSSL's
implementation of Blowfish can't be used either.
Here the author is talking about EdDSA - not ECDSA - but from the rest of the post it sounds like ECDSA over prime finite fields is complete as well.
Quoting the post that follows that one:
Also, I'm not ready to make my code public yet. I just thought I'd post a progress report for anyone interested.
My guess is that this implementation will live in the master branch and not the 2.0 branch. I say that because the DSA changes were in the master branch and not the 2.0 branch. Eventually the master branch (as I understand it) will become 3.0.0 but idk when that'd happen.
I am trying to figure out how to import a symmetric key into NSS for use with encryption at the core crypto boundary. These functions are described
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Reference/NSS_cryptographic_module/FIPS_mode_of_operation
I have been able to do every other type of crypto operation by following the documentation because it mirrors PKCS 11 described here:
http://docs.oasis-open.org/pkcs11/pkcs11-base/v2.40/cos01/pkcs11-base-v2.40-cos01.html
However attempting to import any template where the CK_OBJECT_CLASS" is "CKO_SECRET_KEY" always returns "CKR_ATTRIBUTE_VALUE_INVALID 0x00000013". But I have no problem with assymetric (public/private)
CK_RV crv;
CK_FUNCTION_LIST_PTR pFunctionList;
CK_OBJECT_CLASS keyClass = CKO_SECRET_KEY;
CK_ATTRIBUTE keyTemplate[] = {
{CKA_CLASS, &keyClass, sizeof(keyClass)}
};
crv = pFunctionList->C_CreateObject(hRwSession, keyTemplate, 1, &hKey);
printf("failed with 0x%08X\n", crv);
But according to the documentation this should be returning "CKR_TEMPLATE_INCOMPLETE" as "CKO_SECRET_KEY" is a valid object class.
Again I have had no trouble with assymetric. I should Also point out that my function pointers is for FIPS mode only. Any insight is greatly appreciated!
It looks like the code you pasted is either incomplete or simply wrong. In particular, there's no concrete value for the key you're creating in the template (CKA_VALUE), which can easily cause the error you're getting from C_CreateObject.