How do I map a message from USBSniffer to PyUSB? - usb

I'm trying to figure out how to turn "TrueColor" on for a Microsoft LifeCam. I have a couple big USB logs from USBSniffer, and some sequences which consistently show up when I click the checkbox in Microsoft's software. Here's a request from the beginning of the enable sequence, and the device response (please correct me if I'm misinterpreting this data btw):
[156918643 ms] UsbSnoop - FilterDispatchAny(ac56afd2) : IRP_MJ_INTERNAL_DEVICE_CONTROL
[156918643 ms] UsbSnoop - FdoHookDispatchInternalIoctl(ac56b1ea) : fdo=88b25448, Irp=889911d8, IRQL=0
[156918643 ms] >>> URB 1482 going down >>>
-- URB_FUNCTION_CLASS_INTERFACE:
TransferFlags = 00000001 (USBD_TRANSFER_DIRECTION_IN, ~USBD_SHORT_TRANSFER_OK)
TransferBufferLength = 00000002
TransferBuffer = abb04b86
TransferBufferMDL = 00000000
UrbLink = 00000000
RequestTypeReservedBits = 00000000
Request = 00000085
Value = 00001600
Index = 00000500
[156918644 ms] UsbSnoop - MyInternalIOCTLCompletion(ac56b126) : fido=00000000, Irp=889911d8, Context=894d6c98, IRQL=2
[156918644 ms] <<< URB 1482 coming back <<<
-- URB_FUNCTION_CONTROL_TRANSFER:
PipeHandle = 885639a0
TransferFlags = 0000000b (USBD_TRANSFER_DIRECTION_IN, USBD_SHORT_TRANSFER_OK)
TransferBufferLength = 00000002
TransferBuffer = abb04b86
TransferBufferMDL = 8837fee0
00000000: 40 00
UrbLink = 00000000
SetupPacket =
00000000: a1 85 00 16 00 05 02 00
Now, I'd like to send the sequence of requests I've seen via PyUSB, but when I send that first one, I get:
>>> ret = dev.ctrl_transfer(0x80, 0x85, 0x1600, 0x0500, 8)
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "c:\Python24\Lib\site-packages\usb\core.py", line 702, in ctrl_transfer
self.__get_timeout(timeout)
File "c:\Python24\Lib\site-packages\usb\backend\libusb0.py", line 539, in ctrl_transfer
timeout
File "c:\Python24\Lib\site-packages\usb\backend\libusb0.py", line 387, in _check
raise USBError(errmsg, ret)
usb.core.USBError: (None, 'usb_control_msg: invalid request 0x85')
For what it's worth, a simple GET_STATUS seems to work:
>>> ret = dev.ctrl_transfer(0x80, 0, 0, 0, 8)
>>> print ret
array('B', [0, 0])
Why would a request which gets answered in one case be invalid in another? Syntax? Device state?

From the very helpful USB Made Simple, the request type's highest 3 bits are formatted like so:
D7 Data direction
0 - Host-to-device
1 - Device-to-host
D6:5 Type
0 = Standard
1 = Class
2 = Vendor
3 = Reserved
The log says I'm using USB's Class interface (cf. "URB_FUNCTION_CLASS_INTERFACE"), so the RequestType is specific to the device class (audio, physical... in this case video). The GET_STATUS command works because it is of the Standard type.
So a more correct command is
ret = dev.ctrl_transfer(0xa0, 0x85, 0x1600, 0x0500, 8)
I say more correct because the command returns data, but it's still not the data I expect!
EDIT: for complete verisimilitude, I needed to specify the recipient as the interface, according to the rest of the request type table from USB Made Simple:
D4:0 Recipient
0 = Device
1 = Interface
2 = Endpoint
3 = Other
4-31 = Reserved
...and also set the length to be the same as the sniffed request, so the true version of the command is:
ret = dev.ctrl_transfer(0xa1, 0x85, 0x1600, 0x0500, 2)

Related

Pybullet on colab, cannot connect X server

I am using rl-baselines-zoo 3 to run ddpg with my custom env on colab. After I used show video function in that zoo repo, it said it cannot connect to the server. It works fine on other built-in envs, so I guess it's my env problem. please, need some help...
I set every thing from zoo's tutorials
Traceback:
pybullet build time: Jul 12 2021 20:46:20
/usr/local/lib/python3.7/dist-packages/gym/logger.py:30: UserWarning:
WARN: Box bound precision lowered by casting to float32
startThreads creating 1 threads.
starting thread 0
started thread 0
argc=2
argv[0] = --unused
argv[1] = --start_demo_name=Physics Server
ExampleBrowserThreadFunc started
X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Created GL 3.3 context
Direct GLX rendering context obtained
Making context current
GL_VENDOR=VMware, Inc.
GL_RENDERER=llvmpipe (LLVM 10.0.0, 256 bits)
GL_VERSION=3.3 (Core Profile) Mesa 20.0.8
GL_SHADING_LANGUAGE_VERSION=3.30
pthread_getconcurrency()=0
Version = 3.3 (Core Profile) Mesa 20.0.8
Vendor = VMware, Inc.
Renderer = llvmpipe (LLVM 10.0.0, 256 bits)
b3Printf: Selected demo: Physics Server
startThreads creating 1 threads.
starting thread 0
started thread 0
MotionThreadFunc thread started
ven = VMware, Inc.
ven = VMware, Inc.
Wrapping the env in a VecTransposeImage.
tcmalloc: large alloc 3276800000 bytes == 0x556b03bda000 # 0x7f7cad04a001 0x7f7caa3f554f 0x7f7caa445b58 0x7f7caa449b17 0x7f7caa4e8203 0x556a81194d54 0x556a81194a50 0x556a81209105 0x556a812037ad 0x556a81196c9f 0x556a811d7d79 0x556a811d4cc4 0x556a81196ea1 0x556a81205bb5 0x556a8119630a 0x556a812087f0 0x556a812037ad 0x556a811963ea 0x556a8120460e 0x556a812034ae 0x556a811963ea 0x556a8120532a 0x556a812034ae 0x556a812031b3 0x556a81201660 0x556a81194b59 0x556a81194a50 0x556a81208453 0x556a812034ae 0x556a811963ea 0x556a812043b5
tcmalloc: large alloc 3276800000 bytes == 0x556bc78da000 # 0x7f7cad04a001 0x7f7caa3f554f 0x7f7caa445b58 0x7f7caa449b17 0x7f7caa4e8203 0x556a81194d54 0x556a81194a50 0x556a81209105 0x556a812037ad 0x556a81196c9f 0x556a811d7d79 0x556a811d4cc4 0x556a81196ea1 0x556a81205bb5 0x556a8119630a 0x556a812087f0 0x556a812037ad 0x556a811963ea 0x556a8120460e 0x556a812034ae 0x556a811963ea 0x556a8120532a 0x556a812034ae 0x556a812031b3 0x556a81201660 0x556a81194b59 0x556a81194a50 0x556a81208453 0x556a812034ae 0x556a811963ea 0x556a812043b5
/content/gdrive/My Drive/hsr/rl-baselines3-zoo/logs/ddpg/FoodHuntingHSR-v0_3/videos/final-model-ddpg-FoodHuntingHSR-v0-step-0-to-step-200.mp4
/usr/local/lib/python3.7/dist-packages/gym/logger.py:30: UserWarning:
WARN: Tried to pass invalid video frame, marking as broken: Your frame has data type float32, but we require uint8 (i.e. RGB values from 0-255).
Saving video to /content/gdrive/My Drive/hsr/rl-baselines3-zoo/logs/ddpg/FoodHuntingHSR-v0_3/videos/final-model-ddpg-FoodHuntingHSR-v0-step-0-to-step-200.mp4
numActiveThreads = 0
stopping threads
destroy semaphore
semaphore destroyed
Thread with taskId 0 exiting
Thread TERMINATED
destroy main semaphore
main semaphore destroyed
finished
numActiveThreads = 0
btShutDownExampleBrowser stopping threads
Thread with taskId 0 exiting
Thread TERMINATED
destroy semaphore
semaphore destroyed
destroy main semaphore
main semaphore destroyed
Exception ignored in: <function VecVideoRecorder.__del__ at 0x7f7c2b5cc200>
Traceback (most recent call last):
File "/content/gdrive/My Drive/hsr/stable-baselines3/stable_baselines3/common/vec_env/vec_video_recorder.py", line 114, in __del__
File "/content/gdrive/My Drive/hsr/stable-baselines3/stable_baselines3/common/vec_env/vec_video_recorder.py", line 110, in close
File "/content/gdrive/My Drive/hsr/stable-baselines3/stable_baselines3/common/vec_env/base_vec_env.py", line 278, in close
File "/content/gdrive/My Drive/hsr/stable-baselines3/stable_baselines3/common/vec_env/dummy_vec_env.py", line 67, in close
File "/content/gdrive/My Drive/hsr/stable-baselines3/stable_baselines3/common/monitor.py", line 113, in close
File "/usr/local/lib/python3.7/dist-packages/gym/core.py", line 243, in close
File "/usr/local/lib/python3.7/dist-packages/gym/core.py", line 243, in close
File "/content/gdrive/My Drive/hsr/PyLIS/gym-foodhunting/gym_foodhunting/foodhunting/gym_foodhunting.py", line 538, in close
pybullet.error: Not connected to physics server
class FoodHuntingEnv(gym.Env):
metadata = {'render.modes': ['human','rgb_array']}
GRAVITY = -10.0
BULLET_STEPS = 120 # p.setTimeStep(1.0 / 240.0), so 1 gym step == 0.5 sec.
def __init__(self, render=False, robot_model=R2D2, max_steps=500, num_foods=3, num_fakes=0, object_size=1.0, object_radius_scale=1.0, object_radius_offset=1.0, object_angle_scale=1.0):
"""Initialize environment.
"""
### gym variables
self.observation_space = robot_model.getObservationSpace() # classmethod
self.action_space = robot_model.getActionSpace() # classmethod
self.reward_range = (-1.0, 1.0)
self.seed()
### pybullet settings
self.ifrender = render
self.physicsClient = p.connect(p.GUI if self.ifrender else p.DIRECT)
p.setAdditionalSearchPath(pybullet_data.getDataPath())
### env variables
self.robot_model = robot_model
self.max_steps = max_steps
self.num_foods = num_foods
self.num_fakes = num_fakes
self.object_size = object_size
self.object_radius_scale = object_radius_scale
self.object_radius_offset = object_radius_offset
self.object_angle_scale = object_angle_scale
self.plane_id = None
self.robot = None
self.object_ids = []
### episode variables
self.steps = 0
self.episode_rewards = 0.0
def close(self):
"""Close environment.
"""
p.disconnect(self.physicsClient)
def reset(self):
"""Reset environment.
"""
self.steps = 0
self.episode_rewards = 0
p.resetSimulation()
# p.setTimeStep(1.0 / 240.0)
p.setGravity(0, 0, self.GRAVITY)
self.plane_id = p.loadURDF('plane.urdf')
self.robot = self.robot_model()
self.object_ids = []
for i, (pos, orn) in enumerate(self._generateObjectPositions(num=(self.num_foods+self.num_fakes), radius_scale=self.object_radius_scale, radius_offset=self.object_radius_offset, angle_scale=self.object_angle_scale)):
if i < self.num_foods:
urdfPath = 'food_sphere.urdf'
else:
urdfPath = 'food_cube.urdf'
object_id = p.loadURDF(urdfPath, pos, orn, globalScaling=self.object_size)
self.object_ids.append(object_id)
for i in range(self.BULLET_STEPS):
p.stepSimulation()
obs = self._getObservation()
#print('reset laile')
#self.robot.printAllJointInfo()
return obs
def step(self, action):
"""Apply action to environment, then return observation and reward.
"""
self.steps += 1
self.robot.setAction(action)
reward = -1.0 * float(self.num_foods) / float(self.max_steps) # so agent needs to eat foods quickly
for i in range(self.BULLET_STEPS):
p.stepSimulation()
reward += self._getReward()
self.episode_rewards += reward
obs = self._getObservation()
done = self._isDone()
pos, orn = self.robot.getPositionAndOrientation()
info = { 'steps': self.steps, 'pos': pos, 'orn': orn }
if done:
#print('Done laile')
info['episode'] = { 'r': self.episode_rewards, 'l': self.steps }
# print(self.episode_rewards, self.steps)
#print(self.robot.getBaseRollPosition(), self.robot.getTorsoLiftPosition(), self.robot.getHeadPosition(), self.robot.getArmPosition(), self.robot.getWristPosition(), self.robot.getGripperPosition()) # for HSR debug
#print(self.robot.getHeadPosition(), self.robot.getGripperPosition()) # for R2D2 debug
return obs, reward, done, info
def render(self, mode='human', close=False):
"""This is a dummy function. This environment cannot control rendering timing.
"""
if mode != 'rgb_array':
return np.array([])
return self._getObservation()
def seed(self, seed=None):
"""Set random seed.
"""
self.np_random, seed = seeding.np_random(seed)
return [seed]
def _getReward(self):
"""Detect contact points and return reward.
"""
reward = 0
contacted_object_ids = [ object_id for object_id in self.object_ids if self.robot.isContact(object_id) ]
for object_id in contacted_object_ids:
reward += 1 if self._isFood(object_id) else -1
p.removeBody(object_id)
self.object_ids.remove(object_id)
return reward
def _getObservation(self):
"""Get observation.
"""
obs = self.robot.getObservation()
return obs
def _isFood(self, object_id):
"""Check if object_id is a food.
"""
baseLink, urdfPath = p.getBodyInfo(object_id)
return urdfPath == b'food_sphere.urdf' # otherwise, fake
def _isDone(self):
"""Check if episode is done.
"""
#print(self.object_ids,'self')
available_object_ids = [ object_id for object_id in self.object_ids if self._isFood(object_id) ]
#print(available_object_ids)
return self.steps >= self.max_steps or len(available_object_ids) <= 0
def _generateObjectPositions(self, num=1, retry=100, radius_scale=1.0, radius_offset=1.0, angle_scale=1.0, angle_offset=0.5*np.pi, z=1.5, near_distance=0.5):
"""Generate food positions randomly.
"""
def genPos():
r = radius_scale * self.np_random.rand() + radius_offset
a = -np.pi * angle_scale + angle_offset
b = np.pi * angle_scale + angle_offset
ang = (b - a) * self.np_random.rand() + a
return np.array([r * np.sin(ang), r * np.cos(ang), z])
def isNear(pos, poss):
for p, o in poss:
if np.linalg.norm(p - pos) < near_distance:
return True
return False
def genPosRetry(poss):
for i in range(retry):
pos = genPos()
if not isNear(pos, poss):
return pos
return genPos()
poss = []
for i in range(num):
pos = genPosRetry(poss)
orn = p.getQuaternionFromEuler([0.0, 0.0, 2.0*np.pi*self.np_random.rand()])
poss.append((pos, orn))
return poss

UnicodeDecodeError: While reading a csv file using .csvreader

Here's my code:
import csv
path = "/home/Downloads/sample_email.csv"
with open(path) as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
print(row['first_name'], row['last_name'])
The error is:
UnicodeDecodeError Traceback (most recent call last)
<ipython-input-47-d89ea200a227> in <module>
3 with open(path) as csvfile:
4 reader = csv.DictReader(csvfile)
----> 5 for i in reader:
6 print(i['first_name'], i['last_name'])
/usr/lib/python3.6/csv.py in __next__(self)
109 if self.line_num == 0:
110 # Used only for its side effect.
--> 111 self.fieldnames
112 row = next(self.reader)
113 self.line_num = self.reader.line_num
/usr/lib/python3.6/csv.py in fieldnames(self)
96 if self._fieldnames is None:
97 try:
---> 98 self._fieldnames = next(self.reader)
99 except StopIteration:
100 pass
/usr/lib/python3.6/codecs.py in decode(self, input, final)
319 # decode input (taking the buffer into account)
320 data = self.buffer + input
--> 321 (result, consumed) = self._buffer_decode(data, self.errors, final)
322 # keep undecoded input until the next call
323 self.buffer = data[consumed:]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 596: invalid start byte
When i want to convert this file to df, i have to use :
df = pd.read_csv(path, sep=',', engine = 'python')
Any help?
My aim is , extract the data, and a create a email template. Any guide about this matter is also appriciated.

I'm trying to permanently change fits headers using astropy hdu.writeto has added "None" to my fits file

I want to
read in a fits file
change some of the headers
save it, with the changes, to a new fits file.
So far I think I have achieved this, however the new fits file has "None" when I do hdu.info(). I'm confused what this means, why it's there, and is it bad?
I'll paste my code below with the associated outputs:
from astropy.io import fits
hdulist = fits.open('2D_comb_ff_wavcal_red_sci_2.fits')
hdu = hdulist[0]
hdulist.info()
Filename: 2D_comb_ff_wavcal_red_sci_2.fits
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 287 (1024, 1024) float32
print 'CRVAL1 then = ', hdu.header['CRVAL1']
print 'CRVAL2 then = ', hdu.header['CRVAL2']
CRVAL1 then = 14975.1660156
CRVAL2 then = 1.0
CRVAL1_orig = hdu.header['CRVAL1']
CRVAL2_orig = hdu.header['CRVAL2']
hdu.header['CRVAL1'] = CRVAL2_orig
hdu.header['CRVAL2'] = CRVAL1_orig
print 'CRVAL1 now = ', hdu.header['CRVAL1']
print 'CRVAL2 now = ', hdu.header['CRVAL2']
CRVAL1 now = 1.0
CRVAL2 now = 14975.1660156
hdu.writeto('newheader.fits', overwrite=True)
new = fits.open('newheader.fits')
print new.info()
Filename: newheader.fits
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 287 (1024, 1024) float32
None
The headers have updated (I checked) but why does it say "None" here? Is hdu.writeto() the wrong thing? I've tried reading about flush() but it confused me.
Thanks
That is because you are printing what is returned from .info(). Note in the fourth line you have
hdulist.info()
with no print, yet you get the output you expect. furthermore you do get properly
0 PRIMARY 1 PrimaryHDU 287 (1024, 1024) float32
when you call new.info(). So to sum, it is the method itself which prints, likely not returning anything or returning None by default, and since you print it, it gets displayed as a string.
Just remove the final print.

Why write to BinaryWriter twice?

I'm implementing this tone-generator program and it works great:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/c2b953b6-3c85-4eda-a478-080bae781319/beep-beep?forum=vbgeneral
What I can't figure out, is why the following two lines of code:
BW.Write(Sample)
BW.Write(Sample)
One "write" makes sense, but why the second "write"?
The example is a bit cryptic but the wave file is configured to be 2 channels thus the two writes are simply sending the same audio data to both channels.
The wave header is this hardcoded bit:
Dim Hdr() As Integer = {&H46464952, 36 + Bytes, &H45564157, _
&H20746D66, 16, &H20001, 44100, _
176400, &H100004, &H61746164, Bytes}
Which decoded means:
H46464952 = 'RIFF' (little endian)
36+Bytes = Length of header + length of data
H45564157 = 'WAVE' (little endian)
H20746D66 = 'fmt ' (little endian)
16 = length of fmt chunk (always 16)
H20001 = 0x0001: PCM,
0x0002: 2 channels
44100 = sampleRate
176400 = sampleRate*numChannels*bytesPerSample = 44100*2*2
H100004 = 0x0004: numChannels*bytesPerSample,
0x0010: bitsPerSample (16)
H61746164 = 'data'
Bytes = size of data chunk

Learning Python 3.3 - TypeError: 'int' object is not callable

I was trying to solve Challenge 2 at the end of the classes chapter (Chapter 8) in "Python Programming for the Absolute Beginner" which is stated as:
Write a program that simulates a television by creating it as an object. The user should be able to enter a channel number and raise or lower the volume. Make sure that the channel number and the volume level stay within valid ranges.
I keep getting: TypeError: 'int' object is not callable, which at this stage just isn't very helpful.
I'm a beginner but I've seen something really similar working (see at the bottom right below my code) and nearly went as far as nearly copying that code. Could somebody maybe explain what's wrong with this and how I can get it to work?
Here's the complete error:
Traceback (most recent call last):
File "Untitled 3.py", line 59, in <module>
main()
File "Untitled 3.py", line 50, in main
tv.channel(newchannel = int(input("What channel would you like to set the TV to?")))
TypeError: 'int' object is not callable
My code is below,
Thanks
class Television(object):
"""a TV"""
def __init__(self, channel = 0, volume = 0):
self.channel = channel
self.volume = volume
def channel(self, newchannel = 0):
if newchannel <= 0 or newchannel >9:
print("No negative numbers or numbers higher than 9. Start again from the menu")
else:
self.channel = newchannel
print("You set the TV on channel", self.channel)
def volume(self, newvolume = 0):
if newvolume <= 0 or newvolume >9:
print("No negative numbers or numbers higher than 9. Start again from the menu")
else:
self.volume = newvolume
print("You set the TV on volume", self.volume)
def watch(self):
print("You are watching channel", self.channel, "at volume", self.volume)
def main():
tv = Television()
choice = None
while choice != "0":
print \
("""
TV
0 - Quit
1 - Watch the TV
2 - Change channel
3 - Set the volume
""")
choice = input("Choice: ")
print()
# exit
if choice == "0":
print("Good-bye.")
elif choice == "1":
tv.watching()
elif choice == "2":
tv.channel(newchannel = int(input("What channel would you like to set the TV to?")))
elif choice == "3":
tv.volume(newvolume = int(input("What channel would you like to set the TV to?")))
# some unknown choice
else:
print("\nSorry, but", choice, "isn't a valid choice.")
main()
("\n\nPress the enter key to exit.")
Why does the following work instead? To me, it looks pretty similar to what I've done.
class Critter(object):
"""A virtual pet"""
def __init__(self, hunger = 0, boredom = 0):
self.hunger = hunger
self.boredom = boredom
def eat(self, food = 4):
print("Brruppp. Thank you.")
self.hunger += food
if self.hunger < 0:
self.hunger = 0
crit = Critter()
print(crit.hunger)
crit.eat(food = int(input("how much do you want to feed him?")))
print(crit.hunger)
The problem is you are defining a method with the same name as a property. That is, you're saying Television.channel is an int, but later you are binding a method to that name.