I have a dataframe as shown below
Inspector_ID Sector Waste Fire Traffic
1 A 7 2 1
1 B 0 0 0
1 C 18 2 0
2 A 1 6 3
2 B 1 4 0
2 C 4 14 2
3 A 0 0 0
3 B 2 6 12
3 C 0 1 4
From the above dataframe I would like to calculate the Inspector's expertise score in raising issues in a domain (waste, Fire and Traffic).
For example the the score of inspector-1 for waste is (((7/8)*2) + ((18/22)*3)/2)/2
I1W = Inspector-1 similarity in waste.
Ai = No. of waste issues raised by inspector-1 in sector i
Ti = Total no. of waste issues in sector i
Ni = No of inspectors raised issues in sector i(if all zero then only it is considered as not raised)
TS1 = Total no of sectors the inspector-1 visited.
I1W = Sum((Ai/Ti)*Ni)/TS1
The expected output is below dataframe
Inspector_ID Waste Fire Traffic
1 I1W I1F I1T
2 I2W I2F I2T
3 I3W I3F I3T
TBF = To be filled
You could look into something along the lines of:
newData = []
inspector_ids = df['Inspector_ID'].unique().tolist()
for id in inspector id:
current_data = df.loc[df['Inspector_id'] == id]
#With the data of the current inspector you get the desired values
waste_val = 'I1W'
fire_val = 'I1F'
traffic_val = 'I1T'
newData.append([id,waste_val, fire_val, traffic_val])
new_df = pd.DataFrame(newData, columns = ['Inspector_ID','Waste','Fire','Traffic'])
Some ideas for getting the values you need
#IS1 = Sectors visited by inspector 1.
#After the first loc that filters the inspector
sectors_visited = len(df['Sector'].unique().tolist())
#Ai = No. of waste issues raised by inspector-1 in sector i
waste_Issues_A = current_data.loc[current_data['Sector' == A].value_counts()
#Ti = Total no. of waste issues in sector i
#You can get total number of issues by sector with
df['Sector'].value_counts()
#Ni = No of inspectors raised issues in sector i(if all zero then only it is considered as not raised)
#I dont know if i understand this one correctly, I guess its the number
#of inspectors that raised issues on a sector
inspectors_sector_A = len(df.loc[df['Sector'] == A]['Inspector_ID'].unique().tolist())
The previous was done by memory so take the code with a grain of salt (Specially the Ni one).
I have a KY-022 IR module that I can't get to work on my NodeMCU. I've been searching for some code samples in Lua on the internet with no luck. Can anyone point me in the right direction? Any code samples would be greatly appreciate it.
At the moment I have the following code:
local pin = 4
gpio.mode(pin, gpio.OPENDRAIN, gpio.PULLUP)
gpio.trig(pin, "down", function (level, micro)
print(gpio.read(pin), level, micro)
end)
When I press a button on the remote, I get something like this:
0 0 571940709
0 0 571954086
0 0 571955257
1 0 571958694
1 0 571963275
1 0 571969917
0 0 571974347
0 0 571980989
1 0 571983203
1 0 571987709
0 0 571993359
1 0 572000078
0 0 572004508
0 0 572047513
0 0 572058674
So, how do I get from that to figuring out which key was pressed on the remote?
After a month or so i've reopened this project and played around with it some more. As piglet suggested, I started listening for both high and low signals. The data is still very inconsistent and can't get a stable reading.
(And by the way, thanks for the vote-down piglet, that was greatly appreciated. I wish you could have seen my search history before you decided that i'm ignorant)
I'm going to post my curent code maybe somebody can point out what I'm doing wrong here.
local pin = 4
local prevstate = false
local prevmicro = 0
local prevtime = 0
local count = 0
gpio.mode(pin, gpio.INT)
gpio.trig(pin, "both", function (level, micro)
--local state = gpio.read(pin)
local state = level
if (micro - prevmicro) > 90000 then
prevmicro = 0
prevstate = false
count = 0
print("\n#", "st", "lv", "microtime", "timing")
end
if prevstate ~= state then
time = math.floor((micro - prevmicro)/100)
prevstate = state
prevmicro = micro
if time > 3 and time < 1000 then
if prevtime > 80 and prevtime < 100 then
if time > 17 and time < 25 then
print('Repeat')
elseif time > 40 and time < 50 then
print('Start')
end
else
print(count, gpio.read(pin), level, micro, time)
count = count + 1
end
prevtime = time
end
end
end)
and here are some sample readouts from pushing the same button:
# st lv microtime timing
1 1 1 1504559531 16
2 1 0 1504566995 74
3 0 1 1504567523 5
4 1 0 1504573619 60
5 0 1 1504587422 138
6 1 0 1504588011 5
7 1 1 1504604250 162
8 1 0 1504605908 16
9 1 1 1504659929 540
10 1 0 1504662154 22
# st lv microtime timing
1 1 1 1505483535 16
2 1 0 1505491003 74
3 0 1 1505491558 5
4 1 0 1505497627 60
5 0 1 1505511409 137
6 1 0 1505512023 6
7 1 1 1505518186 61
8 1 0 1505527733 95
9 1 0 1505586167 22
10 1 1 1505586720 5
# st lv microtime timing
1 1 1 1507990937 16
2 1 0 1507998405 74
3 0 1 1507998934 5
4 1 0 1508005029 60
5 0 1 1508018811 137
6 1 0 1508019424 6
7 1 1 1508035641 162
8 1 0 1508037322 16
9 1 1 1508091345 540
10 1 0 1508093570 22
As it turns out, the Lua code required for this is actually quite simple.
Where the code above is falling over is actually the print statements. These are extremely expensive and basically, kill your sampling resolution until it's useless.
You are in essence, writing an interrupt service routine, you have a limited time budget before you have to read the next edge change and if it happens before you are done processing, tough luck! So you need to make the ISR as efficient as you can.
In the example below, we listen to the "both" edge event, when one occurs, we simply record an indication of which edge and what duration.
Periodically (using a timer) we print out the contents of the waveform.
This perfectly matches the waveform on my logic analyzer, you still have the challenge of decoding the signal. Though, there are lots of great protocol docs that explain how to take accurate waveform data and use it to determine the signal being sent. I found that a lot of cheap "brand x" remotes appear to be using the NEC protocol, so this might be a good place to start depending on your project.
IR transmission because of its nature is not completely error-free so you may get a spurious edge signal from time to time but the code below is pretty stable and runs quite well in isolation, I have yet to test it when the Microcontroller is under more load than just listening for IR.
It may turn out that using Lua for this purpose is not the best due to the fact that it is an interpreted language (each command issued is parsed and then executed at runtime, this is not at all efficient.) But I will see how far I can get before I decide to write a c module.
local irpin = 2
local lastTimestamp = 0
local waveform = {}
local i = 1
gpio.mode(irpin,gpio.INT)
gpio.trig(irpin, "both", function(level, ts)
onEdge(level, ts)
end)
function onEdge(level, ts)
waveform[i] = level
waveform[i+1] = ts - lastTimestamp
lastTimestamp = ts
i = i+2
end
-- Print out the waveform
function showWaveform ()
if table.getn(waveform) > 65 then
for k,v in pairs(waveform) do
print(k,v)
end
i = 1;
waveform = {}
end
end
tmr.alarm(0, 1000, 1, showWaveform)
print("Ready")
The following code works for my 17 key remote which came with my cheap KY-022 module. I just finished it and haven't had time to clean it up nor to optimize it, so bear with me.
local IR = 2
local lts, i, wave = 0, 0, {}
local keys = {}
keys['10100010000000100000100010101000'] = '1'
keys['10001010000000100010000010101000'] = '2'
keys['10101010000000100000000010101000'] = '3'
keys['10000010000000100010100010101000'] = '4'
keys['10000000000000100010101010101000'] = '5'
keys['10101000000000100000001010101000'] = '6'
keys['10101010000000000000000010101010'] = '7'
keys['10100010001000000000100010001010'] = '8'
keys['10100000100000000000101000101010'] = '9'
keys['10100000101000000000101000001010'] = '0'
keys['10001010001000000010000010001010'] = '*'
keys['10100010100000000000100000101010'] = '#'
keys['10000000101000000010101000001010'] = 'U'
keys['10000000100000000010101000101010'] = 'L'
keys['10001000101000100010001000001000'] = 'R'
keys['10001000001000100010001010001000'] = 'D'
keys['10000010101000000010100000001010'] = 'OK'
local function getKey()
local data = ''
local len = table.getn(wave)
if len >= 70 then
local pkey = 0
local started = false
for k, v in pairs(wave) do
v = math.floor(v/100)
if (pkey == 87 or pkey == 88 or pkey == 89) and (v > 40 and v < 50) then
started = true
end
pkey = v
if started then
if v > 300 then
started = false
end
--this is just to fix some random skipped edges
if (v > 20 and v < 25) or v == 11 then
if v > 20 and v < 25 then
d = 17
else
d = 6
end
v1 = v - d
data = data .. '' .. math.floor(v1/10)
v2 = v - (v - d)
data = data .. '' .. math.floor(v2/10)
else
if v < 40 then
data = data .. '' .. math.floor(v/10)
end
end
end
end
control = data:sub(0, 32)
if control == '00000000000000000101010101010101' then
data = data:sub(32, 63)
print(len, data, keys[data] or '?')
end
end
lts, i, wave = 0, 0, {}
end
local function onEdge(level, ts)
local time = ts - lts
wave[i] = time
i = i + 1
if time > 75000 then
tmr.alarm(0, 350, 0, getKey)
end
lts = ts
end
gpio.mode(IR,gpio.INT)
gpio.trig(IR, "both", onEdge)
I'm putting this asside and start working on some other parts of my project for the moment, but if anyone has any suggestions on how I could improve it, make it faster, smaller even, leave a comment.
PS: for those that are going to complain about not working for them, you need to adjust the if statement values for the started variable based on your remote timings. In my case it's always 88 or 89 followed by 44.
You have to get the sequence sent by the remote for each button.
Record the IR emitter's on-off sequence by logging the time stamps for high-low and low-high transitions.
Note the various patterns for each button you want to use or emulate.
Here's a in-depth tutorial http://www.instructables.com/id/How-To-Useemulate-remotes-with-Arduino-and-Raspber/
You can find this and similar resources using www.google.com
I need to calculate messages count in telegram or get messages history.
Tell me, is it possible to do this?
I know, that i can see messages count in chat from some member in telegram desktop. Maybe i can do this in any conversation?
Thank you!
You can count messages in telegram, get messages history and do plenty of other things with Telegram API. Here is a brilliant article that describes the process step by step: https://towardsdatascience.com/introduction-to-the-telegram-api-b0cd220dbed2
It worked like a charm for me both for retrieving messages and counting them. Let's look at the code from the article:
counts = {}
# create dictionary of ids to users and chats
users = {}
chats = {}
for u in dialogs.users:
users[u.id] = u
for c in dialogs.chats:
chats[c.id] = c
for d in dialogs.dialogs:
peer = d.peer
if isinstance(peer, PeerChannel):
id = peer.channel_id
channel = chats[id]
access_hash = channel.access_hash
name = channel.title
input_peer = InputPeerChannel(id, access_hash)
elif isinstance(peer, PeerChat):
id = peer.chat_id
group = chats[id]
name = group.title
input_peer = InputPeerChat(id)
elif isinstance(peer, PeerUser):
id = peer.user_id
user = users[id]
access_hash = user.access_hash
name = user.first_name
input_peer = InputPeerUser(id, access_hash)
else:
continue
get_history = GetHistoryRequest(
peer=input_peer,
offset_id=0,
offset_date=None,
add_offset=0,
limit=1,
max_id=0,
min_id=0,
)
history = client(get_history)
if isinstance(history, Messages):
count = len(history.messages)
else:
count = history.count
counts[name] = count
print(counts)
Let's add sorting:
sorted_counts = sorted(counts.items(), key=lambda x: x[1], reverse=True)
for name, count in sorted_counts:
print('{}: {}'.format(name, count))
We got messages counter result:
Group chat 1: 10000
Group chat 2: 3003
Channel 1: 2000
Chat 1: 1500
Chat 2: 300
P.S. Here is a simple (user's) way to get some chats from official wiki, but from my perspective it's very limited and unsuitable for programmatic purposes:
https://telegram.wiki/general/exporting-chats
How to receive OUT report data from HOST PC in STM32f407 discovery board running as HID(USB) in device mode?
Is it possible?
I am thinking to send data from host using hidapi.
There is an official USB library. It is not easy, but you may try to run the examples and adapt them to your needs.
http://www.st.com/en/embedded-software/stsw-stm32046.html
Be careful with clock settings. I experienced problems with that. Here are the values I setup in system_stm32f4.c:
HSE = 8000000
PLL_M = 8
PLL_Q = 7
PLL_N = 336
PLL_P = 4
HSE is the crystal on the board. It replaces the embedded clock on the MCU. The other settings are slightly different from the values in the original configuration file. Here are the calculations of the different clocks:
PLL_VCO = (HSE_VALUE or HSI_VALUE / PLL_M) * PLL_N
= 8000000 / 8 * 336 = 336000000
USB OTG FS, SDIO and RNG Clock = PLL_VCO / PLLQ
= 336000000 / 7
= 48000000
SYSCLK = PLL_VCO / PLL_P
= 336000000 / 4
= 84000000
HCLK = SYSCLK / 1
= 84000000
PCLK2 = HCLK / 1
= 84000000
PCLK1 = HCLK / 2
= 84000000 / 2
= 42000000
The “USB OTG FS” clock HAS to be >= 48MHz if you use USB FS. Otherwise the device won't be recognized.
I am trying to use the following piece of code to connect to a time server and attain the time but have had no luck:
Dim ntpServer As String = "time.windows.com"
Dim ntpData(47) As Byte
Dim addresses = Dns.GetHostEntry(ntpServer).AddressList
Dim EndP As IPEndPoint = New IPEndPoint(addresses(0), 123)
Dim soc As Socket = New Socket(AddressFamily.InterNetwork, _
SocketType.Dgram, ProtocolType.Udp)
soc.Connect(EndP)
soc.Send(ntpData)
soc.Receive(ntpData)
soc.Close()
Tracing through the program I can't get past the following line of code soc.Receive(ntpData). What am I doing wrong?
Thanks
you need to provide some basic information to the server:
ntpData(0) = 27
ntpData(0) contains a section called firstByteBits.
This section needs to be set before sending the data to query for a reply.
First byte is
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|LI | VN |Mode |
LI = leap indicator (0 in sent data)
VN = version number (3, bits 3 and 4 set)
Mode = Mode (client mode = 3, bits 6 and 7 set)
00011011 = 27 = 0x1B
And possibly a better NTP server. The time.windows.com:123 server pool is known to
be slow, sometimes not responding for a while, and of low accuracy. Better: pool.ntp.org:123 (but please read what's written on poo.ntp.org about regular use).
e.g. RFC 5905 for more details.