I wrote code for my micro:bit in makecode.microbit.org. Idea is to use radio communication and send each micro:bit message. So my question at first was how many messages can get just one micro:bit?
Internet say this:
Messages received are read from a queue of configurable size (the larger the queue the more RAM is used). If the queue is full, new messages are ignored. Reading a message removes it from the queue.
So my question is: how to find how much RAM memory is used on my micro:bit using makecode?
edit: https://forum.makecode.com/t/how-to-find-how-much-ram-memory-is-used-on-my-micro-bit-using-makecode/1303/2
How now find length of micro:bit queue?
Related
I was trying to perform LoadTest on the RabbitMQ messaging to see to what extent it can take messages into the queue and transfer it to the target machine over shovel.
Steps i followed:
producer has 20 threads. Each thread sends message to a dedicated queue(Say suppose ProducerQueue1 -- ProducerQueue20).The message is of size 51Mb each. The messages are sent in random interval using java.util.Random(50) seconds.
After each message sent at random seconds(A random second between 1- 50),
there is a sleep of 2 minutes.Therefore each of the producer threads sleep for
2 min after every send.
The messages are sent in a infinite while loop.
There are shovels from each dedicated queue to the consumer side dedicated queues(Say suppose ConsumerQueue1 --- ConsumerQueue20).
The link speed is 100mbps.
Issue observed:
Initially the messages are transferred with no issues, but after some time the NETWORK AT CONSUMER SIDE IS CHOKED.
The reason for choking is that after certain period of time, even if 4/5 out of 20 thread's random second coincides, then the consumer receives close to 250Mb message in one shot. Since the network speed is 100mbps as mentioned above, the network gets choked.
Due to this, the shovels will not be able to exchange heartbeats to stay in "running" state. The leads shovels to move from "running" to "terminated" state. The shovels try to establish a connection depending upon the "reconnect delay".
Due to break in shovels at producer side, the queues at the producer starts getting accumulated.
My Question:
The consumer's rabbitMq memory starts increasing as the queues start accumulating more messages. The memory is crossing the water mark. The purpose of water mark is not served. I have 16gb ram and i have set watermark to 40%(i.e 6.4gb ram). But still the memory shoots up to 10gb and doesnt recover and the producer system hangs.
Can any one please answer my question. and also tell me can there be any other reason for network choking which i mentioned above.
Thanks in advance.
I've made a test, based on the example solution of the activemq-cpp library.
In the test I send 50,000 messages to a queue, and after they're all sent I consume them, with INDIVIDUAL_ACKNOWLEDGE on the session and message->acknowledge() on every consumed message. The consumer is asynchronous.
Memory (private working set) of java.exe before sending messages: 209,320 KB. After sending all messages: 412,548 KB. After consuming all messages: 434,637 KB. Meaning, although queue size is 0, memory was not released.
What am I missing?
Thanks.
Besides the JVM processing mentioned above there are a number of other factors that are in play here. Depending on the state of the broker when you started the producer sending messages there could be a number of resources that where allocated on the broker to create the Queue and various other management objects which will then remain in memory to facilitate message routing etc. To truly analyze the memory usage and check for leaks you should use a tool like Yourkit etc.
I am creating a server on a ST Cortex M3 device. I am using the lwip API and FreeRTOS. All is working, but the response time is way off. I am currently using lwip 1.3.2 and FreeRTOS 7.3.
A single client connects to the server and must have some time-critical data sent frequently. These packets are on the order of 6 or so bytes. Other times, I am sending upwards of 20K.
The problem I am having is that these smaller packets seem to be taking forever to be sent. I assume this is because lwip is waiting for more data to be enqueued to make more efficient transmissions. I cannot wait around for 2 or 3 seconds for the data to be sent; the client is expecting the data nominally in a few micro-seconds or milli-seconds.
I have tried using lwip_send and lwip_write. (I understand that one is the same as the other with a flag passed at the end. Just had to try...) I have tried setting TCP_NODELAY on the socket to no avail. I tried to set SO_SNDLOWAT to '1', but this always returned -1, so I do not think it is supported.
I do not want to redo all of my code using TCP RAW. Is there a way to invoke the tcp_output() function outside of TCP RAW mode? Is there any way to speed things up or is this just how slow lwip TCP with small packets is?
Any and all suggestions are welcome. Thanks.
--EDIT--
I would also like to add that once I am ready to transmit, I make sure that my TX task in FreeRTOS is at the highest priority. There are no other tasks running up to the point at which I call lwip_send/write.
I'm fairly experienced with bare metal lwIP on xilinx and lwip does not wait to send things out. It will pump packets out as fast as your interrupts are acknowledged based on the ethernet hardware. I've been using UDP only. What is coming to mind though, is your problem might be on the receive end. If you are doing TCP, maybe those small packets are coming out late because you are having receive issues. What you need to do is find in the code the lowest level point at which ethernet is transmit, put a general purpose output toggle on that. Then also put a general purpose output toggle on when a ethernet packet is received. Look at the signals on a scope. If it confirms your hypothesis, then move the output toggles around to narrow down the issue. Wash, rinse and repeat until you are down to where the issue its. It's crude and time consuming, but oftentimes this brute force approach solves many "impossible" embedded software problems, due to pure determination. Good luck!
I am coding an application for linux in C, it will receive thousands of UDP messages which may be of variable size. Somehow, I have to get the message size, before reading it into the buffer with recvfrom() syscall. I can not allocate memory for the maximum possible message, because since I use MTU of 9,000 and get thousands of messages a lot of memory will be wasted.
I checked in google, it is possible to find out what is the size of the message with SO_NREAD option for getsockopt syscall, however this works only on BSD. I know the message is located somewhere in Linux kernel because my sockets are working in non-blocking mode and I am being notified by kernel events that the data is available, so, somewhere it has to store the message length, but how can I get it ?
Thanks in advance
You probably want to use ioctl with FIONREAD. It's available for both Linux and BSD.
FIONREAD int Get the number of bytes that are immediately available for reading.
if(ioctl(s, FIONREAD, &bytes) != -1)
printf("%d bytes available", bytes);
On Redhat Linux, I have a multicast listener listening to a very busy multicast data source. It runs perfectly by itself, no packet losses. However, once I start the second instance of the same application with the exactly same settings (same src/dst IP address, sock buffer size, user buffer size, etc.) I started to see very frequent packet losses from both instances. And they lost exact the same packets. If I stop the one of the instances, the remaining one returns to normal without any packet loss.
Initially, I though it is the CPU/kernel load issue, maybe it could not get the packets out of buffer quickly enough. So I did another test. I still keep one instance of the application running. But then started a totally different multicast listener on the same computer but use the second NIC card and listen to a different but even busier multicast source. Both applications run fine without any packet loss.
So it looks like one NIC card is not powerful enough to support two multicast applications, even though they listen to exact the same thing. The possible cause to the packet loss problem might be that, in this scenario, the NIC card driver needs to copy the incoming data to two sock buffers, and this extra copy task is too much for the ether card to handle so it drops packets. Any deeper analysis on this issue and any possible solutions?
Thanks
You are basically finding out that the kernel is inefficient at fan-out of multicast packets. Worst case scenario the code is for every incoming packet allocating two new buffers, the SKB object and packet payload, and copying the NIC buffer twice.
Pick the best case scenario, for every incoming packet a new SKB is allocated but the packet payload is shared between the two sockets with reference counting. Now imagine what happens when two applications, each on their own core and on separate sockets. Every reference to the packet payload is going to cause the memory bus to stall whilst both core caches have to flush and reload, and above that each application is having to kernel context switch back and forth to pass the socket payload. The result is terrible performance.
You aren't the first to encounter such a problem and many vendors have created solutions to it. The basic design is to limit the incoming data to one thread on one core on one socket, then have that thread distribute the data to all other interested threads, preferably using user space code building upon shared memory and lockless data structures.
Examples are TIBCO's Rendezvous and 29 West's Ultra Messaging showing a 660ns IPC bus:
http://www.globenewswire.com/newsroom/news.html?d=194703