First of all I'm developing my own C# library for controlling Philips Hue, which means I'm not using the official SDK. (I'm guessing that the SDK will make sure you won't have any problems)
I'm a little confused about the limitation in the Core concepts page in the API, which states:
We can’t send commands to the lights too fast. If you stick to around 10 commands per second to the /lights resource as maximum you should be fine. For /groups commands you should keep to a maximum of 1 per second.
I intend to respect this limitation, but does the limitation still apply when you are performing GET requests on the /lights resource, or is it only for sending actual commands with PUT requests to /lights/<id>/state that change the state of the light? Same question goes for the /groups resource.
Also is it even possible to damage anything by sending too many requests, or will it just take longer to get all responses?
Edit:
My overall question is: How should I understand the API limitation?
A more specific sub-question is: Should I wait 100 ms before sending another /lights command, relative to when I received a response, or relative to when I sent the previous command?
Another sub-question is: Should I consider this limitation only when using PUT requests on e.g. /lights/<id>/state, or on all request types GET/PUT/POST/DELETE
I don't know if anything was changed in firmware updates, but I have discovered that the bridge might not be so simple as you would think, and that the API description isn't very clear.
I've done a little testing while running firmware 01009914.
The bridge seems to have some kind of queue of incoming commands. I sent {"bri":254} to a group 9 times and 1 final command of {"bri":1}. From the first command to when the light is actually dimmed, takes roughly 3-4 seconds. Each time I sent a command the bridge replied almost instantly with success token.
I did the same small tests sending other commands, 10 of each JSON object:
{"bri":254} 3-4 seconds
{"on":true, "bri":254} 6-7 seconds
{"on":true, "bri":254, "alert":"none", "effect":"none"} 12-13 seconds
This actually shows that each change of attributes takes roughly 0.3 seconds for the bridge to handle.
I will claim that for each attribute we change, the bridge takes about 300 ms to finish, and the limitation of commands should be understood as: As long as you stick with changing one attribute of a group each second, you should be fine.
Note: I only tried with one group consisting of three lights, and I don't know if the bridge actually does have a queue of incoming commands, and in case it does have a queue, I don't know what the limit of items in it is.
Edit:
Now we have some official clarification of the Hue System Performance.
I'm fairly certain that the 10 commands per second is a guideline to prevent failure of the Bridge, and is a technical limitation of the hardware. Any more than that and you're apt to overload the bridge. I believe this applies to commands as well as requests.
Both approaches are reasonable. For laziness' sake, you could wait for 100ms to send a response, but I would only rely on this method if you don't plan on any other interactions with the Bridge.
I consider this limitation on all request types.
You won't damage anything if you send commands too fast. However, if you send commands too fast the bridge might become unresponsive and/or some messages can be ignored.
When it comes to the bridge, the way I think of it is that the bridge is more or less single threaded, so it works best if you make sure you don't send the next command before the previous one has returned.
In practice we've found that this works much better than waiting a fixed time between each request. In fact, you can pretty much send commands as fast as you want as long as you wait for the previous one to finish.
When you send a command to the bridge, the bridge has to then send it to the lamps through Zigbee. Since it's a mesh network in some cases the message has to make a couple of hops from lamp to lamp before it reaches the target. Depending on how many lamps you have and how many hops the signal needs to take, this can take a while. Also, it's possible that some messages randomly take much longer than others.
In general the system is not designed to handle very fast changes, but if you keep the above in mind you can make many cool effects :)
Related
I read there are web servers their behaviors are called blocking whereas Node.js's is said non-blocking.
Would a blocking web server get hung up to the sense it needs restarting, if many http clients send requests at most in parallel?
As a complement, I don't say that it needs restarting while it potentially works fine again after the flood of parallel requests have stopped.
And I currently don't understand how request buffers and overflows work for web servers.
Although technically it could be possible to make a single-thread, single-process blocking server that can only handle 1 request at a time, it doesn't really practically make sense. Concurrency is kind of important.
The three main paradigms for parallelism (that I know of) are:
Multi-process/forking
Threading
Using an event loop/reactor pattern
Node falls in the third category, and also a bit in the second category depending on how you look at it.
Most languages can look at a socket and read from it, and immediately move on if there was nothing to read. Therefore most languages can have this non-blocking behavior.
Good morning!
I have this old Flash app (no worries the question is not about this!) which receives data from .NET server.
The data is a two thousand rows table and basically what I do is: query the DB with vb.net, create a long querystring with the data and send it back to Flash with a response.write method via GET. Then the Flash client parses it accordingly. The app is a parking lot GPS map that our employees uses to locate the vehicles. Believe it or not it has worked fine for the last 12 years and counting!
Long story short now my boss asked me to start over and remake entirely the app in HTML5. One major change is that for the sake of "standartization" the data will be converted from regular table columns to XML format so the chunk of data will grow in size.
Also I confess that I never feel completely happy on moving data back and forth via GET. I can't remember exactly WHY I did it this dirty. Probably by the time we were in a rush to get the app running so it just worked and among a lot of other things to do it was put in the backburner and the rest is history.
Anyway, since we are restarting it fresh I'd like to do it the right way this time. So questions are:
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
Stream the data would be an option or this is a silly idea?
I appreciate suggestions and examples!
Thanks!
First off, I would suggest using JSON over XML. There are two libraries you can use to serialize/deserialize JSON data: either Newtonsoft or System.Text.Json.
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
You should definitely be doing this via a POST request.
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
This really depends. If I were writing this I would add support for server-side pagination so that you know how many total records would be returned, but you're only returning however many records are currently visible. This would dramatically improve speed.
Stream the data would be an option or this is a silly idea?
Just return a JSON response.
What would you recommend for sending data from .NET server to AJAX client? The POST method is the obvious alternative or there is a newest and best way of doing it?
There is no real difference between a GET and a POST, certainly not one that matters to your context anyway; GET would be fine.
A GET might look like this:
GET /api/parkinglot/1234 HTTP/1.1
Host: somehost.com
A POST might look like this:
GET /api/parkinglot HTTP/1.1
Host: somehost.com
{ "id":1234 }
It's a text file, in essence, sent to the server. The server responds. It's not something that is "the way we do things now" or "more modern", POST doesn't "perform better".. It uses trivially more bytes, and is interpreted slightly differently by the server.. That's about it. For what you're describing, GET would be every bit as valid
Should I send the whole XML as a big unique chunk of data and parse it entirely in client or would be better to send it in array format (each item node as an array entry) and parse the array entries? My question here is what would be less CPU intensive for client, considering that machines are tablets and not PCs.
It doesn't really matter. An array isn't necessarily magically more or less of anything than XML; it's all just text, interpreted by the client. You could write a really wasteful array based solution or a lean XML one. What you ought to be throwing away is the idea of sending massive blocks of data to the client. Clients are limited in resource; don't send 2000 anything; what possible use could the user of the device have for 2000 items of data? You can't show it on screen and meaningfully interpret it; if it's a tabular block of data they'll end up panning around it, scrolling, zooming, searching.. Think about redesigning the app so that it sends the data they need when they need it. You might consider that sending 2000 points of data to be rendered as 1000 pins on a map, lat and long, might be a great idea, the client might have a really good rendering engine that can cope with it and make it quick and a pleasure to use.. but really? It sounds like the server needs to do a lot more of the work here
Stream the data would be an option or this is a silly idea?
This is all streaming. Every download or upload is a stream of data. Data gets from A to B in a serial flow so that it pops out the other end the same order it went in. You need to mentally move away from the concept of streaming vs downloading vs sending vs whatever else you think of in terms of getting data around the place. These are not distinct things; start focusing on being really efficient with the data you request, the time it takes to process and emit from the server, and the processing that happens on the client. Decide where it's best to do various calculations; there's no point the client searching for all users called smith, the server sending a million people to the client and the client parsing and searching the data. The server should do most of that. If you want to draw a triangle on screen, you can send 3 points and have the client render it instead of having the server render a 2 million pixel image, sending it and having the client draw the image. In one of these examples the server does a lot, in the other the client does a lot. In both the problem is that there is an excess of data flowing. Focus on the strengths of each resource
I appreciate suggestions and examples!
It isn't really what stackoverflow is for; we don't design your programs for you or write them - you have to do that and we tell you how to fix issues you hit along the way. Questions that ask "what is the best" are typically off topic because they attract opinionated answers.
In writing this answer I haven't really answered any of the questions you've asked in the way you want, because it simply isn't permitted. Instead I've tried to keep to factual observations and points you should consider when forming your own solution. When you hit problems with that solution, we can help but "design and implement my solution for me" is not a problem
I was able to make a working HID USB stack on my "StartUSB for PIC" board for the 18F2550 microcontroller. I based it on one of the MLA libraries, which was made for the 18F45K50 (MLA 2018_11_26, hid_custom, picdem_fs_usb_k50.x), but I converted it to work with the 18F2550 (there might have been easier ways, but only learned to work with PIC about 1 month ago). On the host side, I'm using LibUsbDotNet (also here, there might be easier ways - the documentation on this library really sucks) on a Windows 10 machine.
I'm using the HID class, full speed, and all seems to work. Although, I get some random errors on the host PC (see below), but doing one close/re-open cycle on the host side when getting the error is kind of solving it. Dirty, but it works. So I kind of ignore this now.
Win32Error:Win32Error:GetOverlappedResult Ep 0x01
995:The I/O operation has been aborted because of either a thread exit or an application request.
I'm not an expert on USB (yet). But all examples I'm seeing are based on 1) you send first a command to the device and 2) then you retrieve the answer from the device. I did some performance tests, and see that this indeed shows that I can do about 500 cycles/second. I think that is correct, because each cycle, sending command and retrieving answer, each takes 1 msec.
But do I really need to send a command? Can't I just keep reading endlessly, and when the device has somthing to say, it does send the data in an IN transaction, and when not it ignores which creates a timeout on the host side. That would mean that I can poll at 1000 cycles/second? Unfortunately, I have tried it by changing my implementation on the PIC, but I get very weird results. I think I have issues with suspend mode. That brings me to another question - how can I make the device get out of suspend mode (means that not the host, but the device should be triggering this event). I have searched the MLA library for command such as "wakeup", "resume", ... but couldn't find anything.
So, to summarize, 2 questions:
Conceptual: Can I send data from device to host without being requested for it by a command from the host?
For PIC experts: How can I have a device trigger for a wakeup from suspend mode?
And indeed, the answer is Yes on the first question.
In the meantime, I found another link on the web that contains a Visual Studio C# implementation of a USB library including all the source files.
If you're interested, this is the link
This C# host implementation works as a charm. Without sending a command to the device, I get notified immediately if a button is pressed. Great!
It also proofs that my earlier device implementation based on the original MicroChip MLA, is 100% correct. I stress tested the implementation by sending a "toggle LED command" as fast as I could, and I reach 1000 commands/second. Again great!
I think that LibUsbDotNet isn't that perfect after all. As I wrote above, I get rather unstable communication (Win32Error). But with this implementation, I don't get a single error, even after running for half an hour # 1000 commands/second.
So for me, case closed.
I need to get an understanding of ISO-8583 message platform,lets say i want to perform a authorization of a card transaction,so in real time at a particular instance lets say i got 100000 requests from network(VISA/MASTERCARD) all for authorization,how do i define priority of there request and the response,can the connection pool handle it(in my case its HIKARI),how is it done banks/financial institutions for authorizing a request.Please provide me some insights on how to manage all these requests.Should i go for a MQ?
Tech used are:-spring boot,hibernate,spring-tcp-starter
Your question doesn't seem to be very well researched as there are a ton of switch platforms out there that due this today and many of their technology guides can be found on the web including for major vendors like ACI, FIS, AJB,.. etc if you look yard enough.
I have worked with several iso-interface specifications, commercial switches, and home grown platforms and it is actually pretty consistent in how they do the core realtime processing.
This information on prioritization is generally in each ISO-8583 message processing specification and is made explicitly clear in almost every specification I've ever read written by someone who is familar with ISO-8533 and not just making up their own variant or copying someone elses.
That said.. in general at a high level authorizations / financials (0100, 0200) requests always have high priority than force posts (0x20) messages.
Administrative messages in the 05xx and 06xx and 08xx sometimes also get bumped up above other advices.. but these are still advices and almost always auths/financials are always processed first as they A) Impact the customer B) have much tighter timers than any advice by usually more than double or more.
Most switches I have seen do it entirely in memory without going to MQ and or some other disk for core authorization process to manage these.. but not to say there is not some sort of home grown middle ware sometimes involved.. but non-realtime processes regularly use a MQ process to queue or disk queuing these up into processes not in-line of the approval for this Store-and-forward (SAF) processing.. but many of these still use memory only processing for the front of their queue.
It is important to also differentiate between 100000 requests and 100000 transactions.. the various exchanges both internal and external make a big difference in the number of actual requests/responses in flight at even given time.. a basic transaction can be accomplished in like two messages.. but some of the more complex ones can easily exceed 20 messages just for a pre-authorization or a completion component.
If you are dealing with largely batch transaction bursts.. I can see the challenge of queuing but almost every application I have seen has a max in flight for advices and requests separate of each other.. and sometimes even with different timers.. and the apps pumping the transactions almost always wait for the response back before sending more.. and this tends to work fine for just about everyone.. including big posting batches from retailers and card networks. So if your app doesn't have them.. you probably need to add them.
In fact your 100000 requests should be sorted by (Terminal ID and/or Merchant ID) + (timestamp/local timestamp) + (STAN and/or RRN).
Duplicated transaction requests expected to be rejected.
If you simulating multiple requests from single terminal (or host) with same test card details the increasing of STAN/RRN would be a case.
Please refer to previous answers about STAN and RRN ISO 8583 fields.
In ISO message, what's the use of stan and rrn ?
Long polling has solved 99% of my problems. There is now just one other problem. Imagine a penny auction site, where people bid. On the frontpage, there are several Auctions.
If the user opens three of these auctions, and because javascript is not multithreaded, how would you get the other pages to ever load? Won't they always get bogged down and not load because they are waiting for long polling to end? In practice, I've experienced this and I can't think of a way around it. Any ideas?
There are two ways that javascript gets around some of this.
While javascript is single threaded conceptually, it does its io in separate threads using completion handlers. This means other pieces of javascript can be running while you are waiting for your network request to complete.
Javascript for each page (or even each frame in each page) is isolated from Javascript on the other pages/frames. This means that each copy of javascript can be running in its own thread.
A bigger issue for you is likely to be that browsers often limit the number of concurrent connections to a given site, and it sounds like you want to make many concurrent connections to the same site. In this case you will get a lock up.
If you control both the sever and client, you will need to combined the multiple long-poll request from the client into a single long-poll request to the server.