I have been reading a lot about WCF lately and whenever the subject of implementing a subscriber-broadcast mechanism comes up (as in an instant messaging system), the solution invariably is to use a static dictionary to hold your subscriber channels.
An example can be found in the answer to the following question, but it is a common practice.
Making a list of subscribers available across calls to a service
This seems like a very good solution for "traditional" web programming, but how is this handled in the cloud? Specifically, how do we get around the fact that every computer in the grid has different "static" variables?
I know very little about the different Cloud platforms. Are there different solutions for Azure, Amazon Web Services and VMWare?
For broadcasting/push-type notifications, please look at SignalR (http://signalr.net/). Microsoft is making that part of the ASP.NET platform:
http://channel9.msdn.com/Events/Build/2012/3-034
It has some real nice functionality like gracefully, falling back on multiple mechanisms if advanced things like WebSockets are not supported by the server/client. While it is do-able, you would have to code all of that in WCF.
There are pretty big differences between the cloud vendor platforms. I could post you multiple links but the cloud vendors you mention are changing VERY rapidly with what they offer. Your commitment to a particular cloud vendor is for the long term...don't think of it as well Vendor A has something Vendor B doesn't. There are differences BTW like that...Amazon for example, has specialized VMs: high I/O, high memory, high CPU. While, Azure for example has a much better designed VM layer.
I think of it this way (mu opinion)...Microsoft is a company that owns: .NET, ASP.NET, server platforms: SQL Server, Windows Server, SharePoint, Office Services etc. They are very well positioned against someone like Amazon or VMWare which do not have rich product portfolios like this. Plus Microsoft can price those servers into the cloud, Amazon/RackSpace/VMWare have to pay Microsoft a premium for it. You seem like you are talking about WCF/.NET, which would favor the Microsoft Azure platform.
On Azure you can run Linux VMs; code in python, Java etc. but it favors the Microsoft stack. Conversely, for AWS you can run .NET/Microsoft etc, but it favors the Linux/open source stack. Think of it in long term...because in 2 years both major cloud vendors will be making commitments in those areas. For example, RackSpace is going all-in with their OpenStack platform...they have no choice.
The Windows Azure Service Bus has a a couple of options that you can use for broadcasting WCF events.
The Realy service has a netEventRelayBinding, which will allow subscribing service instances to reeive a one-way service call when a client calls an endpoint. If the clients are discinnected, they will not receive any messages.
http://msdn.microsoft.com/en-us/wazplatformtrainingcourse_eventingonservicebusvs2010_topic2.aspx
Brokered Messaging has tipics and subscriptions where a message can be broeadasct to up to 2,000 subscribers. The messages are stored durably, so if a client is disconnedted they wil receive all the messages when they reconnect.
http://www.cloudcasts.net/devguide/Default.aspx?id=12044
Regards,
Alan
It might be worth you looking into something like RabbitMQ on AppHarbor. It's something I keep meaning to look at but can't find the time. I only mention it because nobody else has ;)
Related
I'd like to create a web service that an application server can contact to add itself to a list of servers implementing the application. Clients could then contact the service to get a list of servers. Something similar to how minecraft's heartbeats work for adding your server to the main server list.
I could implement it myself pretty easily, but I'm hoping someone has already created something like this.
Advanced features would be useful. Things like:
Allowing a client to perform queries on application-specific properties like the number of users currently connected to the server
Distributing the server list across more than one machine
Timing out a server's entry in the list if it hasn't sent a heartbeat within some amount of time
Does anyone know of a service like this? I know there are open protocols and servers for doing local-LAN service discovery, but this would be a WAN service.
The protocols I could find that had any relevance to your intended application are these:
XRDS (eXtensible Resource Descriptor Sequence).
XMPP Service Discovery protocol.
The XRDS documentation is obtuse, but you may be able to push service descriptions in XML format. The service type specification might be generic, but I get a headache from trying to decipher committee-speak.
The XMPP Service Discovery protocol (part of the protocol Formerly Known As Jabber) also looked promising, but it seems that even though you could push your service description, they expect it to be one of the services mentioned on this list. Extending it would make it nonstandard.
Finally, I found something called seap (SErvice Announcement Protocol). It's old, it's rickety, the source may be propriety, it's written in C and Perl, it's a kludge, but it seems to do what you want, kind-of.
It seems like pushing a service announcement pulse is such an application-specific and trivial problem, that almost nobody has considered solving the general case.
My advice? Read the protocols and sources mentioned above for inspiration (I'd start with seap), and then write, implement, and publish a generic (probably xml-based) protocol yourself. All the existing ones seem to be either application-specific, incomprehensible, or a kludge.
Basically, you can write it yourself though I am not aware if anyone has one for public (I wrote one over 10 yrs ago, but for a company).
database (TableCols: auto-counter, svr_name, svr_ip, check_in_time, any-other-data)
code to receive heartbeat (http://<you-app.com>?svr_name=XYZ&svr_ip=P.Q.R.S)
code to list out servers within certain check_in_time
code to do some housecleaning once a while (eg: purge old records)
To send a heartbeat out, you only need to send a http:// call, on Linux use wget* with crontab, on windows use wget.exe with task scheduler.
It is application specific, so even if you wrote one yourself, others can't use it without modifying the source code.
This is a design question, I guess there going to be several suggestions (I hope so!)
We are a small travelling agent office, we are developing a software for our need. We have 30 clients on the local LAN, sometime later some other 5-10 from outside the office. (This will be agents on route).
Would you create a WCF service for this scenario or just leave the DL in the client (for direct access to the SQL)??
Our DataAccess Technology is EntityFramework 4.0 (4.1), ALL clients are Windows 7 based.
About the travelling agents, wouldn't it be much more simple to just use a VPN for the agents connection??
We have plans to supply our guides (outside the office ) with 3G tablets in about 1 year from now.
Developing clients with a DL is of course much more straight forward solution, but our plans for the near future will require some kind of service. The tablets application will have a much more simple interface and functionality. So maybe developing a WCF service dedicated for the tablets will be better?
Our first consideration is time, we are in great need for a working software.
Well, I hope I managed to share with you our thoughts clearly enough.
Thank you for reading.
I think one technology that would be a) very well-suited to your needs, and b) doesn't impose a huge overhead on your development effort would be WCF Data Services.
This is a REST-based WCF service based on a model - like your Entity Framework model.
Check out:
the WCF Data Services Developer Center
MSDN docs on WCF Data Services
I would see clear benefits in architecting a new solution from the get-go to use a service-based approach - even if your local LAN client might end up not using that service layer. Thinking in services, thinking also about the limitations of sending data across a potentially slower link than your LAN usually helps avoid silly mistakes like loading 200'000 customers into a dropdown box and stuff like that...
Having a service layer also makes your client app much less dependent on the actual database technology - that client might be a Winforms, WPF or Silverlight app, a web site, a tablet - you name it. If they all share the service layer, development of the client app will become less of a burden - the majority of your work will be in your data model and your service layer (which is shared across all clients).
I am a newbie to real-time application development and am trying to wrap my head around the myriad options out there. I have read as many blog posts, notes and essays out there that people have been kind enough to share. Yet, a simple problem seems unanswered in my tiny brain. I thought a number of other people might have the same issues, so I might as well sign up and post here on SO. Here goes:
I am building a tiny real-time app which is asynchronous chat + another fun feature. I boiled my choices down to the following two options:
LAMP + RabbitMQ
Node.JS + Redis + Pub-Sub
I believe that I get the basics to start learning and building this out. However, my (seriously n00b) questions are:
How do I communicate with the end-user -> Client to/from Server in both of those? Would that be simple Javascript long/infinite polling?
Of the two, which might more efficient to build out and manage from a single Slice (assuming 100 - 1,000 users)?
Should I just build everything out with jQuery in the 'old school' paradigm and then identify which stack might make more sense? Just so that I can get the product fleshed out as a prototype and then 'optimize' it. Or is writing in one over the other more than mere optimization? ( I feel so, but I am not 100% on this personally )
I hope this isn't a crazy question and won't get flamed right away. Would love some constructive feedback, love this community!
Thank you.
Architecturally, both of your choices are the same as storing data in an Oracle database server for another application to retrieve.
Both the RabbitMQ and the Redis solution require your apps to connect to an intermediary server that handles the data communications. Redis is most like Oracle, because it can be used simply as a persistent database with a network API. But RabbitMQ is a little different because the MQ Broker is not really responsible for persisting data. If you configure it right and use the right options when publishing a message, then RabbitMQ will actually persist the data for you but you can't get the data out except as part of the normal message queueing process. In other words, RabbitMQ is for communicating messages and only offers persistence as a way of recovering from network problems or system crashes.
I would suggest using RabbitMQ and whatever programming languages you are already familiar with. Since the M in LAMP is usually interpreted as MySQL, this means that you would either not use MySQL at all, or only use it for long term storage of data, not for the realtime communications.
The RabbitMQ site has a huge amount of documentation about building apps with AMQP. I suggest that after you install RabbitMQ, you read through the docs for rabbitmqctl and then create a vhost to experiment in. That way it is easy to clean up your experiments without resetting everything. I also suggest using only topic exchanges because you can emulate the behavior of direct and fanout exchanges by using wildcards in the routing_key.
Remember, you only publish messages to exchanges, and you only receive messages from queues. The exchange is responsible for pattern matching the message's routing_key to the queue's binding_key to determine which queues should receive a copy of the message. It is worthwhile learning the whole AMQP model even if you only plan to send messages to one queue with the same name as the routing_key.
If you are building your client in the browser, and you want to build a prototype, then you should consider just using XHR today, and then move to something like Kamaloka-js which is a pure Javascript implementation of AMQP (the AMQ Protocol) which is the standard protocol used to communicate to a RabbitMQ message broker. In other words, build it with what you know today, and then speed it up later which something (AMQP) that has a long term future in your toolbox.
Should I just build everything out with jQuery in the 'old school' paradigm and then identify which stack might make more sense? Just so that I can get the product fleshed out as a prototype and then 'optimize' it. Or is writing in one over the other more than mere optimization? ( I feel so, but I am not 100% on this personally )
This is usually called RAD (rapid application design/development) and it is what I would recommend right now. This lets you build the proof of concept that you can use to work off of later to get what you want to happen.
As for how to talk to the clients from the server, and vice versa, have you read at all on websockets?
Given the choice between LAMP or event based programming, for what you're suggesting, I would tell you to go with the event based programming, so nodejs. But that's just one man's opinion.
Well,
LAMP - Apache create new process for every request. RabbitMQ can be useful with many features.
Node.js - Uses single process to handle all request asynchronously with help of event looping. So, no extra overhead process creation like apache.
For asynchronous chat application,
socket.io + Node.js + redis pub-sup is best stack.
I have already implemented real-time notification using above stack.
All
I am looking for a bit of inspiration here, a client has requested me to build a simple remote process monitoring application with capability for smtp notification, when monitored processes go down or come back on-line. Can anyone point me in the direction of some sample code to get me started. I have briefly looked at .net remoting as a potential development path, from my understanding of how this works, the remoting architecture would require both a client and remote server component, (though I am not a 100% sure on this one), however if this is the case, then it will not fit the bill, as one of the fundamental requirements is that no additional software can be deployed on the servers which are to be monitored.
Alternatively, if you know of a finished 'lightweight' product out there, which would deliver this sort of functionality then this would probably work as well.
Kind Regards
Paul J.
Net-SNMP actually has the ability to monitor processes and send out notifications now. If you look in the snmpd.conf manual page (http://www.net-snmp.org/docs/man/snmpd.conf.html) under "process monitoring" (http://www.net-snmp.org/docs/man/snmpd.conf.html#lbAR) and "disman event-mib" support (http://www.net-snmp.org/docs/man/snmpd.conf.html#lbAX) to get the two features you need. Also the notification sending support (http://www.net-snmp.org/docs/man/snmpd.conf.html#lbAW).
I don't remember the windows support for it and how well it works though (it looks like you're aiming for windows).
Currently im developing a server for satelite monitoring of objects.
In its current state it is very efficient and stable in hi load scenarios.
Server must be able to handle 50+mln messages per day, or more if load balancing is used.
It consists of gps data gateway(singleton), "databroker"(singleton), which is responsible for persisting data, providing it on request, and alerting subscribers about new data, login service(per call) and client service(per session), responsible for subscription and working with web interface and rich client.
At this state i can forsee those problems:
Growing complexity if new services are added.
Tight coupling.
Hard to configure.
Lots of low level code for handling subscriptions etc in future bl services(reporting for example).
To solve those problems i want to use something like ServiceBus.
MS solutions are to expensive for our customers, NServiceBus is frightening me a bit, due to its open source origins(and Class1 in source code=D) and CIO asked to avoid using it.
So i decided to write my own simple bus, and encountered problems with adressing and subscribing diffirent types of services(singleton, per session, per call) and also there is a requirement for load balancing.
I found quite elegant solution for this: use "adapters" for bus - wcf services for incapsulating some specific issues of working with services - like loadbalancing. So bus will only see adapters and route messages between them and they will forward messages further. But im concerned about perfomance and whole idea..
Will be very grateful to hear thoughts about all of this stuff=)
PS Bus and adapters use MSMQ for communication between them, but other services can use http,tcp bindings.
PS2 Sorry for my english, its not my native language=)
I'm probably just bringing the dead back but if you'd still like to implement your own bus these links may come in handy (on design level):
http://msdn.microsoft.com/en-us/magazine/cc500646.aspx
http://msdn.microsoft.com/en-us/magazine/cc546553.aspx
I found answers for some my questions about how dynamically register subservices and route messages from client to them by central router service
You could try using "sql service broker" as a service bus
http://javiercrespoalvez.com/2009/03/using-sql-service-broker-in-net.html
The blog post also has links to other service buses you could try.
Thanks for all replyes=) I was able to convince bosses to use NServiceBus. (After creating a working bus prototype on weekend =). Now it fits quite well into the system=)