I have an application that depends on the reliability of being able to contact the network through Alchemy’s RPC mainnet API.
During the merge, it is crucial that I am able to interact with the PoS chain ASAP. Should I rely on Alchemy’s API for this or do I need a different method or node or something?
Thanks.
Unless I'm misremembering, no. I seem to remember Infura saying that they will have ~12 minutes of downtime around the merge just in case anything goes wrong. I imagine that alchemy will do likewise.
I recommend you run your own geth node.
Related
For a new auction system I am looking for which technology is the best for me.
When there is a new bid, I want to notify the listening users on the auction page. This is something for a pubsub technique, i presume.
First I did take a look at RabbitMQ, and I think this is a good way to build this. But it means I have an extra single point of failure.
So now I am leaning towards Redis PubSub. I know it has disadvantages, because when an user is not listening it won't re-send messages. But that is not a problem. When a user sign in it has all the current bids, and then only want updates. I don't plan to create a chat with a history.
What can you advice? Are there anymore disadvantages to use Redis for this? How about the stability? When a bid occurs, and I want to send the newest price to all listening users, how certain am I everyone gets the message?
Does anyone have experience with this situation?
Thanks
Pro: redis is much simpler than RabbitMQ to set up.
Cons: there is no guarantee of delivery with Redis.
I assume, that by "page" you mean standard HTML page with PHP on backend. If yes, then your main problem is not "should I use Redis or RabbitMQ", because you cannot make direct connection between your user browser and Redis or RabbitMQ.
First you have two answer to yourself, how will you provide updates for the page:
by regular ajax requests asking "is there any new for me"
by using some implementation of websocket
and after selecting answer, you will see that pub/sub mechanism has any use at all in your situation.
I am looking for a pragmatic solution to do Integration testing of our Integration tier based on Mule.
This article here has some excellent pointers about it, but looks a tad outdated. I am reproducing an excellent idea from the article here
Keeping track of the delivery of messages to external systems. Interrogating all the systems that have been contacted with test messages after the suite has run to ensure they all received what was expected would be too tedious to realize. How to keep track of these test messages? One option could be to run Mule ESB with its logging level set to DEBUG and analyze the message paths by tracking them with their correlation IDs. This is very possible. I decided to follow a simpler and coarser approach, which would give me enough certitude about what happened to the different messages I have sent. For this, I decided to leverage component routing statistics to ensure that the expected number of messages where routed to the expected endpoints (including error messages to error processing components). Of course, if two messages get cross-sent to wrong destinations, the count will not notice that. But this error would be caught anyway because each destination will complain about the error, hence raising the count of error messages processed.
Using this technique when I test my integration tier I will not have to stand up all the external systems and can test the integration tier in isolation which would be great.
#David Dassot has provided a reference implementation as well, however I think it was based on Mule 2.X and hence I cannot find the classes in the Mule 3.X codebase.
Looking around I did find FlowConstructStatistics but this is flow specific statistics and I am looking for endpoint specific statistics.
I do agree that as a work around we could wrap all outbound endpoints within sub-flows and get this working, but I would like to avoid doing this ...
Any techniques that help query the endpoint for the number of calls made, payload passed through the endpoints would be great!
First take a look to JMX, perhaps what you need is available right there.
Otherwise, if looging is not enough, and upgrading to the enterprise version is not ok for you. Give it a try to the endpoint level notifications.
I'm trying to understand how to persist data in a Twisted application. Let's say I've decided to write a Twisted server that:
Accepts inbound SMTP requests
Sends the message to a 3rd party system for modification
Relays the modified message to its destination
A typical Twisted tutorial would have you build this app using Deferreds and callbacks, roughly:
A Factory handles inbound requests
Each time a full email is received a call is sent to the remote message processor, returning a deferred
Add an errback that substitutes the original message if anything goes wrong in the modify call.
Add a callback to send the message on to the recipient, which again returns a deferred.
A real server would add/include additional call/errbacks to retry or notify the sender or whatnot. Again for simplicity, assume we consider this an acceptable amount of effort and just log errors.
Of course, this persists NO data in the event of a crash/restart/something else. I get that a solution involves a 3rd party persistent datastore (RabbitMQ is often mentioned) and could probably come up with a dozen random ways to achieve the outcome.
However, I imagine there are a few approaches that work best in a Twisted app. What do they look like? How do they store (and restore in the event of a crash) the in-process messages?
If you found this question, you probably already know that Twisted is event-based. It sounds simple, but the "hardest" part of the answer is to get the persistence platform generating the events we need when we need them. Naturally, you can persist the data in a DB or a message queue, but some platforms don't naturally generate events. For example:
ZeroMQ has (or at least had) no callback for new data. It's also relatively poor at persistence.
In other cases, events are easy but reliability is a problem:
pgSQL can be configured to generate events using triggers, but they're one-time things so you can't resume incomplete events
The light at the end of the tunnel seems to be something like RabbitMQ.
RabbitMQ can persist the message in a database to survive a crash
We can use acknowledgements on both legs (incoming SMTP to RabbitMQ and RabbitMQ to outgoing SMTP) to ensure the application is reliable. Importantly, RabbitMQ supports acknowledgements.
Finally, several of the RabbitMQ clients provide full asynchronous support (see for example pika, txampq, and puka)
It's enough for our purposes that the RabbitMQ client provides us an event-based interface.
At a more theoretical level, however, this need not be the case. In fact, despite the "notification" issue above, ZeroMQ has an event-based client. Even if our software is elegantly event-based, we will run into systems that aren't. In these cases, we have no choice but to fall back on polling. In principle, if not in practice, we just query the message provider for messages. When we exhaust the current queue (and immediately if there are no messages), we use a callLater command to check again in the future. It may feel anti-pattern, but it's (to the best of my knowledge anyway) the right way to handle this particular case.
I am a newbie to real-time application development and am trying to wrap my head around the myriad options out there. I have read as many blog posts, notes and essays out there that people have been kind enough to share. Yet, a simple problem seems unanswered in my tiny brain. I thought a number of other people might have the same issues, so I might as well sign up and post here on SO. Here goes:
I am building a tiny real-time app which is asynchronous chat + another fun feature. I boiled my choices down to the following two options:
LAMP + RabbitMQ
Node.JS + Redis + Pub-Sub
I believe that I get the basics to start learning and building this out. However, my (seriously n00b) questions are:
How do I communicate with the end-user -> Client to/from Server in both of those? Would that be simple Javascript long/infinite polling?
Of the two, which might more efficient to build out and manage from a single Slice (assuming 100 - 1,000 users)?
Should I just build everything out with jQuery in the 'old school' paradigm and then identify which stack might make more sense? Just so that I can get the product fleshed out as a prototype and then 'optimize' it. Or is writing in one over the other more than mere optimization? ( I feel so, but I am not 100% on this personally )
I hope this isn't a crazy question and won't get flamed right away. Would love some constructive feedback, love this community!
Thank you.
Architecturally, both of your choices are the same as storing data in an Oracle database server for another application to retrieve.
Both the RabbitMQ and the Redis solution require your apps to connect to an intermediary server that handles the data communications. Redis is most like Oracle, because it can be used simply as a persistent database with a network API. But RabbitMQ is a little different because the MQ Broker is not really responsible for persisting data. If you configure it right and use the right options when publishing a message, then RabbitMQ will actually persist the data for you but you can't get the data out except as part of the normal message queueing process. In other words, RabbitMQ is for communicating messages and only offers persistence as a way of recovering from network problems or system crashes.
I would suggest using RabbitMQ and whatever programming languages you are already familiar with. Since the M in LAMP is usually interpreted as MySQL, this means that you would either not use MySQL at all, or only use it for long term storage of data, not for the realtime communications.
The RabbitMQ site has a huge amount of documentation about building apps with AMQP. I suggest that after you install RabbitMQ, you read through the docs for rabbitmqctl and then create a vhost to experiment in. That way it is easy to clean up your experiments without resetting everything. I also suggest using only topic exchanges because you can emulate the behavior of direct and fanout exchanges by using wildcards in the routing_key.
Remember, you only publish messages to exchanges, and you only receive messages from queues. The exchange is responsible for pattern matching the message's routing_key to the queue's binding_key to determine which queues should receive a copy of the message. It is worthwhile learning the whole AMQP model even if you only plan to send messages to one queue with the same name as the routing_key.
If you are building your client in the browser, and you want to build a prototype, then you should consider just using XHR today, and then move to something like Kamaloka-js which is a pure Javascript implementation of AMQP (the AMQ Protocol) which is the standard protocol used to communicate to a RabbitMQ message broker. In other words, build it with what you know today, and then speed it up later which something (AMQP) that has a long term future in your toolbox.
Should I just build everything out with jQuery in the 'old school' paradigm and then identify which stack might make more sense? Just so that I can get the product fleshed out as a prototype and then 'optimize' it. Or is writing in one over the other more than mere optimization? ( I feel so, but I am not 100% on this personally )
This is usually called RAD (rapid application design/development) and it is what I would recommend right now. This lets you build the proof of concept that you can use to work off of later to get what you want to happen.
As for how to talk to the clients from the server, and vice versa, have you read at all on websockets?
Given the choice between LAMP or event based programming, for what you're suggesting, I would tell you to go with the event based programming, so nodejs. But that's just one man's opinion.
Well,
LAMP - Apache create new process for every request. RabbitMQ can be useful with many features.
Node.js - Uses single process to handle all request asynchronously with help of event looping. So, no extra overhead process creation like apache.
For asynchronous chat application,
socket.io + Node.js + redis pub-sup is best stack.
I have already implemented real-time notification using above stack.
I designed one On line Trading Application, which uses blazeds & jetty,
in that i used AMF-LongPooling as channel, with following parameter,
Here is the problem is Each message is not reaching all the user,who are connected, messages are missing to few users (300 recieving out of 600)...
what we need to do to provided instant messages to all Online. ??
Please help me one?
Your question is too generic, it's not possible to give an answer because it depends on too many things: network, size of the messages, your system architecture etc. My suggestion is to invest heavily in reading BlazeDS developer guide and to turn the debug messages on (there is a lot of useful information displayed by BlazeDS). It would also help to study BlazeDS source code.
In case of AMF-longpolling the request is parked on the server and if too many requests are parked at a time, they will consume all available threads for the server. And the next client won't be able to connect.
In your case I am assuming the message size is not very big. And the solution can be one of the followings:
To increase the number of available threads. For that you can have multiple server instances and distribute your clients over them.
You can make use of LCDS.
You don't get that problem in LCDS as it makes use of NIO end points that don't block the thread. I have come to know that this thread restriction is not a problem with Servlet 3.0 and in that case you can support more clients with blazeds itself. You can check more about it HERE.