This is in continuation to a question asked by me 7 months back to understand the reactive programming, because I haven't seen any articles which explain the concepts around this other than the use of operators in depth, I feel my knowledge is shallow around this, hence asking what my assumptions are, what I am unclear about, and what I know of the era used before the reactive programming.
I understand that Java before having reactive support used to work like below,
If we need to fetch some results from database,
Client made and http request
Server received it and assigned a thread from the pool.
Database driver interacted with the database, in blocking manner
Database server returned results to the thread.
The waiting thread returned the results to the client and is now freed.
Please see the below image, depicting the same, and correct me If I am wrong anywhere
Now What I know/learned/understand of reactive is,
For a system to be completely reactive, "Every component in the lifecycle of a process (for ex HTTP request to fetch some results) must be reactive."
So I assume from the above statement that below things must have been happening in reactive stack.
Is this a correct understanding?
If yes, then how R2DBC, is making it possible to develop applications with non reactive databases, Is it mimicking only?
Related
I am recently learning Angular 6 with #ngrx/store while one of the tutorial is to use #ngrx/store for state management, however I don't understand the benefit of using #ngrx/store behind the scene.
For example, for a simple login and signup action, previously by using the service(Let's call it AuthService) we might use it to call the backend api, store "userInfo" or "token" in the AuthService, redirect user to "HOME" page and we can inject AuthService in any component where we need to get the userInfo by using DI, which simply that one file AuthService handles everything.
Now if we are using #ngrx/store, we need to define the Action/State/Reducer/Effects/Selector which probably need to write in 4 or 5 files to handle above action or event, then sometimes still we need to call backend api using service, which seems much much more complicated and redundant...
In some other scenario, I even see some page uses #ngrx/store to store the object or list of object like grid data., is that for some kind of in-memory store usage?
So back to the question, why are we using #ngrx/store over service registration store here in Angular project? I know it's for "STATE MANAGEMENT" usage, but what exactly is the "STATE MANAGEMENT"? Is that something like transaction log and When do we need it? Why would we manage it on the front end? Please feel free to share your suggestion or experience in the #ngrx/store area!
I think you should read those two posts about Ngrx store:
Angular Service Layers: Redux, RxJs and Ngrx Store - When to Use a Store And Why?
Ngrx Store - An Architecture Guide
If the first one explains the main issues solved by Ngrx Store, it also quote this statement from the React How-To "that seems to apply equally to original Flux, Redux, Ngrx Store or any store solution in general":
You’ll know when you need Flux. If you aren’t sure if you need it, you don’t need it.
To me Ngrx store solves multiple issues. For example when you have to deal with observables and when responsability for some observable data is shared between different components. In this case store actions and reducer ensure that data modifications will always be performed "the right way".
It also provides a reliable solution for http requests caching. You will be able to store the requests and their responses, so that you could verify that the request you're making has not a stored response yet.
The second post is about what made such solutions appear in the React world with Facebook's unread message counter issue.
Concerning your solution of storing non-obvervable data in services. It works fine when you're dealing with constant data. But when several components will have to update this data you will probably encounter change detection issues and improper update issues, that you could solve with:
observer pattern with private Subject public Observable and next function
Ngrx Store
I'm almost only reading about the benefits of Ngrx and other Redux like store libraries, while the (in my opinion) costly tradeoffs seem to be brushed off with far too much ease. This is often the only reason that I see given: "The only reason not to use Ngrx is if your app is small and simple". This (I would say) is simply incomplete reasoning and not good enough.
Here are my complaints about Ngrx:
You have logic split out into several different files, which makes the code hard to read and understand. This goes against basic code cohesion and locality principles. Having to jump around to different places to read how an operation is performed is mentally taxing and can lead to cognitive overload and exhaustion.
With Ngrx you have to write a lot more code, which increases the chances of bugs. More code -> more places for bugs to appear.
An Ngrx store can become a dumping ground for all things, with no rhyme or reason. It can become a global hodge podge of stuff that no one can get a coherent overview of. It can grow and grow until no one understands it any more.
I've seen a lot of unnecessary deep object cloning in Ngrx apps, which has caused real performance issues. A particular app I was assigned to work on was taking 40 ms to persist data in the store because of deep cloning of a huge store object. This is over two lost render frames if you are trying to hit a smooth 60 fps. Every interaction felt janky because of it.
Most things that Ngrx does can be done much simpler using a basic service/facade pattern that expose observables from rxjs subjects.
Just put methods on services/facades that return observables - such a method replaces the reducer, store, and selector from Ngrx. And then put other methods on the service/facade to trigger data to be pushed on these observables - these methods replace your actions and effects from Ngrx. So instead of reducers+stores+selectors you have methods that return observables. Instead of actions+effects you have methods that produce data the the observables. Where the data comes from is up to you, you can fetch something or compute something, and then just call subject.next() with the data you want to push.
The rxjs knowledge you need in order to use ngrx will already cause you to be competent in using bare rxjs yourself anyways.
If you have several components that depend on some common data, then you still don't need ngrx, as the basic service/facade pattern explicitly handles this already.
If several services depend on common data between them, then you just make a common service between these services. You still don't need ngrx. It's services all the way down, just like it is components all the way down.
For me Ngrx doesn't look so good on the bottom line.
It is essentially a bloated and over engineered Enterprise™🏢👨💼🤮 grade Rxjs Subject, when you could have just used the good old and trusty Rxjs Subject. Listen to me kids, life is too short for unnecessary complexity. Stick to the bare necessities. The simple bare necessities. Forget about your worries and your strife.
I've been working with NgRx for over three years now. I used it on small projects, where it was handy but unnecessary and I used it in applications where it was perfect fit. And meanwhile I had a chance to work on the project which did not use it and I must say it would profit from it.
On the current project I was responsible for designing the architecture of new FE app. I was tasked to completely refactor the existing application which for the same requirements used non NgRx way and it was buggy, difficult to understand and maintain and there was no documentation. I decided to use NgRx there and I did it because of following reasons:
The application has more than one actor over the data. Server uses
the SSE to push state updates which are independent from user
actions.
At the application start we load most of available data which are
then partially updated with SSE.
Various UI elements are enabled/disabled depending on multiple
conditions which come from BE and from user decisions.
UI has multiple variations. Events from BE can change currently
visible UI elements (texts in dialogs) and even user actions might
change how UI looks and works (recurring dialog can be replaced by
snack if user clicked some button).
State of multiple UI elements must be preserved so when user leaves
the page and goes back the same content (or updated via SSE) is
visible.
As you can see the requirements does not meet the standard CRUD operations web page. Doing it the "Angular" way brought such complexity to the code that it became super hard to maintain and what's worst by the time I joined the team the last two original members were leaving without any documentation of that custom made, non NgRx solution.
Now after the year since refactoring the app to use NgRx I think I can sum up the pros and cons.
Pros:
The app is more organized. State representation is easy to read,
grouped by purpose or data origin and is simple to extend.
We got rid of many factories, facades and abstract classes which lost
their purpose. The code is lighter, and components are 'dumber', with
less hidden tricks coming from somewhere else.
Complicated state calculations are made simple using effects and
selectors and most components can be now fully functional just by
injecting the store and dispatching the actions or selecting the
needed slice of the state while handling multiple actions at once.
Because of updated app requirements we were forced to refactor the
store already and it was mostly Ctrl + C, Ctrl + V and some renaming.
Thanks to Redux Dev Tools it is easier to debug and optimize (yep
really)
This is most important - even thought our state itself is unique the
store management we are using is not. It has support, it has
documentation and it's not impossible to find solutions to some
difficult problems on the internet.
Small perk, NgRx is another technology you can put to your CV :)
Cons:
My colleagues were new to the NgRx and it took some time for them to
adapt and fully understand it.
On some occasions we introduced the issue where some actions were
dispatched multiple times and it was difficult to find the cause of
it and fix it
Our Effects are huge, that's true. They can get messy but that's what
we have pull requests for. And if this code wasn't there it would
still end up somewhere else :)
Biggest issue? Actions are differentiated by their string type. Copy
an action, forget to rename it and boom, something different is
happening than you expect, and you have no clue why.
As a conclusion I would say that in our case the NgRx was a great choice. It is demanding at first but later everything feels natural and logical. Also, when you check the requirements, you'll notice that this is a special case. I understand the voices against NgRx and in some cases I would support them but not on this project. Could we have done it using 'Angular' way? Of course, it was done this way before, but it was a mess. It was still full of boiler plate code, things happening in different places without obvious reasons and more.
Anyone who would have the chance to compare those two versions would say the NgRx version is better.
There is also a 3rd option, having data in service and using service directly in html, for instance *ngFor="let item of userService.users". So when you update userService.users in service after add or update action is automatically rendered in html, no need for any observables or events or store.
If the data in your app is used in multiple components, then some kind of service to share the data is required. There are many ways to do this.
A moderately complex app will eventually look like a front end back end structure, with the data handling done in services, exposing the data via observables to the components.
At one point you will need to write some kind of api to your data services, how to get data in and out, queries, etc. Lots of rules like immutability of the data, and well defined single paths to modify the data. Not unlike the server backend, but much quicker and responsive than the api calls.
Your api will end up looking like one of the many state management libraries that already exist. They exist to solve difficult problems. You may not need them if your app is simple.
NGRX sometimes has a lot of files and a lot of duplicate code. Currently working on a fix for this. To make generic type classes for certain NGRX state management situations that are very common inside an Angular project like pagers and object loading from back-ends
In the scala and scalajs library Diode, I have used but not entirely understood the PotAction class and only recently discovered the AsyncAction class, both of which seem to be favored in situations involving, well, asynchronous requests. While I understand that, I don't entirely understand the design decisions and the naming choices, which seem to suggest a more narrow use case.
Specifically, both AsyncAction and PotAction require an initialModel and a next, as though both are modeling an asynchronous request for some kind of refreshable, updateable content rather than a command in the sense of CQRS. I have a somewhat-related question open regarding synchronous actions on form inputs by the way.
I have a few specific use cases in mind. I'd like to know a sketch (not asking for implementation, just the concept) of how you use something like PotAction in conjunction with any of:
Username/password authentication in a conventional flow
OpenAuth-style authentication with a third-party involved and a redirect
Token or cookie authentication behind the scenes
Server-side validation of form inputs
Submission of a command for a remote shell
All of these seem to be a bit different in nature to what I've seen using PotAction but I really want to use it because it has already been helpful when I am, say, rendering something based on the current state of the Pot.
Historically speaking, PotAction came first and then at a later time AsyncAction was generalized out of it (to support PotMap and PotVector), which may explain their relationship a bit. Both provide abstraction and state handling for processing async actions that retrieve remote data. So they were created for a very specific (and common) use case.
I wouldn't, however, use them for authentication as that is typically something you do even before your application is loaded, or any data requested from the server.
Form validation is usually a synchronous thing, you don't do it in the background while user is doing something else, so again Async/PotAction are not a very good match nor provide much added value.
Finally for the remote command use case PotAction might be a good fit, assuming you want to show the results of the command to the user when they are ready. Perhaps PotStream would be even better, depending on whether the command is producing a steady stream of data or just a single message.
In most cases you should use the various Pot structures for what they were meant for, that is, fetching and updating remote data, and maybe apply some of the ideas or internal models (such as the retry mechanism) to other request types.
All the Pot stuff was separated from Diode core into its own module to emphasize that they are just convenient helpers for working with Diode. Developers should feel free to create their own helpers (and contribute back to Diode!) for new use cases.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I've already run several times into the issue of creating a desktop client app for working with some server, and every time I ended with ugly code, which becomes just impossible to support after couple of releases.
I have highlighted the following key points:
All operations must be asynchronous, without any dummy windows for relative fast operations (i.e. less than 30 seconds)
App has to periodically connect with the server and check, for example, user account
All heavy operations must be cancelable
But, most important, all of this must be "naturally" in code, without creating unnecessary difficulties (singletons, hacks, etc)... only really needed code with minimal overhead.
How would you design such kind of app? What pattern would you use? What open source project with good architecture you can recommend?
This seems a little too broad, but instead of flagging I'll try and give an answer as I find the question interesting. I invite you to add more details if they come to mind.
Even though your design concerns the design of the application, there are a number of languages, patterns and technologies that would suit your requirements.
Keeping it general,
If your want your operations to be asynchronous, you are going to
need multiple threads. Their implementation and use may vary
depending on the language that you are using, but the concept behind
is the same. So, just spawn a thread every time you need an
asynchronous task, and implement a way to be noticed when the task is
done (with or without errors). This can be done in a number of ways,
since you asked for pattern I suggest you have a look at
observer.
The second requirement is not completely clear to me, I assume you
want to periodically check that the client's data is aligned with the
server's, and maybe perform security checks ("Are session and
authentication credentials still valid?"). The first solution is to
actually ask the server every n seconds, again using another
thread. This kind of polling might not be the best option though: how
do you factor in the possibility of connectivity issues? Even if your
client cannot operate without a working connection to the server, it
might bother the user to be disconnected and lose his work just
because his Wi-Fi router rebooted. I would suggest you perform
alignment checks at I/O, perhaps distinguishing between critical and
non-critical ones. For example, if you decide the user's profile
has to be aligned, then you would retrieve updated data from the server upon viewing it. On the other hand, if your app offers the
user a list of cooking recipes and you don't care about him not
viewing the one that has been inserted on the server 10 minutes in
the past, you could simply cache these items and refresh them in a
background thread every minute, without even noticing the user in
case the update fails. Last but not least, if you are also
concerned with concurrent modifications of data, again based on your
requirements you can decide to implement locks on data being edited,
to performs checks on save operations to see if the data has
changed in the meanwhile, or to simply let the user overwrite the
data on the server no matter what. All in all, hoping I interpreted
your question correctly, this requirement is nontrivial and has to be
adjusted to your particular use case.
Assuming the data is eventually saved on some sort of database on
the server, one answer are transactions, which allow you to
treat even complex sequences of operations as "all or nothing",
atomic instructions. You might implement your own system to have the
same result, but I don't really see the point of not using this
powerful instrument when possible. Keep in mind one thing: I'm
assuming "cancelable" means "cancelable before some point in time,
and not after" (a sort of "undo"). If you're looking for complete
revertability of any operation of data, the requirement becomes far
more complex, and in general not possible to guarantee.
I believed I already answered in a way that helps you minimize "hacks" in code where possible. To recap:
You are going to need threads, and the observer pattern can help you
keep the code clean.
Again, you can use threads, or focus on check on I/O operations. In
the second case, you might consider an application layer
specifically for client-server synchronization, embed it in one or
more classes, and perform all your checks there. Have a look at the
proxy pattern.
Use transactions to revert operations, and issue a COMMIT only
when you are sure that the operation is confirmed, a ROLLBACK in
every other case. Encapsule this logic in your server's code so that
the client is not aware of the actual transaction system being used,
and your code should be quite clean.
Please comment if my answer is not satisfying or clear.
New to NServiceBus (4.7.5) and just implemented an NSB host.exe hosted service (implementing IWantToRunWhenBusStartsAndStops) that detects changes to database tables and notifies subscribing web apps by publishing events, e.g. "CustomerDataWasUpdatedEvent". In the future we will perform the actual update through messagehandlers receiving commands obviously, but at the moment this publishing service just polls the database etc.
It all works well, however, approaching production, I noticed that David Boike, in his latest edition of "Learning NServiceBus", states that classes implementing
IWantToRunWhenBusStartsAndStops are really mostly for development and rarely used in production. I set up my database change detection in the Start method and it works nicely, does anyone know why this is discouraged?
Here is the comment in the actual book:
https://books.google.se/books?id=rvpzBgAAQBAJ&pg=PA110&lpg=PA110&dq=nservicebus+iwanttorunwhenbusstartsandstops+in+production+david+boike&source=bl&ots=U6sNII0nm3&sig=qIXffOVFhcy-_3qDnSExRpwRlD4&hl=sv&sa=X&ei=lHWRVc2_BKrWywPB65fIBw&ved=0CBsQ6AEwAA#v=onepage&q=nservicebus%20iwanttorunwhenbusstartsandstops%20in%20production%20david%20boike&f=false
The actual quote is:
...it isn't common to have widespread use of in a production system.
Uncommon is not the same thing as discouraged.
That said I do think there is intent here by the author to highlight the fact that further up the page they assert that this is not a good place to be doing lots of coding, as an unhandled exception can cause the whole process to fail.
The author actually does go on to mention a possible use case for when you may want to load a resource(s) to do work within the handler.
Ok, maybe it's just this scenario we have that is a bit uncommon
Agreed - there is nothing fundamentally wrong with your approach. I recently did the same thing as you for wiring up SqlDependency to listen for database events and then publish a message as a result. In these scenarios there is literally nothing else you can do other than to use IWantToRunAtStatup.
Also, David himself often trawls the nservicebus tag, maybe he'll provide a more definitive answer than mine.
I'll copy the answer I gave in the Particular Software Google Group...
I'll quote myself directly here:
An implementation of IWantToRunWhenBusStartsAndStops is a great place to create a quick interface in order to test messages during debugging by allowing you to send messages based on the console input. Apart from this, it isn't common to have widespread use of them in a production system. One possible production use case will be to provision a resource needed by the endpoint at startup and then tear it down when the endpoint stops.
I think if I could add a little bit of emphasis it would be to "widespread use". I'm not trying to say you won't/can't have an IWantToRunWhenBusStartsAndStops in production code or that avoiding them is a best practice. I am trying to say that having a ton of them is probably a code smell.
Above that paragraph in the book, I warn about IWantToRunWhenBusStartsAndStops not having any ambient transactions or try/catch stuff going on. THAT is really the key part. If you end up throwing an exception in an IWantToRunWhenBusStartsAndStops, tyou can run into big problems. If you use something like a .NET Timer and then throw an exception, you can crash your process!
Let me tell you how I screwed up on this in my first-ever NServiceBus system. The system (still in use today, from what I hear) is responsible for ingesting more than 3000 RSS feeds (probably a lot more than that now) into a CMS. So processing each feed, breaking it up into items, resizing images, encoding attached video for mobile ... all those things were handled in NServiceBus message handlers, which was scaled out to multiple servers, and that was all fantastic.
The problem was the scheduler. I implemented that as an IWantToRunWhenBusStartsAndStops (well, actually IWantToRunAtStartup at that time) and it quickly turned into a mess. I kept the whole table worth of feed information in memory so that I could calculate when to fire off the next ProcessFeed command. I was using the .NET Timer class, and IIRC, I eventually had to use threading primitives like ManualResetEvent in order to coordinate the activity. And because I was using .NET Timer, if the scheduler threw an exception, that endpoint failed and had to restart. Lots of weird edge cases and it was always a quagmire of bugs. Plus, this was now a singleton "commander app" so while the feed/item processors could be scaled out, the scheduler could not.
As I got more experienced with NServiceBus, I realized that each feed should have been a saga, starting from a FeedCreated event, controlled through PauseProcessing and ResumeProcessing commands, using timeouts to control the next processing time, and finally (perhaps) ended via a FeedRemoved event. This would have been MUCH more straightforward and everything would have executed inside transactionally-controlled message handlers.
That experience led me to be a little bit distrustful/skeptical of IWantToRunWhenBusStartsAndStops. Not saying it's bad, just something to be aware of. Always be prepared to consider if what you're trying to do couldn't be better accomplished in another way.
I’m currently working on a server application were we have agreed to try and maintain a certain level of service. The level of service we want to guaranty is: if a request is accepted by the server and the server sends on an acknowledgement to the client we want to guaranty that the request will happen, even if the server crashes. As requests can be long running and the acknowledgement time needs be short we implement this by persisting the request, then sending an acknowledgement to the client, then carrying out the various actions to fulfill the request. As actions are carried out they too are persisted, so the server knows the state of a request on start up, and there’s also various reconciliation mechanisms with external systems to check the accuracy of our logs.
This all seems to work fairly well, but we have difficult saying this with any conviction as we find it very difficult to test our fault tolerant code. So far we’ve come up with two strategies but neither is entirely satisfactory:
Have an external process watch the server code and then try and kill it off at what the external process thinks is an appropriate point in the test
Add code the application that will cause it to crash a certain know critical points
My problem with the first strategy is the external process cannot know the exact state of the application, so we cannot be sure we’re hitting the most problematic points in the code. My problem with the second strategy, although it gives more control over were the fault takes, is I do not like have code to inject faults within my application, even with optional compilation etc. I fear it would be too easy to over look a fault injection point and have it slip into a production environment.
I think there are three ways to deal with this, if available I could suggest a comprehensive set of integration tests for these various pieces of code, using dependency injection or factory objects to produce broken actions during these integrations.
Secondly, running the application with random kill -9's, and disabling of network interfaces may be a good way to test these things.
I would also suggest testing file system failure. How you would do that depends on your OS, on Solaris or FreeBSD I would create a zfs file system in a file, and then rm the file while the application is running.
If you are using database code, then I would suggest testing failure of the database as well.
Another alternative to dependency injection, and probably the solution I would use, are interceptors, you can enable crash test interceptors in your code, these would know the state of the application and introduce the above listed failures at the correct time, or any others you may want to create. It would not require changes to your existing code, just some additional code to wrap it.
A possible answer to the first point is to multiply experiments with your external process so that probability to impact problematic parts of code is increased. Then you can analyze core dump file to determine where the code has actually crashed.
Another way is to increase observability and/or commandability by stubbing library or kernel calls, i.e., without modifying your application code.
You can find some resources on Fault Injection page of Wikipedia, in particular in Software Implemented Fault Injection section.
Your concern about fault injection is not a fundamental concern. You merely need a foolproof way to prevent such code ending up in deployment. One way to do so is by designing your fault injector as a debugger. I.e. the faults are injected by a process external to your process. This already provides a level of isolation. Furthermore, most OS'es provide some kind of access control which prevents debugging unless specifially enabled. In the most primitive form, it's by limiting it to root, on other operating systems it requires a specific "debug privilege". Naturally, on production nobody will have that, and thus your fault injector cannot even run on production.
Practially, the fault injector can set breakpoints at specific addresses, i.e. function or even line of code. You can then react to that, e.g. by terminating the process after a certain breakpoint is hit three times.
I was just about to write the same as Justin :)
The component I would suggest to replace during testing could be the logging component (if you have one, if not, I'd strongly suggest to implement one...). It's relatively easy to replace it with code that generates error and the logger usually gets enough information to know the current application state.
Also it seems to be feasible to make sure that the testing code doesn't go into production. I would discourage conditional compilation though but rather go with some configuration file to select the logging component.
Using "random" kills might help to detect errors but is not well suited for systematic testing because of its non-determinism. Therefore I wouldn't use it for automatic tests.