Imagine we have an old good CRUD service over some User entity. We also have kind of event-driven system and we want to emit UserUpdated event every time when User is updating. Event object contains an userId: Int property.
On the event listener side we want to protect our system from event spamming. As far as I know, the most straightforward way to do this in Kotlin is to pass all event through flow with debounce operator. Now I guess if we get 10 events during 1 second we will pass to processing only the last one.
flow
.debounce(1000)
.onEach { doSmthWithEvent(...) }
.launchIn(coroutineScope)
But we don't want to debounce all events indiscriminately. We want to debounce only events related to particular userId. First thing that comes to my mind is to have a dedicated flow for each userId. But since we can have a hundreds of users we will have a hundreds of flows in memory. Flows are lightweight but anyway it looks too bruteforce-ish.
So the question is are there any ways to kind of split a single flow by some event property into subflows and to apply debounce to this new subflow? Something like that
flow
.debounceBy(1000) { it.userId }
.onEach { doSmthWithEvent(...) }
.launchIn(coroutineScope)
I guess you can filter users by particularId and then use debounce if I correctly understand your issue:
flow
.filter { it.userId == particularId }
.debounce(1000)
.onEach { doSmthWithEvent(...) }
.launchIn(coroutineScope)
Related
I have a dashboard with a list of items and a finite number of users. I want to show "an item is being edited" near said item to avoid simultaneous edits and overwrites of data.
This seems to me like updating a flag in the database and relatively simple signalr implementation with the javascript simply adding/removing a css class.
I have seen this:
Prevent multiple people from editing the same form
which describes a method with posting every X minutes and clearing the flag from the database when there are no more update messages from the user.
The issue is:
I was wondering if there was a signalr method (like disconnect; i know it exists but I don't know if it fits this scenario) to do that elegantly rather than running a timer function. If so, is it possible for the server to miss the event and permanently leave the flagged as "editing" when it is not?
you could implement a hub for this, here is a example:
public class ItemAccessHub : Hub
{
public override Task OnConnectedAsync()
{
// your logic to lock the object, set a state in the db
return base.OnConnectedAsync();
}
public override Task OnDisconnectedAsync(Exception exception)
{
// your logic to unlock the object
return base.OnDisconnectedAsync(exception);
}
}
to get information from the query you can access the HttpContext:
Context.GetHttpContext().Request.Query.TryGetValue("item-id", out var itemId)
so you could start a connection when the user is accessing the form and send the id of the item in the query:
/hub/itemAccess?item-id=ITEM_ID
and when the user closes the form then disconnect the connection.
with this method the item is also unlocked when the user loses his network connection.
the on disconnect method is allays invoked when a client disconnects, so you can do your clean up in this method.
in this hub you can than also implement the update function
i hope this is what you are looking for
I am implementing a log system, when scroll down a list a lot of LIST_ITEM_SHOWN action would be dispatched. Then Saga will call the API to send out the log.
I want to make it wait for e.g. 2 second until no further LIST_ITEM_SHOWN is dispatched, and group the LIST_ITEM_SHOWN actions to one to reduce API call.
Can this be done in Saga level? or can only be managed in component/container level?
Yeah, it's pretty easy to implement in saga:
yield takeLatest('LIST_ITEM_SHOWN', watchListItemShown);
so, taking latest LIST_ITEM_SHOWN action to store to logs, so in case of multiple actions - the previous tasks are cancelled (won't go after delay).
// for simplicity storing logs in private variable, consider using redux or something similar
let logs = [];
function* watchListItemShown(action) {
yield call(logs.push, action.payload);
yield call(delay, 2000);
yield call(saveToApi, logs);
logs = [];
}
So on every LIST_ITEM_SHOWN action you store logs into local variable (you can consider using redux or something similar instead, so you'll do better logs management via reducers). And after 2 seconds delay the actual save is called.
I've faced with problems when i try to use CompletableFuture with Axon.
For example:
CompletableFuture future = CompletableFuture.supplyAsync(() -> {
log.info("Start processing target: {}", target.toString());
return new Event();
}, threadPool);
future.thenAcceptAsync(event -> {
log.info("Send Event");
AggregateLifecycle.apply(event);
}, currentExecutor);
in thenAcceptAsync - AggregateLifecycle.apply(event) has unexpected behavior. Some of my #EventSourcingHandler handlers start handling event twice. Does anybody know how to fix it?
I have been reading docs and everything that i got is:
In most cases, the DefaultUnitOfWork will provide you with the
functionality you need. It expects processing to happen within a
single thread.
so, it seems i should use somehow CurrentUnitOfWork.get/set methods but still can't understand Axon API.
You should not apply() events asynchronously. The apply() method will call the aggregate's internal #EventSourcingHandler methods and schedule the event for publication when the unit of work completes (successfully).
The way Axon works with the Unit of Work (which coordinates activity of a single message handler invocation), the apply() method must be invoked in the thread that manages that Unit of Work.
If you want asynchronous publication of Events, use an Event Bus that uses an Async Transport, and use Tracking Processors.
I have read multiple times the documentation on the website. I am reading again and again the same articles and I cannot understand what they are trying to achieve with sagas. Besides, there are almost no resources in internet related to this subject.
But I am completely stuck trying to understand the purpose and benefit of defining so called sagas. I understand handlers (IHandleMessages) - these are interceptors. But I can't understand what Saga is for. The language in the documentation assumes that I am supposed to know something special to grasp that idea, but I dont.
Can someone explain to me in simple words, hopefully with real-life example a situation where I must or should define Saga, and what is the benefit of doing so? I have created an app with multiple endpoints and Saga definition as shown in samples, it works (I guess) but I don't understand what these sagas were defined for... In many samples they use RequestTimeout() method in Saga class. Why, why would anyone want to cause a timeout intentionally? I dont want to put any code fragments here, because its unrelated, I need to understand why I would want to use "Sagas" whatever that means?
Thank you.
NServiceBus Saga is a variant of a Process Manager described in the Enterprise Integration Patterns book.
To understand when to use Saga, one has to need it. Let's assume you're using regular message handlers only to implement new user registration process. At some point in time, you discover that only 40% of the brand-new registrants confirm their email address and becoming active user accounts. There are two things you'd like to address.
Remind new registrants to confirm their email after 24 hours after registration by sending a reminder.
Remove registrant info (email for example) from the data store to be compliant with GDPR within 48 hours.
Now how do you do that with a regular message handler? A handler would receive the initial request (first message, m1) to kick off registration by generating an email with a confirmation link and that's it. Once the handler is done, it's done for good. But your process is not finished. It's a long-running logical process that has to span 48 hours before completed. It's no longer just a single message processing, but a workflow at this point. A workflow with multiple checkpoints. Similar to a state machine. To move from one state to another, a certain condition has to be fulfilled. In case of NServiceBus, those would be messages. A message to send a reminder after 24 hours (let's call it m2) is not going to be triggered by any user action. It's a "system" message. A timed message that should be kicked off automatically. So is with the message to instruct the system to remove registrant information if validation link was not activated. The theme can be observed: need to schedule messages in the future to re-hydrate the workflow and continue from the state it was left last time.
That's what timeouts are. Those are requests to re-hydrate/continue saga/workflow from the point it was left last time at a certain point in time - minutes, hours, days, months, years.
This is what this kind of workflow would look like as a saga (oversimplified and doesn't take into consideration all the edge cases).
class RegistrationWorkflow :
Saga<WorkflowState>,
IAmStartedByMessages<RegisterUser>,
IHandleMessages<ActivationReceived>,
IHandleTimeouts<NoResponseFor24Hours>,
IHandleTimeouts<NoResponseFor48Hours>
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<WorkflowState> mapper)
{
// omitted for simplicity, see message correlation
// https://docs.particular.net/nservicebus/sagas/message-correlation
}
public async Task Handle(RegisterUser message, IMessageHandlerContext context)
{
Data.RegistrationId = message.RegistrationEmail;
await RequestTimeout<NoResponseFor24Hours>(context, TimeSpan.FromHours(24));
}
public async Task Handle(ActivationReceived message, IMessageHandlerContext context)
{
Data.ConfirmationReceived = true;
// email was confirmed and account was activated
await context.Send(new PromoteCandidateToUser
{
CandidateEmail = Data.RegistrationEmail
});
MarkAsComplete()
}
public async Task Timeout(NoResponseFor24Hours timeout, IMessageHandlerContext context)
{
if (Data.ConfirmationReceived)
{
return;
}
await context.Send(new SendReminderEmailToActivateAccount { Email = Data.RegistrationEmail });
await RequestTimeout(context, TimeSpan.FromHours(24), new NoResponseFor48Hours());
}
public async Task Timeout(NoResponseFor48Hours timeout, IMessageHandlerContext context)
{
if (Data.ConfirmationReceived)
{
return;
}
context.Send(new CleanupRegistrationInformationForGDPRCompliancy
{
RegistrationEmail = Data.RegistrationEmail
});
MarkAsComplete();
}
}
Since this is a state machine, the state is persisted between Saga invocations. Invocation would be caused either by a message a saga can handle (RegisterUser and ActivationReceived) or by timeouts that are due (NoResponseFor24Hours and NoResponseFor48Hours). For this specific saga, the state is defined by the following POCO:
class WorkflowState : ContainSagaData
{
public string RegistrationEmail { get; set; }
public bool ConfirmationReceived { get; set; }
}
Timeouts are nothing but plain IMessages that get deferred. The timeouts used in this samples would be
class NoResponseFor24Hours : IMessage {}
class NoResponseFor48Hours : IMessage {}
Hope this clarifies the idea of Sagas in general, what Timeouts are and how they are used. I did not go into Message Correlation, Saga Concurrency, and some other details as those can be found at the documentation site you've referred to. Which bring us to the next point.
I have read multiple times the documentation on their website. It is absolutely terrible. I am reading again and again the same articles and I cannot comprehend what they are trying to achieve.
The site has a feedback mechanism you should absolutely provide.
Besides there almost no resources in internet related to this subject.
Hope to see you posting a blog (or a series of posts) on this topic. By doing so you'll have a positive contribution.
Full disclaimer: I work on NServiceBus
Almost all examples of flux involve data cache on the client side however I don't think I would be able to do this for a lot of my application.
In the system I am thinking about using React/Flux, a single user can have 100's of thousands of the main piece of data we store (and 1 record probably has at least 75 data properties). Caching this much data on the client side seems like a bad idea and probably makes things more complex.
If I were not using Flux, I would just have a ORM like system that can talk to a REST API in which case a request like userRepository.getById(123) would always hit the API regardless if I requested that data in the last page. My idea is to just have the store have these methods.
Does Flux consider it bad that if I were to make request for data, that it always hit the API and never pulls data from a local cache instance? Can I use Flux in a way were a majority of the data retrieval requests are always going to hit an API?
The closest you can sanely get to no caching is to reset any store state to null or [] when an action requesting new data comes in. If you do this you must emit a change event, or else you invite race conditions.
As an alternative to flux, you can simply use promises and a simple mixin with an api to modify state. For example, with bluebird:
var promiseStateMixin = {
thenSetState: function(updates, initialUpdates){
// promisify setState
var setState = this.setState.bind(this);
var setStateP = function(changes){
return new Promise(function(resolve){
setState(changes, resolve);
});
};
// if we have initial updates, apply them and ensure the state change happens
return Promise.resolve(initialUpdates ? setStateP(initialUpdates) : null)
// wait for our main updates to resolve
.then(Promise.params(updates))
// apply our unwrapped updates
.then(function(updates){
return setStateP(updates);
}).bind(this);
}
};
And in your components:
handleRefreshClick: function(){
this.thenSetState(
// users is Promise<User[]>
{users: Api.Users.getAll(), loading: false},
// we can't do our own setState due to unlikely race conditions
// instead we supply our own here, but don't worry, the
// getAll request is already running
// this argument is optional
{users: [], loading: true}
).catch(function(error){
// the rejection reason for our getUsers promise
// `this` is our component instance here
error.users
});
}
Of course this doesn't prevent you from using flux when/where it makes sense in your application. For example, react-router is used in many many react projects, and it uses flux internally. React and related libraries/patters are designed to only help where desired, and never control how you write each component.
I think the biggest advantage of using Flux in this situation is that the rest of your app doesn't have to care that data is never cached, or that you're using a specific ORM system. As far as your components are concerned, data lives in stores, and data can be changed via actions. Your actions or stores can choose to always go to the API for data or cache some parts locally, but you still win by encapsulating this magic.