So when developing an app utilizing Azure Storage Queues and a Web Job, I feel like I need some sort of health check (via API) to ensure my Azure Storage Queue is properly configured for each environment up to prod. I don't have access (directly) to view the Dashboard or Kudu.
My thought thus far was to just create an API route that returns a bool that tells me if I was able to create the queue if it doesn't exist, and peek at a message (even if one doesn't exist), like :
public async Task<bool> StorageQueueHealthCheck()
{
return await _queueManager.HealthCheck();
}
And the implementation:
public async Task<bool> HealthCheck()
{
try
{
CloudQueue queue = _queueClient.GetQueueReference(QueueNames.reportingQueue);
queue.CreateIfNotExists();
CloudQueueMessage peek = await queue.PeekMessageAsync();
return true; // as long as we were able to peek at messages
}
catch (Exception ex)
{
return false;
}
}
Is this a bad approach? Is there another way to "health check" certain Azure functionality when the dashboard is abstracted away? If I absolutely needed I would be able to view the Kudu but would rather just use an API and hit it via Swagger.
Looks good. You can also try CloudQueue.FetchAttributeAsync() since the payload would be smaller when the message size is large.
This is a good approach, please just make sure you do have a retry mechanism so that your healthcheck does not just return false for intermittent failures.
Second Approach,
Instead of api which will perform the job only is trigger there should be a console app (webjob) which does this task on regular interval (1min) and based on some logic lets say all 'creates' in last 10mins threw error sends an email. This can be used in all environments.
Related
I'm accessing a external rest api which is secured using OpenID connect. I must run a job which calls the api several times before it completes. This job may be executed in parallel using several concurrent threads. Thus I use a service for api access which is instantiated for each job:
private AccessAndRefresTokens tokens; // I initially get that from somewhere else
public Mono<Result> callTheApi(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
public Mono<Result> callOtherApiFunction(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
public Mono<Result> callYetAnotherApiFunction(){
return createWebClientWith(tokens.accessToken).executeRequest();
}
As my job is executed it might happen that the access token expires between two api calls. To prevent this from happening, I like to check the validity of the access token before every request and refresh it if necessary.
My first idea was to do the validity check inside a flatMap operator:
Mono.just(tokens).flatMap(tokens -> {
if(accessTokenExpired){
return refreshAccessToken().doOnNext(refreshedTokens -> tokens = refreshedTokens);
}else{
return Mono.just(tokens.accessToken);
}
}).flatMap(accessToken -> createWebClientWith(accessToken));
However it seems to me that this would trigger the refresh several times if multiple threads access the service and the refresh has not yet completed. As a consequence I would end up refreshing the access token several times in a very short time which might fail due to rate limits.
I am new to the whole reactive thing and I suppose using a synchronized block is not a desired option. So I tried to figure out something using reactor's Processor but I could not find a satisfying solution.
So is there a way to ensure the access token is refershed only once? And how do I achieve this without using blocking code?
May be this issue is already reported and resolved .I didn't find the solution and any open issues which talk about this, so creating new one.
I am trying to handle error while publishing data to kafka topic.
With kafka spring steam we are pushing to kafka by using this
if (processor.output().send(messsage , kafkaTimeoutInMS) && acknowledgment != null)
{
LOGGER.debug("Acknowledgment provided");
LOGGER.info("Sending to Kafka successful");
acknowledgment.acknowledge();
}
else
{
LOGGER.error("Sending to Kafka failed", message);
}
Send() method always returns true, I tried stopping kafka manual while running in debug mode, but still it returns true. I have that read it is asynchronous.
I Tried setting
bindings: output: producer: sync: true
This didnt help.
But I see some error which I cant use in my logic to decide whether there is failure or success.
We are manually acknowledging hence we are only supposed to acknowledge when its sent to topic successfully and we need to log all failed messages.
Any suggestions?
I believe you've misinterpreted on how spring-cloud-stream works.
As a framework there is certain contract between the user and the framework and when it comes to messaging the acks, retries, DLQ and many more aspects are handled automatically to ensure the user doesn't have to be exposed to this manually (as you are trying to do).
Consider spending a little time and going through the user guide - https://docs.spring.io/spring-cloud-stream/docs/Fishtown.M3/reference/htmlsingle/
Also, here is the very basic example that will demonstrates a typical interaction of user(developer) with the framework. As you can see, all you're doing is implementing a simple handler which receives and returns a piece of data. The rest (the actual receive from Kafka and send to Kafka or any other messaging system) is handled by the framework provided binders.
#SpringBootApplication
#EnableBinding(Processor.class)
public class ProcessorApplication {
public static void main(String[] args) {
SpringApplication.run(ProcessorApplication.class);
}
#StreamListener(Processor.INPUT)
#SendTo(Processor.OUTPUT)
public String echo(String message) {
return message;
}
}
My aim is to get notifications from google container registry in code whenever any image is updated/inserted/deleted from the registry.
I am following tutorial - https://cloud.google.com/container-registry/docs/configuring-notifications
I am able to pull notification messages from the registry using the google console using command - gcloud alpha pubsub subscriptions pull SUBSCRIPTION
But I want these notification messages to be delivered in code (in java).
If someone can give me any reference to any article or tutorial that will help.
After comment from dsesto i have added following code. This code gave me some messages when i run first. But after that i kept application running and tried to delete/insert images from container registry but it did not gave any message.
Any suggestions.
package com.avaya.ipoffice.mcm.googleconnect;
import org.springframework.stereotype.Service;
import com.google.cloud.pubsub.v1.AckReplyConsumer;
import com.google.cloud.pubsub.v1.MessageReceiver;
import com.google.cloud.pubsub.v1.Subscriber;
import com.google.pubsub.v1.ProjectSubscriptionName;
import com.google.pubsub.v1.PubsubMessage;
#Service
public class RecieveMessagesUtil {
public static void main(String... args) throws Exception {
String projectId = "xxxxx";
String subscriptionId = "prashantsub";
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subscriptionId);
// Instantiate an asynchronous message receiver
MessageReceiver receiver = new MessageReceiver() {
#Override
public void receiveMessage(PubsubMessage message, AckReplyConsumer consumer) {
// handle incoming message, then ack/nack the received message
System.out.println("Id : " + message.getMessageId());
System.out.println("Data : " + message.getData().toStringUtf8());
consumer.ack();
}
};
Subscriber subscriber = null;
try {
// Create a subscriber for "my-subscription-id" bound to the message receiver
subscriber = Subscriber.newBuilder(subscriptionName, receiver).build();
subscriber.startAsync();
// ...
} catch (Exception e) {
System.out.println("Exception while subscribing" + e);
} finally {
// stop receiving messages
if (subscriber != null) {
subscriber.stopAsync();
}
}
}
}
From your question I understand that you have already successfully setup the notifications system of Container Registry using Pub/Sub topics and subscriptions, given that you said that you are already able to retrieve the messages from your Pub/Sub subscription using command gcloud pubsub subscriptions pull. Therefore, it looks like your concern is mostly related to pulling messages from a Subscription programatically.
First of all, I would recommend you to have a look at this documentation page about Subscribers in Pub/Sub. Especially, have a look at the Pull vs. Push comparison, where you will have a better idea of the possibilities available, and you should first decide whether to work with a Pull Subscription (such as the one you are using when calling the gcloud command, where the application initiates requests) or with a Push Subscription (where Pub/Sub initiates the requests to a subscriber application, such as an App Engine application).
Once that is clear (and assuming that you go for the Pull Subscription, which you are already using with gcloud), you can have a look at the documentation on how to perform Asynchronous Pull operations, with a Java-based example. Additionally, you can have a look at the complete subscriber example available in GitHub.
Finally, you should have a look at the Client Libraries documentation, more specifically the Pub/Sub Java reference, where you will find the complete documentation for the Pub/Sub Client Libraries used to work with Pub/Sub programatically.
I would recommend looking at the Cloud Pub/Sub documentation, particularly, the subscriber guide, which gives you the code necessary to receive messages in Java.
I'd like to add a service that executes some initialization operations for the system when it's first created.
I'd imagine it would be a stateless service (with cluster admin rights) that should self-destruct when it's done it's thing. I am under the impression that exiting the RunAsync function allows me to indicate that I'm finished (or in an error state). However, then it still hangs around on the application's context and annoyingly looking like it's "active" when it's not really doing anything at all.
Is it possible for a service to remove itself?
I think maybe we could try using the FabricClient.ServiceManager's DeleteServiceAsync (using parameters based on the service context) inside an OnCloseAsync override but I've not been able to prove that might work and it feels a little funky:
var client = new FabricClient();
await client.ServiceManager.DeleteServiceAsync(new DeleteServiceDescription(Context.ServiceName));
Is there a better way?
Returning from RunAsync will end the code in RunAsync (indicate completion), so SF won't start RunAsync again (It would if it returned an exception, for example). RunAsync completion doesn't cause the service to be deleted. As mentioned, for example, the service might be done with background work but still listening for incoming messages.
The best way to shut down a service is to call DeleteServiceAsync. This can be done by the service itself or another service, or from outside the cluster. Services can self-delete, so for services whose work is done we typically see await DeleteServiceAsync as the last line of RunAsync, after which the method just exits. Something like:
RunAsync(CancellationToken ct)
{
while(!workCompleted && !ct.IsCancellationRequested)
{
if(!DoneWithWork())
{
DoWork()
}
if(DoneWithWork())
{
workCompleted == true;
await DeleteServiceAsync(...)
}
}
}
The goal is to ensure that if your service is actually done doing the work it cleans itself up, but doesn't trigger its own deletion for the other reasons that a CancellationToken can get signaled, such as shutting down due to some upgrade or cluster resource balancing.
As mentioned already, returning from RunAsync will end this method only, but the service will continue to run and hence not be deleted.
DeleteServiceAsync certainly is the way to go - however it's not quite as simple as just calling it because if you're not careful it will deadlock on the current thread (especially in local developer cluster). You would also likely get a few short-lived health warnings about RunAsync taking a long time to terminate and/or target replica size not being met.
In any case - solution is quite simple - just do this:
private async Task DeleteSelf(CancellationToken cancellationToken)
{
using (var client = new FabricClient())
{
await client.ServiceManager.DeleteServiceAsync(new DeleteServiceDescription(this.Context.ServiceName), TimeSpan.FromMinutes(1), cancellationToken);
}
}
Then, in last line of my RunAsync method I call:
await DeleteSelf(cancellationToken).ConfigureAwait(false);
The ConfigureAwait(false) will help with deadlock issue as it will essentially return to a new thread synchronization context - i.e. not try to return to "caller context".
I have read multiple times the documentation on the website. I am reading again and again the same articles and I cannot understand what they are trying to achieve with sagas. Besides, there are almost no resources in internet related to this subject.
But I am completely stuck trying to understand the purpose and benefit of defining so called sagas. I understand handlers (IHandleMessages) - these are interceptors. But I can't understand what Saga is for. The language in the documentation assumes that I am supposed to know something special to grasp that idea, but I dont.
Can someone explain to me in simple words, hopefully with real-life example a situation where I must or should define Saga, and what is the benefit of doing so? I have created an app with multiple endpoints and Saga definition as shown in samples, it works (I guess) but I don't understand what these sagas were defined for... In many samples they use RequestTimeout() method in Saga class. Why, why would anyone want to cause a timeout intentionally? I dont want to put any code fragments here, because its unrelated, I need to understand why I would want to use "Sagas" whatever that means?
Thank you.
NServiceBus Saga is a variant of a Process Manager described in the Enterprise Integration Patterns book.
To understand when to use Saga, one has to need it. Let's assume you're using regular message handlers only to implement new user registration process. At some point in time, you discover that only 40% of the brand-new registrants confirm their email address and becoming active user accounts. There are two things you'd like to address.
Remind new registrants to confirm their email after 24 hours after registration by sending a reminder.
Remove registrant info (email for example) from the data store to be compliant with GDPR within 48 hours.
Now how do you do that with a regular message handler? A handler would receive the initial request (first message, m1) to kick off registration by generating an email with a confirmation link and that's it. Once the handler is done, it's done for good. But your process is not finished. It's a long-running logical process that has to span 48 hours before completed. It's no longer just a single message processing, but a workflow at this point. A workflow with multiple checkpoints. Similar to a state machine. To move from one state to another, a certain condition has to be fulfilled. In case of NServiceBus, those would be messages. A message to send a reminder after 24 hours (let's call it m2) is not going to be triggered by any user action. It's a "system" message. A timed message that should be kicked off automatically. So is with the message to instruct the system to remove registrant information if validation link was not activated. The theme can be observed: need to schedule messages in the future to re-hydrate the workflow and continue from the state it was left last time.
That's what timeouts are. Those are requests to re-hydrate/continue saga/workflow from the point it was left last time at a certain point in time - minutes, hours, days, months, years.
This is what this kind of workflow would look like as a saga (oversimplified and doesn't take into consideration all the edge cases).
class RegistrationWorkflow :
Saga<WorkflowState>,
IAmStartedByMessages<RegisterUser>,
IHandleMessages<ActivationReceived>,
IHandleTimeouts<NoResponseFor24Hours>,
IHandleTimeouts<NoResponseFor48Hours>
{
protected override void ConfigureHowToFindSaga(SagaPropertyMapper<WorkflowState> mapper)
{
// omitted for simplicity, see message correlation
// https://docs.particular.net/nservicebus/sagas/message-correlation
}
public async Task Handle(RegisterUser message, IMessageHandlerContext context)
{
Data.RegistrationId = message.RegistrationEmail;
await RequestTimeout<NoResponseFor24Hours>(context, TimeSpan.FromHours(24));
}
public async Task Handle(ActivationReceived message, IMessageHandlerContext context)
{
Data.ConfirmationReceived = true;
// email was confirmed and account was activated
await context.Send(new PromoteCandidateToUser
{
CandidateEmail = Data.RegistrationEmail
});
MarkAsComplete()
}
public async Task Timeout(NoResponseFor24Hours timeout, IMessageHandlerContext context)
{
if (Data.ConfirmationReceived)
{
return;
}
await context.Send(new SendReminderEmailToActivateAccount { Email = Data.RegistrationEmail });
await RequestTimeout(context, TimeSpan.FromHours(24), new NoResponseFor48Hours());
}
public async Task Timeout(NoResponseFor48Hours timeout, IMessageHandlerContext context)
{
if (Data.ConfirmationReceived)
{
return;
}
context.Send(new CleanupRegistrationInformationForGDPRCompliancy
{
RegistrationEmail = Data.RegistrationEmail
});
MarkAsComplete();
}
}
Since this is a state machine, the state is persisted between Saga invocations. Invocation would be caused either by a message a saga can handle (RegisterUser and ActivationReceived) or by timeouts that are due (NoResponseFor24Hours and NoResponseFor48Hours). For this specific saga, the state is defined by the following POCO:
class WorkflowState : ContainSagaData
{
public string RegistrationEmail { get; set; }
public bool ConfirmationReceived { get; set; }
}
Timeouts are nothing but plain IMessages that get deferred. The timeouts used in this samples would be
class NoResponseFor24Hours : IMessage {}
class NoResponseFor48Hours : IMessage {}
Hope this clarifies the idea of Sagas in general, what Timeouts are and how they are used. I did not go into Message Correlation, Saga Concurrency, and some other details as those can be found at the documentation site you've referred to. Which bring us to the next point.
I have read multiple times the documentation on their website. It is absolutely terrible. I am reading again and again the same articles and I cannot comprehend what they are trying to achieve.
The site has a feedback mechanism you should absolutely provide.
Besides there almost no resources in internet related to this subject.
Hope to see you posting a blog (or a series of posts) on this topic. By doing so you'll have a positive contribution.
Full disclaimer: I work on NServiceBus