I'm trying to wrap my head around how to manage controlled and uncontrolled results/failures in Azure Functions triggered by Event Grid events. I'm sure I'm missing some fundamental aspect but I can't really make sense of the available Microsoft documentation.
Using the current Visual Studio 2019 Azure Function templates for Event Grid triggers, we get C# methods that looks similar to this:
[FunctionName("UserCreated")]
public static void UserCreated([EventGridTrigger]EventGridEvent evt, ILogger log)
{
...
}
Q1. What is the proper way of returning statuses and error codes from these methods so that the event can be either retried or passed to the dead letter blob? Lets say I want to return a Status 400 Bad Request because the custom data payload in the event that was passed in wasn't up to speed. I've tried this:
[FunctionName("UserCreated")]
public static async Task<IActionResult> UserCreated([EventGridTrigger]EventGridEvent evt, ILogger log)
{
...
return new BadRequestObjectResult("ouch");
}
... which just results in this error when running the function locally:
Cannot bind parameter '$return' to type IActionResult&. Make sure the parameter Type is supported by the binding.
I don't understand this error and have no idea how to solve it.
Q2. How do we properly catch exceptions and return them in an orderly fashion? Let's say the UserCreated method above requires the passed in event to have a few custom datapoints, and that one of those datapoints are missing. E.g. we have this:
[JsonObject(ItemRequired = Required.Always)]
private class CustomerAndContactIds
{
public int CustomerId { get; set; }
public int ContactId { get; set; }
}
... and when we convert some event data that is missing e.g. the ContactID field, like so:
private static T ExtractCustomPayloadFromEvent<T>(EventGridEvent evt)
{
return JObject.FromObject(evt.Data).ToObject<T>();
}
... we get this in the logs:
System.Private.CoreLib: Exception while executing function: UserCreated. Newtonsoft.Json: Required property 'ContactPersonId' not found in JSON. Path ''.
However, all we get in e.g. Postman is:
Exception while executing function: UserCreated
What's the proper way of making this a bit more legible for consumers that aren't privy to e.g. the Azure log stream or Azure Insights?
To Q1:
Currently version of the EventGridTrigger function doesn't have a specific exception object to handle for its wrapper back a HttpStatusCode value. In other words, any exception in the azure function will force a retry pattern when it is enabled.
As a workaround for this issue, we can use a HttpTrigger function for subscriber handler with the HttpStatusCode return value.
Related
I have a question about handling deserialization exceptions in Spring Cloud Stream while processing batches (i.e. batch-mode: true).
Per the documentation here, https://docs.spring.io/spring-kafka/docs/2.5.12.RELEASE/reference/html/#error-handling-deserializer, (looking at the implementation of FailedFooProvider), it looks like this function should return a subclass of the original message.
Is the intent here that a list of both Foo's and BadFoo's will end up at the original #StreamListener method, and then it will be up to the code (i.e. me) to sort them out and handle separately? I suspect this is the case, as I've read that the automated DLQ sending isn't desirable for batch error handling, as it would resubmit the whole batch.
And if this is the case, what if there is more than one message type received by the app via different #StreamListener's, say Foo's and Bar's. What type should the value function return in that case? Below is the pseudo code to illustrate the second question?
#StreamListener
public void readFoos(List<Foo> foos) {
List<> badFoos = foos.stream()
.filter(f -> f instanceof BadFoo)
.map(f -> (BadFoo) f)
.collect(Collectors.toList());
// logic
}
#StreamListener
public void readBars(List<Bar> bars) {
// logic
}
// Updated to return Object and let apply() determine subclass
public class FailedFooProvider implements Function<FailedDeserializationInfo, Object> {
#Override
public Object apply(FailedDeserializationInfo info) {
if (info.getTopics().equals("foo-topic") {
return new BadFoo(info);
}
else if (info.getTopics().equals("bar-topic") {
return new BadBar(info);
}
}
}
Yes, the list will contain the function result for failed deserializations; the application needs to handle them.
The function needs to return the same type that would have been returned by a successful deserialization.
You can't use conditions with batch listeners. If the list has a mixture of Foos and Bars, they all go to the same listener.
I've been trying in vain to load an aggregate using the load method of the repository class. I know that the load() method requires a string argument but my aggregate identifier is integer.
I have tried to convert to string, yet I'm not able to load the aggregate. I keep getting null.
My code is shown below
#Autowired
private Repository<Product> repository;
#CommandHandler
public Order(CreateOrderCommand cmd){
try{
//This line throws an error
Product product = (Product) repository.load(Integer.toString(cmd.getProductId()));
}
catch (Exception ex){
System.out.println("Oops! Error Occured: " + ex.getMessage());
}
apply(
new OrderCreatedEvent(
cmd.getOrderId(), cmd.getPrice(), cmd.getNumber(), cmd.getProductId()
)
);
}
I'm providing an more updates:
I simply want to update the products stock, when a new order is placed. So what I did was that in the CommandHandler for creating a new Order, I simply find the Product aggregate and call it's UpdateStock method, which is a command handler.
Is there something I really don't understand?
These are the files below:
The Product aggregate:
#Aggregate
public class Product {
#AggregateIdentifier
private String id;
private Double price;
private Integer stock;
private String description;
public Product() {
}
#CommandHandler
public Product(AddProductCommand cmd){
apply( new ProductAddedEvent(
cmd.getId(),
cmd.getPrice(),
cmd.getStock(),
cmd.getDescription()
)
);
}
#CommandHandler
public void updateStock (UpdateStockCommand cmd){
if(this.stock >= cmd.getStock()){
apply(new StockUpdatedEvent(cmd.getId(), cmd.getStock()));
}
else {
throw new RuntimeException("Out of Stock!");
}
}
#EventSourcingHandler
public void on(StockUpdatedEvent evt){
id = evt.getId();
stock = stock - evt.getStock();
}
#EventSourcingHandler
public void on(ProductAddedEvent evt){
id = evt.getId();
price = evt.getPrice();
stock = evt.getStock();
description = evt.getDescription();
}
}
The Order aggregate:
#Aggregate
public class Order {
#AggregateIdentifier
private String orderId;
private Double price;
private Integer number;
private String productId;
//Repository provides an abstraction for storage of Aggregates
#Autowired
private Repository<Product> repository;
#CommandHandler
public Order(CreateOrderCommand cmd){
try{
//This line throws an error
Product product = (Product) repository.load(cmd.getProductId());
}
catch (Exception ex){
System.out.println("Oops! Error Occured: " + ex.getMessage());
}
apply(
new OrderCreatedEvent(
cmd.getOrderId(), cmd.getPrice(), cmd.getNumber(), cmd.getProductId()
)
);
}
#EventSourcingHandler
public void on(OrderCreatedEvent evt){
orderId = evt.getOrderId();
price = evt.getPrice();
number = evt.getNumber();
productId = evt.getProductId();
}
}
As you have noticed, Axon will enforce a String for the aggregate identifier. This doesn't mean you need to provide a String directly. It'll work perfectly fine as long as you have a sensible toString() method implemented. Hence, using Integer as the aggregate identifier will result in the toString() result of that field to be used as the aggregate identifier.
Having said that, I'm surprised you are unable to load an aggregate based on this.
Maybe we need to do some follow up based on that, but firstly I want to comment on the snippet you're sharing.
It seems like you have a constructor command handler on the Order class. Moreover, that signals to me that the Order class is an aggregate.
I'd like to state that by no means would I ever recommend to load in an Aggregate from within another Aggregate.
Doing so ties in the lock on the Order aggregate with the Product aggregate, thus blocking a larger portion of your system than the sender of CreateOrderCommand would ever expect.
Communication between aggregate should be kept asynchronous at all times, through means of event messages.
That thus requires a dedicated Event Handling Component which does the coordination between both instances based on the published event.
I'd thus strongly suggest a rewrite of your logic on the matter.
When it comes to loading in an aggregate, I'd be hard pressed to give you a reasoning right now.
Hence, let me ask a couple of follow up questions.
I'd suggest to update your original request with the responses by the way; to maintain a nice thread of continuity.
Which version of Axon are you using any way?
Are you using any interesting configurations around the Product aggregate?
Are you pairing your application with Axon Server? If so, standard or enterprise, and which version?
Do you use the Configuration API directly or the axon-spring-boot-starter?
Update
Thanks for updating your question Kindson, let me update my answer too.
Firstly, as I already stated previously, I would never load in an Aggregate from within another Aggregate. Both your Order class and Product class are Aggregates and the snippets make overly clear that you are consolidating the "product Repository" from within the Order aggregate.
This approach does not only prolong locks on your aggregates, thus imposing strains on your users, but following this approach might event incur deadlocks. As a rule of thumb, always follow an asynchronous approach to communicating between aggregate instances. Thus a component reacting on an aggregate's events and dispatching commands to another. This can be done in a regular Event Handling Component, or a Saga for example.
Apart from the above, I have tested locally whether I could use an Integer as an Aggregate Identifier. Thus, as the #AggregateIdentifier annotated field in the aggregate and as the #TargetAggregateIdentifier annotated field on your command message. On top of that, I tried this out by having the framework call the Repository#load(String) operation (thus just dispatching a command on the CommandGateway/CommandBus) and by calling it directly as you did.
Sadly, this works out fine for me. Thus, as another follow up, I'd suggest to share the actual exception you are getting when you are performing the Repository#load(String) operation.
I am implementing a MassTransit middleware in my receive end point to record the performance of handling the message, i want to get the message type from the PipeContext, how can i get it?
public async Task Send(T context, IPipe<T> next)
{
// I want to know the message type from here so that i can log it
using (_logger.BeginTimedOperation("Time for handling message", null, LogEventLevel.Debug))
{
await next.Send(context);
}
}
You would need to intercept at the ConsumeContext, which has a property for the message types from the serialization envelope.
Then, you can get the supported message types using:
IEnumerable<string> SupportedMessageTypes {get;}
That should get you what you need to log the message type with the duration.
So a filter along the lines of:
public class LogMessageTypeFilter :
IFilter<ConsumeContext>
{
}
Implement the send method, call next within the method, and then take action after the next pipe completes.
The problem: I'm crashing when I want to render my incoming data which was retrieved asynchronously.
The app starts and displays some dialog boxes using XAML. Once the user fills in their data and clicks the login button, the XAML class has in instance of a worker class that does the HTTP stuff for me (asynchronously using IXMLHTTPRequest2). When the app has successfully logged in to the web server, my .then() block fires and I make a callback to my main xaml class to do some rendering of the assets.
I am always getting crashes in the delegate though (the main XAML class), which leads me to believe that I cannot use this approach (pure virtual class and callbacks) to update my UI. I think I am inadvertently trying to do something illegal from an incorrect thread which is a byproduct of the async calls.
Is there a better or different way that I should be notifying the main XAML class that it is time for it to update it's UI? I am coming from an iOS world where I could use NotificationCenter.
Now, I saw that Microsoft has it's own Delegate type of thing here: http://msdn.microsoft.com/en-us/library/windows/apps/hh755798.aspx
Do you think that if I used this approach instead of my own callbacks that it would no longer crash?
Let me know if you need more clarification or what not.
Here is the jist of the code:
public interface class ISmileServiceEvents
{
public: // required methods
virtual void UpdateUI(bool isValid) abstract;
};
// In main XAML.cpp which inherits from an ISmileServiceEvents
void buttonClick(...){
_myUser->LoginAndGetAssets(txtEmail->Text, txtPass->Password);
}
void UpdateUI(String^ data) // implements ISmileServiceEvents
{
// This is where I would render my assets if I could.
// Cannot legally do much here. Always crashes.
// Follow the rest of the code to get here.
}
// In MyUser.cpp
void LoginAndGetAssets(String^ email, String^ password){
Uri^ uri = ref new URI(MY_SERVER + "login.json");
String^ inJSON = "some json input data here"; // serialized email and password with other data
// make the HTTP request to login, then notify XAML that it has data to render.
_myService->HTTPPostAsync(uri, json).then([](String^ outputJson){
String^ assets = MyParser::Parse(outputJSON);
// The Login has returned and we have our json output data
if(_delegate)
{
_delegate->UpdateUI(assets);
}
});
}
// In MyService.cpp
task<String^> MyService::HTTPPostAsync(Uri^ uri, String^ json)
{
return _httpRequest.PostAsync(uri,
json->Data(),
_cancellationTokenSource.get_token()).then([this](task<std::wstring> response)
{
try
{
if(_httpRequest.GetStatusCode() != 200) SM_LOG_WARNING("Status code=", _httpRequest.GetStatusCode());
String^ j = ref new String(response.get().c_str());
return j;
}
catch (Exception^ ex) .......;
return ref new String(L"");
}, task_continuation_context::use_current());
}
Edit: BTW, the error I get when I go to update the UI is:
"An invalid parameter was passed to a function that considers invalid parameters fatal."
In this case I am just trying to execute in my callback is
txtBox->Text = data;
It appears you are updating the UI thread from the wrong context. You can use task_continuation_context::use_arbitrary() to allow you to update the UI. See the "Controlling the Execution Thread" example in this document (the discussion of marshaling is at the bottom).
So, it turns out that when you have a continuation, if you don't specify a context after the lambda function, that it defaults to use_arbitrary(). This is in contradiction to what I learned in an MS video.
However by adding use_currrent() to all of the .then blocks that have anything to do with the GUI, my error goes away and everything is able to render properly.
My GUI calls a service which generates some tasks and then calls to an HTTP class that does asynchronous stuff too. Way back in the HTTP classes I use use_arbitrary() so that it can run on secondary threads. This works fine. Just be sure to use use_current() on anything that has to do with the GUI.
Now that you have my answer, if you look at the original code you will see that it already contains use_current(). This is true, but I left out a wrapping function for simplicity of the example. That is where I needed to add use_current().
In classic ASP.NET I’d persist data extracted from a web service in base class property as follows:
private string m_stringData;
public string _stringData
{ get {
if (m_stringData==null)
{
//fetch data from my web service
m_stringData = ws.FetchData()
}
return m_stringData;
}
}
This way I could simply make reference to _stringData and know that I’d always get the data I was after (maybe sometimes I’d use Session state as a store instead of a private member variable).
In Silverlight with a WCF I might choose to use Isolated Storage as my persistance mechanism, but the service call can't be done like this, because a WCF service has to be called asynchronously.
How can I both invoke the service call and retrieve the response in one method?
Thanks,
Mark
In your method, invoke the service call asynchronously and register a callback that sets a flag. After you have invoked the method, enter a busy/wait loop checking the flag periodically until the flag is set indicating that the data has been returned. The callback should set the backing field for your method and you should be able to return it as soon as you detect the flag has been set indicating success. You'll also need to be concerned about failure. If it's possible to get multiple calls to your method from different threads, you'll also need to use some locking to make your code thread-safe.
EDIT
Actually, the busy/wait loop is probably not the way to go if the web service supports BeginGetData/EndGetData semantics. I had a look at some of my code where I do something similar and I use WaitOne to simply wait on the async result and then retrieve it. If your web service doesn't support this then throw a Thread.Sleep -- say for 50-100ms -- in your wait loop to give time for other processes to execute.
Example from my code:
IAsyncResult asyncResult = null;
try
{
asyncResult = _webService.BeginGetData( searchCriteria, null, null );
if (asyncResult.AsyncWaitHandle.WaitOne( _timeOut, false ))
{
result = _webService.EndGetData( asyncResult );
}
}
catch (WebException e)
{
...log the error, clean up...
}
Thanks for your help tvanfosson. I followed your code and have also found a pseudo similar solution that meets my needs exactly using a lambda expression:
private string m_stringData;
public string _stringData{
get
{
//if we don't have a list of departments, fetch from WCF
if (m_stringData == null)
{
StringServiceClient client = new StringServiceClient();
client.GetStringCompleted +=
(sender, e) =>
{
m_stringData = e.Result;
};
client.GetStringAsync();
}
return m_stringData;
}
}
EDIT
Oops... actually this doesn't work either :-(
I ended up making the calls Asynchronously and altering my programming logic to use MVVM pattern and more binding.