I'm creating integration tests for my application using Spek. I have a set of Providers to test and each one makes a request and parses a response.
describe("Providers")
{
for(provider in providers)
{
on("Provider: $providerName")
{
try
{
//...
val responsePromise = when (provider)
{
is HtmlProvider -> connectionService.reactiveGetForHtml(connectionRequest)
is JsonProvider<*> -> connectionService.reactiveGetForJson(connectionRequest, provider.getJsonClass())
else -> throw IllegalStateException("Provider must be either Html or JSON")
}
val response = runBlocking { responsePromise.await() }
Both connectionService.reactiveGetForHtml() reactiveGetForJson() return a RxJava2's SingleSource.
The problem: one of Providers throws a bunch of exceptions (it has to parse a big JSON file and due to problems with data model Jackson throws huge number of exceptions). This is fine, test are designed to handle and report such errors. In this case, problematic Provider fails its own test with a Jackson exception. But, when runner executes remaining test cases, strange things happen. All next Providers' tests fail because of the same exceptions. runBlocking keeps throwing exceptions from faulty Provider on and on, no matter what responsePromise is. Don't know if this is a bug or some strange behaviour of .await(), but I don't hava any idea how to overcome this.
All test cases of Providers that don't throw multiple exceptions run well and report their own errors, but after executing that faulty Provider everything fails. It's not the case of this exact Provider, if I removed this particular test case, other Providers that thrown multiple exceptions caused simiar problem.
Related
Background
I have a legacy application where I need to return a List<Item>
There are many different Service classes each belonging to an ItemType.
Each service class calls a few different backend APIs and collects the responses to create a SubType of the Item.
So we can say, each service class implementation returns an Item
All backend API access code is using WebClient which returns Mono of some type, and I can zip all Mono within the service to create an Item
The user should be able to look up many different types of items in one call. This requires many backend calls
So for performance sake, I wanted to make this all asynchronous using reactor, so I introduced Spring Reactive code.
Problem
If my endpoint had to return Flux<Item> then this code work fine,
But this is some service code which is used by other legacy code caller.
So eventually I want to return the List<Item> but When I try to convert my Flux into the List I get an error
"message": "block()/blockFirst()/blockLast() are blocking,
which is not supported in thread reactor-http-nio-3",
Here is the service, which is calling a few other service classes.
Flux<Item> itemFlux = Flux.fromIterable(searchRequestByItemType.entrySet())
.flatMap(e ->
getService(e.getKey()).searchItems(e.getValue()))
.subscribeOn(Schedulers.boundedElastic());
Mono<List<Item>> listMono = itemFlux
.collectList()
.block(); //This line throws error
Here is what the above service is calling
default Flux<Item> searchItems(List<SingleItemSearchRequest> requests) {
return Flux.fromIterable(requests)
.flatMap(this::searchItem)
.subscribeOn(Schedulers.boundedElastic());
}
Here is what a single-item search is which is used by above
public Mono<Item> searchItem(SingleItemSearchRequest sisr) {
return Mono.zip(backendApi.getItemANameApi(sisr.getItemIdentifiers().getItemId()),
sisr.isAddXXXDetails()
?backendApi.getItemAXXXApi(sisr.getItemIdentifiers().getItemId())
:Mono.empty(),
sisr.isAddYYYDetails()
?backendApi.getItemAYYYApi(sisr.getItemIdentifiers().getItemId())
:Mono.empty())
.map(tuple3 -> Item.builder()
.name(tuple3.getT1())
.xxxDetails(tuple3.getT2())
.yyyDetails(tuple3.getT3())
.build()
);
}
Sample project to replicate the problem..
https://github.com/mps-learning/spring-reactive-example
I’m new to spring reactor, feel free to pinpoint ALL errors in the code.
UPDATE
As per Patrick Hooijer Bonus suggestion, updating the Mono.zip entries to always contain some default.
#Override
public Mono<Item> searchItem(SingleItemSearchRequest sisr) {
System.out.println("\t\tInside " + supportedItem() + " searchItem with thread " + Thread.currentThread().toString());
//TODO: how to make these XXX YYY calls conditionals In clear way?
return Mono.zip(getNameDetails(sisr).defaultIfEmpty("Default Name"),
getXXXDetails(sisr).defaultIfEmpty("Default XXX Details"),
getYYYDetails(sisr).defaultIfEmpty("Default YYY Details"))
.map(tuple3 -> Item.builder()
.name(tuple3.getT1())
.xxxDetails(tuple3.getT2())
.yyyDetails(tuple3.getT3())
.build()
);
}
private Mono<String> getNameDetails(SingleItemSearchRequest sisr) {
return mockBackendApi.getItemCNameApi(sisr.getItemIdentifiers().getItemId());
}
private Mono<String> getYYYDetails(SingleItemSearchRequest sisr) {
return sisr.isAddYYYDetails()
? mockBackendApi.getItemCYYYApi(sisr.getItemIdentifiers().getItemId())
: Mono.empty();
}
private Mono<String> getXXXDetails(SingleItemSearchRequest sisr) {
return sisr.isAddXXXDetails()
? mockBackendApi.getItemCXXXApi(sisr.getItemIdentifiers().getItemId())
: Mono.empty();
}
Edit: Below answer does not solve the issue, but it contains useful information about Thread switching. It does not work because .block() is no problem for non-blocking Schedulers if it's used to switch to synchronous code.
This is because the block operator inherited the reactor-http-nio-3 Thread from backendApi.getItemANameApi (or one of the other calls in Mono.zip), which is non-blocking.
Most operators continue working on the Thread on which the previous operator executed, this is because the Thread is linked to the emitted item. There are two groups of operators where the Thread of the output item differs from the input:
flatMap, concatMap, zip, etc: Operators that emit items from other Publishers will keep the Thread link they received from this inner Publisher, not from the input.
Time based operators like delayElements, interval, buffer(Duration), etc. will schedule their tasks on the provided Scheduler, or Schedulers.parallel() if none provided. The emitted items will then be linked to the Thread the task was scheduled on.
In your case, Mono.zip emits items from backendApi.getItemANameApi linked to reactor-http-nio-3, which gets propagated downstream, goes outside both the flatMap in searchItems and in itemFlux, until it reaches your block operator.
You can solve this by placing a .publishOn(Schedulers.boundedElastic()), either in searchItem, searchItems or itemFlux. This will cause the item to switch to a Thread in the provided Scheduler.
Bonus: Since you requested to pinpoint errors: Your Mono.zip will not work if sisr.isAddXXXDetails() is false, as Mono.zip discards any element it could not zip. Since you return a Mono.empty() in that case, no items can be zipped and it will return an empty Mono.
If we have only spring-boot-starter-webflux defined as application dependency, then springbok spin up a `Netty server.
One is not expected to block() in a reactive application using a non-blocking server.
However, once we add spring-boot-starter-web dependency then even with the presence of spring-boot-starter-webflux, springboot spinup a tomcat server. Which is a thread-per-request model and is expected to have blocking calls
So to solve my problem, all I had to do above is, to add spring-boot-starter-web dependency in pom.xml. After that applications is started in Tomcat
with timcat .collectList().block() works in Controller class to return the List<Item>.
Whereas with the Netty server I could return only Flux<Item> not List<Item>, which is expected.
i have a problem that i don't exactly know how to solve. I'm implementing an E2E test in which using selenium i need to click in a Link and check that is sending me to the right URL.
Here starts the problem...
There are 3 possibilities, mix of 2 types of links, just one type of link or the other type of link. No problems with the situations in which there are both types of links but when there is just one type when it searches for the identifier we use for the links that are not in page it gives me a timeoutException. This is not a failure because it's a posible situation but i will like to know if there is a way in which to check that if it finds no links it asserts that the exception is thrown.
I though using a runCatching (or try catch) wait for the link to appear and if it doesn't appear the test asserts that when i look for the element the timeout exception is thrown again.
It smells a bit for me this way of doing it and i don't know if it's correctly done.
EDIT: Im ussing AssertK and JUnit5 for testing.
EDIT 2: I've done this, i dont know if it a correct way of doing it
runCatching {
driver.waitFor(numberOfWidgetsToBeMoreThan(BrowserSelector.cssSelector(OFFERS_WITH_PRICE_AND_DATE), 0), ofMillis(2000))
}.onFailure {
assertThrows<WaitTimeoutException> {
findLink(OFFERS_WITH_PRICE_AND_DATE)
}
}.onSuccess {
val widget = findLink(OFFERS_WITH_PRICE_AND_DATE)
widget.click()
assertThat(driver.url).contains(NO_DATE_TEXT)
}
I'm not sure I understood your problem correctly, but you can use assertFails to assert that a piece of code throws an exception:
#Test
fun test() {
val exception = assertFails {
// some code that should throw
}
// some more assertions on the type of exception etc. may go here
}
I'm writing a small, simple web app in go using the gorm ORM.
Since the database can fail independently of the web application, I'd like to be able to identify errors that correspond to this case so that I can reconnect to my database without restarting the web application.
Motivating example:
Consider the following code:
var mrs MyRowStruct
db := myDB.Model(MyRowStruct{}).Where("column_name = ?", value).First(&mrs)
return &mrs, db.Error
In the event that db.Error != nil, how can I programmatically determine if the error stems from a database connection problem?
From my reading, I understand that gorm.DB does not represent a connection, so do I even have to worry about reconnecting or re-issuing a call to gorm.Open if a database connection fails?
Are there any common patterns for handling database failures in Go?
Gorm appears to swallow database driver errors and emit only it's own classification of error types (see gorm/errors.go). Connection errors do not currently appear to be reported.
Consider submitting an issue or pull request to expose the database driver error directly.
[Original]
Try inspecting the runtime type of db.Error per the advice in the gorm readme "Error Handling" section.
Assuming it's an error type returned by your database driver you can likely get a specific code that indicates connection errors. For example, if you're using PostgreSQL via the pq library then you might try something like this:
import "github.com/lib/pq"
// ...
if db.Error != nil {
pqerr, ok := err.(*pq.Error)
if ok && pqerr.Code[0:2] == "08" {
// PostgreSQL "Connection Exceptions" are class "08"
// http://www.postgresql.org/docs/9.4/static/errcodes-appendix.html#ERRCODES-TABLE
// Do something for connection errors...
} else {
// Do something else with non-pg error or non-connection error...
}
}
In regards to Error handling in PHP -- As far I know there are 3 styles:
die()or exit() style:
$con = mysql_connect("localhost","root","password");
if (!$con) {
die('Could not connect: ' . mysql_error());
}
throw Exception style:
if (!function_exists('curl_init')) {
throw new Exception('need the CURL PHP extension.
Recomplie PHP with curl');
}
trigger_error() style:
if(!is_array($config) && isset($config)) {
trigger_error('Error: config is not an array or is not set', E_USER_ERROR);
}
Now, in the PHP manual all three methods are used.
What I want to know is which style should I prefer & why?
Are these 3 drop in replacements of each other & therefore can be used interchangeably?
Slightly OT: Is it just me or everyone thinks PHP error handling options are just too many to the extent it confuses php developers?
The first one should never be used in production code, since it's transporting information irrelevant to end-users (a user can't do anything about "Cannot connect to database").
You throw Exceptions if you know that at a certain critical code point, your application can fail and you want your code to recover across multiple call-levels.
trigger_error() lets you fine-grain error reporting (by using different levels of error messages) and you can hide those errors from end-users (using set_error_handler()) but still have them be displayed to you during testing.
Also trigger_error() can produce non-fatal messages important during development that can be suppressed in production code using a custom error handler. You can produce fatal errors, too (E_USER_ERROR) but those aren't recoverable. If you trigger one of those, program execution stops at that point. This is why, for fatal errors, Exceptions should be used. This way, you'll have more control over your program's flow:
// Example (pseudo-code for db queries):
$db->query('START TRANSACTION');
try {
while ($row = gather_data()) {
$db->query('INSERT INTO `table` (`foo`,`bar`) VALUES(?,?)', ...);
}
$db->query('COMMIT');
} catch(Exception $e) {
$db->query('ROLLBACK');
}
Here, if gather_data() just plain croaked (using E_USER_ERROR or die()) there's a chance, previous INSERT statements would have made it into your database, even if not desired and you'd have no control over what's to happen next.
I usually use the first way for simple debugging in development code. It is not recommended for production. The best way is to throw an exception, which you can catch in other parts of the program and do some error handling on.
The three styles are not drop-in replacements for each other. The first one is not an error at all, but just a way to stop the script and output some debugging info for you to manually parse. The second one is not an error per se, but will be converted into an error if you don't catch it. The last one is triggering a real error in the PHP engine which will be handled according to the configuration of your PHP environment (in some cases shown to the user, in other cases just logged to a file or not saved at all).
What's the best practice for handling exceptions in NHibernate?
I've got a SubjectRepository with the following:
public void Add(Subject subject)
{
using (ISession session = HibernateUtil.CurrentSession)
using (ITransaction transaction = session.BeginTransaction())
{
session.Save(subject);
transaction.Commit();
}
}
And a Unit Test as follows:
[Test]
public void TestSaveDuplicate()
{
var subject = new Subject
{
Code = "En",
Name = "English"
};
_subjectRepository.Add(subject);
var duplicateSubject = new Subject
{
Code = "En",
Name = "English1"
};
_subjectRepository.Add(duplicateSubject);
}
I got to the point of handling the error generated by the unit test and got a bit stuck. This fails as expected, though with a GenericADOException, I was expecting a ConstraintViolationException or something similar (there is a uniqueness constraint on the subject code at database level).
The ADOException wraps a MySQL Exception that has a sensible error message but I don't want to start breaking encapsulation by just throwing the inner exception. Particularly as MySQL isn't finalised as the back end for this project.
Ideally I'd like to be able to catch the exception and return a sensible error to the user at this point. Are there any documented best practice approaches to handling NHibernate Exceptions and reporting back up to the user what went wrong and why?
Thanks,
Matt
I would handle it in the Add method as such:
public void Add(Subject subject)
{
using (ISession session = HibernateUtil.CurrentSession)
using (ITransaction transaction = session.BeginTransaction())
{
try
{
session.Save(subject);
transaction.Commit();
}
catch (Exception ex)
{
transaction.Rollback();
// log exception
throw;
}
}
}
In the catch block, you should first rollback the transaction and log the exception. Then your options are:
Rethrow the same exception, which is what my version does
Wrap it in your own exception and throw that
Swallow the exception by doing nothing, which is very rarely a good idea
You don't have any real options for handling the exception in this method. Assuming that the UI calls this method, it should call it in its own try..catch and handle it by displaying a meaningful error message to the user. You can make your unit test pass by using the ExpectedException(type) attribute.
To answer your question directly, you should create your own "sensible error" by extending Exception and throw that with the original exception as its InnerException. That's the exception wrapping technique I listed in (2).
All the Nhibernate exceptions are non recoverable, you could revisit the design of the app/data layer if you are trying to recover from nhibernate exceptions .
You can also Take a look at spring.net 's exception translation implementaion
Also you manually handling transactions on exceptions is tedious and error prone, take a look at nhibernate's contextual sessions .
Spring.net also has some nice helpers around nhibernate .
The general question is going to be, what do you want to tell the user, and who is the user?
If the user will sometimes be another computer (i.e., this is a web service), then you would want to use the appropriate mechanism to return a SOAP Fault or HTTP error.
If the user will sometimes be a UI of some sort, then you may want to display a message to the user, but what would you tell the user so he can do something about it? For instance, most web sites will say, "sorry, we had an unexpected error", no matter what the reason. That's because there's usually nothing the user could do about the error.
But in either case, the choice of how to tell "the user" is a matter for the Presentation layer (UI tier), not for the DAL. You should possibly wrap exceptions from the DAL in another exception type, but only if you're going to change the message. You don't need your own exception class, unless your callers would do something different if it's a data access exception rather than some other kind.
I'd probably validate the input before saving the object; that way you can implement whatever validation you like (e.g. check the length of the Subject Code as well as the fact that there aren't any duplicates), and pass back meaningful validation errors back to the user.
The logic is as follows; exceptions are used to indicate exceptional circumstances that your program doesn't cater for. A user entering a duplicate Subject Code in your example above is something your program should be catering for; so, rather than handling an exception because a DB constraint gets violated (which is an exceptional event and shouldn't be occurring), you'd want to handle that scenario first and only attempt to save the data when you know that the data you're saving is correct.
The advantage with implementing all validation rules in your DAL is that you can then ensure that the data going into your DB is valid as per your business processes in a consistent manner, rather than relying on the constraints in your DB to catch those for you.