I'm writing a small, simple web app in go using the gorm ORM.
Since the database can fail independently of the web application, I'd like to be able to identify errors that correspond to this case so that I can reconnect to my database without restarting the web application.
Motivating example:
Consider the following code:
var mrs MyRowStruct
db := myDB.Model(MyRowStruct{}).Where("column_name = ?", value).First(&mrs)
return &mrs, db.Error
In the event that db.Error != nil, how can I programmatically determine if the error stems from a database connection problem?
From my reading, I understand that gorm.DB does not represent a connection, so do I even have to worry about reconnecting or re-issuing a call to gorm.Open if a database connection fails?
Are there any common patterns for handling database failures in Go?
Gorm appears to swallow database driver errors and emit only it's own classification of error types (see gorm/errors.go). Connection errors do not currently appear to be reported.
Consider submitting an issue or pull request to expose the database driver error directly.
[Original]
Try inspecting the runtime type of db.Error per the advice in the gorm readme "Error Handling" section.
Assuming it's an error type returned by your database driver you can likely get a specific code that indicates connection errors. For example, if you're using PostgreSQL via the pq library then you might try something like this:
import "github.com/lib/pq"
// ...
if db.Error != nil {
pqerr, ok := err.(*pq.Error)
if ok && pqerr.Code[0:2] == "08" {
// PostgreSQL "Connection Exceptions" are class "08"
// http://www.postgresql.org/docs/9.4/static/errcodes-appendix.html#ERRCODES-TABLE
// Do something for connection errors...
} else {
// Do something else with non-pg error or non-connection error...
}
}
Related
I'm learning go and trying to understand how to get more detailed error information out of the generic error type. The example I'll use is from the net package, specifically the DialTimeout function.
The signature is
func DialTimeout(network, address string, timeout time.Duration) (Conn, error)
The error type only defines an Error() string function. If I want to find out exactly why DialTimeout failed, how can I get that information? I found out that I can use type assertion to get the net.Error specific error:
con, err := net.DialTimeout("tcp", net.JoinHostPort(address, "22"),
time.Duration(5) * time.Second)
if err != nil {
netErr, ok := err.(net.Error)
if ok && netErr.Timeout() {
// ...
}
}
but that only tells me whether or not I had a timeout. For example, say I wanted to differentiate between a refused connection and no route to host. How can I do that?
Maybe DialTimeout is too high-level to give me that kind of detail, but even looking at syscall.Connect, I don't see how to get the specific error. It just says it returns the generic error type. Compare that to the Posix connect, which will let me know why it failed with the various return codes.
My general question is: how am I supposed to pull out error details from the generic error type if the golang docs don't tell me what type of errors may be returned?
Most networking operations return a *OpError which holds detailed information about the error
and implements the net.Error interface. So for most use cases it is sufficient to use net.Error
as you already did.
But for your case you'd want to assert the returned error to be *net.OpError and
use the internal error:
if err != nil {
if oerr, ok := err.(*OpError); ok {
// Do something with oerr.Err
}
}
As soon as you're doing this you are in the land of platform dependency as syscalls under Linux
can fail differently to those under Windows. For Linux you'd do something like this:
if oerr.Err == syscall.ECONNREFUSED {
// Connection was refused :(
}
The syscall package contains the important error constants for your platform. Unfortunately
the golang website only shows the syscall package for Linux amd64. See here for ECONNREFUSED.
Finding out types
The next time you're wondering what is actually returned by some function and you can't make
heads and tails of it, try using the %#v format specified in fmt.Printf (and friends):
fmt.Printf("%#v\n", err)
// &net.OpError{Op:"dial", Net:"tcp", Addr:(*net.TCPAddr)(0xc20006d390), Err:0x6f}
It will print detailed type information and is generally quite helpful.
I'm using SQL Azure in a Windows Azure app running as a cloud service. Most of the time my database actions works completely fine (that is, after handling all sorts of timeouts and what not), however i'm running into a problem that seems
using (var connection = new SqlConnection(m_connectionString))
{
m_ConnectionRetryPolicy.ExecuteAction(() => connection.Open());
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT * FROM X WHERE Y = Z";
var reader = m_CommandRetryPolicy.ExecuteAction(() => command.ExecuteReader());
return LoadData(reader).FirstOrDefault();
}
}
The line that fails is the Command.ExecuteReader with an:
ExecuteReader requires an open and available Connection. The connection's current state is closed
Things that i have already considered
I'm not "reusing" an old connection or saving a connection is a member variable
There should be no concurrency issues - the repository class that these methods belong to is created each time it is needed
Have anyone else experienced this? I could of course just add this to the list of exception which would yield a retry, but I'm not very comfortable with that as
I had a bunch of these errors a few days ago (West Europe) on my production deployment, but they went away by themselves. At the same time I was seeing timeouts, throttling and other errors from SQL Azure. I assume that there was a temporary problem with the platform (or at least the server that I am running on).
You probably aren't doing anything wrong in your code, but are suffering from degraded performance on SQL Azure. Try and handle the errors, perform retries, exponential back-off, queues (to reduce concurrency), splitting your load across databases — that sort of thing.
write every thing within try and catch,finally block.
as follows:
try
{
con.open();
m_ConnectionRetryPolicy.ExecuteAction(() => connection.Open());
using (var command = connection.CreateCommand())
{
command.CommandText = "SELECT * FROM X WHERE Y = Z";
var reader = m_CommandRetryPolicy.ExecuteAction(() => command.ExecuteReader());
return LoadData(reader).FirstOrDefault();
}
con.close();
}
catch(exception ex)
{
}
finally
{
con.close();
}
Remember to close connection in finally block as well.
There is an Enterprise Library that MS has produced specifically for SQL Azure, here are some examples from their patterns and Practice.
It's similar to what you are doing, however it does more on the reliability (and these examples show how to get a reliable connection)
http://msdn.microsoft.com/en-us/library/hh680899(v=pandp.50).aspx
Are you sure it's the reader that's failing and not the opening of the connection? I'm encountering an exception when I wrap the connection.Open() in the m_ConnectionRetryPolicy.ExecuteAction().
However it works just fine for me if I skip the ExecuteAction wrapper and open the connection using connection.OpenWithRetry(m_ConnectionRetryPolicy).
And I'm also using command.ExecuteReaderWithRetry(m_ConnectionRetryPolicy) which is working for me.
I have no idea though why it's not working when wrapped in ExecuteAction though.
I believe this means that Azure has closed the connection behind the scenes, without telling the connection pooler. This is by design. So, the connection pooler gives you what it thinks is an available, open connection, but when you try to use it, it finds out it's not open after all.
This seems very clunky to me, but it's the way Azure is at the moment.
In regards to Error handling in PHP -- As far I know there are 3 styles:
die()or exit() style:
$con = mysql_connect("localhost","root","password");
if (!$con) {
die('Could not connect: ' . mysql_error());
}
throw Exception style:
if (!function_exists('curl_init')) {
throw new Exception('need the CURL PHP extension.
Recomplie PHP with curl');
}
trigger_error() style:
if(!is_array($config) && isset($config)) {
trigger_error('Error: config is not an array or is not set', E_USER_ERROR);
}
Now, in the PHP manual all three methods are used.
What I want to know is which style should I prefer & why?
Are these 3 drop in replacements of each other & therefore can be used interchangeably?
Slightly OT: Is it just me or everyone thinks PHP error handling options are just too many to the extent it confuses php developers?
The first one should never be used in production code, since it's transporting information irrelevant to end-users (a user can't do anything about "Cannot connect to database").
You throw Exceptions if you know that at a certain critical code point, your application can fail and you want your code to recover across multiple call-levels.
trigger_error() lets you fine-grain error reporting (by using different levels of error messages) and you can hide those errors from end-users (using set_error_handler()) but still have them be displayed to you during testing.
Also trigger_error() can produce non-fatal messages important during development that can be suppressed in production code using a custom error handler. You can produce fatal errors, too (E_USER_ERROR) but those aren't recoverable. If you trigger one of those, program execution stops at that point. This is why, for fatal errors, Exceptions should be used. This way, you'll have more control over your program's flow:
// Example (pseudo-code for db queries):
$db->query('START TRANSACTION');
try {
while ($row = gather_data()) {
$db->query('INSERT INTO `table` (`foo`,`bar`) VALUES(?,?)', ...);
}
$db->query('COMMIT');
} catch(Exception $e) {
$db->query('ROLLBACK');
}
Here, if gather_data() just plain croaked (using E_USER_ERROR or die()) there's a chance, previous INSERT statements would have made it into your database, even if not desired and you'd have no control over what's to happen next.
I usually use the first way for simple debugging in development code. It is not recommended for production. The best way is to throw an exception, which you can catch in other parts of the program and do some error handling on.
The three styles are not drop-in replacements for each other. The first one is not an error at all, but just a way to stop the script and output some debugging info for you to manually parse. The second one is not an error per se, but will be converted into an error if you don't catch it. The last one is triggering a real error in the PHP engine which will be handled according to the configuration of your PHP environment (in some cases shown to the user, in other cases just logged to a file or not saved at all).
In a previous ticket i asked about logging PHP errors in MySQL which gives me:
function myErrorHandler($errno, $errstr, $errfile, $errline)
{
// mysql connect etc here...
$sql = "INSERT INTO `error_log` SET
`number` = ".mysql_real_escape_string($errno).",
`string` = ".mysql_real_escape_string($errstr).",
`file` = ".mysql_real_escape_string($errfile).",
`line` = ".mysql_real_escape_string($errline);
mysql_query($sql);
// Don't execute PHP internal error handler
return true;
}
// set to the user defined error handler
$new_error_handler = set_error_handler("myErrorHandler");
I can make this work but only if it is triggerred like this:
trigger_error("message here");
However, I also want the error handler to be called for all errors such as syntax errors like:
echo "foo;
But these errors are just outputted to the screen, what am i doing wrong?
You can only handle runtime errors with a custom error handler. The echo "foo error in your example happens when parsing (i.e. reading in) the source. Since PHP can not fully parse the code, it can also not run your error handler on this error.
If You're forced to test if syntax is correct, You can use php_check_syntax function, with filename parameter PHP Manual php_check_syntax
php_check_syntax also provides second parameter, witch when used will be populated by the error string, as far as i remember
That's indeed terrible way of error logging
You don't need not a single advantage of a database. Would you make a database lookup for the certain line number? Or order your results by file name?
database is a subject of many errors itself.
You've been told already that it's impossible to catch a parse error at the program logic level, because a syntactically wrong program will never run.
Let's take your code as an example. It will raise a MySQL error (because of poorly formed query) which you will never see. As well as any other errors occurred. That's what I am talking about.
What's the best practice for handling exceptions in NHibernate?
I've got a SubjectRepository with the following:
public void Add(Subject subject)
{
using (ISession session = HibernateUtil.CurrentSession)
using (ITransaction transaction = session.BeginTransaction())
{
session.Save(subject);
transaction.Commit();
}
}
And a Unit Test as follows:
[Test]
public void TestSaveDuplicate()
{
var subject = new Subject
{
Code = "En",
Name = "English"
};
_subjectRepository.Add(subject);
var duplicateSubject = new Subject
{
Code = "En",
Name = "English1"
};
_subjectRepository.Add(duplicateSubject);
}
I got to the point of handling the error generated by the unit test and got a bit stuck. This fails as expected, though with a GenericADOException, I was expecting a ConstraintViolationException or something similar (there is a uniqueness constraint on the subject code at database level).
The ADOException wraps a MySQL Exception that has a sensible error message but I don't want to start breaking encapsulation by just throwing the inner exception. Particularly as MySQL isn't finalised as the back end for this project.
Ideally I'd like to be able to catch the exception and return a sensible error to the user at this point. Are there any documented best practice approaches to handling NHibernate Exceptions and reporting back up to the user what went wrong and why?
Thanks,
Matt
I would handle it in the Add method as such:
public void Add(Subject subject)
{
using (ISession session = HibernateUtil.CurrentSession)
using (ITransaction transaction = session.BeginTransaction())
{
try
{
session.Save(subject);
transaction.Commit();
}
catch (Exception ex)
{
transaction.Rollback();
// log exception
throw;
}
}
}
In the catch block, you should first rollback the transaction and log the exception. Then your options are:
Rethrow the same exception, which is what my version does
Wrap it in your own exception and throw that
Swallow the exception by doing nothing, which is very rarely a good idea
You don't have any real options for handling the exception in this method. Assuming that the UI calls this method, it should call it in its own try..catch and handle it by displaying a meaningful error message to the user. You can make your unit test pass by using the ExpectedException(type) attribute.
To answer your question directly, you should create your own "sensible error" by extending Exception and throw that with the original exception as its InnerException. That's the exception wrapping technique I listed in (2).
All the Nhibernate exceptions are non recoverable, you could revisit the design of the app/data layer if you are trying to recover from nhibernate exceptions .
You can also Take a look at spring.net 's exception translation implementaion
Also you manually handling transactions on exceptions is tedious and error prone, take a look at nhibernate's contextual sessions .
Spring.net also has some nice helpers around nhibernate .
The general question is going to be, what do you want to tell the user, and who is the user?
If the user will sometimes be another computer (i.e., this is a web service), then you would want to use the appropriate mechanism to return a SOAP Fault or HTTP error.
If the user will sometimes be a UI of some sort, then you may want to display a message to the user, but what would you tell the user so he can do something about it? For instance, most web sites will say, "sorry, we had an unexpected error", no matter what the reason. That's because there's usually nothing the user could do about the error.
But in either case, the choice of how to tell "the user" is a matter for the Presentation layer (UI tier), not for the DAL. You should possibly wrap exceptions from the DAL in another exception type, but only if you're going to change the message. You don't need your own exception class, unless your callers would do something different if it's a data access exception rather than some other kind.
I'd probably validate the input before saving the object; that way you can implement whatever validation you like (e.g. check the length of the Subject Code as well as the fact that there aren't any duplicates), and pass back meaningful validation errors back to the user.
The logic is as follows; exceptions are used to indicate exceptional circumstances that your program doesn't cater for. A user entering a duplicate Subject Code in your example above is something your program should be catering for; so, rather than handling an exception because a DB constraint gets violated (which is an exceptional event and shouldn't be occurring), you'd want to handle that scenario first and only attempt to save the data when you know that the data you're saving is correct.
The advantage with implementing all validation rules in your DAL is that you can then ensure that the data going into your DB is valid as per your business processes in a consistent manner, rather than relying on the constraints in your DB to catch those for you.