Handling connection reset errors in Go - error-handling

On a plain Go HTTP handler, if I disconnect a client while still writting to the response, http.ResponseWritter.Write will return an error with a message like write tcp 127.0.0.1:60702: connection reset by peer.
Now from the syscall package, I have sysca.ECONNRESET, which has the message connection reset by peer, so they're not exactly the same.
How can I match them, so I know not to panic if it occurs ? On other ocasions I have been doing
if err == syscall.EAGAIN {
/* handle error differently */
}
for instance, and that worked fine, but I can't do it with syscall.ECONNRESET.
Update:
Because I'm desperate for a solution, for the time being I'll be doing this very dirty hack:
if strings.Contains(err.Error(), syscall.ECONNRESET.Error()) {
println("it's a connection reset by peer!")
return
}

The error you get has the underlying type *net.OpError, built here, for example.
You should be able to type-assert the error to its concrete type like this:
operr, ok := err.(*net.OpError)
And then access its Err field, which should correspond to the syscall error you need:
operr.Err.Error() == syscall.ECONNRESET.Error()

The answer by #zian is more useful than the accepted answer, but now on Go 1.13+ it is preferable to avoid manually unwrapping the errors:
if errors.Is(opErr,syscall.ECONNRESET) {
fmt.Println("Found a ECONNRESET")
}
This has the benefit that you can also use it more generally, such as after:
resp, err := http.Get("http://127.0.0.1:4444")
Here this err would otherwise have an extra layer of wrapping (*url.Error) and would be missed by the condition #zian used without explicitly unwrapping it a third time.

I came across this issue and the accepted answer was sufficient to point me in the right direction. However, the code it provides to check if the Error embedded inside *net.OpError is ECONNRESET is not complete, at least not for Golang 1.9.
The error embedded at OpError.Err is actually of type *os.SyscallError (https://golang.org/pkg/os/#SyscallError). The Write() function implemented by struct *net.netFD (which is what's being written to when sending a response over the network) looks like this:
func (fd *netFD) Write(p []byte) (nn int, err error) {
nn, err = fd.pfd.Write(p)
runtime.KeepAlive(fd)
return nn, wrapSyscallError("write", err)
}
And wrapSyscallError:
func wrapSyscallError(name string, err error) error {
if _, ok := err.(syscall.Errno); ok {
err = os.NewSyscallError(name, err)
}
return err
}
The error inside the *os.SyscallError struct can be directly compared against syscall.ECONNRESET.
So, given an error returned from a network write (e.g. a call to http.ResponseWritter.Write), the full code block to determine if that error is ECONNRESET is:
if opErr, ok := err.(*net.OpError); ok {
if syscallErr, ok := opErr.Err.(*os.SyscallError); ok {
if syscallErr.Err == syscall.ECONNRESET {
fmt.Println("Found a ECONNRESET")
}
}
}

#zian - thanks for your good solution to João Pinto's (and my) question : How can I match them, so I know not to panic if it occurs ?
As at go version 1.13, an improvement is to use the errors.Is function which does error unwrapping and testing sequentially 'under the hood'. For example :
if errors.Is(opErr,syscall.ECONNRESET) {
fmt.Println("Found a ECONNRESET")
}
#SteveCoffman - adding to your good answer, cheers!
Working with Errors in Go 1.13 - The Go Blog - Golang

Related

Testing log.Fatalf in go?

I'd like to achieve 100% test coverage in go code. I am not able to cover the following example - can anyone help me with that?
package example
import (
"io/ioutil"
"log"
)
func checkIfReadable(filename string) (string, error) {
_, err := ioutil.ReadFile(filename)
if err != nil {
log.Fatalf("Cannot read the file... how to add coverage test for this line ?!?")
}
return "", nil
}
func main() {
checkIfReadable("dummy.txt")
}
Some dumy test for that:
package example
import (
"fmt"
"testing"
)
func TestCheckIfReadable(t *testing.T) {
someResult, err := checkIfReadable("dummy.txt")
if len(someResult) > 0 {
fmt.Println("this will not print")
t.Fail()
}
if err != nil {
fmt.Println("this will not print")
t.Fail()
}
}
func TestMain(t *testing.T) {
...
}
The issue is that log.Fatalf calls os.Exit and go engine dies.
I could modify the code and replace built-in library with my own - what makes the tests less reliable.
I could modify the code and create a proxy and a wrapper and a .... in other words very complex mechanism to change all calls to log.Fatalf
I could stop using built-in log package... what is equal to asking "how much is go built-in worth?"
I could live with not having 100% coverage
I could replace log.Fataf with something else - but then what is the point for built-in log.Fatalf?
I can try to mangle with system memory and depending on my OS replace memory address for the function (...) so do something obscure and dirty
Any other ideas?
Use log.Print instead of log.Fatal and return the error value that you declared in signature of function checkIfReadable. Or don't the error it and return it to some place that knows better how to handle it.
The function log.Fatal is strictly for reporting your program's final breath.
Calling log.Fatal is a bit worse than calling panic (there is also log.panic), because it does not execute deferred calls. Remember, that overusing panic in Go is considered a bad style.
A good way to get 100% test coverage and not fail at the same time is to use recover() to catch the panic that is thrown by log.Fatalf().
Here are the docs for recover. I think it fits your use case nicely.

How would I test this method?

Essentially I've begun to work on a wrapper for the Riot Games API and I'm struggling with how to test it. I've got the repository plugged into Travis so on push it runs go test but I'm not sure how to go about testing it since the API_KEY required for the requests changes daily and I can't auto-regenerate it, i'd have to manually add it every day if I tested the endpoints directly.
So I was wondering if it was possible to mock the responses, but in that case I'm guessing I'd need to refactor my code?
So i've made a struct to represent their SummonerDTO
type Summoner struct {
ID int64 `json:"id"`
AccountID int64 `json:"accountId"`
ProfileIconID int `json:"profileIconId"`
Name string `json:"name"`
Level int `json:"summonerLevel"`
RevisionDate int64 `json:"revisionDate"`
}
That struct has a method:
func (s Summoner) ByName(name string, region string) (summoner *Summoner, err error) {
endpoint := fmt.Sprintf("https://%s.api.riotgames.com/lol/summoner/%s/summoners/by-name/%s", REGIONS[region], VERSION, name)
client := &http.Client{}
req, err := http.NewRequest("GET", endpoint, nil)
if err != nil {
return nil, fmt.Errorf("unable to create new client for request: %v", err)
}
req.Header.Set("X-Riot-Token", API_KEY)
resp, err := client.Do(req)
if err != nil {
return nil, fmt.Errorf("unable to complete request to endpoint: %v", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
return nil, fmt.Errorf("request to api failed with: %v", resp.Status)
}
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, fmt.Errorf("unable to read response body: %v", err)
}
if err := json.Unmarshal([]byte(respBody), &summoner); err != nil {
return nil, fmt.Errorf("unable to unmarshal response body to summoner struct: %v", err)
}
return summoner, nil
}
Is it a case that the struct method doesn't have a single responsibility, in a sense it's building the endpoint, firing off the request and parsing the response. Do I need to refactor it in order to make it testable, and in which case what's the best approach for that? Should I make a Request and Response struct and then test those?
To clarify the API Keys used are rate limited and need to be regenerated daily, Riot Games do not allow you to use a crawler to auto-regenerate your keys. I'm using Travis for continuous integration so I'm wondering if there's a way to mock the request/response.
Potentially my approach is wrong, still learning.
Hopefully that all makes some form of sense, happy to clarify if not.
Writing unit tests consists of:
Providing known state for all of your inputs.
Testing that, given all meaning combinations of those inputs, you receive the expected outputs.
So you need to first identify your inputs:
s Summoner
name string
region string
Plus any "hidden" inputs, by way of globals:
client := &http.Client{}
And your outputs are:
summoner *Summoner
err error
(There can also be "hidden" outputs, if you write files, or change global variables, but you don't appear to do that here).
Now the first three inputs are easy to create from scratch for your tests: Just provide an empty Summoner{} (since you don't read or set that at all in your function, there's no need to set it other than to an empty value). name and region can simply be set to strings.
The only part remaining is your http.Client. At minimum, you should probably pass that in as an argument. Not only does this give you control over your tests, but it allows you to use easily use different client in production in the future.
But to ease testing, you might consider actually passing in a client-like interface, which you can easily mock. The only method you call on client is Do, so you could easily create a Doer interface:
type doer interface {
Do(req *Request) (*Response, error)
}
Then change your function signature to take that as one argument:
func (s Summoner) ByName(client doer, name string, region string) (summoner *Summoner, err error) {
Now, in your test you can create a custom type that fulfills the doer interface, and responds with any http.Response you like, without needing to use a server in your tests.

How to maintain good Go package test coverage when dealing with obscure errors?

I'm trying to maintain 100% code coverage on some of my Go packages. This isn't viable everywhere, even with some tests that I select with a -integration build tag on a build system, but it should be possible for my relatively isolated library packages.
I'm having trouble dealing coverage for obscure error paths, though.
Here is an example of one of my methodss that's part of an integration test where there's a real filesystem:
func (idx Index) LoadPost(title string) (*PostSpec, string, error) {
postFolder := strings.Replace(strings.ToLower(title), " ", "_", -1)
spec, err := idx.getSpec(postFolder)
if err != nil {
return nil, "", err
}
f, err := os.Open(path.Join(idx.blogdir, postFolder, "content.html"))
if err != nil {
return nil, "", err
}
defer f.Close()
b, err := ioutil.ReadAll(f)
if err != nil {
return nil, "", err
}
return spec, string(b), nil
}
Here's what it looks like in go tool -cover:
Hitting that block is not easy. I can't think of any way to do it other than creating a special test directory where the file it's trying to open is something other than a regular file. That seems like a lot of complexity.
This isn't too much of a deal on its own, but it means that I have to remember that 97.3% coverage is the right figure. If I see that number go down, does it mean I've broken my tests and there's now more uncovered code? Or just that I've managed to improve my package through simplification and removal or dead code? It leads to second guessing.
More importantly to some, in a business context it's an obstacle to a nice build dashboard.
io/ioutil/ioutil_test.go does test that error simply by calling ioutil.ReadFile() function with a non-existing file.
That shouldn't require any setup.
filename := "rumpelstilzchen"
contents, err := ReadFile(filename)
if err == nil {
t.Fatalf("ReadFile %s: error expected, none found", filename)
}

How do you get a Golang program to print the line number of the error it just called?

I was trying to throw errors in my Golang program with log.Fatal but, log.Fatal does not also print the line where the log.Fatal was ran. Is there no way of getting access to the line number that called log.Fatal? i.e. is there a way to get the line number when throwing an error?
I was trying to google this but was unsure how. The best thing I could get was printing the stack trace, which I guess is good but might be a little too much. I also don't want to write debug.PrintStack() every time I need the line number, I am just surprised there isn't any built in function for this like log.FatalStackTrace() or something that isn't costume.
Also, the reason I do not want to make my own debugging/error handling stuff is because I don't want people to have to learn how to use my special costume handling code. I just want something standard where people can read my code later and be like
"ah ok, so its throwing an error and doing X..."
The less people have to learn about my code the better :)
You can set the Flags on either a custom Logger, or the default to include Llongfile or Lshortfile
// to change the flags on the default logger
log.SetFlags(log.LstdFlags | log.Lshortfile)
Short version, there's nothing directly built in, however you can implement it with a minimal learning curve using runtime.Caller
func HandleError(err error) (b bool) {
if err != nil {
// notice that we're using 1, so it will actually log where
// the error happened, 0 = this function, we don't want that.
_, filename, line, _ := runtime.Caller(1)
log.Printf("[error] %s:%d %v", filename, line, err)
b = true
}
return
}
//this logs the function name as well.
func FancyHandleError(err error) (b bool) {
if err != nil {
// notice that we're using 1, so it will actually log the where
// the error happened, 0 = this function, we don't want that.
pc, filename, line, _ := runtime.Caller(1)
log.Printf("[error] in %s[%s:%d] %v", runtime.FuncForPC(pc).Name(), filename, line, err)
b = true
}
return
}
func main() {
if FancyHandleError(fmt.Errorf("it's the end of the world")) {
log.Print("stuff")
}
}
playground
If you need exactly a stack trace, take a look at https://github.com/ztrue/tracerr
I created this package in order to have both stack trace and source fragments to be able to debug faster and log errors with much more details.
Here is a code example:
package main
import (
"io/ioutil"
"github.com/ztrue/tracerr"
)
func main() {
if err := read(); err != nil {
tracerr.PrintSourceColor(err)
}
}
func read() error {
return readNonExistent()
}
func readNonExistent() error {
_, err := ioutil.ReadFile("/tmp/non_existent_file")
// Add stack trace to existing error, no matter if it's nil.
return tracerr.Wrap(err)
}
And here is the output:

Exit with error code in go?

What's the idiomatic way to exit a program with some error code?
The documentation for Exit says "The program terminates immediately; deferred functions are not run.", and log.Fatal just calls Exit. For things that aren't heinous errors, terminating the program without running deferred functions seems extreme.
Am I supposed to pass around some state that indicate that there's been an error, and then call Exit(1) at some point where I know that I can exit safely, with all deferred functions having been run?
I do something along these lines in most of my real main packages, so that the return err convention is adopted as soon as possible, and has a proper termination:
func main() {
if err := run(); err != nil {
fmt.Fprintf(os.Stderr, "error: %v\n", err)
os.Exit(1)
}
}
func run() error {
err := something()
if err != nil {
return err
}
// etc
}
In Python I commonly use a pattern, which being converted to Go looks like this:
func run() int {
// here goes
// the code
return 1
}
func main() {
os.Exit(run())
}
I think the most clear way to do it is to set the exitCode at the top of main, then defer closing as the next step. That lets you change exitCode anywhere in main, and it's last value will be exited with:
package main
import (
"fmt"
"os"
)
func main() {
exitCode := 0
defer func() { os.Exit(exitCode) }()
// Do whatever, including deferring more functions
defer func() {
fmt.Printf("Do some cleanup\n")
}()
func() {
fmt.Printf("Do some work\n")
}()
// But let's say something went wrong
exitCode = 1
// Do even more work/cleanup if you want
// At the end, os.Exit will be called with the last value of exitCode
}
Output:
Do some work
Do some cleanup
Program exited: status 1.
Go Playgroundhttps://play.golang.org/p/AMUR4m_A9Dw
Note that an important disadvantage of this is that you don't exit the process as soon as you set the error code.
As mentioned by fas, you have func Exit(exitcode int) from the os package.
However, if you need the defered function to be applied, you always can use the defer keyword like this:
http://play.golang.org/p/U-hAS88Ug4
You perform all your operation, affect a error variable and at the very end, when everything is cleaned up, you can exit safely.
Otherwise, you could also use panic/recover:
http://play.golang.org/p/903e76GnQ-
When you have an error, you panic, end you cleanup where you catch (recover) it.
Yes, actually. The os package provides this.
package main
import "os"
func main() {
os.Exit(1)
}
http://golang.org/pkg/os/#Exit
Edit: so it looks like you know of Exit. This article gives an overview of Panic which will let deferred functions run before returning. Using this in conjunction with an exit may be what you're looking for. http://blog.golang.org/defer-panic-and-recover
Another good way I follow is:
if err != nil {
// log.Fatal will print the error message and will internally call System.exit(1) so the program will terminate
log.Fatal("fatal error message")
}