How to check a log/output in go test? - testing

I have this function that logs the error in some cases:
func readByte(/*...*/){
// ...
if err != nil {
fmt.Println("ERROR")
log.Print("Couldn't read first byte")
return
}
// ...
}
Now, in the test file, I want to check the output error from this function:
c.Assert(OUTPUT, check.Matches, "teste")
How can I access the log? I tried to put a buffer but it didn't work. What is the right way to catch this log without change my readByte function code?

For example,
readbyte_test.go:
package main
import (
"bytes"
"fmt"
"io"
"log"
"os"
"testing"
)
func readByte( /*...*/ ) {
// ...
err := io.EOF // force an error
if err != nil {
fmt.Println("ERROR")
log.Print("Couldn't read first byte")
return
}
// ...
}
func TestReadByte(t *testing.T) {
var buf bytes.Buffer
log.SetOutput(&buf)
defer func() {
log.SetOutput(os.Stderr)
}()
readByte()
t.Log(buf.String())
}
Output:
$ go test -v readbyte_test.go
=== RUN TestReadByte
ERROR
--- PASS: TestReadByte (0.00s)
readbyte_test.go:30: 2017/05/22 16:41:00 Couldn't read first byte
PASS
ok command-line-arguments 0.004s
$

Answer for Concurrent Tests
If your test is running concurrently (for example, when testing an http Server or Client), you may encounter a race between writing to the buffer and reading from it. Instead of the buffer, we can redirect output to an os.Pipe and use a bufio.Scanner to block until output has been written by using the Scan() method.
Here is an example of creating an os.Pipe and setting the stdlib log package to use the pipe. Note my use of the testify/assert package here:
func mockLogger(t *testing.T) (*bufio.Scanner, *os.File, *os.File) {
reader, writer, err := os.Pipe()
if err != nil {
assert.Fail(t, "couldn't get os Pipe: %v", err)
}
log.SetOutput(writer)
return bufio.NewScanner(reader), reader, writer
}
The *os.File objects are returned so they can be properly closed with a deferred function. Here I'm just printing to stdout since if there was some strange error on close I personally wouldn't want to fail the test. However, this could easily be another call to t.Errorf or similar if you wanted:
func resetLogger(reader *os.File, writer *os.File) {
err := reader.Close()
if err != nil {
fmt.Println("error closing reader was ", err)
}
if err = writer.Close(); err != nil {
fmt.Println("error closing writer was ", err)
}
log.SetOutput(os.Stderr)
}
And then in your test you would have this pattern:
scanner, reader, writer := mockLogger(t) // turn this off when debugging or developing as you will miss output!
defer resetLogger(reader, writer)
// other setup as needed, getting some value for thing below
go concurrentAction()
scanner.Scan() // blocks until a new line is written to the pipe
got := scanner.Text() // the last line written to the scanner
msg := fmt.Sprintf("your log message with thing %v you care about", thing)
assert.Contains(t, got, msg)
And finally, the concurrentAction() function is calling a log function (or method if using a log.logger, the package actually behaves the same way with log.SetOutput() call above either way) like:
// doing something, getting value for thing
log.Printf("your log message with the thing %v you care about", thing)

Related

go-libp2p pubsub example not working in multithreaded environment

I am a beginner and am currently playing with the pubsub example from libp2p given here https://github.com/libp2p/go-libp2p/tree/master/examples/pubsub/basic-chat-with-rendezvous
I have been able to build the code and run the binary in different terminals and it works.
I am trying to automate this process from the main.go program itself where I can create a few threads to spin up new agents where they
publish messages to the network and the rest of the peers subscribe to it.
I have provided the modified code I have built currently but it doesnt seem to work. The peers cannot discover each other.
func main() {
help := flag.Bool("help", false, "Display Help")
cfg := parseFlags()
if *help {
fmt.Printf("Simple example for peer discovery using mDNS. mDNS is great when you have multiple peers in local LAN.")
fmt.Printf("Usage: \n Run './chat-with-mdns'\nor Run './chat-with-mdns -host [host] -port [port] -rendezvous [string] -pid [proto ID]'\n")
os.Exit(0)
}
fmt.Printf("[*] Listening on: %s with port: %d\n", cfg.listenHost, cfg.listenPort)
var wg sync.WaitGroup
for i := 0; i < 5; i++ {
// Spawn a thread for each iteration in the loop.
// Pass 'i' into the goroutine's function
// in order to make sure each goroutine
// uses a different value for 'i'.
wg.Add(5)
go func(i int) {
// At the end of the goroutine, tell the WaitGroup
// that another thread has completed.
defer wg.Done()
ctx := context.Background()
r := rand.Reader
// Creates a new RSA key pair for this host.
prvKey, _, err := crypto.GenerateKeyPairWithReader(crypto.RSA, 2048, r)
if err != nil {
panic(err)
}
// 0.0.0.0 will listen on any interface device.
sourceMultiAddr, _ := multiaddr.NewMultiaddr(fmt.Sprintf("/ip4/%s/tcp/%d", cfg.listenHost, cfg.listenPort))
// libp2p.New constructs a new libp2p Host.
// Other options can be added here.
host, err := libp2p.New(
libp2p.ListenAddrs(sourceMultiAddr),
libp2p.Identity(prvKey),
)
if err != nil {
panic(err)
}
// Set a function as stream handler.
// This function is called when a peer initiates a connection and starts a stream with this peer.
host.SetStreamHandler(protocol.ID(cfg.ProtocolID), handleStream)
fmt.Printf("\n[*] Your Multiaddress Is: /ip4/%s/tcp/%v/p2p/%s\n", cfg.listenHost, cfg.listenPort, host.ID().Pretty())
peerChan := initMDNS(host, cfg.RendezvousString)
for { // allows multiple peers to join
peer := <-peerChan // will block untill we discover a peer // the code currently hangs here
fmt.Println("Found peer:", peer, ", connecting")
if err := host.Connect(ctx, peer); err != nil {
fmt.Println("Connection failed:", err)
continue
}
//** this part of the code is experimental and is not accessed by any thread yet **//
stream, err := host.NewStream(ctx, peer.ID, protocol.ID(cfg.ProtocolID))
if err != nil {
fmt.Println("Stream open failed", err)
} else {
rw := bufio.NewReadWriter(bufio.NewReader(stream), bufio.NewWriter(stream))
go writeData(rw)
go readData(rw)
fmt.Println("Connected to:", peer)
}
//** this part of the code is experimental and is not accessed by any thread yet **//
}
}(i)
}
fmt.Println("exit")
wg.Wait()
fmt.Println("Finished for loop")
}
But this doesn't seem to work. Are there any examples I can look at currently for solving this error.

how to deal with the "fmt" golang library package for CLI testing

Disclaimer: I wish you a merry XMas and I hope my question does not disturb you!
sample.go:
package main
import(
"fmt"
"os"
)
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Println("Error: something is wrong!")
os.Exit(1)
} else {
fmt.Println("May the force be with you!")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: something is wrong!
// exit status 1
I did a lot of research on how to use interfaces in golang testing. But so far I was not able to wrap my head around this completely. At least I can not see how interfaces help me when I need to "mock" (apologies for using this word) golang std. library packages like "fmt".
I came up with two scenarios:
use os/exec to test the command line interface
wrap fmt package so I have control and am able to check the output strings
I do not like both scenarios:
I experience going through the actual command line a convoluted and not-performant (see below). Might have portability issues, too.
I believe this is the way to go but I fear that wrapping the fmt package might be a lot of work (at least wrapping the time package for testing turned out a non-trivial task (https://github.com/finklabs/ttime)).
Actual Question here: Is there another (better/simpler/idiomatic) way?
Note: I want to do this in pure golang, I am not interested in the next testing framework.
cli_test.go:
package main
import(
"os/exec"
"testing"
)
func TestCli(t *testing.T) {
out, err := exec.Command("go run sample.go").Output()
if err != nil {
t.Fatal(err)
}
if string(out) != "May the force be with you!\nError: this is broken and not useful!\nexit status 1" {
t.Fatal("There is something wrong with the CLI")
}
}
Chapter 11 of Kerningham's Book gives a good solution to this question.
The trick is to change the calls to fmt.Printline() to calls to
fmt.Fprint(out, ...) where out is initialised to os.Stdout
This can be overwritten in the test harness to new(bytes.Buffer) allowing the
test to capture the output.
See https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo.go and
https://github.com/adonovan/gopl.io/blob/master/ch11/echo/echo_test.go
edited by OP...
sample.go:
package main
import(
"fmt"
"os"
"io"
)
var out io.Writer = os.Stdout // modified during testing
var exit func(code int) = os.Exit
type sample struct {
value int64
}
func (s sample) useful() {
if s.value == 0 {
fmt.Fprint(out, "Error: something is wrong!\n")
exit(1)
} else {
fmt.Fprint(out, "May the force be with you!\n")
}
}
func main() {
s := sample{42}
s.useful()
s.value = 0
s.useful()
}
// output:
// May the force be with you!
// Error: this is broken and not useful!
// exit status 1
cli_test.go:
package main
import(
"bytes"
"testing"
)
func TestUsefulPositive(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
s := sample{42}
s.useful()
if out.(*bytes.Buffer).String() != "May the force be with you!\n" {
t.Fatal("There is something wrong with the CLI")
}
}
func TestUsefulNegative(t *testing.T) {
bak := out
out = new(bytes.Buffer)
defer func() { out = bak }()
code := 0
osexit := exit
exit = func(c int) { code = c }
defer func() { exit = osexit }()
s := sample{0}
s.useful()
if out.(*bytes.Buffer).String() != "Error: something is wrong!\n" {
t.Fatal("There is something wrong with the CLI")
}
if code != 1 {
t.Fatal("Wrong exit code!")
}
}
Am I missing something here or are you talking of testable examples?
Basically, it works like this: In a *_test.go file, you need to adhere to the convention Example[[T][_M]] where T is a placeholder for the type and M a placeholder for the method you want to display the testable example as example code in the Godoc. If the function is just called Example(), the code will be shown as a package example.
Below the last line of the code of your example, you can put a comment like this
// Output:
// Foo
Now go test will make sure that the testable example function either exactly puts out everything below // Output: (including whitespace) or it will make the test fail.
Here is an actual example for an testable example
func ExampleMongoStore_Get() {
sessionId := "ExampleGetSession"
data, err := ms.Get(sessionId)
if err == sessionmw.ErrSessionNotFound {
fmt.Printf("Session '%s' not found\n", sessionId)
data = make(map[string]interface{})
data["foo"] = "bar"
ms.Save(sessionId, data)
}
loaded, _ := ms.Get(sessionId)
fmt.Printf("Loaded value '%s' for key '%s' in session '%s'",
loaded["foo"],
"foo", sessionId)
// Output:
// Session 'ExampleGetSession' not found
// Loaded value 'bar' for key 'foo' in session 'ExampleGetSession'
}
Edit: Have a look at the output of above example at godoc.org

Cannot convert *tls.listener to *net.TCPListener on golang

I created a TLS server on GO.
func main() {
Log("Hello server!")
cert, err := tls.LoadX509KeyPair("cert.pem", "key.pem")
if err != nil {
Log("server: loadkeys: ", err);
return;
}
config := tls.Config{Certificates: []tls.Certificate{cert}}
config.Rand = rand.Reader
service := "0.0.0.0:9988"
listener, err := tls.Listen("tcp", service, &config)
if err != nil {
Log("server: listen: %s", err)
return
}
defer listener.Close()
for {
Log("Waiting for clients")
connection, error := listener.Accept()
if error != nil {
Log("Client error: ", error)
} else {
//connection.SetLinger(0) //error here
go ClientHandler(connection)
}
}
}
I cannot call SetLinger function, because tls.Listen function returns net.Listener. I need net.TCPListener.
I tried:
tpcListener := listener.(*net.TCPListener)
Error: panic: interface conversion: net.Listener is *tls.listener, not *net.TCPListener
There is ListenTCP on net package but there is not ListenTCP on tls package.
You explicitly mention SetLinger but that's called on a TCPConn not on a TCPListener. Have you checked if the accepted net.Conn can be asserted to *net.TCPConn?
The crypto/tls package implements the net.Listener interface via a non-exported type so you cannot access the "inner" listener that way.
If you really need a net.TCPListener:
As with all the other Go standard packages that provide convenience functions that create listeners for you, there are lower level functions that let you create your own. So call net.ListenTCP, then whatever methods you like on it, then pass that to the tls package via tls.NewListener.
See the source for crypto/tls's Listen for further guidance.

golang upload file err runtime error index out of range

I've put together a golang func that takes an uploaded file and saves it to folder.
Just before os.Create() I am getting the following error :
http: panic serving [::1]:64373: runtime error: index out of range
My golang function is:
func webUploadHandler(w http.ResponseWriter, r *http.Request) {
file, header, err := r.FormFile("file") // the FormFile function takes in the POST input id file
if err != nil {
fmt.Fprintln(w, err)
return
}
defer file.Close()
// My error comes here
messageId := r.URL.Query()["id"][0]
out, err := os.Create("./upload/" + messageId + ".mp3")
if err != nil {
fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")
return
}
defer out.Close()
// write the content from POST to the file
_, err = io.Copy(out, file)
if err != nil {
fmt.Fprintln(w, err)
}
fmt.Fprintf(w,"File uploaded successfully : ")
fmt.Fprintf(w, header.Filename)
}
any ideas? much appreciate
You should at least check if r.URL.Query()["id"] has actually one element.
If len(r.URL.Query()["id"]), you could consider not accessing the index 0.
Easier, Ainar-G suggests in the comments to use the Get() method
Get gets the first value associated with the given key.
If there are no values associated with the key, Get returns the empty string.
To access multiple values, use the map directly.

How to read a binary file in Go

I'm completely new to Go and I'm trying to read a binary file, either byte by byte or several bytes at a time. The documentation doesn't help much and I cannot find any tutorial or simple example (by the way, how could Google give their language such an un-googlable name?). Basically, how can I open a file, then read some bytes into a buffer? Any suggestion?
For manipulating files, the os package is your friend:
f, err := os.Open("myfile")
if err != nil {
panic(err)
}
defer f.Close()
For more control over how the file is open, see os.OpenFile() instead (doc).
For reading files, there are many ways. The os.File type returned by os.Open (the f in the above example) implements the io.Reader interface (it has a Read() method with the right signature), it can be used directly to read some data in a buffer (a []byte) or it can also be wrapped in a buffered reader (type bufio.Reader).
Specifically for binary data, the encoding/binary package can be useful, to read a sequence of bytes into some typed structure of data. You can see an example in the Go doc here. The binary.Read() function can be used with the file read using the os.Open() function, since as I mentioned, it is a io.Reader.
And there's also the simple to use io/ioutil package, that allows you to read the whole file at once in a byte slice (ioutil.ReadFile(), which takes a file name, so you don't even have to open/close the file yourself), or ioutil.ReadAll() which takes a io.Reader and returns a slice of bytes containing the whole file. Here's the doc on ioutil.
Finally, as others mentioned, you can google about the Go language using "golang" and you should find all you need. The golang-nuts mailing list is also a great place to look for answers (make sure to search first before posting, a lot of stuff has already been answered). To look for third-party packages, check the godoc.org website.
HTH
This is what I use to read an entire binary file into memory
func RetrieveROM(filename string) ([]byte, error) {
file, err := os.Open(filename)
if err != nil {
return nil, err
}
defer file.Close()
stats, statsErr := file.Stat()
if statsErr != nil {
return nil, statsErr
}
var size int64 = stats.Size()
bytes := make([]byte, size)
bufr := bufio.NewReader(file)
_,err = bufr.Read(bytes)
return bytes, err
}
For example, to count the number of zero bytes in a file:
package main
import (
"fmt"
"io"
"os"
)
func main() {
f, err := os.Open("filename")
if err != nil {
fmt.Println(err)
return
}
defer f.Close()
data := make([]byte, 4096)
zeroes := 0
for {
data = data[:cap(data)]
n, err := f.Read(data)
if err != nil {
if err == io.EOF {
break
}
fmt.Println(err)
return
}
data = data[:n]
for _, b := range data {
if b == 0 {
zeroes++
}
}
}
fmt.Println("zeroes:", zeroes)
}
You can't whimsically cast primitive types to (char*) like in C, so for any sort of (de)serializing of binary data use the encoding/binary package.
http://golang.org/pkg/encoding/binary .
I can't improve on the examples there.
Here is an example using Read method:
package main
import (
"io"
"os"
)
func main() {
f, e := os.Open("a.go")
if e != nil {
panic(e)
}
defer f.Close()
for {
b := make([]byte, 10)
_, e = f.Read(b)
if e == io.EOF {
break
} else if e != nil {
panic(e)
}
// do something here
}
}
https://golang.org/pkg/os#File.Read