I've put together a golang func that takes an uploaded file and saves it to folder.
Just before os.Create() I am getting the following error :
http: panic serving [::1]:64373: runtime error: index out of range
My golang function is:
func webUploadHandler(w http.ResponseWriter, r *http.Request) {
file, header, err := r.FormFile("file") // the FormFile function takes in the POST input id file
if err != nil {
fmt.Fprintln(w, err)
return
}
defer file.Close()
// My error comes here
messageId := r.URL.Query()["id"][0]
out, err := os.Create("./upload/" + messageId + ".mp3")
if err != nil {
fmt.Fprintf(w, "Unable to create the file for writing. Check your write access privilege")
return
}
defer out.Close()
// write the content from POST to the file
_, err = io.Copy(out, file)
if err != nil {
fmt.Fprintln(w, err)
}
fmt.Fprintf(w,"File uploaded successfully : ")
fmt.Fprintf(w, header.Filename)
}
any ideas? much appreciate
You should at least check if r.URL.Query()["id"] has actually one element.
If len(r.URL.Query()["id"]), you could consider not accessing the index 0.
Easier, Ainar-G suggests in the comments to use the Get() method
Get gets the first value associated with the given key.
If there are no values associated with the key, Get returns the empty string.
To access multiple values, use the map directly.
Related
I have this function that logs the error in some cases:
func readByte(/*...*/){
// ...
if err != nil {
fmt.Println("ERROR")
log.Print("Couldn't read first byte")
return
}
// ...
}
Now, in the test file, I want to check the output error from this function:
c.Assert(OUTPUT, check.Matches, "teste")
How can I access the log? I tried to put a buffer but it didn't work. What is the right way to catch this log without change my readByte function code?
For example,
readbyte_test.go:
package main
import (
"bytes"
"fmt"
"io"
"log"
"os"
"testing"
)
func readByte( /*...*/ ) {
// ...
err := io.EOF // force an error
if err != nil {
fmt.Println("ERROR")
log.Print("Couldn't read first byte")
return
}
// ...
}
func TestReadByte(t *testing.T) {
var buf bytes.Buffer
log.SetOutput(&buf)
defer func() {
log.SetOutput(os.Stderr)
}()
readByte()
t.Log(buf.String())
}
Output:
$ go test -v readbyte_test.go
=== RUN TestReadByte
ERROR
--- PASS: TestReadByte (0.00s)
readbyte_test.go:30: 2017/05/22 16:41:00 Couldn't read first byte
PASS
ok command-line-arguments 0.004s
$
Answer for Concurrent Tests
If your test is running concurrently (for example, when testing an http Server or Client), you may encounter a race between writing to the buffer and reading from it. Instead of the buffer, we can redirect output to an os.Pipe and use a bufio.Scanner to block until output has been written by using the Scan() method.
Here is an example of creating an os.Pipe and setting the stdlib log package to use the pipe. Note my use of the testify/assert package here:
func mockLogger(t *testing.T) (*bufio.Scanner, *os.File, *os.File) {
reader, writer, err := os.Pipe()
if err != nil {
assert.Fail(t, "couldn't get os Pipe: %v", err)
}
log.SetOutput(writer)
return bufio.NewScanner(reader), reader, writer
}
The *os.File objects are returned so they can be properly closed with a deferred function. Here I'm just printing to stdout since if there was some strange error on close I personally wouldn't want to fail the test. However, this could easily be another call to t.Errorf or similar if you wanted:
func resetLogger(reader *os.File, writer *os.File) {
err := reader.Close()
if err != nil {
fmt.Println("error closing reader was ", err)
}
if err = writer.Close(); err != nil {
fmt.Println("error closing writer was ", err)
}
log.SetOutput(os.Stderr)
}
And then in your test you would have this pattern:
scanner, reader, writer := mockLogger(t) // turn this off when debugging or developing as you will miss output!
defer resetLogger(reader, writer)
// other setup as needed, getting some value for thing below
go concurrentAction()
scanner.Scan() // blocks until a new line is written to the pipe
got := scanner.Text() // the last line written to the scanner
msg := fmt.Sprintf("your log message with thing %v you care about", thing)
assert.Contains(t, got, msg)
And finally, the concurrentAction() function is calling a log function (or method if using a log.logger, the package actually behaves the same way with log.SetOutput() call above either way) like:
// doing something, getting value for thing
log.Printf("your log message with the thing %v you care about", thing)
I am trying to create a bot and retrieve the list of channels.
I used the bot example in repository and it is mostly working, except for the part where it has to get the list of channels.
Either I am doing something silly or GetChannels API really does not work the way it is described in bot_sample.go .
I made a smaller separate function to test that part.
Adding code here for better readability:
func mattermostPrintChannels(client *mattermost.Client) {
channelsResult, err := client.GetChannels("")
if err != nil {
fmt.Print("Couldn't get channels: ", err)
return
}
channelList := channelsResult.Data.(*mattermost.ChannelList)
fmt.Print("Channels:")
for _, channel := range channelList.Channels {
fmt.Printf("%s -> %s", channel.Id, channel.DisplayName)
}
}
This code gives me the error:
./mattermost.go:30: channelList.Channels undefined (type
*model.ChannelList has no field or method Channels)
Now if I just print the contents of ChannelList variable (using spew), I get the following:
channelList: : ([]interface {}) (len=1 cap=1) {
(*model.ChannelList)(<nil>)
}
JimB is correct. The model.ChannelList type used to be a struct, but it recently changed to []*model.Channel. You'll want to change
for _, channel := range channelList.Channels {
to
for _, channel := range *channelList {
I just read the blog written by Rob Pike. I have a small question regarding this and may be I can be wrong too but would still like to get feedback and understand properly the Go.
In the blog there was a snippet code (which actually was written by #jxck_)
_, err = fd.Write(p0[a:b])
if err != nil {
return err
}
_, err = fd.Write(p1[c:d])
if err != nil {
return err
}
_, err = fd.Write(p2[e:f])
if err != nil {
return err
}
// and so on
a) As per my understanding the above code will return if error occurred at fd.Write(p0[a:b]), and will never execute fd.Write(p1[c:d]) , right?
And Rob suggested to write something like this
var err error
write := func(buf []byte) {
if err != nil {
return
}
_, err = w.Write(buf)
}
write(p0[a:b])
write(p1[c:d])
write(p2[e:f])
// and so on
if err != nil {
return err
}
b) Based on the above, looks like the error will return from the sub function. So this means if the error occurs at write(p0[a:b]) then still it will execute write(p1[c:d]), right? So this means logically both are not same, right?
Anybody explain.
No, they are the same. If an error occurs at fd.Write(p0[a:b]), the err variable will hold its value.
Now if you call write(p1[c:d]), then the write() func will first check if err != nil but since it already stores the error which occured in the previous call, it will return immediately and will not execute further code.
a) Yes, you are correct. If the error occures in the first write, it will return.
b) No. The write in this example is a closure. The err inside of it is the same as in the outer scope. So if the first write fails, the other will simply return, because the outer err is not nil anymore.
When a user uploads a file using r.FormFile("file") you get a multipart.File, a multipart.FileHeader and an error.
My question is how to just obtain information about the uploaded file . For example, its size, its dimensions if it's an image, and so on and so forth.
I have literally got no idea on where to start so any help would be great.
To get the file size and MIME type:
// Size constants
const (
MB = 1 << 20
)
type Sizer interface {
Size() int64
}
func Sample(w http.ResponseWriter, r *http.Request) error {
if err := r.ParseMultipartForm(5 * MB); err != nil {
return err
}
// Limit upload size
r.Body = http.MaxBytesReader(w, r.Body, 5*MB) // 5 Mb
//
file, multipartFileHeader, err := r.FormFile("file")
// Create a buffer to store the header of the file in
fileHeader := make([]byte, 512)
// Copy the headers into the FileHeader buffer
if _, err := file.Read(fileHeader); err != nil {
return err
}
// set position back to start.
if _, err := file.Seek(0, 0); err != nil {
return err
}
log.Printf("Name: %#v\n", multipartFileHeader.Filename)
log.Printf("Size: %#v\n", file.(Sizer).Size())
log.Printf("MIME: %#v\n", http.DetectContentType(fileHeader))
}
Sample output:
2016/12/01 15:00:06 Name: "logo_35x30_black.png"
2016/12/01 15:00:06 Size: 18674
2016/12/01 15:00:06 MIME: "image/png"
The file name and MIME type can be obtained from the returned multipart.FileHeader.
Most further meta-data will depend on the file type. If it's an image, you should be able to use the DecodeConfig functions in the standard library, for PNG, JPEG and GIF, to obtain the dimensions (and color model).
There are many Go libraries available for other file types as well, which will have similar functions.
EDIT: There's a good example on the golang-nuts mail group.
You can get approximate information about the size of file from Content-Length header. This is not recommended, because this header can be changed.
A better way is to use ReadFrom method:
clientFile, handler, err := r.FormFile("file") // r is *http.Request
var buff bytes.Buffer
fileSize, err := buff.ReadFrom(clientFile)
fmt.Println(fileSize) // this will return you a file size.
Another way I've found pretty simple for this type of testing is to place test assets in a test_data directory relative to the package. Within my test file I normally create a helper that creates an instance of *http.Request, which allows me to run table test pretty easily on multipart.File, (errors checking removed for brevity).
func createMockRequest(pathToFile string) *http.Request {
file, err := os.Open(pathToFile)
if err != nil {
return nil
}
defer file.Close()
body := &bytes.Buffer{}
writer := multipart.NewWriter(body)
part, err := writer.CreateFormFile("file", filepath.Base(pathToFile))
if err != nil {
return nil
}
_, _ = io.Copy(part, file)
err = writer.Close()
if err != nil {
return nil
}
// the body is the only important data for creating a new request with the form data attached
req, _ := http.NewRequest("POST", "", body)
req.Header.Set("Content-Type", writer.FormDataContentType())
return req
}
I'm completely new to Go and I'm trying to read a binary file, either byte by byte or several bytes at a time. The documentation doesn't help much and I cannot find any tutorial or simple example (by the way, how could Google give their language such an un-googlable name?). Basically, how can I open a file, then read some bytes into a buffer? Any suggestion?
For manipulating files, the os package is your friend:
f, err := os.Open("myfile")
if err != nil {
panic(err)
}
defer f.Close()
For more control over how the file is open, see os.OpenFile() instead (doc).
For reading files, there are many ways. The os.File type returned by os.Open (the f in the above example) implements the io.Reader interface (it has a Read() method with the right signature), it can be used directly to read some data in a buffer (a []byte) or it can also be wrapped in a buffered reader (type bufio.Reader).
Specifically for binary data, the encoding/binary package can be useful, to read a sequence of bytes into some typed structure of data. You can see an example in the Go doc here. The binary.Read() function can be used with the file read using the os.Open() function, since as I mentioned, it is a io.Reader.
And there's also the simple to use io/ioutil package, that allows you to read the whole file at once in a byte slice (ioutil.ReadFile(), which takes a file name, so you don't even have to open/close the file yourself), or ioutil.ReadAll() which takes a io.Reader and returns a slice of bytes containing the whole file. Here's the doc on ioutil.
Finally, as others mentioned, you can google about the Go language using "golang" and you should find all you need. The golang-nuts mailing list is also a great place to look for answers (make sure to search first before posting, a lot of stuff has already been answered). To look for third-party packages, check the godoc.org website.
HTH
This is what I use to read an entire binary file into memory
func RetrieveROM(filename string) ([]byte, error) {
file, err := os.Open(filename)
if err != nil {
return nil, err
}
defer file.Close()
stats, statsErr := file.Stat()
if statsErr != nil {
return nil, statsErr
}
var size int64 = stats.Size()
bytes := make([]byte, size)
bufr := bufio.NewReader(file)
_,err = bufr.Read(bytes)
return bytes, err
}
For example, to count the number of zero bytes in a file:
package main
import (
"fmt"
"io"
"os"
)
func main() {
f, err := os.Open("filename")
if err != nil {
fmt.Println(err)
return
}
defer f.Close()
data := make([]byte, 4096)
zeroes := 0
for {
data = data[:cap(data)]
n, err := f.Read(data)
if err != nil {
if err == io.EOF {
break
}
fmt.Println(err)
return
}
data = data[:n]
for _, b := range data {
if b == 0 {
zeroes++
}
}
}
fmt.Println("zeroes:", zeroes)
}
You can't whimsically cast primitive types to (char*) like in C, so for any sort of (de)serializing of binary data use the encoding/binary package.
http://golang.org/pkg/encoding/binary .
I can't improve on the examples there.
Here is an example using Read method:
package main
import (
"io"
"os"
)
func main() {
f, e := os.Open("a.go")
if e != nil {
panic(e)
}
defer f.Close()
for {
b := make([]byte, 10)
_, e = f.Read(b)
if e == io.EOF {
break
} else if e != nil {
panic(e)
}
// do something here
}
}
https://golang.org/pkg/os#File.Read