I'm new to Go and I'm trying to write a little program to save enumerated values to a database.
The way I declare my values is as follows:
type FileType int64
const (
movie FileType = iota
music
book
etc
)
I use these values in my struct like this:
type File struct {
Name string
Type FileType
Size int64
}
I use gorp for my database stuff, but I guess the use of gorp isn't relevant to my problem. I put stuff in my DB like this:
dbmap.Insert(&File{"MyBook.pdf",movie,1000})
but when I try to retrieve stuff…
dbmap.Select(&dbFiles, "select * from Files")
I get the following error:
panic: reflect.Set: value of type int64 is not assignable to type main.FileType
When I use int64 as the type for the const(...) and for the File.Type field, everything works fine, but I'm new to Go and want to understand the problem.
The way I see it, I have two problems:
Why can't Go convert this stuff successfully? I looked at the source code of the Go reflection and sql packages and there are methods for this kind of conversion, but they seem to fail. Is this a bug? What is the problem?
I figured out, that one can implement the sql.Scanner interface by implementing the following method:
Scan(src interface{}) error
I tried to implement the method and I even was able to get the right value from src and convert it to a FileType, but I was confused if I should implement the method for "(f *FileType) or (f FileType). Either way the method gets invoked, however I'm not able to overwrite f (or at least the update gets lost later) and the File instances read from the DB always had a "0" as value for File.Type.
Do you have any ideas on those two points?
I recently had the same need, and the solution is to implement two interfaces:
sql/driver.Valuer
sql.Scanner
Here's a working example:
type FileType int64
func (u *FileType) Scan(value interface{}) error { *u = FileType(value.(int64)); return nil }
func (u FileType) Value() (driver.Value, error) { return int64(u), nil }
Slightly off-topic, but may be useful to others as I kept revisiting this question/answer when solving a similar problem when working with postgres enum fields in golang (which are returned as bytes).
// Status values
const (
incomplete Status = "incomplete"
complete Status = "complete"
reject Status = "reject"
)
type Status string
func (s *Status) Scan(value interface{}) error {
asBytes, ok := value.([]byte)
if !ok {
return errors.New("Scan source is not []byte")
}
*s = Status(string(asBytes))
return nil
}
func (s SubjectStatus) Value() (driver.Value, error) {
// validation would go here
return string(s), nil
}
Go needs to be specific with types, which can be a pain sometimes.
(f FileType) is cheaper than (f *FileType) for "native" types, pretty much unless you have a complex type, it's almost always better to not use a pointer.
What do you mean it doesn't overwrite it? did you resave the struct after you modified it?
Related
I'm learning Rust and trying to solve some basic algorithm problems with it. In many cases, I want to read lines from stdin, perform some transformation on each line and return a vector of resulting items. One way I did this was like this:
// Fully working Rust code
let my_values: Vec<u32> = stdin
.lock()
.lines()
.filter_map(Result::ok)
.map(|line| line.parse::<u32>())
.filter_map(Result::ok)
.map(|x|x*2) // For example
.collect();
This works but of course silently ignores any errors that may occur. Now what I woud like to do is something along the lines of:
// Pseudo-ish code
let my_values: Result<Vec<u32>, X> = stdin
.lock()
.lines() // Can cause std::io::Error
.map(|line| line.parse::<u32>()) // Can cause std::num::ParseIntError
.map(|x| x*2)
.collect();
Where X is some kind of error type that I can match on afterwards. Preferably I want to perform the whole operation on one line at a time and immediately discard the string data after it has been parsed to an int.
I think I need to create some kind of Enum type to hold the various possible errors, possibly like this:
#[derive(Debug)]
enum InputError {
Io(std::io::Error),
Parse(std::num::ParseIntError),
}
However, I don't quite understand how to put everything together to make it clean and avoid having to explicitly match and cast everywhere. Also, is there some way to automatically create these enum error types or do I have to explicilty enumerate them every time I do this?
You're on the right track.
The way I'd approach this is by using the enum you've defined,
then add implementations of From for the error types you're interested in.
That will allow you to use the ? operator on your maps to get the kind of behaviour you want.
#[derive(Debug)]
enum MyError {
IOError(std::io::Error),
ParseIntError(std::num::ParseIntError),
}
impl From<std::io::Error> for MyError {
fn from(e:std::io::Error) -> MyError {
return MyError::IOError(e)
}
}
impl From<std::num::ParseIntError> for MyError {
fn from(e:std::num::ParseIntError) -> MyError {
return MyError::ParseIntError(e)
}
}
Then you can implement the actual transform as either
let my_values: Vec<_> = stdin
.lock()
.lines()
.map(|line| -> Result<u32,MyError> { Ok(line?.parse::<u32>()?*2) } )
.collect();
which will give you one entry for each input, like: {Ok(x), Err(MyError(x)), Ok(x)}.
or you can do:
let my_values: Result<Vec<_>,MyError> = stdin
.lock()
.lines()
.map(|line| -> Result<u32,MyError> { Ok(line?.parse::<u32>()?*2) } )
.collect();
Which will give you either Err(MyError(...)) or Ok([1,2,3])
Note that you can further reduce some of the error boilerplate by using an error handling crate like snafu, but in this case it's not too much.
I'm currently learning golang (mostly a Java/C# developer) and I hit an issue with pointers and defer.
I'm trying to wrap the writes to a CSV file under a struct in a OO-like style. From the examples I found online, it seems that creating "methods" on a struct could be done like so:
type MyObject struct {
fp *os.File
csv *csv.Writer
}
func (mo MyObject) Open(filepath string) {
println(&mo)
var err error
mo.fp, err = os.Create(filepath)
if err != nil {
panic(err)
}
mo.csv = csv.NewWriter(mo.fp)
}
The problem I hit was once I left the Open method, the pointers for fp and csv went back to null. Subsequent calls to this class would throw a nil error. A full example can be found here.
After a lot of googling, I ended up looking at how golang implemented their logger. They used a pointer to the object like so:
type MyObject struct {
fp *os.File
csv *csv.Writer
}
func New() *MyObject {
return &MyObject{}
}
func (mo *MyObject) Open(filepath string) {
println(&mo)
var err error
mo.fp, err = os.Create(filepath)
if err != nil {
panic(err)
}
mo.csv = csv.NewWriter(mo.fp)
}
A refactoring of my code (see here) shows it works as expected. I'm still confused though why the first method didn't work. I'm guessing I'm misunderstanding something on how structs, pointers, and/or defer work. What am I missing?
It didn't work in the first case, because func (mo MyObject) Open(filepath string) only got a local copy of MyObject...and all changes made to it remained within that context.
But after you added * to the receiver, i.e (mo *MyObject) the changes within the function affected the original MyObject.
you can check here for more info
hope this helps
libc's error handling is usually to return something < 0 in case of an error. I find myself doing this over and over:
let pid = fork()
if pid < 0 {
// Please disregard the fact that `Err(pid)`
// should be a `&str` or an enum
return Err(pid);
}
I find it ugly that this needs 3 lines of error handling, especially considering that these tests are quite frequent in this kind of code.
Is there a way to return an Err in case fork() returns < 0?
I found two things which are close:
assert_eq!. This needs another line and it panics so the caller cannot handle the error.
Using traits like these:
pub trait LibcResult<T> {
fn to_option(&self) -> Option<T>;
}
impl LibcResult<i64> for i32 {
fn to_option(&self) -> Option<i64> {
if *self < 0 { None } else { Some(*self) }
}
}
I could write fork().to_option().expect("could not fork"). This is now only one line, but it panics instead of returning an Err. I guess this could be solved using ok_or.
Some functions of libc have < 0 as sentinel (e.g. fork), while others use > 0 (e.g. pthread_attr_init), so this would need another argument.
Is there something out there which solves this?
As indicated in the other answer, use pre-made wrappers whenever possible. Where such wrappers do not exist, the following guidelines might help.
Return Result to indicate errors
The idiomatic Rust return type that includes error information is Result (std::result::Result). For most functions from POSIX libc, the specialized type std::io::Result is a perfect fit because it uses std::io::Error to encode errors, and it includes all standard system errors represented by errno values. A good way to avoid repetition is using a utility function such as:
use std::io::{Result, Error};
fn check_err<T: Ord + Default>(num: T) -> Result<T> {
if num < T::default() {
return Err(Error::last_os_error());
}
Ok(num)
}
Wrapping fork() would look like this:
pub fn fork() -> Result<u32> {
check_err(unsafe { libc::fork() }).map(|pid| pid as u32)
}
The use of Result allows idiomatic usage such as:
let pid = fork()?; // ? means return if Err, unwrap if Ok
if pid == 0 {
// child
...
}
Restrict the return type
The function will be easier to use if the return type is modified so that only "possible" values are included. For example, if a function logically has no return value, but returns an int only to communicate the presence of error, the Rust wrapper should return nothing:
pub fn dup2(oldfd: i32, newfd: i32) -> Result<()> {
check_err(unsafe { libc::dup2(oldfd, newfd) })?;
Ok(())
}
Another example are functions that logically return an unsigned integer, such as a PID or a file descriptor, but still declare their result as signed to include the -1 error return value. In that case, consider returning an unsigned value in Rust, as in the fork() example above. nix takes this one step further by having fork() return Result<ForkResult>, where ForkResult is a real enum with methods such as is_child(), and from which the PID is extracted using pattern matching.
Use options and other enums
Rust has a rich type system that allows expressing things that have to be encoded as magic values in C. To return to the fork() example, that function returns 0 to indicate the child return. This would be naturally expressed with an Option and can be combined with the Result shown above:
pub fn fork() -> Result<Option<u32>> {
let pid = check_err(unsafe { libc::fork() })? as u32;
if pid != 0 {
Some(pid)
} else {
None
}
}
The user of this API would no longer need to compare with the magic value, but would use pattern matching, for example:
if let Some(child_pid) = fork()? {
// execute parent code
} else {
// execute child code
}
Return values instead of using output parameters
C often returns values using output parameters, pointer parameters into which the results are stored. This is either because the actual return value is reserved for the error indicator, or because more than one value needs to be returned, and returning structs was badly supported by historical C compilers.
In contrast, Rust's Result supports return value independent of error information, and has no problem whatsoever with returning multiple values. Multiple values returned as a tuple are much more ergonomic than output parameters because they can be used in expressions or captured using pattern matching.
Wrap system resources in owned objects
When returning handles to system resources, such as file descriptors or Windows handles, it good practice to return them wrapped in an object that implements Drop to release them. This will make it less likely that a user of the wrapper will make a mistake, and it makes the use of return values more idiomatic, removing the need for awkward invocations of close() and resource leaks coming from failing to do so.
Taking pipe() as an example:
use std::fs::File;
use std::os::unix::io::FromRawFd;
pub fn pipe() -> Result<(File, File)> {
let mut fds = [0 as libc::c_int; 2];
check_err(unsafe { libc::pipe(fds.as_mut_ptr()) })?;
Ok(unsafe { (File::from_raw_fd(fds[0]), File::from_raw_fd(fds[1])) })
}
// Usage:
// let (r, w) = pipe()?;
// ... use R and W as normal File object
This pipe() wrapper returns multiple values and uses a wrapper object to refer to a system resource. Also, it returns the File objects defined in the Rust standard library and accepted by Rust's IO layer.
The best option is to not reimplement the universe. Instead, use nix, which wraps everything for you and has done the hard work of converting all the error types and handling the sentinel values:
pub fn fork() -> Result<ForkResult>
Then just use normal error handling like try! or ?.
Of course, you could rewrite all of nix by converting your trait to returning Results and including the specific error codes and then use try! or ?, but why would you?
There's nothing magical in Rust that converts negative or positive numbers into a domain specific error type for you. The code you already have is the correct approach, once you've enhanced it to use a Result either by creating it directly or via something like ok_or.
An intermediate solution would be to reuse nix's Errno struct, perhaps with your own trait sugar on top.
so this would need another argument
I'd say it would be better to have different methods: one for negative sentinel values and one for positive sentinel values.
I have this code
fn get_last_commit () -> String {
Command::new("git")
.arg("rev-parse")
.arg("HEAD")
.output()
.map(|output| {
String::from_utf8(output.stdout).ok().expect("error reading into string")
})
.ok().expect("error invoking git rev-parse")
}
I'd like to be able to cut the ok().expect(..) a bit down so that I ideally have something like that:
fn get_last_commit () -> String {
Command::new("git")
.arg("rev-parse")
.arg("HEAD")
.output()
.and_then(|output| {
String::from_utf8(output.stdout)
})
.ok().expect("error invoking git rev-parse")
}
However, that doesn't work because the errors don't line up leaving me with:
mismatched types:
expected `core::result::Result<_, std::io::error::Error>`,
found `core::result::Result<collections::string::String, collections::string::FromUtf8Error>`
(expected struct `std::io::error::Error`,
found struct `collections::string::FromUtf8Error`)
I know the error handling changed quite a bit within the last month and I have the feeling there should be away to get them aligned without too much hassle. I seem unable to figure it out though.
The problem is that the closure passed to the and_then needs to return a Result with the same error type as the Result that and_then was called on; otherwise, there's no single type that and_then could return; and_then maps one Ok type to another, but keeps the error type the same.
Since you are just throwing away the error value by converting it to an option with ok() that you unwrap anyhow, you can do that before calling and_then, and within the closure, as the Option type returned by and_then on an Option only depends on the value returned by the closure:
fn get_last_commit () -> String {
Command::new("git")
.arg("rev-parse")
.arg("HEAD")
.output()
.ok()
.and_then(|output| {
String::from_utf8(output.stdout).ok()
})
.expect("error invoking git rev-parse")
}
If you actually cared about the error value, you would need to define your own error type that could contain either of the two types of errors, and wrap either of the errors up. The FromError trait and try! macro offer a convenient way to is wrap up the value and return it from one of several places in your function, though in this case map_err would probably be a better way to do that as you are doing it all via chained method calls rather than separate statements.
enum MyError {
Io(IoError),
Utf8(FromUtf8Error)
}
fn get_last_commit () -> Result<String,MyError> {
Command::new("git")
.arg("rev-parse")
.arg("HEAD")
.output()
.map_err(MyError::Io)
.and_then(|output| {
String::from_utf8(output.stdout)
.map_err(MyError::Utf8)
})
}
If you notice, this parallels the earlier solution fairly closely, coercing both of the result types into a single common type; in the first solution, it just throws away the error value by using ok(), while in the second, it preserves the error value so you can return it, but you now need the extra machinery of a type that could wrap either.
I have two queries against a particular table in Go - one to retrieve a single item and the other to return a list. The first one uses sql.DB.QueryRow because it only needs to retrieve a single row, and the second one uses sql.DB.Query to return a few different results.
The problem is that serialization takes some work and I'd like to DRY it up by having a single method that scans from a database row and reads it into a Go type. My code right now looks like:
// Some type which varies considerably from its DB representation, and takes some effort to serialize.
type Foo struct {
Baz *Baz
Board [8][8]int
}
// Get one foo by its id
func GetFoo(id int) {
row := db.QueryRow("select * from foo where id = ?", id)
// Work that's duplicated from below...
var foo Foo
row.Scan(&foo.blah, &foo.etc)
// Do more work to serialize the type...
}
// Get all of the fooes
func GetFooes() {
rows, err := db.Query("select * from foo")
for rows.Next() {
// Work that's duplicated from above...
var foo Foo
rows.Scan(&foo.blah, &foo.etc)
// Do more work to serialize the type...
}
}
However combining row.Scan and rows.Scan is proving to be a little tricky. I thought I could use something like:
func serializeFoo(scanner sql.Scanner) (*Foo, error) {
}
though sql.Scanner takes a single (value interface{}) and not a list of (...value interface{}).
Any advice here? Another solution would be to convert the single QueryRow call into a db.Query.
db.QueryRow is a convenience function. There is no reason to use it unless it will save on typing/code-complexity. In this case, it doesn't so I recommend you just use db.Query.
See http://golang.org/src/pkg/database/sql/sql.go?s=25740:25802#L966 for more details
As you mentioned, neither Row nor Rows implement the Scanner interface. Scanner is used for arguments of the variatic scan functions.
If you want to have a parameter that allows either Row or Rows, you need to make your own interface For example:
func serializeFoo(scanner interface{Scan(dest ...interface{}) error}) (*Foo, error) {
}