I have two queries against a particular table in Go - one to retrieve a single item and the other to return a list. The first one uses sql.DB.QueryRow because it only needs to retrieve a single row, and the second one uses sql.DB.Query to return a few different results.
The problem is that serialization takes some work and I'd like to DRY it up by having a single method that scans from a database row and reads it into a Go type. My code right now looks like:
// Some type which varies considerably from its DB representation, and takes some effort to serialize.
type Foo struct {
Baz *Baz
Board [8][8]int
}
// Get one foo by its id
func GetFoo(id int) {
row := db.QueryRow("select * from foo where id = ?", id)
// Work that's duplicated from below...
var foo Foo
row.Scan(&foo.blah, &foo.etc)
// Do more work to serialize the type...
}
// Get all of the fooes
func GetFooes() {
rows, err := db.Query("select * from foo")
for rows.Next() {
// Work that's duplicated from above...
var foo Foo
rows.Scan(&foo.blah, &foo.etc)
// Do more work to serialize the type...
}
}
However combining row.Scan and rows.Scan is proving to be a little tricky. I thought I could use something like:
func serializeFoo(scanner sql.Scanner) (*Foo, error) {
}
though sql.Scanner takes a single (value interface{}) and not a list of (...value interface{}).
Any advice here? Another solution would be to convert the single QueryRow call into a db.Query.
db.QueryRow is a convenience function. There is no reason to use it unless it will save on typing/code-complexity. In this case, it doesn't so I recommend you just use db.Query.
See http://golang.org/src/pkg/database/sql/sql.go?s=25740:25802#L966 for more details
As you mentioned, neither Row nor Rows implement the Scanner interface. Scanner is used for arguments of the variatic scan functions.
If you want to have a parameter that allows either Row or Rows, you need to make your own interface For example:
func serializeFoo(scanner interface{Scan(dest ...interface{}) error}) (*Foo, error) {
}
Related
I've a method X that's getting data from the server via pub sub. This method returns a flow. I've another method that subscribes to the flow by method X but only wants to take the first 3 values max from the flow if the data is distinct compared to previous data. I've written the following code
fun subscribeToData() : Flow<List<MyData>> {
....
//incoming data
emit(list)
}
fun getUptoFirst3Items() {
subscribeToData()
.take(ITEM_COUNT) // ITEM_COUNT is 3
.distinctUntilChange() //only proceed if the data is different from the previous top 3 items
.mapIndex {
//do transformation
}
.collect { transformedListOf3Elements ->
}
}
Problem:
In collect{} I'm not getting 3 elements but rather I'm getting all the data that's coming in the flow.
I'm not sure what's wrong here? Can someone help me?
You have a Flow<List<MyData>> here, which means every element of this flow is itself a list.
The take operator is applied on the flow, so you will take the 3 first lists of the flow. Each individual list is not limited, unless you use take on the list itself.
So the name transformedListOf3Elements is incorrect, because the list is of an unknown number of elements, unless you filter it somehow in the map.
#Joffrey answer already explained why you get the whole list returned and suggested you use take() on the list itself.
If you want to take just the first ITEM_COUNT elements from every list that is emitted/observed, then you have to map the result and only take ITEM_COUNT items from the list each time, instead of taking ITEM_COUNT items from the flow.
fun getUptoFirst3Items() {
subscribeToData()
.map {
// in Kotlin stdlib Iterable<T> has an extension method take(n: Int)
// that will return a List<T> containing the first n element from the iterable
it.take(ITEM_COUNT)
// alternatively you can also use subList, but the semantics are not the same,
// so check the subList documentation, before using it
it.subList(0, ITEM_COUNT)
}
.distinctUntilChange() //only proceed if the data is different from the previous top 3 items
.mapIndex {
//do transformation
}
.collect { transformedListOf3Elements ->
}
}
I have a method which performs an NHibernate query, and returns the values as an IEnumerable<long>. It is running a future query so the result is actually of type IFutureEnumerable<long>.
public static IEnumerable<long> GetQueryResults(IEnumerable<long> idsToFilterOn)
{
if((idsToFilterOn == null) || !(idsToFilterOn.Any()))
{
return Enumerable.Empty<long>();
}
else
{
IQueryOver<MyTable> query = GenerateTheBigQuery(idsToFilterOn);
return query.Future<long>();
}
}
I want this result to return IFutureEnumerable<long>, but I still want to first check the parameters, and if I know the result will be empty I want to just return an empty value without running a query.
If I just change the return type to IFutureEnumerable<long>, the line of code that returns Enumerable.Empty<long>() generates a compiler error (Cannot implicitly convert type...)
Is there some static method like FutureEnumerable.Empty<long>() which generates an IFutureEnumerable that returns an empty list?
Looking at the code, there doesn't appear to be any native support for that concept. IFutureEnumerable is implemented by two types, one of which is deprecated and neither offer the notion of emptiness.
I suppose that leaves it up to you to create a type that implements IFutureEnumerable<T> that supports emptiness.
libc's error handling is usually to return something < 0 in case of an error. I find myself doing this over and over:
let pid = fork()
if pid < 0 {
// Please disregard the fact that `Err(pid)`
// should be a `&str` or an enum
return Err(pid);
}
I find it ugly that this needs 3 lines of error handling, especially considering that these tests are quite frequent in this kind of code.
Is there a way to return an Err in case fork() returns < 0?
I found two things which are close:
assert_eq!. This needs another line and it panics so the caller cannot handle the error.
Using traits like these:
pub trait LibcResult<T> {
fn to_option(&self) -> Option<T>;
}
impl LibcResult<i64> for i32 {
fn to_option(&self) -> Option<i64> {
if *self < 0 { None } else { Some(*self) }
}
}
I could write fork().to_option().expect("could not fork"). This is now only one line, but it panics instead of returning an Err. I guess this could be solved using ok_or.
Some functions of libc have < 0 as sentinel (e.g. fork), while others use > 0 (e.g. pthread_attr_init), so this would need another argument.
Is there something out there which solves this?
As indicated in the other answer, use pre-made wrappers whenever possible. Where such wrappers do not exist, the following guidelines might help.
Return Result to indicate errors
The idiomatic Rust return type that includes error information is Result (std::result::Result). For most functions from POSIX libc, the specialized type std::io::Result is a perfect fit because it uses std::io::Error to encode errors, and it includes all standard system errors represented by errno values. A good way to avoid repetition is using a utility function such as:
use std::io::{Result, Error};
fn check_err<T: Ord + Default>(num: T) -> Result<T> {
if num < T::default() {
return Err(Error::last_os_error());
}
Ok(num)
}
Wrapping fork() would look like this:
pub fn fork() -> Result<u32> {
check_err(unsafe { libc::fork() }).map(|pid| pid as u32)
}
The use of Result allows idiomatic usage such as:
let pid = fork()?; // ? means return if Err, unwrap if Ok
if pid == 0 {
// child
...
}
Restrict the return type
The function will be easier to use if the return type is modified so that only "possible" values are included. For example, if a function logically has no return value, but returns an int only to communicate the presence of error, the Rust wrapper should return nothing:
pub fn dup2(oldfd: i32, newfd: i32) -> Result<()> {
check_err(unsafe { libc::dup2(oldfd, newfd) })?;
Ok(())
}
Another example are functions that logically return an unsigned integer, such as a PID or a file descriptor, but still declare their result as signed to include the -1 error return value. In that case, consider returning an unsigned value in Rust, as in the fork() example above. nix takes this one step further by having fork() return Result<ForkResult>, where ForkResult is a real enum with methods such as is_child(), and from which the PID is extracted using pattern matching.
Use options and other enums
Rust has a rich type system that allows expressing things that have to be encoded as magic values in C. To return to the fork() example, that function returns 0 to indicate the child return. This would be naturally expressed with an Option and can be combined with the Result shown above:
pub fn fork() -> Result<Option<u32>> {
let pid = check_err(unsafe { libc::fork() })? as u32;
if pid != 0 {
Some(pid)
} else {
None
}
}
The user of this API would no longer need to compare with the magic value, but would use pattern matching, for example:
if let Some(child_pid) = fork()? {
// execute parent code
} else {
// execute child code
}
Return values instead of using output parameters
C often returns values using output parameters, pointer parameters into which the results are stored. This is either because the actual return value is reserved for the error indicator, or because more than one value needs to be returned, and returning structs was badly supported by historical C compilers.
In contrast, Rust's Result supports return value independent of error information, and has no problem whatsoever with returning multiple values. Multiple values returned as a tuple are much more ergonomic than output parameters because they can be used in expressions or captured using pattern matching.
Wrap system resources in owned objects
When returning handles to system resources, such as file descriptors or Windows handles, it good practice to return them wrapped in an object that implements Drop to release them. This will make it less likely that a user of the wrapper will make a mistake, and it makes the use of return values more idiomatic, removing the need for awkward invocations of close() and resource leaks coming from failing to do so.
Taking pipe() as an example:
use std::fs::File;
use std::os::unix::io::FromRawFd;
pub fn pipe() -> Result<(File, File)> {
let mut fds = [0 as libc::c_int; 2];
check_err(unsafe { libc::pipe(fds.as_mut_ptr()) })?;
Ok(unsafe { (File::from_raw_fd(fds[0]), File::from_raw_fd(fds[1])) })
}
// Usage:
// let (r, w) = pipe()?;
// ... use R and W as normal File object
This pipe() wrapper returns multiple values and uses a wrapper object to refer to a system resource. Also, it returns the File objects defined in the Rust standard library and accepted by Rust's IO layer.
The best option is to not reimplement the universe. Instead, use nix, which wraps everything for you and has done the hard work of converting all the error types and handling the sentinel values:
pub fn fork() -> Result<ForkResult>
Then just use normal error handling like try! or ?.
Of course, you could rewrite all of nix by converting your trait to returning Results and including the specific error codes and then use try! or ?, but why would you?
There's nothing magical in Rust that converts negative or positive numbers into a domain specific error type for you. The code you already have is the correct approach, once you've enhanced it to use a Result either by creating it directly or via something like ok_or.
An intermediate solution would be to reuse nix's Errno struct, perhaps with your own trait sugar on top.
so this would need another argument
I'd say it would be better to have different methods: one for negative sentinel values and one for positive sentinel values.
var take = R.curry(function take(count, o) {
return R.pick(R.take(count, R.keys(o)), o);
});
This function takes count keys from an object, in the order, in which they appear. I use it to limit a dataset which was grouped.
I understand that there are placeholder arguments, like R.__, but I can't wrap my head around this particular case.
This is possible thanks to R.converge, but I don't recommend going point-free in this case.
// take :: Number -> Object -> Object
var take = R.curryN(2,
R.converge(R.pick,
R.converge(R.take,
R.nthArg(0),
R.pipe(R.nthArg(1),
R.keys)),
R.nthArg(1)));
One thing to note is that the behaviour of this function is undefined since the order of the list returned by R.keys is undefined.
I agree with #davidchambers that it is probably better not to do this points-free. This solution is a bit cleaner than that one, but is still not to my mind as nice as your original:
// take :: Number -> Object -> Object
var take = R.converge(
R.pick,
R.useWith(R.take, R.identity, R.keys),
R.nthArg(1)
);
useWith and converge are similar in that they accept a number of function parameters and pass the result of calling all but the first one into that first one. The difference is that converge passes all the parameters it receives to each one, and useWith splits them up, passing one to each function. This is the first time I've seen a use for combining them, but it seems to make sense here.
That property ordering issue is supposed to be resolved in ES6 (final draft now out!) but it's still controversial.
Update
You mention that it will take some time to figure this out. This should help at least show how it's equivalent to your original function, if not how to derive it:
var take = R.converge(
R.pick,
R.useWith(R.take, R.identity, R.keys),
R.nthArg(1)
);
// definition of `converge`
(count, obj) => R.pick(R.useWith(R.take, R.identity, R.keys)(count, obj),
R.nthArg(1)(count, obj));
// definition of `nthArg`
(count, obj) => R.pick(R.useWith(R.take, R.identity, R.keys)(count, obj), obj);
// definition of `useWith`
(count, obj) => R.pick(R.take(R.identity(count), R.keys(obj)), obj);
// definition of `identity`
(count, obj) => R.pick(R.take(count, R.keys(obj)), obj);
Update 2
As of version 18, both converge and useWith have changed to become binary. Each takes a target function and a list of helper functions. That would change the above slightly to this:
// take :: Number -> Object -> Object
var take = R.converge(R.pick, [
R.useWith(R.take, [R.identity, R.keys]),
R.nthArg(1)
]);
I'm new to Go and I'm trying to write a little program to save enumerated values to a database.
The way I declare my values is as follows:
type FileType int64
const (
movie FileType = iota
music
book
etc
)
I use these values in my struct like this:
type File struct {
Name string
Type FileType
Size int64
}
I use gorp for my database stuff, but I guess the use of gorp isn't relevant to my problem. I put stuff in my DB like this:
dbmap.Insert(&File{"MyBook.pdf",movie,1000})
but when I try to retrieve stuff…
dbmap.Select(&dbFiles, "select * from Files")
I get the following error:
panic: reflect.Set: value of type int64 is not assignable to type main.FileType
When I use int64 as the type for the const(...) and for the File.Type field, everything works fine, but I'm new to Go and want to understand the problem.
The way I see it, I have two problems:
Why can't Go convert this stuff successfully? I looked at the source code of the Go reflection and sql packages and there are methods for this kind of conversion, but they seem to fail. Is this a bug? What is the problem?
I figured out, that one can implement the sql.Scanner interface by implementing the following method:
Scan(src interface{}) error
I tried to implement the method and I even was able to get the right value from src and convert it to a FileType, but I was confused if I should implement the method for "(f *FileType) or (f FileType). Either way the method gets invoked, however I'm not able to overwrite f (or at least the update gets lost later) and the File instances read from the DB always had a "0" as value for File.Type.
Do you have any ideas on those two points?
I recently had the same need, and the solution is to implement two interfaces:
sql/driver.Valuer
sql.Scanner
Here's a working example:
type FileType int64
func (u *FileType) Scan(value interface{}) error { *u = FileType(value.(int64)); return nil }
func (u FileType) Value() (driver.Value, error) { return int64(u), nil }
Slightly off-topic, but may be useful to others as I kept revisiting this question/answer when solving a similar problem when working with postgres enum fields in golang (which are returned as bytes).
// Status values
const (
incomplete Status = "incomplete"
complete Status = "complete"
reject Status = "reject"
)
type Status string
func (s *Status) Scan(value interface{}) error {
asBytes, ok := value.([]byte)
if !ok {
return errors.New("Scan source is not []byte")
}
*s = Status(string(asBytes))
return nil
}
func (s SubjectStatus) Value() (driver.Value, error) {
// validation would go here
return string(s), nil
}
Go needs to be specific with types, which can be a pain sometimes.
(f FileType) is cheaper than (f *FileType) for "native" types, pretty much unless you have a complex type, it's almost always better to not use a pointer.
What do you mean it doesn't overwrite it? did you resave the struct after you modified it?