I can reproduce the problem I am currently having with Cargo.toml and lib.rs as shown below.
use ndarray::{ArrayD, ArrayViewD, ArrayViewMutD};
use numpy::{IntoPyArray, PyArrayDyn, PyReadonlyArrayDyn};
use pyo3::prelude::{pymodule, PyModule, PyResult, Python};
#[pymodule]
fn rust_ext(_py: Python<'_>, m: &PyModule) -> PyResult<()> {
// immutable example
fn axpy(a: f64, x: ArrayViewD<'_, f64>, y: ArrayViewD<'_, f64>) -> ArrayD<f64> {
a * &x + &y
}
// mutable example (no return)
fn mult(a: f64, mut x: ArrayViewMutD<'_, f64>) {
x *= a;
}
// wrapper of `axpy`
#[pyfn(m, "axpy")]
fn axpy_py<'py>(
py: Python<'py>,
a: f64,
x: PyReadonlyArrayDyn<f64>,
y: PyReadonlyArrayDyn<f64>,
) -> &'py PyArrayDyn<f64> {
let x = x.as_array();
let y = y.as_array();
axpy(a, x, y).into_pyarray(py)
}
// wrapper of `mult`
#[pyfn(m, "mult")]
fn mult_py(_py: Python<'_>, a: f64, x: &PyArrayDyn<f64>) -> PyResult<()> {
let x = unsafe { x.as_array_mut() };
mult(a, x);
Ok(())
}
Ok(())
}
This code is from the README.md of rust-numpy.
We also created a Cargo.toml as shown below.
[package]
name = "pyotest"
version = "0.1.0"
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
pyo3 = { version = "0.13",features=["extension-module"] }
numpy = "0.13"
ndarray = "0.14"
[lib]
name="preprocess"
crate-type = ["cdylib"]
If you do this, two versions of ndarray will be installed, resulting in version conflicts, so it will not compile.
The error message is as follows
Compiling pyotest v0.1.0 (/home/bokutotu/pyotest)
error[E0308]: mismatched types
--> src/lib.rs:50:17
|
50 | axpy(a, x, y).into_pyarray(py)
| ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
|
= note: expected struct `ArrayBase<ViewRepr<&f64>, Dim<IxDynImpl>>`
found struct `ndarray::ArrayBase<ndarray::ViewRepr<&f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
= note: perhaps two different versions of crate `ndarray` are being used?
error[E0308]: mismatched types
--> src/lib.rs:50:20
|
50 | axpy(a, x, y).into_pyarray(py)
| ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
|
= note: expected struct `ArrayBase<ViewRepr<&f64>, Dim<IxDynImpl>>`
found struct `ndarray::ArrayBase<ndarray::ViewRepr<&f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
= note: perhaps two different versions of crate `ndarray` are being used?
error[E0599]: no method named `into_pyarray` found for struct `ArrayBase<OwnedRepr<f64>, Dim<IxDynImpl>>` in the current scope
--> src/lib.rs:50:23
|
50 | axpy(a, x, y).into_pyarray(py)
| ^^^^^^^^^^^^ method not found in `ArrayBase<OwnedRepr<f64>, Dim<IxDynImpl>>`
error[E0308]: mismatched types
--> src/lib.rs:57:17
|
57 | mult(a, x);
| ^ expected struct `ArrayBase`, found struct `ndarray::ArrayBase`
|
= note: expected struct `ArrayBase<ViewRepr<&mut f64>, Dim<IxDynImpl>>`
found struct `ndarray::ArrayBase<ndarray::ViewRepr<&mut f64>, ndarray::dimension::dim::Dim<ndarray::dimension::dynindeximpl::IxDynImpl>>`
= note: perhaps two different versions of crate `ndarray` are being used?
warning: unused import: `IntoPyArray`
--> src/lib.rs:25:13
|
25 | use numpy::{IntoPyArray, PyArrayDyn, PyReadonlyArrayDyn};
| ^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
error: aborting due to 4 previous errors; 1 warning emitted
Some errors have detailed explanations: E0308, E0599.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `pyotest`
To learn more, run the command again with --verbose.
You can also check Cargo.lock to see that it is trying to install two versions of ndarray.
....
[[package]]
name = "ndarray"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c0d5c9540a691d153064dc47a4db2504587a75eae07bf1d73f7a596ebc73c04"
dependencies = [
"matrixmultiply 0.2.4",
"num-complex 0.3.1",
"num-integer",
"num-traits",
"rawpointer",
]
...
[[package]]
name = "ndarray"
version = "0.15.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08e854964160a323e65baa19a0b1a027f76d590faba01f05c0cbc3187221a8c9"
dependencies = [
"matrixmultiply 0.3.1",
"num-complex 0.4.0",
"num-integer",
"num-traits",
"rawpointer",
]
...
[[package]]
name = "numpy"
version = "0.13.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a996bcd58fb29bef9debf717330cd8876c3b4adbeea4939020a5326a3afad4d9"
dependencies = [
"cfg-if 0.1.10",
"libc",
"ndarray 0.15.3",
"num-complex 0.4.0",
"num-traits",
"pyo3",
]
As you can see, I have installed a strange version due to rust-numpy. What should I do in this case?
I want to use 0.14 for ndarray-linalg because of the version of ndarray.
Related
I have two modules with unit tests and I want to access constants/variables with expected test results from both of those modules.
File: src/lib.rs
mod other;
#[cfg( test )]
mod tests {
mod expected;
use crate::tests::expected::FOUR;
#[test]
fn it_works() {
let result = 2 + 2;
assert_eq!( result, FOUR );
}
}
File: src/other.rs
#[cfg( test )]
mod tests {
use crate::tests::expected::EIGHT;
#[test]
fn it_works() {
let result = 4 + 4;
assert_eq!( result, EIGHT );
}
}
File src/tests/expected.rs
pub const FOUR: i32 = 4;
pub const EIGHT: i32 = 8;
From src/lib.rs I can access the constants within expected.rs. But this is not true from src/other.rs. I get the follwing error:
error[E0603]: module `expected` is private
--> src/other.rs:3:20
|
3 | use crate::tests::expected::EIGHT;
| ^^^^^^^^ private module
|
note: the module `expected` is defined here
--> src/lib.rs:5:2
|
5 | mod expected;
| ^^^^^^^^^^^^^
I have no idea how to make expected public. Where would I place expected.rs or how would I make it accessible from the test code of other modules of the library?
I'm coding a hash in Rust for practice. The code looks like this:
pub fn get_fnv1a32(to_hash:&str) -> u32{
const OFFSET_BASIS:u32 = 2_166_136_261;
const PRIME:u32 = 16_777_619;
if !to_hash.is_empty(){
let mut hash = OFFSET_BASIS;
for b in to_hash.bytes(){
hash = hash ^ (b as u32);
hash = hash.wrapping_mul(PRIME);
}
hash
}
else
{
0
}
}
And this is the code I'm trying to use to test this:
mod fnv;
#[cfg(test)]
mod tests {
#[test]
fn get_correct_hash(){
assert_eq!(0x7a78f512, fnv::get_fnv1a32("Hello world!"));
}
#[test]
fn hash_handles_empty_string_correctly(){
assert_eq!(0, fnv::get_fnv1a32(""));
}
}
The test code is in lib.rs and the get_fnv1a32 function is in fnv.rs. They're both in the same directory. But when I try to run cargo test I keep getting these messages:
Compiling hashes v0.1.0 (U:\skunkworks\rust\hashes)
warning: function is never used: `get_fnv1a32`
--> src\fnv.rs:1:8
|
1 | pub fn get_fnv1a32(to_hash:&str) -> u32{
| ^^^^^^^^^^^
|
= note: `#[warn(dead_code)]` on by default
error[E0433]: failed to resolve: use of undeclared type or module `fnv`
--> src\lib.rs:7:32
|
7 | assert_eq!(0x7a78f512, fnv::get_fnv1a32("Hello world!"));
| ^^^ use of undeclared type or module `fnv`
error[E0433]: failed to resolve: use of undeclared type or module `fnv`
--> src\lib.rs:12:23
|
12 | assert_eq!(0, fnv::get_fnv1a32(""));
| ^^^ use of undeclared type or module `fnv`
error: aborting due to 2 previous errors
I can't figure out what I'm doing wrong. I tried changing the mod fnv; line at the top to pub mod fnv; and that gets rid of the dead code warning but it doesn't fix the two errors. What do I need to do to get the get_fnv1a32 function to be visible in the lib.rs file?
Not that I would think it would matter but the version of rustc is rustc 1.41.0 (5e1a79984 2020-01-27)
The test module is separate from the outer module. Add
use super::*;
or an equivalent statement like use crate::fnv inside the tests module to make the fnv module visible.
I have a file that I wish to read and filter the data into two different sets and determine the number of items in each set.
use std::io::{self, BufRead};
fn main() {
let cursor = io::Cursor::new(b"pillow\nbrick\r\nphone");
let lines = cursor.lines().map(|l| l.unwrap());
let soft_count = lines.filter(|line| line.contains("pillow")).count();
let hard_count = lines.filter(|line| !line.contains("pillow")).count();
}
Playground
GitHub
However, the borrow checker gives me an error:
error[E0382]: use of moved value: `lines`
--> src/main.rs:14:22
|
8 | let lines = cursor.lines().map(|l| l.unwrap());
| ----- move occurs because `lines` has type `std::iter::Map<std::io::Lines<std::io::Cursor<&[u8; 19]>>, [closure#src/main.rs:8:36: 8:50]>`, which does not implement the `Copy` trait
9 |
10 | let soft_count = lines
| ----- value moved here
...
14 | let hard_count = lines
| ^^^^^ value used here after move
I tried getting around this using reference counting to allow multiple ownership:
use std::io::{self, BufRead};
use std::rc::Rc;
fn main() {
let cursor = io::Cursor::new(b"pillow\nbrick\r\nphone");
let lines = Rc::new(cursor.lines().map(|l| l.unwrap()));
let soft_count = Rc::clone(&lines)
.filter(|line| line.contains("pillow"))
.count();
let hard_count = Rc::clone(&lines)
.filter(|line| !line.contains("pillow"))
.count();
}
Playground
Github
I get a similar error message:
error[E0507]: cannot move out of an `Rc`
--> src/main.rs:11:22
|
11 | let soft_count = Rc::clone(&lines)
| ^^^^^^^^^^^^^^^^^ move occurs because value has type `std::iter::Map<std::io::Lines<std::io::Cursor<&[u8; 19]>>, [closure#src/main.rs:9:44: 9:58]>`, which does not implement the `Copy` trait
error[E0507]: cannot move out of an `Rc`
--> src/main.rs:15:22
|
15 | let hard_count = Rc::clone(&lines)
| ^^^^^^^^^^^^^^^^^ move occurs because value has type `std::iter::Map<std::io::Lines<std::io::Cursor<&[u8; 19]>>, [closure#src/main.rs:9:44: 9:58]>`, which does not implement the `Copy` trait
You cannot. Instead, you will need to clone the iterator, or some building block of it. In this case, the highest thing you can clone is the Cursor:
use std::io::{self, BufRead};
fn main() {
let cursor = io::Cursor::new(b"pillow\nbrick\r\nphone");
let lines = cursor.clone().lines().map(|l| l.unwrap());
let lines2 = cursor.lines().map(|l| l.unwrap());
let soft_count = lines.filter(|line| line.contains("pillow")).count();
let hard_count = lines2.filter(|line| !line.contains("pillow")).count();
}
For an actual File, you will need to use try_clone as it might fail. In either case, you will be referring to the same data twice and only the iterator information will be kept.
For your specific case, you don't need any of this. In fact, iterating over the data twice is inefficient. The simplest built-in thing you can do is to partition the iterator:
let (softs, hards): (Vec<_>, Vec<_>) = lines.partition(|line| line.contains("pillow"));
let soft_count = softs.len();
let hard_count = hards.len();
This is still a bit inefficient as you don't need the actual values. You could create your own type that implements Extend and discards the values:
#[derive(Debug, Default)]
struct Count(usize);
impl<T> std::iter::Extend<T> for Count {
fn extend<I>(&mut self, iter: I)
where
I: IntoIterator,
{
self.0 += iter.into_iter().count();
}
}
let (softs, hards): (Count, Count) = lines.partition(|line| line.contains("pillow"));
let soft_count = softs.0;
let hard_count = hards.0;
You could also just use a for loop or build something on top of fold:
let (soft_count, hard_count) = lines.fold((0, 0), |mut state, line| {
if line.contains("pillow") {
state.0 += 1;
} else {
state.1 += 1;
}
state
});
I have a trait
trait B {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
I want to get all the Indexes within bounds:
fn iterate<T: B>(it: &T) {
let (low, high) = it.bounds();
for i in low..high {}
}
This won't work since there's no constraint that the type T can be "ranged" over, and the compiler says as much:
error[E0277]: the trait bound `<T as B>::Index: std::iter::Step` is not satisfied
--> src/main.rs:8:5
|
8 | for i in low..high {}
| ^^^^^^^^^^^^^^^^^^^^^ the trait `std::iter::Step` is not implemented for `<T as B>::Index`
|
= help: consider adding a `where <T as B>::Index: std::iter::Step` bound
= note: required because of the requirements on the impl of `std::iter::Iterator` for `std::ops::Range<<T as B>::Index>`
I tried adding the Step bound to Index
use std::iter::Step;
trait B {
type Index: Sized + Copy + Step;
fn bounds(&self) -> (Self::Index, Self::Index);
}
but apparently it isn't stable:
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:1:5
|
1 | use std::iter::Step;
| ^^^^^^^^^^^^^^^
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:4:32
|
4 | type Index: Sized + Copy + Step;
| ^^^^
Am I missing something or is it just not possible to do so right now?
If you want to require that a Range<T> can be iterated over, just use that as your trait bound:
trait Bounded {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
fn iterate<T>(it: &T)
where
T: Bounded,
std::ops::Range<T::Index>: IntoIterator,
{
let (low, high) = it.bounds();
for i in low..high {}
}
fn main() {}
To do this kind of thing generically the num crate is helpful.
extern crate num;
use num::{Num, One};
use std::fmt::Debug;
fn iterate<T>(low: T, high: T)
where
T: Num + One + PartialOrd + Copy + Clone + Debug,
{
let one = T::one();
let mut i = low;
loop {
if i > high {
break;
}
println!("{:?}", i);
i = i + one;
}
}
fn main() {
iterate(0i32, 10i32);
iterate(5u8, 7u8);
iterate(0f64, 10f64);
}
Given this definition for foo:
let foo = vec![vec![1, 2, 3], vec![4, 5, 6], vec![7, 8, 9]];
I'd like to be able to write code like this:
let result: Vec<_> = foo.iter()
.enumerate()
.flat_map(|(i, row)| if i % 2 == 0 {
row.iter().map(|x| x * 2)
} else {
std::iter::empty()
})
.collect();
but that raises an error about the if and else clauses having incompatible types. I tried removing the map temporarily and I tried defining an empty vector outside the closure and returning an iterator over that like so:
let empty = vec![];
let result: Vec<_> = foo.iter()
.enumerate()
.flat_map(|(i, row)| if i % 2 == 0 {
row.iter() //.map(|x| x * 2)
} else {
empty.iter()
})
.collect();
This seems kind of silly but it compiles. If I try to uncomment the map then it still complains about the if and else clauses having incompatible types. Here's part of the error message:
error[E0308]: if and else have incompatible types
--> src/main.rs:6:30
|
6 | .flat_map(|(i, row)| if i % 2 == 0 {
| ______________________________^
7 | | row.iter().map(|x| x * 2)
8 | | } else {
9 | | std::iter::empty()
10 | | })
| |_________^ expected struct `std::iter::Map`, found struct `std::iter::Empty`
|
= note: expected type `std::iter::Map<std::slice::Iter<'_, {integer}>, [closure#src/main.rs:7:28: 7:37]>`
found type `std::iter::Empty<_>`
Playground Link
I know I could write something that does what I want with some nested for loops but I'd like to know if there's a terse way to write it using iterators.
Since Rust is statically typed and each step in an iterator chain changes the result to a new type that entrains the previous types (unless you use boxed trait objects) you will have to write it in a way where both branches are covered by the same types.
One way to convey conditional emptiness with a single type is the TakeWhile iterator implementation.
.flat_map(|(i, row)| {
let iter = row.iter().map(|x| x * 2);
let take = i % 2 == 0;
iter.take_while(|_| take)
})
If you don't mind ignoring the edge-case where the input iterator foo could have more than usize elements you could also use Take instead with either 0 or usize::MAX. It has the advantage of providing a better size_hint() than TakeWhile.
In your specific example, you can use filter to remove unwanted elements prior to calling flat_map:
let result: Vec<_> = foo.iter()
.enumerate()
.filter(|&(i, _)| i % 2 == 0)
.flat_map(|(_, row)| row.iter().map(|x| x * 2))
.collect();
If you ever want to use it with map instead of flat_map, you can combine the calls to filter and map by using filter_map which takes a function returning an Option and only keeps elements that are Some(thing).