Julia: Make Dataframe from iterator output - iterator

Question

I am attempting to create a data frame in Julia with two columns, x and y, representing the cartesian product of x values and y values. I think in summary this could be described as using iterator output, a 2D array of tuples, as the argument for DataFrame.
Here's code for obtaining an array of the product tuples (Julia 1.4.2):
x = [0:0.1:2;]
y = [0:1.5:30;]
product = collect(Iterators.product(x, y))
I want something like this:
x y
Float64 Float64
1 0.0 0.0
2 0.1 1.5
3 0.2 3.0
4 0.3 4.5
5 0.4 6.0
6 0.5 7.5
Many thanks for looking.

Answer 1

Cartesian product is a set of ordered tuples, so in fact Iterators.product returns in such case matrix iterator over tuples, i.e. calling collect on Iterators.product(x,y) will return Matrix{Tuple{Float64,Float64}}
But DataFrame constructor can eat iterator as an argument and return desired result
x = 0:0.1:2
y = 0:1.5:30
product = Iterators.product(x, y)
df = DataFrame(product)
rename!(df, [:x, :y])
Note that you cannot specify names of df in constructor like DataFrame(product, [:x, :y]) because there's no such method

Answer 2

Actually it is even easier. Just write:
julia> rename!(DataFrame(vec(product)), [:x, :y])
441×2 DataFrame
│ Row │ x │ y │
│ │ Float64 │ Float64 │
├─────┼─────────┼─────────┤
│ 1 │ 0.0 │ 0.0 │
│ 2 │ 0.1 │ 0.0 │
⋮
│ 439 │ 1.8 │ 30.0 │
│ 440 │ 1.9 │ 30.0 │
│ 441 │ 2.0 │ 30.0 │
Another nice pattern for two columns is:
julia> flatten(DataFrame(x=x, y=Ref(y)), :y)
441×2 DataFrame
│ Row │ x │ y │
│ │ Float64 │ Float64 │
├─────┼─────────┼─────────┤
│ 1 │ 0.0 │ 0.0 │
│ 2 │ 0.0 │ 1.5 │
⋮
│ 439 │ 2.0 │ 27.0 │
│ 440 │ 2.0 │ 28.5 │
│ 441 │ 2.0 │ 30.0 │
If you use Iterators.product without materializing it first it is a bit faster, and materializing Iterators.product is fastest (but uses a bit more memory):
julia> #benchmark rename!(DataFrame(Iterators.product($x, $y)), [:x, :y])
BenchmarkTools.Trial:
memory estimate: 10.98 KiB
allocs estimate: 56
--------------
minimum time: 9.400 μs (0.00% GC)
median time: 10.000 μs (0.00% GC)
mean time: 13.129 μs (8.31% GC)
maximum time: 5.644 ms (99.56% GC)
--------------
samples: 10000
evals/sample: 1
julia> #benchmark flatten(DataFrame(x=$x, y=Ref($y)), :y)
BenchmarkTools.Trial:
memory estimate: 17.72 KiB
allocs estimate: 80
--------------
minimum time: 10.299 μs (0.00% GC)
median time: 11.300 μs (0.00% GC)
mean time: 14.268 μs (7.56% GC)
maximum time: 5.400 ms (99.58% GC)
--------------
samples: 10000
evals/sample: 1
julia> #benchmark rename!(DataFrame(vec(collect(Iterators.product($x, $y)))), [:x, :y])
BenchmarkTools.Trial:
memory estimate: 18.03 KiB
allocs estimate: 58
--------------
minimum time: 6.600 μs (0.00% GC)
median time: 7.160 μs (0.00% GC)
mean time: 9.286 μs (11.73% GC)
maximum time: 1.104 ms (98.91% GC)
--------------
samples: 10000
evals/sample: 5

Answer 3

You can use Ranges just directly in the DataFrame constructor:
julia> using DataFrames
julia> df = DataFrame(x=0:0.1:2, y=0:1.5:30)
21×2 DataFrame
│ Row │ x │ y │
│ │ Float64 │ Float64 │
├─────┼─────────┼─────────┤
│ 1 │ 0.0 │ 0.0 │
│ 2 │ 0.1 │ 1.5 │
│ 3 │ 0.2 │ 3.0 │
│ 4 │ 0.3 │ 4.5 │
│ 5 │ 0.4 │ 6.0 │
│ 6 │ 0.5 │ 7.5 │
⋮
│ 20 │ 1.9 │ 28.5 │
│ 21 │ 2.0 │ 30.0 │