Why do generators not support map()? - iterator

It seems utterly natural to me that generators, which function very much like Arrays, should support the very basic list operations, like map(), filter(), and reduce(). Am I missing something?
I wrote the code for map and it seems simple enough, but it would be much better to have all the functions embedded in all the generators:
let fancyGen = g => {
let rv = function*() {
for (let x of g)
yield x;
}
rv.map = function*(p) {
for (let x of g)
yield p(x);
}
return rv;
}
I'm new to generators, so any comments on the code are welcome. In particular, is that the best way to write "the identity generator"?

Why do generators not support map()?
Because it's too easy to fill in as a userland implementation. ES3 didn't include Array iteration methods either, maybe will see transformers for iterators in ES7 :-)
generators, which function very much like Arrays
No, please stop and distinguish iterators from generators:
An iterator is an object with a .next() method that conforms to the iterator protocol.
A generator is an iterator created by a generator function (function*). Its .next() method takes an argument which is the result of each yield inside the generator function. It also has .return() and .throw() methods.
You'll mostly be interested in iterators, where we don't pass values to next, and don't care about the end result - just like for of loops do. We can extend them with the desired methods easily:
var IteratorPrototype = Object.getPrototypeOf(Object.getPrototypeOf([][Symbol.iterator]()));
IteratorPrototype.map = function*(f) {
for (var x of this)
yield f(x);
};
IteratorPrototype.filter = function*(p) {
for (var x of this)
if (p(x))
yield x;
};
IteratorPrototype.scan = function*(f, acc) {
for (var x of this)
yield acc = f(acc, x);
return acc;
};
IteratorPrototype.reduce = function(f, acc) {
for (var x of this)
acc = f(acc, x);
return acc;
};
These should suffice for the start, and most common use cases. A proper library will extend this to generators so that values are passed through appropriately, and also will deal with the problem that iterators can be used only once before they are exhausted (in contrast to arrays).

Related

What is this anti-pattern called (using parent scopes to pass state)?

I'm trying to describe to a colleague issues I have with how their code is structured, and I'm looking for the name of the anti-pattern he's implemented (bonus points for the software principals it violates). I'm using JS to demonstrate, but this isn't JS specific.
function x() {
var a, b, c;
var doWork = function(){
a = 1;
b = 2;
addAB();
return c;
};
var addAB = function(){
c = a + b;
};
var result = doWork();
}
He's passing information into and out of functions/methods using the parent scope. It makes understanding the code very difficult.
I don't know that there is an official name for it but, the issue you are describing is creating functions with side effects.
You don't want to have any function that modifies anything outside of its own scope. Having a shared member (in this case a, b, & c) that can be modified by any other function can lead to unknown and/or inconsistent states and/or behaviors.
I believe that your concerns aren't applicable to JavaScript (and many other programming languages). Your code and your team mates are using closures:
Closures are functions that refer to independent (free) variables
(variables that are used locally, but defined in an enclosing scope).
In other words, these functions 'remember' the environment in which
they were created.
In JavaScript and many other languages which can create closures is very common to access parent scope's references and it provides more power to code rather than pain. Obviously, a wrongly used tool produces pain, but I should analyze your mates' code to be sure that it's not that you're just against closures.
In summary, closures aren't an anti-pattern. They're a language feature.
For example, your code could be an actual use case like DOM event handling:
var text = "";
document.getElementById("someButton").addEventListener(function() {
text = document.getElementById("someInput").value;
});
And some developers implement something like private functions defining them inside a constructor function:
function A() {
this.text = "";
var that = this;
function fillText() {
that.text = "hello world";
}
fillText();
}
var a = new A();
console.log(a.text); // "hello world"

Whats the best way to write an iterator supporting multiple logic branches?

In Rust I've started writing iterators, converting them from code which took a callback function.
I ran into the problem where the code that used a callback in multiple branches of the function didn't convert so cleanly into a Rust iterator.
To give some pseudo-code.
// function using callbacks where the caller can exit at any time,
// can be used in a similar way to an iterator.
fn do_stuff(args, callback_fn(cb_args)) {
// define a, b, c... args
if callback_fn(a, b, 0) == false { return; }
for i in 0..n {
if callback_fn(c, d, i) == false { return; }
}
if callback_fn(e, f, -1) == false { return; }
}
Converting this to an iterator was rather awkward since I needed to store some state representing each branch.
impl Iterator for MyStruct {
fn next(&mut self) -> Option<MyResult> {
let out = match (self.state) {
0 => {
self.state += 1;
Some(MyResult(self.a, self.b, 0))
},
1 => {
self.i += 1;
if self.i == self.n {
self.state += 1;
}
Some(MyResult(self.c, self.d, self.i - 1))
},
2 => {
self.state += 1;
Some(MyResult(self.e, self.f, -1))
},
_ => {
None
},
}
return out;
}
// --- snip
With the example above, this is arguably acceptable, (if a little awkward). Consider cases with multiple for loops, variable scopes, where its much harder to track state.
While I didn't try these, I imagine there are some ways to achieve this which in most cases are less-then-ideal workarounds:
Using the callback version, building a vector, then iterating over it... (works but defeats the purpose of using an iterator, no way to early exit and avoid creating the entire data set for eg).
Writing an iterator which communicates with a thread that uses similar logic to the callback version.(while possible, the overhead of creating OS threads makes it a poor choice in many cases).
Besides the workarounds above:
Are there ways to write iterators like the example given, with less convoluted logic?Ideally more like the example that uses callbacks.
Otherwise are there other ways to handle this?
Or is this simply not supported in Rust?
Note, the same logic applies coming from Python generators (using yield instead of a callback, using callbacks as an example here since they're ubiquitous with first class functions).
Languages like C# and Python provide a way to generate iterators from methods written using a special yield keyword. As of Rust 1.11, there is no such feature in the language. However, such a feature is planned (see RFC) (indeed, yield is a reserved keyword!) and would likely work as in C# (i.e. the compiler would generate a struct with the necessary state and implementation for Iterator).
In the meantime, you could try Stateful, a project that attempts to provide this feature. (This blog post explains how Stateful works, and the challenges involved.)

How to rewrite this in terms of R.compose

var take = R.curry(function take(count, o) {
return R.pick(R.take(count, R.keys(o)), o);
});
This function takes count keys from an object, in the order, in which they appear. I use it to limit a dataset which was grouped.
I understand that there are placeholder arguments, like R.__, but I can't wrap my head around this particular case.
This is possible thanks to R.converge, but I don't recommend going point-free in this case.
// take :: Number -> Object -> Object
var take = R.curryN(2,
R.converge(R.pick,
R.converge(R.take,
R.nthArg(0),
R.pipe(R.nthArg(1),
R.keys)),
R.nthArg(1)));
One thing to note is that the behaviour of this function is undefined since the order of the list returned by R.keys is undefined.
I agree with #davidchambers that it is probably better not to do this points-free. This solution is a bit cleaner than that one, but is still not to my mind as nice as your original:
// take :: Number -> Object -> Object
var take = R.converge(
R.pick,
R.useWith(R.take, R.identity, R.keys),
R.nthArg(1)
);
useWith and converge are similar in that they accept a number of function parameters and pass the result of calling all but the first one into that first one. The difference is that converge passes all the parameters it receives to each one, and useWith splits them up, passing one to each function. This is the first time I've seen a use for combining them, but it seems to make sense here.
That property ordering issue is supposed to be resolved in ES6 (final draft now out!) but it's still controversial.
Update
You mention that it will take some time to figure this out. This should help at least show how it's equivalent to your original function, if not how to derive it:
var take = R.converge(
R.pick,
R.useWith(R.take, R.identity, R.keys),
R.nthArg(1)
);
// definition of `converge`
(count, obj) => R.pick(R.useWith(R.take, R.identity, R.keys)(count, obj),
R.nthArg(1)(count, obj));
// definition of `nthArg`
(count, obj) => R.pick(R.useWith(R.take, R.identity, R.keys)(count, obj), obj);
// definition of `useWith`
(count, obj) => R.pick(R.take(R.identity(count), R.keys(obj)), obj);
// definition of `identity`
(count, obj) => R.pick(R.take(count, R.keys(obj)), obj);
Update 2
As of version 18, both converge and useWith have changed to become binary. Each takes a target function and a list of helper functions. That would change the above slightly to this:
// take :: Number -> Object -> Object
var take = R.converge(R.pick, [
R.useWith(R.take, [R.identity, R.keys]),
R.nthArg(1)
]);

Implement DISTINCT in for comprehension

In the penultimate lecture of his Coursera course, Prof. Odersky offered the following for comprehension as the final step in a lovely case study:
def solutions(target: Int): Stream[Path] =
for {
pathSet <- pathSets
path <- pathSet
if path.endState contains target
} yield path
In an earlier lecture he drew some analogies between for comprehensions and SQL.
What I'm looking for is a way to yield only those paths that have a DISTINCT endState.
Is there a way to refer back from within a filter clause of the same comprehension to the items that have already been yielded?
Another approach might be to convert pathSets to a Map from endState to path before the for statement, then convert it back to a Stream before returning it. However, this would seem to lose the lazy computation benefits of using a Stream.
An earlier method from the same case study accomplished similar goals, but it was already a recursive function, while this one doesn't (seem to) need to be recursive.
It looks like I could use a mutable Set to track the endStates that get yielded, but that feels unsatisfying, since the course has successfully avoided using mutability so far.
Is there a way to refer back from within a filter clause of the same comprehension to the items that have already been yielded?
Your for comprehension desugars to something more or less like
pathSets flatMap {
pathSet => pathSet filter {
path => path.endState contains target
}
} map {path => path}
The last map with an identity function is your yield. I can't remember if the spec allows that map to be elided when it's an identity function.
Anyway, I hope this shows more clearly why there's no "reaching back" with that structure.
You can write a lazy, recursive distinctBy function
implicit class DistinctStream[T](s: Stream[T]) {
def distinctBy[V](f: T => V): Stream[T] = {
def distinctBy(remainder: Stream[T], seen:Set[V]): Stream[T] =
remainder match {
case head #:: tail =>
val value = f(head)
if (seen contains value) distinctBy(tail, seen)
else Stream.cons(head, distinctBy(tail, seen + value))
case empty => empty
}
distinctBy(s, Set())
}
}
And use it like so
def solutions(target: Int): Stream[Path] =
(for {
pathSet <- pathSets
path <- pathSet
if path.endState contains target
} yield path) distinctBy (_.endState)
Yeah, now there's recursion. But there already was because Stream's map, flatMap, and filter functions are all lazy recursive functions already.

How do I use Strategy Pattern in this context?

Let me begin by saying I am a mathematician and not a coder. I am trying to code a linear solver. There are 10 methods which I coded. I want the user to choose which solver she wishes to use, like options.solver_choice='CG'.
Now, I have all 10 methods coded in a single class. How do I use the strategy pattern in this case?
Previously, I had 10 different function files which I used to use in the main program using a switch case.
if options.solver_choice=='CG'
CG(A,x,b);
if options.solver_choice=='GMRES'
GMRES(A,x,b);
.
.
.
This isn't the most exact of answers, but you should get the idea.
Using the strategy pattern, you would have a solver interface that implements a solver method:
public interface ISolver {
int Solve();
}
You would implement each solver class as necessary:
public class Solver1 : ISolver {
int Solve() {
return 1;
}
}
You would then pass the appropriate solver class when it's time to do the solving:
public int DoSolve(ISolver solver) {
return solver.solve();
}
Foo.DoSolve(new Solver1());
TL;DR
As I've always understood the strategy pattern, the idea is basically that you perform composition of a class or object at run-time. The implementation details vary by language, but you should be able to swap out pieces of behavior by "plugging in" different modules that share an interface. Here I present an example in Ruby.
Ruby Example
Let's say you want to use select a strategy for how the #action method will return a set of results. You might begin by composing some modules named CG and GMRES. For example:
module CG
def action a, x, b
{ a: a, x: x, b: b }
end
end
module GMRES
def action a, x, b
[a, x, b]
end
end
You then instantiate your object:
class StrategyPattern
end
my_strategy = StrategyPattern.new
Finally, you extend your object with the plug-in behavior that you want. For example:
my_strategy.extend GMRES
my_strategy.action 'q', nil, 1
#=> ["q", nil, 1]
my_strategy.extend GMRES
my_strategy.action 'q', nil, 1
#=> {:a=>"q", :x=>nil, :b=>1}
Some may argue that the Strategy Pattern should be implemented at the class level rather than by extending an instance of a class, but this way strikes me as easier to follow and is less likely to screw up other instances that need to choose other strategies.
A more orthodox alternative would be to pass the name of the module to include into the class constructor. You might want to read Russ Olsen's Design Patterns in Ruby for a more thorough treatment and some additional ways to implement the pattern.
Other answers present the pattern correctly, however I don't feel they are clear enough. Unfortunately the link I've provided does the same, so I'll try to demonstrate what's the Strategy's spirit, IMHO.
Main thing about strategy is to have a general procedure, with some of its details (behaviours) abstracted, allowing them to be changed transparently.
Consider an gradient descent optimization algorithm - basically, it consists of three actions:
gradient estimation
step
objective function evaluation
Usually one chooses which of these steps they need abstracted and configurable. In this example it seems that evaluation of the objective function is not something that you can do in more than one way - you always just ... evaluate the function.
So, this introduces two different strategy (or policy) families then:
interface GradientStrategy
{
double[] CalculateGradient(Function objectiveFunction, double[] point);
}
interface StepStrategy
{
double[] Step(double[] gradient, double[] point);
}
where of course Function is something like:
interface Function
{
double Evaluate(double[] point);
}
interface FunctionWithDerivative : Function
{
double[] EvaluateDerivative(double[] point);
}
Then, a solver using all these strategies would look like:
interface Solver
{
double[] Maximize(Function objectiveFunction);
}
class GradientDescentSolver : Solver
{
public Solver(GradientStrategy gs, StepStrategy ss)
{
this.gradientStrategy = gs;
this.stepStrategy = ss;
}
public double[] Maximize(Function objectiveFunction)
{
// choosing starting point could also be abstracted into a strategy
double[] currentPoint = ChooseStartingPoint(objectiveFunction);
double[] bestPoint = currentPoint;
double bestValue = objectiveFunction.Evaluate(bestPoint);
while (...) // termination condition could also
// be abstracted into a strategy
{
double[] gradient = this.gradientStrategy.CalculateGradient(
objectiveFunction,
currentPoint);
currentPoint = this.stepStrategy.Step(gradient, currentPoint);
double currentValue = objectiveFunction.Evaluate(currentPoint);
if (currentValue > bestValue)
{
bestValue = currentValue;
bestPoint = currentPoint;
}
else
{
// terminate or step back and reduce step size etc.
// this could also be abstracted into a strategy
}
}
return bestPoint;
}
private GradientStrategy gradientStrategy;
private StepStrategy stepStrategy;
}
So the main point is that you have some algorithm's outline, and you delegate particular, general steps of this algorithm to strategies or policies. Now you could implement GradientStrategy which works only for FunctionWithDerivative (casts down) and just uses function's analytical derivative to obtain the gradient. Or you could have another one implementing stochastic version of gradient estimation. Note, that the main solver does not need to know about how the gradient is being calculated, it just needs the gradient. The same thing goes for the StepStrategy - it can be a typical step policy with single step-size:
class SimpleStepStrategy : StepStrategy
{
public SimpleStepStrategy(double stepSize)
{
this.stepSize = stepSize;
}
double[] Step(double[] gradient, double[] point)
{
double[] result = new double[point.Length];
for (int i = 0;i < result.Length;++i)
{
result[i] = point[i] + this.stepSize * gradient[i];
}
return result;
}
private double stepSize;
}
, or a complicated algorithm adjusting the step-size as it goes.
Also think about the behaviours noted in the comments in the code: TerminationStrategy, DeteriorationPolicy.
Names are just examples - they're probably not the best, but I hope they give the intent. Also, usually best to stick with one version (Strategy or Policy).
PHP Examples
You'd define your strategies that implement only singular method called solve()
class CG
{
public function solve($a, $x, $y)
{
//..implementation
}
}
class GMRES
{
public function solve($a, $x, $y)
{
// implementation..
}
}
Usage:
$solver = new Solver();
$solver->setStratery(new CG());
$solver->solve(1,2,3); // the result of CG
$solver->setStrategy(new GMRES());
$solver->solve(1,2,3); // the result of GMRES
class Solver
{
private $strategy;
public function setStrategy($strategy)
{
$this->strategy = $strategy;
}
public function solve($a, $x, $y)
{
return $this->strategy->solve($a, $x, $y);
}
}