Imagine that I have integers, n,q and vectors/arrays with these dimensions:
import numpy as np
n = 100
q = 102
A = np.random.normal(size=(n,n))
B = np.random.normal(size=(q, ))
C = np.einsum("i, jk -> ijk", B, A)
D = np.einsum('ijk, ikj -> k', C, C)
which is working fine if all intermediate arrays fit in memory.
Now assume that I can store in memory arrays of size (n,n), (q,n) but not any three dimensional arrays such as with shape (n,n,q). I cannot store in memory array C above. Instead, to compute D,
D1 = np.einsum('i, jk, i, kj -> k', B, A, B, A, optimize='optimal')
works fine and np.einsum is typically smart enough to find a einsum_path so that no 3d array is ever constructed. Great!
Now let's complicate things slightly:
C = np.einsum("i, jk -> ijk", B, A) # as before
Y2 = np.random.normal(size=(n, ))
Z2 = np.random.normal(size=(q, n))
C2 = np.einsum("j, ik -> ijk", Y2, Z2)
E = np.einsum('ijk, ikj -> k', C+C2, C+C2)
Here I cannot find a reasonable way (reasonable, as in short/readable code) to construct E without constructing intermediate 3d arrays such as C and C2.
Questions:
is there a np.einsum one liner that would construct E, without constructing the intermediate 3d arrays C and C2?
The following appears to work by expanding into four terms, but is rather impractical compared to the hypothetical API in question 2...
E_CC = np.einsum('i, jk, i, kj -> k', B, A, B, A, optimize='optimal') # as D before
E_C2C2 = np.einsum('j, ik, k, ij -> k', Y2, Z2, Y2, Z2, optimize='optimal')
E_CC2 = np.einsum('i, jk, k, ij -> k', B, A, Y2, Z2, optimize='optimal')
E_C2C = np.einsum('j, ik, i, kj -> k', Y2, Z2, B, A, optimize='optimal')
E_new = E_CC + E_C2C2 + E_CC2 + E_C2C
np.isclose(E_new, E) # all True!
Is there a ''lazy'' version of np.einsum that would wait before the final call to find an optimal einsum_path throughout the composition of several lazy einsum, including sums as in the above example? For instance, with an hypothetical einsum_lazy, the following would construct E without storing a 3d array (such as C or C2) in memory:
C = np.einsum_lazy("i, jk -> ijk", B, A) # nothing has been computed yet!
C2 = np.einsum_lazy("j, ik -> ijk", Y2, Z2) # nothing has been computed yet!
E = np.einsum('ijk, ikj -> k', C+C2, C+C2) # expand the sums and uses optimal einsum_path to compute E
This is a really fascinating question - as #s-m-e mentioned, numpy does not offer a lazy einsum computations, but it does offer a lower level function called np.einsum_path, which np.einsum uses to actually find the optimal contractions.
What if you did this:
C_path = np.einsum_path("i, jk -> ijk", B, A)[0]
C2_path = np.einsum_path("j, ik -> ijk", Y2, Z2)[0]
CC2_path = C_path + C2_path[1:]
And somehow used the path in a final computation? The biggest issue here is that you're summing C and C2, and elementwise addition is not currently supported by einsum, so it's hard to optimize that.
Take a look at #Eelco Hoogendoorn's response to a similar question: maybe breaking it up into smaller computations isn't such a bad idea :)
Targeting question 2:
There is no lazy version of einsum, unfortunately. einsum simply returns a numpy ndarray object - which is precisely what a subsequent call to einsum would expect as a parameter in your scenario. However, you can exploit Python itself by using generators. In your case, the following would do the trick:
C1 = (np.einsum_lazy("i, jk -> ijk", b, a) for a, b in ((A, B),))
C2 = (np.einsum_lazy("j, ik -> ijk", y2, z2) for y2, z2 in ((Y2, Z2),))
def _einsum(v, w):
u = v + w # no need to do this twice
return np.einsum('ijk, ikj -> k', u, u)
E = (_einsum(c1, c2) for c1, c2 in ((C1, C2),))
for e in E: # only HERE C1, C2 and E are actually computed
print(e)
The above example used chained generator expressions. It's the final for loop, which triggers the actual evaluation of the chain. It's lazy, more or less. There is also another downside: From a memory perspective, C1 and C2 are in fact constructed/created (temporarily).
If memory consumption is your primary concern and if you are doing multiple similar operations, you may have a look at the out parameter of einsum. In fact, most numpy ufuncs happen to have an out parameter, which allows you to specify a "preexisting" numpy ndarray as a target for the result of the operation. Therefore, no new memory needs to be allocated, which is also speeding up your computation as a side-effect.
what's the difference between
L1 = {m ∈ (a + b)*} L1 = {m ∈ (a , b)*}
i feel like there is no difference but i cant really understand the difference between this + and the plus in this expression for example (a,b)+
You are correct, there is no difference. a+b translates to a union b and (a , b) is another (albeit much less common) way that some researchers refer to a union b. If you so choose to continue your study of automata you will find that there is an incredible variety of ways that different researchers refer to the same concepts. In this instance alone, off the top of my head I know this can be written as:
L1 = {m ∈ (a + b)*}
L1 = {m ∈ (a , b)*}
L1 = {m ∈ (a | b)*}
L1 = {m ∈ (a U b)*}
Consider a dependent type
Inductive dep (n: nat) :=
mkDep : dep n.
Now, consider a simple theorem I wish to prove:
Theorem equalTypes (n n': nat): n = n' -> dep n = dep n'.
Proof.
intros.
Abort.
How do I show that two dependent types are equal? What is a notion of type equality?
Worse, consider this "theorem" (which does not compile)
Theorem equalInhabitants (n n' : nat): n = n' -> mkDep n = mkDep n'.
Abort.
This very statement is wrong, because the types mkDep n and mkDep n' don't match. However, in some sense, this statement is true, because they are the same value under the assumption n = n'.
I wish to understand how to formalize and prove statements about dependent types (specifically, their equality and notions thereof)
How do I show that two dependent types are equal?
In this case, you can prove it with apply f_equal; assumption or subst; reflexivity (or destruct H; reflexivity or case H; reflexivity or induction H; reflexivity or exact (eq_rect n (fun n' => dep n = dep n') eq_refl n' H)).
What is a notion of type equality?
The same as any other equality; Print eq. gives:
Inductive eq (A : Type) (x : A) : A -> Prop := eq_refl : x = x
which says that the only special fact you have to construct a proof of equality is that x = x for any x. The way to use a proof of equality, eq_rect, is that, if you have x = y, to prove a property P of y, it suffices to prove P of x. In this case, since we have n = n', to prove dep n = dep n', it suffices to prove dep n = dep n (where P := fun n' => dep n = dep n').
There is a deeper sense in which this question can be asked, because it turns out that equality of types in Coq is under-constrained. Given
Inductive unit1 := tt1.
Inductive unit2 := tt2.
you can not prove unit1 = unit2, nor can you prove unit1 <> unit2. In fact, it turns out that the only type inequalities T1 <> T2 that you can prove are cases where you can prove that T1 and T2 are not isomorphic. The Univalence axiom is a way of "filling in the details" of type equality to say that any isomorphic types are equal. There are other consistent interpretations, though (for example, I believe that it's consistent to assume A * B = C * D -> A = C /\ B = D, though this contradicts univalence).
Worse, consider this "theorem" (which does not compile)
Theorem equalInhabitants (n n' : nat): n = n' -> mkDep n = mkDep n'.
Right. This is because we do not have an equality reflection rule in Coq, and judgmental/definitional equality is not the same as propositional equality. The way to state this is to "cast" the term mkDep n across the proof of equality.
Import EqNotations.
Theorem equalInhabitants (n n' : nat): forall H : n = n', rew H in mkDep n = mkDep n'.
intros.
subst; reflexivity.
Qed.
Note that rew binds more tightly than =, and is a notation for eq_rect. This says that for any proof H of n = n', the term mkDep n, when transported across H to become a term of type dep n', is equal to mkDep n'. (Note also that we could just as well have used destruct H or induction H or case H (but not apply f_equal) instead of subst.)
Say you have a record:
Record Example := {
fieldA : nat;
fieldB : nat
}.
Can we prove:
Lemma record_difference : forall (e1 e2 : Example),
e1 <> e2 ->
(fieldA e1 <> fieldA e2)
\/
(fieldB e1 <> fieldB e2).
If so, how?
On the one hand, it looks true, since Records are absolutely defined by their fields. On the other, without knowing what made e1 different from e2 in the first place, how are we supposed to decide which side of the disjunction to prove?
As a comparison, note that if there is only one field in the record, we are able to prove the respective lemma:
Record SmallExample := {
field : nat
}.
Lemma record_dif_small : forall (e1 e2 : SmallExample),
e1 <> e2 -> field e1 <> field e2.
Proof.
unfold not; intros; apply H.
destruct e1; destruct e2; simpl in H0.
f_equal; auto.
Qed.
On the other, without knowing what made e1 different from e2 in the first place, how are we supposed to decide which side of the disjunction to prove?
That is precisely the point: we need to figure out what makes both records different. We can do this by testing whether fieldA e1 = fieldA e2.
Require Import Coq.Arith.PeanoNat.
Record Example := {
fieldA : nat;
fieldB : nat
}.
Lemma record_difference : forall (e1 e2 : Example),
e1 <> e2 ->
(fieldA e1 <> fieldA e2)
\/
(fieldB e1 <> fieldB e2).
Proof.
intros [n1 m1] [n2 m2] He1e2; simpl.
destruct (Nat.eq_dec n1 n2) as [en|nen]; try now left.
right. intros em. congruence.
Qed.
Here, Nat.eq_dec is a function from the standard library that allows us to check whether two natural numbers are equal:
Nat.eq_dec : forall n m, {n = m} + {n <> m}.
The {P} + {~ P} notation denotes a special kind of boolean that gives you a proof of P or ~ P when destructed, depending on which side it lies on.
It is worth stepping through this proof to see what is going on. On the third line of the proof, for instance, executing intros em leads to the following goal.
n1, m1, n2, m2 : nat
He1e2 : {| fieldA := n1; fieldB := m1 |} <> {| fieldA := n2; fieldB := m2 |}
en : n1 = n2
em : m1 = m2
============================
False
If en and em hold, then the two records must be equal, contradicting He1e2. The congruence tactic simply instructs Coq to try to figure this out by itself.
Edit
It is interesting to see how far one can get without decidable equality. The following similar statement can be proved trivially:
forall (A B : Type) (p1 p2 : A * B),
p1 = p2 <-> fst p1 = fst p2 /\ snd p1 = snd p2.
By contraposition, we get
forall (A B : Type) (p1 p2 : A * B),
p1 <> p2 <-> ~ (fst p1 = fst p2 /\ snd p1 = snd p2).
It is here that we get stuck without a decidability assumption. De Morgan's laws would allow us to convert the right-hand side to a statement of the form ~ P \/ ~ Q; however, their proof appeals to decidability, which is not generally available in Coq's constructive logic.
I have a simple lemma:
Lemma map2_comm: forall A (f:A->A->B) n (a b:t A n),
(forall x y, (f x y) = (f y x)) -> map2 f a b = map2 f b a.
which I was able to prove using standard equality (≡). Now I am need to prove the similar lemma using setoid equality (using CoRN MathClasses). I am new to this library and type classes in general and having difficulty doing so. My first attempt is:
Lemma map2_setoid_comm `{Equiv B} `{Equiv (t B n)} `{Commutative B A}:
forall (a b: t A n),
map2 f a b = map2 f b a.
Proof.
intros.
induction n.
dep_destruct a.
dep_destruct b.
simpl.
(here '=' is 'equiv'). After 'simpl' the goal is "(nil B)=(nil B)" or "[]=[]" using VectorNotations. Normally I would finish it using 'reflexivity' tactics but it gives me:
Tactic failure: The relation equiv is not a declared reflexive relation. Maybe you need to require the Setoid library.
I guess I need somehow to define reflexivity for vector types, but I am not sure how to do that. Please advise.
First of all the lemma definition needs to be adjusted to:
Lemma map2_setoid_comm : forall `{CO:Commutative B A f} `{SB: !Setoid B} ,
forall n:nat, Commutative (map2 f (n:=n)).
To be able to use reflexivity:
Definition vec_equiv `{Equiv A} {n}: relation (vector A n) := Vforall2 (n:=n) equiv.
Instance vec_Equiv `{Equiv A} {n}: Equiv (vector A n) := vec_equiv.