Background: I have a simulation model which has unobserved parameters. I created a metamodel using artificial neural networks (ANN) because the runtime was very long for the simulation model. I am trying to estimate the unobserved parameters using Bayesian calibration, where priors are based on current knowledge, and the likelihood of observing data is being estimated from the metamodel.
Query: I have two random variables X and Y for which I am trying to get the posterior distribution using STAN. The prior distribution of X is uniform, U(0,2). The prior for Y is also uniform, but it will always exceed X i.e., Y ~ U(X,2). Since Y is linked to X, how can I define the prior distribution for Y in STAN such that the constraint Y>X holds? I am new to STAN, so I would appreciate any suggestions or guidance on how to proceed. Thank you so much!
Stan's ordered vectors are what you need. Create an ordered vector of length 2 (I'll call it beta) in the parameters block, like this:
parameters {
ordered<lower=0,upper=2>[2] beta;
}
Ordered vectors are constrained such that each element is greater than the previous element. So beta[1] will be your estimate of X and beta[2] will be your estimate of Y.
(To make sure I understand your model correctly: you have two parameters, X and Y, and your only prior knowledge about them is that they both lie in [0, 2] and Y > X. X and Y describe some aspect of the distribution of your data - for example, maybe X is the mean of some other random variable Z, for which you have observations. Do I have that right?)
I believe Stan's priors are uniform by default, but you can make sure of this by specifying a prior for beta in the model block:
model {
beta ~ uniform(0, 2);
...
}
Related
I'm trying to understand the variational inference module in tensorflow; I have a particular use case I'm hoping to use it for.
I want to make a custom distribution, the RV of which is a transformation of a vector of independent gamma RV's. This transformation removes one degree of freedom.
For simplicity's sake, let's consider the Dirichlet distribution. If x is an independent gamma vector with shape parameter vector a, then y = x / sum(x) is a dirichlet vector with the same shape vector, and sums to 1. Thus it loses 1 degree of freedom in the transformation.
Let's say I want to implement this distribution as a tfp.distributions.TransformedDistribution. Would that be possible? The Bijector class assumes implementation of both forward and inverse transformations, which, after the sum is integrated out, is no longer possible.
How would I go about implementing the Dirichlet in TransformedDistribution?
If my latent representation of variational autoencoder(VAE) is r, and my dataset is x, does vae's latent representation follows normalization based on r or x?
If r= 10, that means it has 10 means and variance (multi-gussain) and distribution comes from data whole data x?
Or r = 10 constructs one distribution based on r, and every sample try to follow this distribution
I'm confused about which one is correct
VAE constructs a mapping e(x) -> Z (encoder), and d(z) -> X (decoder). This means that every elements of your input space x will be mapped through an encoder e(x) into a single, r-dimensional Gaussian. It is not a "mixture", it is just a single gaussian with diagonal covariance matrix.
I'll add my 2 cents to #lejlot answer.
Your encoder in VAE will map your sample to a distribution, that in your case has 10 dimensions... that distribution is used to say "ok my best estimate of this property of this sample is mu, but I'm not too sure, so consider that it might have variance sigma"
Therefore, you have a distribution for each sample.
However, in order to make sampling easier in VAE, we ask the VAE to keep the distributions as close to a known one, that is the standard normal distribution (we know "where the distributions are located", if you check the latent space in a normal AE you will see that you will have groups far from eachother).
I'm using Tensorflow (2.4) and Keras to build my neural network model. It takes two tensors as inputs and gives a scalar output. The network is already trained and, from now on, it has fixed weights. It is possible, given one of the two inputs, to find the value of the other input that maximise the output value?
Thank you in advance
In theory, yes.
Lets call your network model f. It takes two inputs x and y and outputs f(x, y). Then, assuming x and f are fixed, you can find the value y* that maximize f(x, y) as follows:
calculate the gradient of f with respect to y. Then, there are two possibilities.
there exists stationary points. Just set df/dy = 0 and solve for y. This gives the y* at which there is either a maximum or a minimum. Compute f(x, y*) to check weather y* gives a maximum or a minimum.
there are no stationary points (or there is no maximum). Here, you need to study where f decreases or increases if y varies. To do this, look for df/dy > 0 (increases) and df/dy < 0 (decreases). You will find that, asymptotically, the function increases. Simply take y*=a, where a is the closest value to such asymptote that you can take (given your data type precision).
I would like to estimate the parameters of a directed Bayes net using PyMC. I came across one particular example that implements the sprinkler network, which has 3 random variables and a conditional probability distribution (CPD) defined for each node.
However, this example has the CPD encoded using deterministic variables.
Is it possible to provide the joint or marginal distribution over 2 or 3 random variables as the observed data to a deterministic PyMC variable?
In other words, if my network is of the form X -> Z <- Y, is it possible to provide a set of tuples of the form 'x1,y1,z1' as observed data, to learn the parameters of the CPD (Z|X,Y)?
The sprinkler example is really setting "Static" probability values. In this line:
p_G = mc.Lambda('p_G', lambda S=S, R=R: pl.where(S, pl.where(R, .99, .9), pl.where(R, .8, 0.)),
doc='Pr[G|S,R]')
to my understanding, i think we would require to set learn one parameter for each value of the parent. so if we want to learn P(Z/X,Y), we will need for each combination of values of X and Y, learn one parameter set for Z.
so lets say X and Y take boolean values and Z is a bernoulli distribution.
for each value of (X,Y) , ie: (0,0),(0,1),(1,0),(1,1) we have parameters, p1,p2,p3,p4. And then Z has 4 pymc observed variables: Z1 with parameter p1, Z2 with parameter p2 , Z3 with parameter p3 and Z4 with parameter p4.
Thus:
P(Z=0/X=0,Y=0) is the mcmc estimated mean of p1.
P(Z=1/X=0,Y=0) = 1-p1
P(Z=0/X=1,Y=0) = p2 and so on....
I have a related question here:
How to use pymc to parameterize a probabilistic graphical model?
I used to code my MCMC using C. But I'd like to give PyMC a try.
Suppose X_n is the underlying state whose dynamics following a Markov chain and Y_n is the observed data. In particular,
Y_n has Poisson distribution with mean depending on X_n and a multidimensional unknown parameter theta
X_n | X_{n-1} has distribution depending on theta
How should I describe this model using PyMC?
Another question: I can find conjugate priors for theta but not for X_n. Is it possible to specify which posteriors are updated using conjugate priors and which using MCMC?
Here is an example of a state-space model in PyMC on the PyMC wiki. It basically involves populating a list and allowing PyMC to treat it as a container of PyMC nodes.
As for the second part of the question, you could certainly calculate some of your conjugate posteriors ahead of time and put them into the model. For example, if you observed binomial data x=4, n=10 you could insert a Beta node p = Beta('p', 5, 7) to represent that posterior (its really just a prior, as far as the model is concerned, but it is the posterior given data x). Then PyMC would draw a sample for this posterior at every iteration to be used wherever it is needed in the model.