Vectors of multivariate variables

[fonnesbeck]

It would be useful if we could model multiple independent multivariate variables in the same statement. For example, if I wanted four multivariate normal vectors with the same prior, I should be able to specify:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=(4,3))

but it currently returns a ValueError complaining of non-aligned matrices.

[jsalvatier]

will it be obvious what dimension is the multivariate dimension?

On Fri, May 2, 2014 at 10:16 AM, Chris Fonnesbeck
notifications@github.comwrote:

It would be useful if we could model multiple independent multivariate
variables in the same statement. For example, if I wanted four multivariate
normal vectors with the same prior, I should be able to specify:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=(4,3))

but it currently returns a ValueError complaining of non-aligned matrices.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/535
.

[fonnesbeck]

It should be intuitive, if not obvious. I think most people would expect a vector of variables, which implies that the first dimension is the number of variable elements and the remaining dimension(s) the size of each variable.

[fonnesbeck]

We at least need to be able to do the analog of this:

m = [pm.MvNormal('m_{}'.format(i), mu, Tau, value=[0]*3) for i in range(len(unique_studies))]

This has been a show-stopper for me trying to use PyMC 3 for new work, so I'm going to try to set aside some time to work on this.

Thinking about it some more, however, I think that shape is not the appropriate way to specify the dimension of a multivariate variable -- that should be reserved for the size of the vector of variables. Might be best to have:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), dim=3)

for a single variable and:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=4, dim=3)

for a vector containing 4 MvNormals of dimension 3. Better yet, we ought to be able to infer the dimension of the MvNormal from its arguments.

[jsalvatier]

That makes some sense. The words shape and dim seem very close, so it seems
confusing to have both

On Thu, May 29, 2014 at 1:30 PM, Chris Fonnesbeck
notifications@github.comwrote:

We at least need to be able to do the analog of this:

m = [pm.MvNormal('m_{}'.format(i), mu, Tau, value=[0]*3) for i in range(len(unique_studies))]

This has been a show-stopper for me trying to use PyMC 3 for new work, so
I'm going to try to set aside some time to work on this.

Thinking about it some more, however, I think that shape is not the
appropriate way to specify the dimension of a multivariate variable -- that
should be reserved for the size of the vector of variables. Might be best
to have:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), dim=3)

for a single variable and:

f = pm.MvNormal('f', np.zeros(3), np.eye(3), shape=4, dim=3)

for a vector containing 4 MvNormals of dimension 3. Better yet, we ought
to be able to infer the dimension of the MvNormal from its arguments.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/535#issuecomment-44581060
.

[fonnesbeck]

Perhaps using plates here would be clearer, since this is common terminology in graphical models. The we could generalize the business of generating vectors of variables.

[fonnesbeck]

Just bumping this one. I come up against it frequently in epidemiological analyses.

[aflaxman]

I like the originally proposed notation, shape=(4,3), since that will be the shape of f.value. Am I stuck in a PyMC2 way of thinking?

[fonnesbeck]

I'd be happy with that. We would just have to adopt the convention that the last dimension is always the size of the individual multivariate node, and not the size of the array containing the nodes. The tricky part comes when you have, say, a vector of Wisharts that is itself multidimensional, so the total shape could be (4,4,3,3) for a 4x4 array of 3x3 variables.

[twiecki]

I would imagine it's a rare case but can't hurt to consider it and come up with a sane way to handle. In the end, complex things will be complex in code but defaulting to the last dimensions is an easy rule to keep in mind.

[aflaxman]

+1 for shape=(4,4,3,3) to get a 4x4 array of 3x3 wisharts. So

C = pm.WishartCov('C', C=np.eye(3), n=5)
C.value.shape == (3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,3,3))
C.value.shape == (4,3,3)

C = pm.WishartCov('C', C=np.eye(3), n=5, shape=(4,4,3,3))
C.value.shape == (4,4,3,3)

[fonnesbeck]

Multivariate classes could have the appropriate dimension specified in the class to know how to deal with the shape argument. Wisharts will always be 2-dimensional, for example, so any remaining dimensions will always be how many wisharts are in the set. Multinomials will always be a 1-d vector, etc.