Repository navigation
tf.contrib.distributions: undocumented behaviour in Multinomial and Categorical #10149
Description
System information
- Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes?
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS Sierra 10.12.3
- TensorFlow installed from (source or binary): binary, via pip
- TensorFlow version (use command below): 1.1.0 (v1.1.0-rc0-61-g1ec6ed5)
- Bazel version (if compiling from source): N/A
- CUDA/cuDNN version: N/A
- GPU model and memory: N/A
- Exact command to reproduce:
import tensorflow as tf
categorical = tf.contrib.distributions.Categorical(probs = [0.25, 0.5, 0.25])
multinomial = tf.contrib.distributions.Multinomial(total_count = 1., probs = [0.25, 0.5, 0.25])
mvn = tf.contrib.distributions.MultivariateNormalDiag(loc = [0., 0., 0.], scale_diag= [1., 1., 1.])
# expected values (points 1 and 2)
# The docs for Categorical say value should be float or double, but it expects an int
categorical.log_prob([0, 0, 1])
# <tf.Tensor 'Categorical_2/log_prob/Neg:0' shape=(3,) dtype=float32>
categorical.log_prob([0., 0., 1.])
# TypeError: Value passed to parameter 'labels' has DataType float32 not in list of allowed values: int32, int64
# The docs for Categorical say value should be float or double, which is how it behaves (though this is unlike categorical)
multinomial.log_prob([0, 0, 1])
# ValueError: Tensor conversion requested dtype int32 for Tensor with dtype float32: 'Tensor("Multinomial_2/log_prob/Log:0", shape=(3,), dtype=float32)'
multinomial.log_prob([0., 0., 1.])
# <tf.Tensor 'Multinomial_1/log_prob/sub:0' shape=() dtype=float32>
# output shape (points 3 and 4)
# The docs for both say that the output should be:
# "a Tensor of 'shape sample_shape(x) + self.batch_shape' with values of type self.dtype"
# though sample_shape doesn't seem to be relevant here, it's an argument to param_shapes() and sample()
# for Categorical (with int value), the result is a vector, matching the shape of value
categorical.log_prob([0, 0, 1])
# <tf.Tensor 'Categorical_2/log_prob/Neg:0' shape=(3,) dtype=float32>
# for Multinomial (with float value), the result is a scalar
multinomial.log_prob([0., 0., 1.])
# <tf.Tensor 'Multinomial_1/log_prob/sub:0' shape=() dtype=float32>
# for Multivariate Normal the result is a scalar
mvn.log_prob([0.1, 0.2, 0.3])
# <tf.Tensor 'MultivariateNormalDiag_2/log_prob/add:0' shape=() dtype=float32>Describe the problem
There are four related issues:
-
The expected type of
valuefor thelog_prob()method intf.contrib.distributions.Categoricalis inconsistent with the documentation. -
The expected values for
tf.contrib.distributions.Categoricalandtf.contrib.distributions.Multinomialare inconsistent with one another, which is odd as the categorical distribution is a special case of the multinomial, withtotal_count = 1 -
The output dimensions for
tf.contrib.distributions.Categoricalandtf.contrib.distributions.Multinomialare inconsistent with the documentation -
The output dimensions for
tf.contrib.distributions.Categoricalare a vector, which doesn't really make sense for a multivariate distribution, and is inconsistent withtf.contrib.distributions.Multinomialandtf.contrib.distributions.MultivariateNormal*
Details are in the code snippet above
Activity
jart commentedon May 26, 2017
Passing documentation issue off to contrib/distributions czar @langmore.
jart commentedon May 26, 2017
Punting this issue to @jvdillon. PTAL.
jvdillon commentedon May 26, 2017
Thanks for reporting this issue! I'll put together a fix right now and report back.
jvdillon commentedon Jun 2, 2017
Sorry for the delay! Small update: we implemented a fix and it's currently under review.
jvdillon commentedon Jun 22, 2017
Fix submitted. Should be in master in a few days.
Branch 160346151 (#11094)
Branch 160346151 (tensorflow#11094)