Which part of the code is actually used in gmminf?

[jgrizou]

Me spamming again!

I looked quickly into gmminf.py

Are all the part of the code actually in use?

The conditional functions seem not in use inside gmminf as well as not in use in ilogmm. There are the following:

def conditional(mean, covar, dims_in, dims_out, covariance_type='full'):
    """ Return a function f such that f(x) = p(dims_out | dims_in = x) (f actually returns the mean and covariance of the conditional distribution
    """
    in_in = covar[ix_(dims_in, dims_in)]
    in_out = covar[ix_(dims_in, dims_out)]
    out_in = covar[ix_(dims_out, dims_in)]
    out_out = covar[ix_(dims_out, dims_out)]
    in_in_inv = inv(in_in)
    out_in_dot_in_in_inv = out_in.dot(in_in_inv)

    cond_covar = out_out - out_in_dot_in_in_inv.dot(in_out)
    cond_mean = lambda x: mean[dims_out] + out_in_dot_in_in_inv.dot(x - mean[dims_in])
    return lambda x: [cond_mean(x), cond_covar]

and

    def conditional(self, in_dims, out_dims):
        conditionals = []

        for k, (weight_k, mean_k, covar_k) in enumerate(self):
            conditionals.append(conditional(mean_k, covar_k,
                                            in_dims, out_dims,
                                            self.covariance_type))

        cond_weights = lambda v: [(weight_k * Gaussian(mean_k[in_dims].reshape(-1,),
                                  covar_k[ix_(in_dims, in_dims)]).normal(v.reshape(-1,)))
                                  for k, (weight_k, mean_k, covar_k) in enumerate(self)]

        def res(v):
            gmm = GMM(n_components=self.n_components,
                      covariance_type=self.covariance_type,
                      random_state=self.random_state, thresh=self.thresh,
                      min_covar=self.min_covar, n_iter=self.n_iter, n_init=self.n_init,
                      params=self.params, init_params=self.init_params)
            gmm.weights_ = cond_weights(v)
            means_covars = [f(v) for f in conditionals]
            gmm.means_ = array([mc[0] for mc in means_covars]).reshape(self.n_components,
                                                                       -1)
            gmm._set_covars(array([mc[1] for mc in means_covars]))
            return gmm

        return res

        self.in_dims = array(in_dims)
        self.out_dims = array(out_dims)
        means = zeros((self.n_components, len(out_dims)))
        covars = zeros((self.n_components, len(out_dims), len(out_dims)))
        weights = zeros((self.n_components,))
        sig_in = []
        inin_inv = []
        out_in = []
        mu_in = []
        for k, (weight_k, mean_k, covar_k) in enumerate(self):
            sig_in.append(covar_k[ix_(in_dims, in_dims)])
            inin_inv.append(matrix(sig_in).I)
            out_in.append(covar_k[ix_(out_dims, in_dims)])
            mu_in.append(mean_k[in_dims].reshape(-1, 1))

            means[k, :] = (mean_k[out_dims] +
                           (out_in *
                            inin_inv *
                            (value - mu_in)).T)

            covars[k, :, :] = (covar_k[ix_(out_dims, out_dims)] -
                               out_in *
                               inin_inv *
                               covar_k[ix_(in_dims, out_dims)])
            weights[k] = weight_k * Gaussian(mu_in.reshape(-1,),
                                             sig_in).normal(value.reshape(-1,))
        weights /= sum(weights)

        def p(value):
            # hard copy of useful matrices local to the function
            pass

        return p

The second one is specially cryptic to me, I don't understand what is going on there. Maybe those are archaeological remainings :D ?

[clement-moulin-frier]

Yes they are not used inside gmminf. The idea behind conditional was the following.

Imagine you have a GMM modeling a joint prob distribution P(A^B) and you want to compute P(A | B) for several values of B. If you use the classical ìnference method, you will recompute everything for each new value (and in particular the expensive matrix inversions), whereas most of the computation is actually the exact same one.

This is why conditional returns a function, which corresponds to f(x) = P(A | B=x). However, from what I remember it didn't improved the computational time significantly, for some reasons I hadn't clearly understood.

So in short: those functions are probably buggy and not used elsewhere in the code (from my old memories). However, continuing investigating this issue is very useful I think (you have probably noticed that ilo-gmm is slow, I think this could improve it a lot, e.g. when you simply want to make a series of predictions).