Comments on: What inputs do Monte Carlo algorithms need? http://brenocon.com/blog/2013/04/monte-carlo-algorithm-inputs/ cognition, language, social systems; statistics, visualization, computation Tue, 25 Nov 2025 13:11:20 +0000 hourly 1 By: brendano http://brenocon.com/blog/2013/04/monte-carlo-algorithm-inputs/#comment-327396 brendano Mon, 22 Apr 2013 03:01:50 +0000 http://brenocon.com/blog/?p=2007#comment-327396 Yeah SGD requires decomposition into minibatch gradients you can sum together for the big gradient (and thus the gradient from a minibatch is in expectation same as the full gradient). I’m not aware of SGD for non-sum situations…

]]> By: John Myles White http://brenocon.com/blog/2013/04/monte-carlo-algorithm-inputs/#comment-327274 John Myles White Sun, 21 Apr 2013 22:13:18 +0000 http://brenocon.com/blog/?p=2007#comment-327274 Very nice summary.

One question: does SGD actually work in cases in which the objective function doesn’t decompose in an obvious way into the sum of one-example objective functions? My vague recollection of the proofs of convergence is that they make heavy use of the decomposition of the objective functions.

]]>