
Archives
 November 2017
 May 2017
 April 2017
 December 2016
 September 2015
 December 2014
 February 2014
 January 2014
 September 2013
 September 2012
 January 2012
 November 2011
 October 2011
 July 2011
 May 2011
 March 2011
 November 2009
 October 2009
 August 2009
 June 2009
 May 2009
 March 2009
 February 2009
 January 2009
 December 2008
 November 2008
 October 2008
 August 2008
 July 2008
 June 2008

Meta
Tag Archives: optimization
Truncated BiLevel Optimization
In 2012, I wrote a paper that I probably should have called “truncated bilevel optimization”. I vaguely remembered telling the reviewers I would release some code, so I’m finally getting around to it. The idea of bilevel optimization is quite … Continue reading
Posted in Uncategorized
Tagged crossvalidation, machine learning, matlab, optimization, regularization
5 Comments
Fitting an inference algorithm instead of a model
One recent trend seems to be the realization that one can get better performance by tuning a CRF (Conditional Random Field) to a particular inference algorithm. Basically, forget about the distribution that the CRF represents, and instead only care how … Continue reading
What GaussSeidel is Really Doing
I’ve been reading Alan Sokal’s lecture notes “Monte Carlo Methods in Statistical Mechanics: Foundations and New Algorithms” today. Once I learned to take the word “Hamiltonian” and mentally substitute “function to be minimized”, they are very clearly written. Anyway, the … Continue reading
HessianVector products
You have some function . You have figured out how to compute it’s gradient, . Now, however, you find that you are implementing some algorithm (like, say, Stochastic Meta Descent), and you need to compute the product of the Hessian … Continue reading
Why does regularization work?
When fitting statistical models, we usually need to “regularize” the model. The simplest example is probably linear regression. Take some training data, . Given a vector of weights , the total squared distance is So to fit the model, we … Continue reading