EECS260 Optimization (Fall semester 2016): project 3

Project 3 consists of participating in class discussions of several papers that apply optimisation algorithms to specific problems. You should read each paper ahead of time and be prepared to ask questions (or to answer them). The first thing to understand is the optimisation problem, i.e., what the variables are and what the objective function is (and why it makes sense to formulate the problem in that way). Example questions (not all may apply): is the problem constrained/unconstrained/smooth/line-search/trust-region...? If it uses a line search, does it satisfy the Wolfe (or other) conditions? Does the paper use a standard optimisation algorithm, or are there ad-hoc modifications, and if so what are the consequences? Are there better choices of algorithm (take into account ease of implementation, scalability wrt problem size, etc.)? Is the empirical evaluation fair and conclusive (competing methods are often presented at a disadvantage)? What is the stopping condition or convergence criterion? Does the problem have a unique optimum? How important is the optimisation part in the paper's main objectives? Be critical.

• Oct. 13. Presenters: Omar DeGuchy, Yerlan Idelbayev.

  • Erway, J. B. and Marcia, R. F.: "On efficiently computing the eigenvalues of limited-memory quasi-Newton matrices". SIAM J. Matrix Anal. and Apps., 2015.
  • DeGuchy, O. et al.: "Compact representation of the full Broyden class of quasi-Newton updates". Numerical Linear Algebra with Applications, 2018.
  • Dauphin, Y. N. et al.: "Identifying and attacking the saddle point problem in high-dimensional non-convex optimization". NIPS 2014.

• Oct. 20. Presenters: Mohammadkazem Ebrahimpour, Tapish Rathore.

  • Bottou, L., Curtis, F. E. and Nocedal, J.: Optimization Methods for Large-Scale Machine Learning", arXiv:1606.04838 and ICML 2016 tutorial on "Stochastic gradient methods for large-scale machine learning".
  • Andrychowicz, M. et al.: "Learning to learn by gradient descent by gradient descent". NIPS 2016 and arXiv:1606.04474.

• Oct. 27. Presenters: Sebastian Rodriguez.

  • Boyd, S., Diaconis, P. and Lin, X.: "Fastest mixing Markov chain on a graph". SIAM Review, 2004.

• Nov. 3. Presenter: Ramin Raziperchikolaei.

  • SVM optimisation: CSE176 lecture notes.
  • Wang, W. and Carreira-Perpiñán, M. Á.: "The role of dimensionality reduction in classification". AAAI 2014 and arXiv:1405.6444.

• Nov. 10. Presenter: Ramin Raziperchikolaei.

  • Carreira-Perpiñán, M. Á. and Raziperchikolaei, R.: "Hashing with binary autoencoders". CVPR 2015 and arXiv:1501.00756.
  • Raziperchikolaei, R. and Carreira-Perpiñán, M. Á.: "Optimizing affinity-based binary hashing using auxiliary coordinates". NIPS 2016 and arXiv:1501.05352.

• Nov. 17. Presenter: Suryabhan Singh Hada, Thomas Thayer.

  • M. Jaggi: "Revisiting Frank-Wolfe: projection-free sparse convex optimization". ICML 2013 and ICML 2013 tutorial.
  • Richter, C., Bry, A. and Roy, N.: "Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments". Proc. Int. Symposium of Robotics Research (ISRR 2013).

• Nov. 21. Presenter: Pooya Tavallali.

  • Norouzi et al.: "Efficient non-greedy optimization of decision trees". NIPS 2015.

• Nov. 30. Presenter: Mohammadkazem Ebrahimpour.

  • LASSO optimisation: chapter 5 of Hastie, T. Tibshirani, R. and Wainwright, M.: "Statistical Learning with Sparsity: The Lasso and Generalizations". CRC Press, 2015.

UC Merced | EECS | MACP's Home Page