Teaching:

CPSC 114 Internet Programming
CPSC 230 Machine Organization
CPSC 415 Special Topics: Machine Learning

Research:

I'm interested in machine learning and sequential decision making under uncertainty: reinforcement learning, classical decision theory, Bayesian methods and Monte-Carlo schemes.

• Niv, Y., Duff, M., & Dayan, P. (2005) "Dopamine, Uncertainty, and TD-learning," Behavioral & Brain Functions.

• Design for an Optimal Probe, Proceedings of the 20th International Conference on Machine Learning, 2003.

• Diffusion Approximation for Bayesian Markov Chains, ICML 2003.

• Optimal Learning: Computational procedures for Bayes-adaptive Markov decision processes, PhD Thesis, University of Massassachusetts Amherst, January 2002.

• Monte-Carlo algorithms for the improvement of finite-state controllers , Proc. AI and Statistics, Key West, January 2001.

• Local bandit approximation for optimal learning problems (with Andy Barto), NIPS-10, 1997.

• Q-learning for Bandit Problems, Proceedings of Machine Learning 1995.

• Reinforcement Learning for Semi-Markov Processes (with Steve Bradtke), Neural Information Processing Systems -- 7, 1995.

• Monte Carlo Matrix Inversion and Reinforcement Learning (with Andy Barto), Neural Information Processing Systems -- 6, 1994, pp. 687-694.

• Solving Bellman's Equation by the Method of Continuation, American Control Conference, 1994.

• A Control Variable Perspective for the Optimal Combination of Truncated Corrected Returns, Technical Note, 1994.

• Backpropagation and Bach's 5th Cello Suite (Sarabande), Proceedings IEEE Conference on Neural Networks.

• Optimal Design of an Electrostatic Lens (with M. Szilagyi and S.J Yakowitz), Applied Physics Letters.

I was a Senior Research Fellow last year with the Gatsby Computational Neuroscience Unit, University College, London.

I received my PhD from UMASS in 2002.