Psych 711 Syllabus

Section I: Fundamentals

September 9: Course overview

Sept 14, 16: Overview and basic principles

Connectionist models. In Squire, L. (Ed.), Encyclopedia of Neuroscience, volume 3, pp. 75-82. Oxford: Academic Press.

McClelland (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1 (2009), 11-38.

Optional: 

Rogers, T. T. and McClelland (2014) PDP at 25: Further explorations in the microstructure of cognition.

Rogers, T. T. (2020) Neural networks as a critical level of description in cognitive neuroscience. Current Directions in Behavioral Sciences, 32, 167-173.

Lab 0: Introduction to LENS

Slides here

Sept 21, 23: Constraint satisfaction and schemas

Rumelhart, D. E. (1989). The architecture of mind: A connectionist approach. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 133-159). Cambridge, MA: MIT Press.

Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. PDP2, Chapter 14.

Related contemporary papers:

Botvinick, M., & Braver, T. (2015). Motivation and cognitive control: from behavior to neural mechanism. Annual review of psychology66, 83-113. Review of influential view that “Goodness” in model states provides a measure of conflict that executive systems can use to determine when top-down control is needed. This is one domain of work where connectionist models of behavior led directly to a new theory about the role of different brain areas in a highly complex aspect of human cognition.

McClelland, J. L. and Kumaran, D. (2012). Generalization through the recurrent interaction of episodic memories: A model of the hippocampal system. Psychological Review, 119(3), 573-616. This paper uses a Jets-n-Sharks like memory mechanism to understand patterns of hippocampally-mediated generalization in transitive inference.

Lab 1: The Jets-and-Sharks model

Slides here

Homework 1: Constraint satisfaction models.
Part 1: The Jets-n-Sharks model.
Part 2: The Room Schema
Homework download: schema.zip

Jets-n-Sharks / IAC-style activation function.

Sept 28, 30: Simple learning: Hebb and delta rules

McClelland, J. L., and Rumelhart, D. E. (1988). The pattern associator. PDP Handbook, Chapter 4 (pp. 83-96).

McClelland, J. L. & Rumelhart, D. E. (1986). A distributed model of human learning and memory. PDP2, Chapter 17.

Lab 2: Learning in LENS

Hebb demo Excel file

Demo illustrating gradient descent in error for delta rule (perceptron) learning 

Slides here

Homework 2: Hebb and Delta rules
HW 2: Download assignment here
Download network files for HW2

Cool neural network thing of the day: Online demo of backprop for XOR

Contemporary papers:

Competitive Hebbian learning through spike-timing-dependent synaptic plasticity

Oct 5, 7: Backpropagation (HW 1 due!)

Rumelhart, D. E., Hinton, G. and Williams, R. (1986). Learning internal representations by backpropagating errors PDP1, Chapter 8.

Here is a nice contemporary introduction to backpropagation.

Lab 3: The XOR problem, building your own network

Download Excel demo illustrating linear separability constraint on delta rule

Slides here

Online cool neural network thing of the day: See this great video on backpropagation, part of a v useful series by Three Blue One Brown.

Homework 3: Backpropagation
Download backpropagation assignment here
Download XOR network files

Contemporary papers

Whittington and Bogacz (2019). Theories of error back-propagation in the brain. Trends in Cognitive Sciences 

Also essentially any paper that trains a deep network–they all use some tweak on backpropagation.

Section 2: Flavors

Oct 12, 14: Simple recurrent networks.

Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211

Lab: Building and testing simple recurrent networks

slides here

Cool neural network thing of the day: Here is a Java-based implementation of an LSTM that learns to predict upcoming characters from a large corpus of text, and here is a Python/TensorFlow/Jupyter implementation of an LSTM trained to generate (terrible) scripts for Simpsons episodes.

Contemporary papers

Rabovsky, Hansen and McClelland (2018). Modelling the N400 brain potential as a change in probabilistic representation of meaning.  Nature Human Behavior. This cool paper uses a classic SRN model of sentence level meaning to explain a host of effects in the study of the semantic N400.

Lab4: Simple recurrent networks

Oct 19, 21: Fully recurrent networks. HW 2 due!

Hinton, G. E. & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann Machines. PDP1, Chapter 7.

Optional:

Williams & Zipser (1995). Gradient based learning algorithms for recurrent networks and their computational complexity. This is a pretty mathy paper for those who want to understand the approach in significant detail.

Lab 5: Building and testing fully recurrent networks

slides here

 

Oct 26, 28: Deep convolutional networks. HW 3 due!

Read this online tutorial introduction

Here is another one that might be helpful

Lab 6: Building networks in Python using TensorFlow/Keras

Contemporary papers

Krishevsky et al. (2012), ImageNet classification with deep convolutional neural networks. The original imagenet classifier

Cool neural network stuff

Trains a deep image classifier in your browser, with configurable parameters and visualizations.

Click here to learn about / work with the ImageNet dataset.

Nov 2, 4: Long short-term memory. Project proposal due!

Read this online tutorial introduction

Here is another one that might be helpful

Lab 7: Recurrent networks in Python using TensorFlow/Keras

Contemporary papers

Kirov and Cotterel (2018),Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate. Do contemporary language models solve the issues raised by Pinker and Prince?

Cool neural network stuff

Cool online demo of character generation from LSTM.

Start a drawing and let an RNN complete it

Section 3: Applications

Nov 9, 11: Perception and inference.

McClelland et al. (2014), Interactive activation and mutual constraint satisfaction in perception and cognition. Cognitive Science, 38, 1139-1189.

Lab 8: Looking at model data in R part 1

slides here

Nov 16, 14: Language

Seidenberg, M. S. and Plaut, D. C. (2014). Quasi-regularity and its discontents: The legacy of the past-tense debate. Cognitive Science, 38,

Lab 9: Looking at model data in R part 2

Nov 23: Memory

O’Reilly, R. C. et al. (2014). Complementary learning systems. Cognitive Science, 38, 1229-1248.

THANKSGIVING, NO LAB

Nov 30, Dec 2: Cognitive control

Botvinick, M. M. and Cohen, J. (2014). The computational and neural basis of cognitive control: Charted territory and new frontiers. Cognitive Science, 38, 1249-1285.

Class project progress reports

Dec 7, 9: Cognitive control

Class project progress reports

Class project progress reports

Dec 14: Consciousness

Class project progress reports