Section I: Fundamentals
September 9: Course overview
Sept 14, 16: Overview and basic principles
Connectionist models. In Squire, L. (Ed.), Encyclopedia of Neuroscience, volume 3, pp. 75-82. Oxford: Academic Press.
McClelland (2009). The place of modeling in cognitive science. Topics in Cognitive Science, 1 (2009), 11-38.
Optional:
Rogers, T. T. and McClelland (2014) PDP at 25: Further explorations in the microstructure of cognition.
Rogers, T. T. (2020) Neural networks as a critical level of description in cognitive neuroscience. Current Directions in Behavioral Sciences, 32, 167-173.
Lab 0: Introduction to LENS
Sept 21, 23: Constraint satisfaction and schemas
Rumelhart, D. E. (1989). The architecture of mind: A connectionist approach. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 133-159). Cambridge, MA: MIT Press.
Rumelhart, D. E., Smolensky, P., McClelland, J. L., & Hinton, G. E. (1986). Schemata and sequential thought processes in PDP models. PDP2, Chapter 14.
Related contemporary papers:
Botvinick, M., & Braver, T. (2015). Motivation and cognitive control: from behavior to neural mechanism. Annual review of psychology, 66, 83-113. Review of influential view that “Goodness” in model states provides a measure of conflict that executive systems can use to determine when top-down control is needed. This is one domain of work where connectionist models of behavior led directly to a new theory about the role of different brain areas in a highly complex aspect of human cognition.
McClelland, J. L. and Kumaran, D. (2012). Generalization through the recurrent interaction of episodic memories: A model of the hippocampal system. Psychological Review, 119(3), 573-616. This paper uses a Jets-n-Sharks like memory mechanism to understand patterns of hippocampally-mediated generalization in transitive inference.
Lab 1: The Jets-and-Sharks model
Homework 1: Constraint satisfaction models.
Part 1: The Jets-n-Sharks model.
Part 2: The Room Schema
Homework download: schema.zip
Jets-n-Sharks / IAC-style activation function.
Sept 28, 30: Simple learning: Hebb and delta rules
McClelland, J. L., and Rumelhart, D. E. (1988). The pattern associator. PDP Handbook, Chapter 4 (pp. 83-96).
McClelland, J. L. & Rumelhart, D. E. (1986). A distributed model of human learning and memory. PDP2, Chapter 17.
Lab 2: Learning in LENS
Demo illustrating gradient descent in error for delta rule (perceptron) learning
Homework 2: Hebb and Delta rules
HW 2: Download assignment here
Download network files for HW2
Cool neural network thing of the day: Online demo of backprop for XOR
Contemporary papers:
Competitive Hebbian learning through spike-timing-dependent synaptic plasticity
Oct 5, 7: Backpropagation (HW 1 due!)
Rumelhart, D. E., Hinton, G. and Williams, R. (1986). Learning internal representations by backpropagating errors PDP1, Chapter 8.
Here is a nice contemporary introduction to backpropagation.
Lab 3: The XOR problem, building your own network
Download Excel demo illustrating linear separability constraint on delta rule
Online cool neural network thing of the day: See this great video on backpropagation, part of a v useful series by Three Blue One Brown.
Homework 3: Backpropagation
Download backpropagation assignment here
Download XOR network files
Contemporary papers
Also essentially any paper that trains a deep network–they all use some tweak on backpropagation.
Section 2: Flavors
Oct 12, 14: Simple recurrent networks.
Elman, J. L. (1990). Finding structure in time. Cognitive Science, 14, 179-211
Lab: Building and testing simple recurrent networks
Cool neural network thing of the day: Here is a Java-based implementation of an LSTM that learns to predict upcoming characters from a large corpus of text, and here is a Python/TensorFlow/Jupyter implementation of an LSTM trained to generate (terrible) scripts for Simpsons episodes.
Contemporary papers
Rabovsky, Hansen and McClelland (2018). Modelling the N400 brain potential as a change in probabilistic representation of meaning. Nature Human Behavior. This cool paper uses a classic SRN model of sentence level meaning to explain a host of effects in the study of the semantic N400.
Lab4: Simple recurrent networks
Oct 19, 21: Fully recurrent networks. HW 2 due!
Hinton, G. E. & Sejnowski, T. J. (1986). Learning and relearning in Boltzmann Machines. PDP1, Chapter 7.
Optional:
Williams & Zipser (1995). Gradient based learning algorithms for recurrent networks and their computational complexity. This is a pretty mathy paper for those who want to understand the approach in significant detail.
Lab 5: Building and testing fully recurrent networks
Oct 26, 28: Deep convolutional networks. HW 3 due!
Read this online tutorial introduction
Here is another one that might be helpful
Lab 6: Building networks in Python using TensorFlow/Keras
Contemporary papers
Krishevsky et al. (2012), ImageNet classification with deep convolutional neural networks. The original imagenet classifier
Cool neural network stuff
Trains a deep image classifier in your browser, with configurable parameters and visualizations.
Click here to learn about / work with the ImageNet dataset.
Nov 2, 4: Long short-term memory. Project proposal due!
Read this online tutorial introduction
Here is another one that might be helpful
Lab 7: Recurrent networks in Python using TensorFlow/Keras
Contemporary papers
Kirov and Cotterel (2018),Recurrent Neural Networks in Linguistic Theory: Revisiting Pinker and Prince (1988) and the Past Tense Debate. Do contemporary language models solve the issues raised by Pinker and Prince?
Cool neural network stuff
Cool online demo of character generation from LSTM.
Start a drawing and let an RNN complete it
Section 3: Applications
Nov 9, 11: Perception and inference.
McClelland et al. (2014), Interactive activation and mutual constraint satisfaction in perception and cognition. Cognitive Science, 38, 1139-1189.
Lab 8: Looking at model data in R part 1
Nov 16, 14: Language
Seidenberg, M. S. and Plaut, D. C. (2014). Quasi-regularity and its discontents: The legacy of the past-tense debate. Cognitive Science, 38,
Lab 9: Looking at model data in R part 2
Nov 23: Memory
O’Reilly, R. C. et al. (2014). Complementary learning systems. Cognitive Science, 38, 1229-1248.
THANKSGIVING, NO LAB
Nov 30, Dec 2: Cognitive control
Botvinick, M. M. and Cohen, J. (2014). The computational and neural basis of cognitive control: Charted territory and new frontiers. Cognitive Science, 38, 1249-1285.
Class project progress reports
Dec 7, 9: Cognitive control
Class project progress reports
Class project progress reports
Dec 14: Consciousness
Class project progress reports