Paper guidelines

For proposal

Please write a brief (one page maximum) description of your proposed project. Your proposal should include a short (one paragraph) high-level overview of the general issue that interests you. This could be an empirical finding or phenomenon in cognition or neuroscience, a computational question relating to neural networks, or a replication of an existing model and study in the literature. You can then proceed along two paths:
  • Path 1: you have a good idea of what you would like to do. In this case, lay out your plan: what behavior will you try to explore/capture with your model; what kind of model do you plan to use; show a picture if you have one; how will you generate training and test examples; what behavior will you look at to answer your principle question.
  • Path 2: you aren’t sure exactly how to pursue your interest. In this case, provide a little more information about what you want to do and where you are getting stuck. Specifically, think about the following questions and consider which are uncertain and why. In your proposal indicate for which questions you have a good plan, and for which you are uncertain. The questions are:
      • Is there a specific behavioral or neural phenomenon you are interested in understanding and can imagine simulating? If so, explain this in detail. If not, provide more information about the general domain of interest and the kinds of phenomena people study.
      • What kind of model is best suited to your project: feed forward, simple recurrent, fully recurrent, self-organizing, or other?
      • What model architecture to use: what are the inputs, outputs, and internal layers and connections that will allow you to simulate behavior in your task?
      • What is the model environment? How will input and output patterns be structured to capture important properties of the task?
      • How will you implement the model—LENS, or some other software environment?
      • How will you build the environment?
      • How will you test the model? How will you measure model success/failure?
Note: You don’t have to be Shakespeare for this assignment, and you don’t need to have clear answers to all questions. The goal is to provide me with an early perspective into your plans/thinking, so I can give you advice/guidance as needed. Just try to write down, as clearly and concisely as possible, your current thoughts.

For the final paper

Your final paper should provide a brief report describing (a) the motivation for the project, (b) the approach you have taken and how it relates to material covered in class, (c) a detailed description of the model and simulation details, (d) a report of the results with evidence from the model, and (e) an interpretation of the results relating these back to the main question of interest.

As in any writing assignment, the main constraint on length is to be as clear as you can without being too verbose. People vary greatly in their preferred writing style; some people can be clear and very concise; others (like me) sometimes feel they need more words to clearly get the point across. My strong preference is for you to write what you feel you need to write to clearly explain what you are up to and what you observed, rather than trying to fit some arbitrary word limit.

With that in mind, students almost always ask me “Yeah, but still, how long should it be?” So for those students, and keeping the above paragraph in mind, as a rough guideline each of the above sections should be between 500 and 1500 words, so the full paper should be between 2500 and 7500 words total.

Don’t forget to provide evidence from the simulations! This is critically important to a good grade. Working out a single example of how the network processes a single pattern can be very useful for helping the reader understand what is going on, but is not sufficient evidence for generalizations about the network’s behavior. If you say something like “the network is slower to learn atypical patterns,” for instance, you need to support this by reporting, say, the average number of epochs taken to learn *all* atypical patterns compared to *all* typical patterns. If you say something like “the network succeeds on active-voice sentences but not passive-voice sentences,” then report statistics across *all* such sentences, and not just one or two.

Also: it is important, of course, to clearly explain how the network behaves—but it is equally important to explain *why* the network is doing what it is doing. This almost always involves considering other possible reasons for the network’s behavior, and ruling these out with evidence. If you think the model is good at learning typical patterns and is struggling with atypical patterns, you’d better make sure that the effect doesn’t really arise because the typical patterns are more frequent, or involve fewer active units, or some other confound. Just like in an experiment, try to think of alternative reasons for the behavior, then run a simulation to test the alternative explanation. If you can rule it out, report these “diagnostic” simulations in the paper.

Also also: Don’t assume I know everything. To the contrary, you will do better to err on the side of pretending I am your grandmother and you need to explain things slowly and carefully. A common mis-step in these assignments is for the student to omit explaining anything we covered in class, assuming I know this stuff already. That may be true, but I need to know that *you* know it. So imagine that your paper is not being written for me, but for someone else who maybe has not taken the class and needs things to be explained. The trick of writing, of course, is to balance this requirement with the “don’t be too verbose” admonishment above.

Finally, failure can be just as interesting as success. If you weren’t able to get your network to do what you were trying to do, clearly explaining what you tried and why you think it is failing is perfectly acceptable. It is even more acceptable if you can then comment on the implications of the failure for your central question.

 A checklist for your paper:

 i) What is your problem and why is it interesting? (500-1500 words)

  • Can be psychological or computational
  • Need to explain why it is interesting from the perspective of cognitive psychology or computational theory or both.
  • Need to clearly explain how problem relates to content covered in the course.

ii) What is your approach and model? (500-1500 words)

  • How are you going to use neural network models to tackle your project?
  • What is the architecture of your model? Show a picture!
  • How does this model relate to your problem?
    • How does it “do” the task? What is the model analog of the behavior you are interested in?

iii)                Model details (500-1500 words)

  • How are the input and output representations constructed, and how does this relate to assumptions about the “real” representations in the system?
    • If representational structure is important to your insight about how the model will behave, make this structure clear here!
  • If your model learns, what learning rule are you going to employ and why?
  • What aspects of your model’s behavior are you going to look at as an analog of behavior?
    • Time to activate correct response?
    • Accuracy? How are you determining accuracy and why?
    • Behavior over the course of learning, or after fully trained?
    • Behavior in different training regimens?
  • What evidence will bear on the question you are asking?
    • What counts as “success”?

iv) Modeling results (500-1500 words)

  • Boring parameterization details.
    • How long did you train it? What learning rate? How did you decide when to stop? Etc.
  • Basic behavior
    • How did the network do on the examples you trained it with? If it fails to learn them all, which patterns does it fail to learn and why? Are these failures important and interesting or trivial?
  • Interesting behavior
    • Now report the key behaviors of interest, which you explicitly noted in the previous section.

v) Discussion / interpretation. (500-1500 words).

  • Relate observations back to original problem
    • How do you interpret the network behavior with respect to deriving a theory of the behavior?
    • If the network failed, why do you think it failed?
    • If the network succeeded, what does this imply about behavior in the domain?
  • If you are reporting multiple simulations, repeat iii) and iv)
  • Now summarize main points and indicate briefly what future work might look like.