NEXT: Replicating a triplet experiment

Download zip file of materials: NEXT-tutorial.zip
Materials include: init.yaml file, zip file of 20 image stimuli, example text file with experiment details.

Note: this tutorial requires you to set up an account (with credit card to pay for server time) at AWS (Amazon Web Services). If you have not done this, go to the AWS website and click Sign Up. Follow the instructions to set up an account. It may take Amazon 24 hours to verify your credit card, so when it has been verified, return here for the tutorial.

Using NEXT for psychology experiments

The majority of NEXT documentation is on its github wiki, which we will reference throughout the tutorial. Because NEXT was developed outside of the Knowledge and Concepts Lab, any questions that are unanswered by our pages here or are not in the NEXT wiki should be directed to the NEXT development team.

There are 3 key steps to getting your experiment to run on NEXT. Click each link to go to that section of the tutorial.

Step 1: Set up and launch a machine running the NEXT software using one of the remote machines available from AWS.

Step 2: Once the NEXT instance has been launched successfully, set up and launch an experiment

Step 3: Once your experiment has been successfully launched, manage the experiment URL and download data

 

Notes

How many queries?

  • No-noise model: 2*n*d*log(n)
  • Standard estimate: 10*n*d*log(n)
  • High-noise model: 30*n*d*log(n)

How much does AWS cost?

Costs are not completely straightforward to calculate, because the charges depend on the storage requirements, computing power of the next server, market demand, and others. As an example, though: for a c3.8xlarge, 200gb server running 3 triplet experiments with simple image stimuli, charges after running for 24 hours were approximately $40.

This section will be updated as we learn more about the AWS cost structure.

Things AWS does charge for

  • S3 bucket (data, for instance your stimuli)
  • EC2 instances
  • EC2 volumes

Things AWS does not charge for

  • Security groups