Import Meta Reviewer Suggestions

Neil D. Lawrence 13th April 2014

Get the meta reviewer names from the google spreadsheet. Then update the reviewer data base, reviewer.db with meta reviewers, then export as invites into CMT. Need to make a list of the meta reviewers only from the local data base to upload to CMT for the invites. Potential Area Chairs were stored in a Google doc spreadsheet. For access to the google spreadsheet to work, you need to set, in cmtutils the email and password variables to settings that can access the relevant google doc.

The Google doc contained one entry per meta reviewer in the form of

Name | Institute | Subjects | Email | Answer
In [ ]:
import cmtutils
from config import config
# Download our meta-reviewer names from google doc.
spreadsheet_key = config.get('google docs', 'program_committee_key')
spreadsheet_name = '2014-01-19_NIPS_ACs'
ds = cmtutils.drive_store(spreadsheet_key, spreadsheet_name)
column_fields={'1':'Name', '2':'Institute', '3':'Subjects', '4':'Email', '5':'Answer'}
ds.read(column_fields=column_fields)

Now take a look at the suggested area chairs to make sure they were read in correctly.

In [ ]:
ds.reviewers

To try and keep track of everything we have a local SQLite data base. Here we load that data base in and add invites, labelling them as 'FirstRoundAreaChairInvite'.

In [ ]:
# Load in the reviewer data base.
db = cmtutils.reviewerdb('test.db')
fieldname = 'FirstRoundAreaChairInvite'
db.add_field(fieldname) # Add the field if it doesn't exist already

Now add the users to the data base.

In [ ]:
db.add_users(ds.reviewers, fieldname=fieldname)

We run an SQL command on the data base to count how many invites we have where there is no reply (we assume that the column Metareviewer is set to NULL when there is no reply).

In [ ]:
# This is number of non replied first round invitees.
print db._execute_sql('SELECT COUNT(Active) FROM Reviewers WHERE ' + fieldname + '=1 AND Metareviewer is 0;')
# This exports reviewers into a file called cmt_export.tsv ready for invite.
#db.export_reviewers(sql='FirstRoundInvite=1')

Second Wave of Invites

We proceeded with a second wave of invites on March the 10th, this script loads those invites in, they are in the same spreadsheet file, but on a different sheet.

In [ ]:
spreadsheet_name = '2014-03-10_NIPS_ACs'
ds = cmtutils.drive_store(spreadsheet_name, spreadsheet_key)
ds.read(column_fields=column_fields)
fieldname='SecoundRoundAreaChairInvite'

Once again we can check users have been loaded from the spreadsheet properly.

In [ ]:
ds.reviewers

And now we can add them to the data base.

In [ ]:
db.add_users(ds.reviewers, fieldname=fieldname)

We can check how many reviewers we've just added who are not currently meta reviewers with the following SQL command.

In [ ]:
print db._execute_sql("""SELECT COUNT(Active) FROM Reviewers WHERE ' + fieldname + '=1 AND Metareviewer is NULL;""")
db.export_reviewers(sql=fieldname + '=1') # this exports those reviews into CMT for an upload.

Counting Number of Metareviewers

This command gives us the total number of meta reviewers by calling SQL on the data base.

In [ ]:
db._execute_sql("""SELECT COUNT(Active) FROM Reviewers WHERE Metareviewer=1;""")