# Reviewer Expertise¶

### 23rd July 2014 Neil D. Lawrence¶

In response to a question from Christoph Lippert about the expertise of reviewers, this notebook loads in reviewer expertise and summarizes it in a couple of different ways.

In [23]:
import cmtutils
import pylab as plt
%matplotlib inline
plt.rcParams['figure.figsize'] = (10., 8.)
plt.rcParams['font.size'] = 16


Load in the reviewer data from 24th July alongside the subject areas.

In [1]:
reviewers = cmtutils.reviewers(filename='2014-07-24_reviewer_export.xls',
subject_file='2014-07-24_reviewerSubjectAreas.xls')

Loaded Users.


A little helper function for histogramming papers.

In [106]:
def histogram_papers(papers, title):
papers = papers.fillna(0)
a= plt.hist(papers, bins=papers.max()-papers.min(), range=(papers.min(), papers.max()), align='mid')
plt.title(title)
plt.grid(False)
print papers.max()


### Reviewers: Papers Since 2007¶

Now let's start plotting. First we histogram the reviewing body showing the number of reviewers who've had the corresponding numbers of papers (since 2007).

In [109]:
histogram_papers(reviewers.users[reviewers.users.IsMetaReviewer=='No'].PapersSince2007,
'Reviewers: Number of NIPS papers since 2007')

22.0


### Reviewers: Papers Since 2012¶

Same idea, but more recent expertise, how many papers have reviewers had since 2012?

In [110]:
histogram_papers(reviewers.users[reviewers.users.IsMetaReviewer=='No'].PapersSince2012,
'Reviewers: Number of NIPS papers since 2012')

10.0


### Area Chairs: Papers Since 2007¶

Now the same for Area Chairs, how many papers have they had since 2007?

In [103]:
histogram_papers(reviewers.users[reviewers.users.IsMetaReviewer=='Yes'].PapersSince2007,
'Area Chairs: Number of NIPS papers since 2007')


### Area Chairs: Papers Since 2012¶

And finally the recent experience of Area Chairs. How many papers have they had since 2012?

In [104]:
histogram_papers(reviewers.users[reviewers.users.IsMetaReviewer=='Yes'].PapersSince2012,
'Area Chairs: Number of NIPS papers since 2012')