This repository contains dataset and code of the "A Method for Automatically Estimating the Informativeness of Peer Reviews" Authors: Prabhat Kumar Bharti, Tirthankar Ghoshal, Mayank Agrawal, Asif Ekbal, Affiliation: Indian Institute of Technology, Patna, India
baseline_res :
- contains the output csv files from the baseline ICLR 2018 reviews analysis.
-
- contains the output csv files from the qualitative NIPS 2018 reviews analysis.
- contains the output csv files from the qualitative NIPS 2018 reviews analysis.
-
graphs :
- contains the graphs and other pictorial charts obtained from the reviews analysis.
- contains the graphs and other pictorial charts obtained from the reviews analysis.
-
- contains the codes for initialising the Baseline ICLR 2018 analysis; extracting the annotated labels, finding their coverage and distribution and hence, calculating the Section Scores and Aspect Scores; and also finding features such as "sentence count", "word count", "avg. sentence length", "avg. word length", "vocab length", "hedge word count", "non-hedge word count", and "hedge ratio", for use in future work.
- contains the codes for initialising the Baseline ICLR 2018 analysis; extracting the annotated labels, finding their coverage and distribution and hence, calculating the Section Scores and Aspect Scores; and also finding features such as "sentence count", "word count", "avg. sentence length", "avg. word length", "vocab length", "hedge word count", "non-hedge word count", and "hedge ratio", for use in future work.
-
- contains the codes for finding the Hedge Score using Tirthankar Ghoshal HedgePeer Model for the baseline ICLR 2018 analysis.
- contains the codes for finding the Hedge Score using Tirthankar Ghoshal HedgePeer Model for the baseline ICLR 2018 analysis.
-
- contains the codes for calculating the Informativeness Score for baseline ICLR 2018 analysis, along with a few other features such as "VADER compound sentiment" and "POS features" for future work.
- contains the codes for calculating the Informativeness Score for baseline ICLR 2018 analysis, along with a few other features such as "VADER compound sentiment" and "POS features" for future work.
-
04_Qualitative ipynb notebook :
- contains the codes for performing Qualitative Analysis on NIPS 2018 reviews, by extracting their labels, finding all the previously mentioned features, and calculating the Section Score, Aspect Score, Hedge Score and hence, the Informativeness Score..
-
- contains the weights for each Section and Aspect label obtained from their frequency distribution of overall coverage in all papers, that will be used to calculate the Section and Aspect Scores.