Beyond the Review Tool: A Content Analysis of Narrative Review Reports Produced by the Balochistan Textbook Board

Abstract

Textbook manuscripts submitted to the Balochistan Textbook Board (BTBB) are evaluated by panel of experts whose judgment is recorded largely as narrative, free-text comments in addition to the BTBB review tool. Though the review tools used in this process have received a little attention in earlier work, the descriptive comments themselves given by the reviewers have not been studied directly. This current paper reports a qualitative content analysis of the descriptive reports produced by the reviewers. A total of 574 individual reviewer comments extracted from eleven BTBB Internal Review Committee (IRC) reports of English, Mathematics, and Computer Science manuscripts across Grades 9 to 12, were analysed. The comments were coded under four dimensions namely subject-matter focus, objectivity, evidence base, and actionability. Analysis of the data shows that curriculum and SLO alignment along with general content accuracy are leading concerns of reviewers with (23%) and (23%) values respectively. Additionally it was found that purely subjective, unjustified comments are rare (0.2%), but that a large majority of comments (74%) are semi-objective and only 22% cite a specific, verifiable reference. The third variable, actionability, also shows noticeable differences depending on how the report is structured. Reports that present findings alongside their recommendations tend to provide clearer and more practical guidance than those that list findings and recommendations separately. Overall, the consistency of the reports seems to be influenced more by the subject area and the format of the report than by individual reviewers. The paper presents recommendations for strengthening and improving evidence-based BTBB review practice and suggests recommendations for further research.