Design of a conceptual inventory in fundamental ecology

Background/Question/Methods

Developing useful assessments requires careful design and evaluation, emphasizing the importance of collaboration among educators. Here I present methods and data from the design, implementation and evaluation of a conceptual inventory.  Seventeen conceptual questions were designed and given as an online test at the beginning of a third year introductory course in ecology in the fall of 2009 at the University of British Columbia, Vancouver, Canada. Validation of questions was done through interviewing 35 students and experts in ecology. Ten additional questions were designed and added to the inventory to ensure important course topics were represented. A total of 27 questions were given as a post test at the end of the fall semester. Evaluation of the inventory was done by comparing students’ post-test scores with final exam scores and by comparing the pre and post-test scores of students in different programs.

Results/Conclusions

280 out of 388 students took both pre and post tests. Overall, students significantly improved their scores over the semester. The gain however was small (5.0%). I found a significant, but weak (r=0.21) correlation between post-test scores and final exam scores. Students performing well (>75%) on the final exam scored significantly higher on the post test than students who performed <75% (41.2% vs. 47.5%). The inventory also revealed that students in different programs differed significantly in their understanding of ecological concepts as they entered the introductory course.  Students enrolled in the general science program scored significantly lower on the pre-test than did students in the animal biology program (31.4% vs. 40.2%). Students in other programs, i.e. in general biology, cell biology and genetics and ecology and environmental biology scored in between these two programs. Students enrolled in the animal biology program also finished the course with a significantly higher score than did students in general science (34.5% vs. 46.6%). Similar significant differences were found between students who had taken previous courses in ecology and those who had no previous experience as well as between students who planned to take advanced courses in ecology and those how did not. My results suggest that the designed inventory can be used to gauge students’ understanding of ecological concepts as they enter and finish courses in ecology. However, the low overall score gain raises questions such as: Do students take on-line inventories seriously?  Why do students who perform well on final exam still score low on the post test?