Position Overview
The Assessment System Analyst collaborates with Undergraduate Medical Education leadership, course and clerkship directors, and faculty on the development of clinical assessments (e.g., EPAs), objective structured clinical examinations (OSCEs), and simulation. This person will assist in the implementation of testing software, will collaborate on building exams, and work with dashboards and assessment systems and software, including Item Response Theory and computer adaptive testing (CAT).
The Assessment System Analyst will be knowledgeable and experienced in the use of statistical and analytic software such as SPSS, R, Python, etc. and be able to conduct statistical and qualitative data analyses.
The Assessment System Analyst will train and support students, faculty, and staff on the use of medical school assessment tools. This person will serve as a contact point with collaborators in the office of information technology on issues related to assessment systems and software.
Duties/Responsibilities
30%: Development of Clinical Assessments
- Analyze and refine the Clinical Skills Assessment (CSA) protocol
- Assemble EPA assessment data and reports as requested
- Communicate effectively with students, staff, faculty, clerkship directors and leadership on the results of data analysis
30%: Develop and Maintain Dashboards and Assessment Systems
- Prepare data tables, charts, statistical significance
- Analyze and interpret data tables, charts, statistical significance, dashboards using software such as SPSS, R, Python
- Prepare narrative reports
20%: Implementation and Management of Testing Software
- Manage ExamSoft item analysis
- Explore the use of Fastest for potential use in (CAT)
10%: Collaborate on Building Exams
- Consult with Foundational Course directors, instructional designers and instructors regarding effective test construction and results of item analysis for continuous improvement
- Advise Clerkship Directors
10%: Maintain Clinical Exams
- Analyze results of objective structured clinical examinations (OSCEs) and collaborate with faculty and assessment colleagues to refine OSCEs based on data