Examinations

We embarked on and completed a series of web scans to check if our exam content has been breached and released on the internet. We’re pleased to report that this does not appear to be the case. During this process we learned a lot about candidate’s desire for additional support, and we have scoped a project to refresh our exam information on our website and subsequently develop additional support materials for candidates of the KAPS exam in particular.  We continue to actively observe to detect potential leakage of examination information and this will continue into 2022/23.

New process for exam questions is continuing to be developed and implemented – Pairwise Scaling : We will be presenting a paper on this work at the CLEAR conference in September. This innovative new process alleviates the bottleneck of adding questions to our scaled exams.

We are constantly assessing how effective our exam content is.  We have noticed that developing case studies with 5 effective questions where all the associated questions perform to an acceptable level psychometrically is extremely difficult. Questions arising from case studies also provide an increased level of complexity in terms of psychometric analysis.

As a result, we have determined to phase out the inclusion of case studies and the associated questions in the KAPS exam. We will develop alternative standalone questions across all subject areas that include an increased level of patient specific information that will require candidates to undertake a focused evaluation of the material presented before moving to answer the associated question.

We saw the number of exam attempts return to typical numbers this year despite the interruptions of the pandemic. The introduction of online proctored exams and Rasch modelling methodology assisted with this.

Intern Written Exam

This exam tests the competency of pharmacy interns in Australia. Candidates sitting the exam must have completed at least 75% of the required supervised practice hours in Australia. Pharmacy Board of Australia – Internships

In 2020 we introduced the option of online remote proctored examinations for this year, to enable candidates an opportunity to sit the exam without travelling to an examination centre. This was a successful program, and after evaluation we have continued this. Each intern candidate can choose to sit remotely or in a test centre.

Of the 1502 Intern Written Exam attempts in 2021/22, 33% were supervised remotely via OnVUE (online).

CAOP (Competency Assessment of Overseas Pharmacists) exam

This exam tests the competency of pharmacists to practise in an Australian setting. Candidates sitting the exam are pharmacists trained and registered in:

• Canada
• Ireland
• United Kingdom (UK), OR
• United States of America (USA)

In June 2021, following successful implementation of online remote proctoring for the Intern Written Examination, we commenced offering CAOP by remote or test centre.

KAPS (Knowledge Assessment of Pharmaceutical Sciences) exam

This exam tests the knowledge of current pharmaceutical sciences in an Australian setting. Candidates sitting the exam are from countries other than those listed above. This examination is offered in test centres across 28 countries.

The 2021-2022 financial year has been a busy one for the Examinations Team. In the second half of 2021 the retirement of Peter Robinson led us to restructure how we handle the psychometric analysis of our exams to ensure their quality and consistency, leading us to strengthen our relationship with Professor Jim Tognolini of the University of Sydney who now serves as our external consultant for this purpose. Jim’s advice had already led us to explore new approaches for regularly refreshing our exam question bank, and in October 2021 we embarked on a trial of ‘pairwise scaling’, an approach we have proven successful and now intend to use ongoing. Jim and his team will also assist us as we embark on the next stages of our ambitions to further examine opportunities.  This work will be presented at the CLEAR conference in September in Louisville USA by our team.

At the turn of the new year we scoped several projects for 2022. We completed a series of web scans to check if our exam content has been breached and released on the internet and are happy to report that this does not appear to be the case. During this process we learned a lot about the candidates desire for additional support, and we have scoped a project to refresh our exam information on our website and subsequently develop additional support materials for candidates of the KAPS exam in particular. We are also undertaking to simplify the process we use for processing KAPS results after each exam session, strengthening our operational resilience.

On the technical side we are working to improve our access to and use of operational data, allowing us to report more effectively on our practices and ask deeper questions about the trends emerging amongst our candidates and our exams and improve our processes.

Finally, we are strengthening our relationship with our Fijian counterparts by improving the services we offer to their candidates to allow them to streamline their administrative work and give a smoother candidate journey.