Project objectives

In late 2019, the Australian National Audit Office (ANAO) reviewed the implementation of the My Health Record (MHR) system. The ANAO, while noting that the implementation of MHR has largely been effective to date, made recommendations for further improving risk management and evaluation of the system, including a recommendation that the Australian Digital Health Agency “develop and implement a program evaluation plan for My Health Record, including forward timeframes and sequencing of measurement and evaluation activities across the coming years, and report on the outcomes of benefits evaluation.”

HPA partnered with Taylor Fry to develop a 10-year evaluation plan for the MHR that addressed this recommendation.

Australian Digital Health Agency

Our approach

To develop the plan, we reviewed existing ADHA documentation and evaluation activities, reviewed literature to investigate international approaches to evaluating similar digital health initiatives and consulted with a broad range of internal and external stakeholders. External stakeholders included government agencies, subject experts and various other clinical and consumer groups.

We developed a program theory as a framework for the evaluation plan. The theory shows how the MHR and its use could achieve its intended outcomes. Since many of these intended objectives are long-term outcomes, the program theory facilitates a structured approach to identifying and evaluating shorter-term measures with an attributable line of sight to these outcomes. The program theory was used to frame and develop evaluation questions that would test, measure, monitor and validate key aspects of the MHR. The questions are grouped into domains; context (including the digital ecosystem, organisational, regulatory, legal, and other factors), technology, information, adopters and adoption, meaningful use, outcomes and impact.

In the plan, we also detailed quantitative and qualitative methods to answer the evaluation questions that took into consideration the ADHA’s current and planned monitoring and evaluation activities. The methods include observational studies (such as natural experiments) controlled trials, descriptive analyses, qualitative analyses, and synthesis of published literature. We included a project plan within the evaluation plan which set forth timelines for specific evaluation activities and recommendations on governance for the evaluation.

Outcome

The ADHA is using the plan to guide its evaluation activities and monitor the MHR’s progress in achieving its intended objectives over the next 10 years.

Let's work together

Contact us