EQA Methodology

EQAs are led by U.S. Government (USG) personnel. The optimal number of people on the EQA team is a function of the number of VMMC sites that will be visited. Each site visit should be performed by not fewer than four members and should include individuals with relevant skills and experience (e.g., team lead, clinician, demand generation & counseling expert, M&E practitioner, other clinical/non-clinical staff). Note that the EQA team usually includes staff from various organizations, including USG/OGAC/DOD/PEPFAR, World Health Organization, National Department of Health/Ministry of Health, and local implementing partners. At each assessed site, the EQA team performs:

  • Document and chart reviews
  • Observations of facilities and VMMC procedures
  • Observation of equipment and clients
  • Discussions with key personnel

During the EQA visit, data are collected on:

  • Number of circumcisions performed in the past 12 months
  • HIV status
  • Adverse events
  • Client record completion
  • Drug and supplies storage
  • Readiness for emergency procedures
  • Availability of standard operating procedures, policies and job aids

The EQA team also observes in-person communication, group and individual counseling, VMMC procedures, and follow-up reviews.

EQAs generate both quantitative findings in the form of scores from each tool for each site and qualitative findings in the form of violations of PEPFAR policy, weaknesses, strengths, and best practices. These latter findings are determined based on explicit criteria:

  • Violations of PEPFAR policy:
    • Require immediate remediation
    • Not appropriate for the site to continue services before this is addressed
  • Weaknesses:
    • Areas of performance not meeting minimally acceptable levels
    • Performance below 70% (where a score is applicable)
    • Require remediation
  • Strengths:
    • Areas of performance that meet all standards and demonstrate full compliance with quality standards
  • Best practices:
    • Superior performance that represents a benchmark exceeding relevant quality standards or expectations

Examples of policy violations found in VMMC EQAs include:

  • No sedation or general anesthesia use
  • Lack of emergency supplies, equipment and/or trained staff
  • Clients not provided with written instructions on post-procedure wound care

Commonly identified weaknesses in VMMC programs detected in EQA visits include:

  • Client vitals not consistently measured/documented:
  • Some sites with no provision to register temperature
  • Blood pressure not routinely taken; sometimes staff not aware why they have to perform this
  • Weight not consistently recorded: Sometimes not taken, other times taken but not registered (without documentation of patient weight, it is questionable how anesthetic dosing is calculated)
  • Details often omitted from documentation include: prolonged bleeding, age, time of surgery, provider, and VMMC method
  • Type and volume of anesthetic dosing and/or client weight not registered
  • Inconsistent adverse event documentation; clients without AE not captured
  • Missing emergency equipment and training for emergency management; many sites equipped with emergency trolleys but often located outside the surgical theatre
  • Staff training often not complete

EQA findings are summarized in country-level and site-level reports. As part of the EQA site visit, the assessment team provides a verbal debrief with the site-level VMMC team. As part of the overall country EQA visit, the EQA assessment team presents and discusses the EQA findings with all relevant stakeholders, including Ministry of Health/Department of Health officials, implementing partners, and USG staff.

Guidance on team composition, an approximate timeline of major activities encompassed in the EQA, and outputs of the EQA are found in the resource below.

VMMC EQA Mobile Application: VMMC QUAL

To facilitate VMMC EQAs, USAID funded ASSIST to develop, pilot test, and deploy a VMMC EQA mobile application: VMMC QUAL. Developed in 2015, VMMC QUAL:

  • Uses a dashboard to compare sites, countries or different years of EQAs performed
  • Enables multiple inputs simultaneously
  • Has offline and online capabilities

VMMC QUAL can be downloaded to a tablet or android phone. For more information, contact assist-info@urc-chs.com.

The mobile app offers a number of advantage over the use of paper-based tools alone. The use of the app makes it easier to provide quantitative results immediately at the assessed site and complete all reporting of results within 2-3 weeks. The VMMC QUAL app allows for real-time collection and analysis of data and delivers more specific feedback to sites for action planning. Another advantage of the mobile app is that it incorporates VMMC Site Improvement through Monitoring System (SIMS) assessment within the EQA, allowing US Government staff to accomplish a SIMS site visit at the same time as conducting the EQA.

screenshot of the VMMC QUAL login page

Resources

This guidance was developed as part of the USAID-led EQA training carried out in Malawi in March 2017.

Year: