External Evaluator Orbis International

Orbis International is a nonprofit organization that works in Africa, Asia, Latin America, and the Caribbean. Our vision is to transform lives through the prevention and treatment of blindness. Without network of partners, we mentor, train and inspire local communities so they can save sight in their communities.

In Zambia, Orbis is a partner of the Ministry of Health and provides technical support to eye care teams to provide quality eye care in Lusaka, Copperbelt and Northwestern provinces.

Orbis seeks the services of a legally registered and qualified individual external evaluator to conduct an end of project evaluation for the “Strengthening Eye Health Capacity and Care in Lusaka” project.

The consultant must have a license and Taxpayer Registration Certificate to justify the 15% withholding tax deduction or willing to have 15% of the overall payment retained with Orbis International Zambia as withholding tax and ultimately paid to the Zambia Revenue Authority.

If eligible, please submit your interest and technical and financial proposal in response to the TORs below by email within 7 working days of this announcement to the address seen below. The closing date for receiving the applications is close of business, 11th March 2022.

o   Interested applicants can look at the detailed Terms of Reference (TOR)             attached to this advert and send their application to                                                 info.zambia@orbis.org.  copying in tendai.nhandu@orbis.org.

o   The successful consultant will be notified by 23rd March 2022.

Orbis International Zambia reserves the right to reject any or all bids and proposals.

Orbis International Zambia

15 Kabulonga Road, Kabulonga, Lusaka.

Tel +260 211 265902/ Mobile 0763590963 | Zambia.

TOR for End-term Evaluation

Project Title: Strengthening Eye Health Capacity and Care in Lusaka, Zambia

1. Introduction and Background

1.1 Introduction:

This Terms of Reference (TOR) outlines relevant details for Orbis International Zambia to hire an external evaluator to lead an end-term evaluation of Strengthening Eye Health Capacity and Care in Lusaka project.

Orbis International is a non-aligned, non-profit, global development organization on a mission to transform lives through access to quality eye health. To achieve its mission, it builds the capabilities of its partners through education and training, eye care management systems development, technical and financial support, and research. Orbis International (OI) manages four global programs – a Flying Eye Hospital, Hospital Based Programs, Fellowships, and Cybersight – and has country programs in Asia, Africa, Latin America, and the Caribbean.

Orbis International Country Office/Program: Orbis International Zambia (OIZ) is branch of OI which was established in 2016 and works in partnership with the Ministry of Health. OIZ’s mission is to transform lives through the prevention and treatment of blindness and a vision of networking with partners, mentor, train and inspire local teams so that they can save sight in their communities. In Lusaka Province, Orbis works across all levels of eye health throughout the province’s six district with a focus to support the University Teaching Hospital (UTH)-Eye Hospital’s priority capacity and equipment needs to ensure it has the infrastructure, operational capacity and staffing to provide comprehensive and equitable adult and pediatric eye health care services. Orbis has also employed learnings and proven models from previous work in Zambia to roll out community awareness and primary/district level capacity strengthening interventions in Lusaka province.

1.2  Background:

UTH-Eye Hospital is the first public independent eye hospital in Zambia. In 2017, UTH in Lusaka was restructured into 5 specialized hospitals, including an eye hospital (UTH-Eye Hospital).  Operating previously as a relatively small eye unit within UTH, UTH-Eye Hospital is now earmarked to be a national center of excellence – the leading institution in eye health care coordination, clinical care, residency training, research, and eye health promotion in Zambia.

A core focus of this project was to support UTH-Eye Hospital to improve the structure of their residency program and enhance faculty capacity building through Hospital Based Trainings and sub specialist fellowships. UTH was also supported with equipment and consumables.  In its last year of implementation, the project extended the support to the district hospitals and 1st level hospitals, supporting them with equipment, consumables, training and outreach services.

A mid-term review was conducted virtually in May 2021 to assess and see how program activities were being delivered, which actors were involved, and which processes were being used. This informed the implementation of the rest of remaining activities.

2.1 Project goal: To ensure quality eye health for all in Lusaka province

1.2 Project objectives:

a) To strengthen the implementation and management of the residency program at UTH-Eye Hospital

b) To improve capacity at UTH-Eye Hospital to provide comprehensive quality ophthalmology services.

c) To drive demand and increase uptake of quality eye health services in Lusaka district.

d) To advocate for the development and implementation of favorable eye health policies in Zambia

2.3 Target population and project sites:

The University Teaching Eye Hospital is the highest referral hospital in the country. The targeted beneficiaries of the project were both the residents and the faculty being capacity built while at the same time those that came to seek eye health services at this facility. Being a referral hospital, patients from all over the country were accessing its eye health services. Patients are referred from district hospitals and 1st level hospitals from within Lusaka province.

Project Sites

• UTH Eye Hospital: responsible for coordinating the outreach activities of District hospitals, and the performance of static services and at the district facilities during outreach camps.
• 1st level hospitals: responsible for initial screening of patients as they receive referrals from the health posts. They also carry out community outreach screening and treating those that don’t need to be referred to UTH.
• Six district hospitals (secondary level): deliver primary eye care and carry out outreach screenings in the community, and coordinates activities of the local Health Posts and Health Centers.

2.4 Project duration: May 2019 to 31st March 2022.

2.5 Project partners: The Project partnered with the Provincial Ministry of Health to roll out the project at all levels of health care and supported service deliver in the four 1st level hospitals and six district hospitals. The project implemented through the Ministry of health structures and staff, offering capacity building (including training), systems strengthening (including equipment and consumables) as well as outreach and referral.

2.6 Project strategy and approach:

Over the period 2019-2021, with funding support from the David and Molly Pyott Foundation, Orbis set out to implement key strategic interventions to ensure that:

1. UTH-Eye Hospital delivers a high-quality residency training program that results in increasing numbers of competent ophthalmology graduates.

2. UTH-Eye Hospital has improved clinical and operational (i.e., pricing structures, workflow efficiencies, sustainability) capacity to provide comprehensive quality adult and pediatric ophthalmology services.

3. Demand for, and uptake of services at UTH-Eye Hospital and key health facilities in Lusaka district increases.

4. National eye health issues are prioritized and promoted through robust engagement with key influencers, including national, regional, and local government officials, civic leaders and other community-based NGOs working in the region.

2. Objectives and Outcomes of the Evaluation

3.1 Evaluation objectives:

• Assess the performance and results of the project in relation to the project goal.
• Assess the effects of the program on target groups, partner institutions and hospitals.
• Generate evidence of organizational learning and accountability.
• Make recommendations to inform the design of a revised program.

3.2 Evaluation outcome:

This evaluation is meant to document how relevant, effective and efficient the project was in meeting the planned objectives and goal, as well as its impact. The outcome of the evaluation is to generate lessons learnt, conclusions and recommendations that Orbis can use to improve on the implementation of the second phase of the project.

3.3 Evaluator deliverables:

1. An inception package, outlining:

a. Evaluation design and methodology. In the methodology section, detail:

• data collection tools and organization
• sampling procedures
• quality assurance
• data analysis
• ethical considerations

b. Work-plan

2. A presentation to Orbis International Zambia highlighting significant findings and draft recommendations before the end of field work.

3. A draft report to be reviewed and commented on by Orbis International Zambia.

4. A final report of maximum 40 pages, excluding annexes, but including an executive summary of maximum two pages.

5. A power-point presentation of the main findings, conclusions, and recommendations, suitable for Orbis International Zambia to use in presentation to stakeholders.

3.3.1 Structure of the final report

The report should be written in English and include the following sections:

• Acknowledgments
• Table of contents
• Executive summary
• List of abbreviations
• Introduction
• Background: overview of project, strategy, and activities
• Methodology: methods, limitations
• Findings: overall, by objectives, and by partner hospital
• Strengths, gaps/challenges, and lessons learned
• Discussion
• Recommendations
• Annexes: TOR; data collection tools; lists of interviewees, documents reviewed, sites visited; and disclosure of any evaluation team member’s conflict of interest, etc.

3. Evaluation Scope, Questions, and Methodology

4.1 Scope of the evaluation:

This review will cover the period May 2019 – March 31st, 2022, and the evaluator is expected to:

• Outline evaluation design and methodology including data collection instruments and detailed work plan.
• Collect, compile, and analyze the data.
• Produce draft and final review reports.
• Present the main findings, conclusions, and recommendations to Orbis Zambia and partner institutions.

The evaluation will cover all the sampled districts which will be a representative of the 7 districts including Lusaka, Chongwe, Chirundu, Kafue, Luangwa, Rufunsa and Chilanga where the project is working in as mentioned above and will seek to establish if all the four project objectives as listed below were met.

a) To strengthen the implementation and management of the residency program at UTH-Eye Hospital.

b) To improve capacity at UTH-Eye Hospital to provide comprehensive quality ophthalmology services.

c) To drive demand and increase uptake of quality eye health services in Lusaka district.

d) To advocate for the development and implementation of favorable eye health policies in Zambia.

4.2 Evaluation questions:

1. How effective and efficient were projects in achieving their: (i) targets and (ii) objectives (allowing for external factors potentially contributing to these results and noting changes in objectives during the life of projects)?

2. What were the strengths, gaps/challenges, and lessons learned with respect to results achieved (including distribution of results), implementation, and application of learning to project design and interventions?

3. Did the projects have other notable effects on the facilities and beneficiaries? These may be positive or negative, and planned or unplanned

4. What are the recommendations for a revised project with the same focus?

4.3 Evaluation methodology:

The evaluator is expected to design an appropriate methodology to answer the evaluation questions. The methodology and data collection tools will be finalized in consultation with Orbis. Typically, the evaluator is required to travel to project sites and have face-to-face discussions with various project stakeholders to gather required information. The evaluator will have contact meetings with respondents. The methodology should include:

A. Review of secondary data including but not limited to the project plan; M&E operational plan, data collection instruments, quarterly and annual progress reports; financial reports; medical/hospital data; and event reports, case studies, and special assessments/surveys.

B. Generate primary data via methods such as the following:

• Retrieve core indicator data directly from the HMIS/records and analyse to unearth trends in volume of patient services provided (e.g., # of patient visits, # of spectacles prescribed, # of surgeries performed etc.) from partner hospitals, as well as validate data that has already been reported.
• Collect and analyze data from eye health facilities (e.g., vision centres, etc.)
• Interview health facility staff, management, and other key informants
• Consult with relevant Orbis staff
• Collect case stories to show the outcome or impact of the project. Case stories can be patients who received services from the partner hospitals, or hospital staff who received training
• Conduct partner and beneficiaries’ satisfaction surveys using Orbis tools.

4.4 Selection of sites or partners for the evaluation:

The evaluator is required to sample some sites using a method that will help select a representative sample of the total sites. The project is implemented in urban, peri urban and rural districts. This therefore means the health facilities are at different levels. Some districts have ophthalmic staff and others don’t have and this should be considered during sampling. The following factors should be considered while sampling.

• Service delivery level (e.g., primary, secondary, or tertiary)
• Geographical representations (e.g., rural vs. urban)
• Facilities with Ophthalmic staff (Ophthalmologists, Ophthalmic Clinical Officers and Ophthalmic Nurses)

The following sites are the project implementation areas.

DISTRICT
ROUTE
GEOGRAPHICAL REPRESENTATION
SERVICE DELIVERY LEVEL
OPHTHALMIC STAFF
Lusaka
Provincial HQ
Urban
Primary, secondary & Tertiary
Ophthalmologists, OCO & ONs
Chirundu
South
Peri-urban
Primary & Secondary
None
Kafue
South
Urban
Primary & Secondary
OCO & ONs
Chilanga
South
Peri-urban
Primary
None
Chongwe
East
Peri-urban
Primary & Secondary
OCO & ONs
Rufunsa
East
Rural
Primary & Secondary
None
Luangwa
East
Rural
Primary & Secondary
OCO
Chilenje
Lusaka
Urban
Primary
ONs
Chawama
Lusaka
Urban
Primary
ONs
Matero
Lusaka
Urban
Primary
ON
Mandevu
Lusaka
Urban
Primary
OCO & ON
Bauleni
Lusaka
Urban
Primary
ONs
Chelstone
Lusaka
Urban
Primary
None
Kanyama
Lusaka
Urban
Primary
None

4. Timeline and Tasks

SL
Task
Time frame
1
TOR shared with potential evaluators
4th March, 2022
2
Proposals received from interested evaluators
11th March, 2022
3
Evaluator selected
23rd March, 2022
4
Evaluation design and tools developed
30th March, 2022
5
Inception meeting
4th April, 2022
5
Virtual consultation, key informant interviews, and secondary data analysis completed
11th April, 2022
6
Primary data analysis completed, and draft report submitted
28th April, 2022
7
Findings presented by evaluator
29th April, 2022
9
Orbis feedback on draft report provided
6th May, 2022
10
Final report submitted
13th May, 2022
5.  Expected Competencies of and Selection Criteria for the Evaluator

6.1 Expected Competencies of the Evaluator

The evaluator should have:

• At least a master’s degree in the field of public health/medicine/development studies/social science or other relevant discipline.
• Experience in research and evaluation in health systems, public health, community health, community development, and eye health care services
• Professional familiarity with hospital settings
• Strong methodological skills in the development and implementation of data collection tools, data management, and analysis
• Excellent written English and report writing skills
• Oral and written proficiency in the local language

6.2 Evaluator Selection Criteria

Criteria
Weight in Points
Technical proposal

Personal/Organizational Background

Relevant academic qualification (e.g., ophthalmology, public health, M&E, etc.)
5
Relevant experience (e.g., evaluation/ research/ survey in eye health)
15
Team (preferred over an individual evaluator)
5
Proposal Developed by Evaluator

Proposed methodology
20
Proposed data management and analysis plan
10
Structure and presentation of proposal
5
Work plan/ time frame
10
Total
70
Financial proposal

Proposed budget
30
Grand Total
100
Technical proposals must earn at least 52.5 points (75%) to be considered.

6. Submission Checklist for Proposal

The evaluator will submit a detailed proposal. The proposal must describe the methodology, tools, and analysis plan in detail. The proposal should be divided into two parts: technical and financial. The financial proposal will include the evaluation’s estimated cost and list the evaluator’s daily rate. The technical part will contain the following sections:

Background of evaluation company or organization
Detailed evaluation methodology
Detailed work plan/timeframe
Evaluator’s CV listing relevant experience
References
Proposals will be subject to a price and quality comparison.

7. Payment Schedule

1st installment (40%) will be made upon signing of the contract.

2nd installment (40%) will be made upon submission of first draft.

3rd installment (20%) will be made upon submission of final report.

8. How to Apply

Interested applicants are requested to summit proposal (technical and financial) and required documents on or before 11th March 2022. The application should be submitted via email to info.zambia@orbis.org, copying in tendai.nhandu@orbis.org g. Please mention “Pyott TOR” in the email subject line.

9.    Considerations

10.1 Ethical Considerations:

The evaluation will respect human subjects’ considerations such as confidentiality of responses in interviews or discussions as well as any personal patient information (which will not include identifiers) and obtaining informed oral consent from patients participating in interviews or surveys.

10.2 Child Protection Policy:

The firm/individual shall comply with the Child Protection Policy of Orbis International. Any violation /deviation in complying with Orbis’ child protection policy will result in termination of the agreement.

10.3 Binding:

All documents, papers, and data produced during the assessment are to be treated as Orbis property and restricted for public use. The contracted agency/consultant will submit all original documents, materials, and data to Orbis International Zambia.