Performance Measurement
A provincial education department developed an accountability pillar framework to measure performance within the educational system. Over a series of years, HarGroup consultants were involved in the development and assessment of measurement tools used within the accountability pillar framework.
From analyzing options to conduct province-wide surveys of parents, teachers, and students, to designing surveying instruments, to testing implementation processes, HarGroup consultants assisted the educational department in many different areas of the measurement process. A summary of activities in which HarGroup consultants were involved, include:
- interviewing school jurisdictions to identify existing approaches to gathering input and feedback about school performance
- conducting benchmark and best practices research with other provincial jurisdictions about performance measurement systems being employed
- preparing a satisfaction survey manual to assist the department in working with school jurisdictions
- designing survey instruments in conjunction with department and school jurisdiction representatives through workshops
- testing survey instruments with students and teachers in classrooms and parents through mail survey processes
- introducing and obtaining feedback about the accountability framework measurement process from school jurisdictions throughout the province
- pilot testing online survey processes with students and teachers in school classrooms
- reviewing and evaluating the survey measures and methodologies after three years of implementation including contributions of survey data in department decision making processes
For over a decade, HarGroup consultants conducts an annual Customer Satisfaction Survey to assess Calgarians' use and non-use of Transit services. The surveys involved 500 telephone interviews with existing transit customers and 500 interviews with non-users of the transit service. Various key measures comprised the satisfaction surveys such as transit use, relevancy of transit in the lives of citizens, customer loyalty and commitment, and an assortment of service attributes. The measurement process also examined service offerings that were being considered by the transit service and methods of communication to reach out to customers.
Another component of the transit satisfaction survey was measures associated with safety, security, and cleanliness. This survey involved 500 residents who used the transit service on a regular basis. Key measures examined safety ratings, rider experiences at different times of travel, possible influences of perceptions, exposure to nuisances and crimes, and cleanliness of vehicles, stations, and stops.
In both cases, detailed statistical analysis was performed to identify drivers of satisfaction and importance to performance comparisons.
Periodically, a Community Services department of a municipality engaged HarGroup consultants to conduct a sense of community survey to assess residents’ feelings of community, belonging, efficacy, neighborliness, safety, civic pride and satisfaction. The findings of the survey are used to enhance programs and services within the town.
Using the results from different waves of the survey, changes in perceptions about sense of civic pride, and satisfaction and belonging in the community were examined and understood. These aspects of sense of community were measured and compared for the community as a whole, as well as within the neighbourhoods where residents live.
Participatory and engagement facets of community such as volunteering, attending events and activities, engaging in social relationships and connections with neighbors, and depending and relying on other residents were also examined in the survey.
Opinions about safety and security of living in the community were measured and compared to other measures to gain insight into factors that contribute to and influence sense of community.
The community is comprised of permanent and non-permanent residents and the analysis examined differences and changes in perceptions over the various survey waves.
Ultimately, the survey results enabled Community Services to understand residents' awareness and responses to programs and services that were developed and delivered by the town.
HarGroup consultants have applied these sense of community measures in many other projects it has conducted for municipalities throughout the years.