Measuring Impact

Email us at info@thelearningbar.com to receive a copy of these slides. slides.

OurSCHOOL Pre-Survey Strategy Session
OurSCHOOL Data Deep Dive
OurSCHOOL Assessing Your Impact
OurSCHOOL Building a Thriving School
OurSCHOOL Parent Survey Debrief
OurSCHOOL Teacher Survey Debrief
OurSCHOOL Data Deep Dive for School Teams
EYE-TA Data Deep Dive for Districts

Request a demo

Make literacy your bottom line

Efficient reading instruction is at the center of community transformation, and children’s future opportunities. Learn about the financial benefits of reading success, or access resources to understand the literacy needs of your community now.

How much could grade 3 reading success save you in the next

Education system costs

What do you pay annually for reading interventions and interventionists in grades 3+ ?

How much does an additional year of schooling cost per student?

How many students take an additional year or more to complete high school?

Community costs

How many students drop out of school each year?*

EFFICIENT READING INSTRUCTION ADDS UP

* 20,000$ per student - Estimate based on Canadian Council on Learning (2009). No “drop” in the bucket: the high costs of dropping out.

The Confident Learners program provides a rigorous curriculum aligned with teaching activities, assessments and quality professional development. The program covers all pre-literacy to literacy skills required to achieve reading fluency by the end of Grade 3. The foundational model of literacy and language development that the Confident Learners Literacy Program uses, draws upon is the widely recognized and validated ‘Simple View of Reading.’ This model has two critical and complementary dimensions:

Code-related skill: Concepts about print, phonological awareness, letter knowledge, and reading fluency (decoding and spelling, recognition and spelling, speed and prosody).

Language skill: Vocabulary, receptive language, expressive language, and written language.

Children must master both to read proficiently.

Click to view a Code-Related Instructional Activity or Language-Related Instructional Activity.

Measures versus single questions – what’s the big deal?

The OurSCHOOL survey only uses multi-question measures, as opposed to single questions as found on most other school surveys. When developing our measures, we take steps to ensure they are the most representative of the construct under investigation. We begin with the broad construct (or domain) – for example, the concept of Anxiety – then we conduct a thorough examination of the existing literature and consult with subject matter experts.

A bank of questions is then developed that are intended to reflect the construct and are rigorously tested. We know that measures are more reliable when they include a number of well-formulated questions as this allows us to more accurately differentiate between students.

Click here for a more detailed breakdown of how are measures are constructed.

Customized Secondary School Surveys

MEASURES FOR GRADES 7-12

Social – Emotional Outcomes

– Social Engagement – Emotional Health

– Institutional Engagement – Career Pathways

– Intellectual Engagement

Physical Health Outcomes

– Nutrition – Risky Behaviours

– Physical Fitness – Sexual Health

Academic

Outcomes

Demographic

Factors

Drivers Of Student Outcomes

– Quality Instruction – Classroom Context

– School Context – Family Context

Dependent on selected measures within a domain, thematic reports are also available.

Find out how the RTI classification acts as a prediction model of a child being a successful reader at age 8 or 9.

The RTI classification includes three tiers, which are based on a prediction model derived from longitudinal data that estimated the likelihood of a child being a successful reader at age 8 or 9, based on his or her EYE-TA domain scores. Children with a greater than 80% chance of becoming a successful reader are classified as having Tier 1 learning needs; those with a 50% to 80% chance of becoming a successful reader are classified as having Tier 2 learning needs; and those with less than a 50% chance are considered to have Tier 3 learning needs. The prediction model takes into account the age of the child at the time of the assessment and his or her skill level in each of the five domains. The results derived from the model weights some skills more heavily than others. Skills in the Cognitive and Language and Communication domains figure most prominently; children’s Awareness of Self and Environment, Social Skills and Approaches to Learning, and Fine Motor skills also contribute to the classification.

An RTI classification is not permanent or necessarily long term, nor is it a label assigned to a child. As children are developing their literacy skills, their progress should be monitored on a regular basis, using a variety of formal and informal tests. In practice, teachers use the domain-specific results to plan their instruction. Schools with more than one kindergarten or Grade 1 classroom can use the RTI score to ensure the prevalence of vulnerable children is evenly distributed among classrooms.

Also, children with Tier 2 or Tier 3 learning needs do not necessarily have a specific learning or intellectual disability. The classification simply indicates that a child may need extra instructional time and resources during the primary school years.

Leading indicators versus trailing Indicators

School monitoring programs and national and international studies such as PISA provide trailing indicators of student’s outcomes. The primary aim of trailing indicators is to assess changes in system performance across schools and districts, and over time.

Educators need leading indicators to inform timely school policy and instructional decisions. Leading indicator data allow you to not only focus on what is happening in your school now, but also to predict what will happen as a result (e.g. high bullying rates now will lead to high anxiety rates later). Leading indicator data are collected before learning occurs or as it is occurring. They typically employ short tests and administer surveys more frequently. The data can be used to:

provide a framework for intervention (i.e., a response-to-intervention (RTI) program);
guide school policy and practice in order to improve school climate;
stimulate discussion about school reform;
identify issues relevant to particular students or groups of students;
identify students who need extra support;
involve parents in meaningful ways; and
increase student and teacher engagement.

We enable schools to continuously monitor several leading indicators that can help guide school policy and practice and assess the effects of school reforms.

Why do measures need to be valid?

Validity refers to whether a set of survey questions is measuring what it is intended to measure.

Steps for establishing the validity of an OurSCHOOL measure

Determine a theoretical definition of the construct by reviewing literature, and by seeking expert opinion.
Identify the questions that reflect the construct through thorough discussions with experts, as well as teachers and students in focus groups, and a review of other instruments.

Data are analyzed using two psychometric techniques: factor analysis and item response theory (IRT). The analysis provides an indication of how well the questions hold together to provide a measure of a single unified construct, and whether each question contributes to the reliable measurement of the construct.

Click here to delve deeper into validity in the OurSCHOOL context.

How do you make sure your data is accurate?

High accuracy of reported measures is best achieved by having 100% of students participate. However, a reasonable estimate can be achieved by having as few as 30 students participate.

Click here to get more details on the data accuracy generated by the OurSCHOOL survey.

Why do measures need to be reliable?

Reliability refers to the consistency of the measurement process and is one of the key criteria for a sound measure of a schooling outcome or school process.

High reliability is dependent on having multiple well-formulated questions contributing to a measure.
Reliability is usually assessed with a coefficient that ranges from 0 to 1.0, with 1.0 being perfectly reliable. Most of the measures used in our surveys have reliability coefficients ranging from 0.80 to 0.95.

Learn more about the level of reliability in the OurSCHOOL Survey.

Enhance your school planning

Join the OurSCHOOL Engagement Team in one of our personalized interactive sessions. Each session was designed to reach all levels of data literacy, meaning you don’t need to be a data expert or statistician to attend. Let our team of experts guide you in finding your data story!

1

PRE-SURVEY

STRATEGY SESSION

2

DATA

DEEP DIVE

3

ASSESSING

YOUR IMPACT

4

BUILDING A

THRIVING SCHOOL

Explore topics such as:

In this year-end online debrief, the

Engagement Team will provide

you with tips and resources you

can use to:

A school is thriving when teachers,

students, parents / guardians, and

the community are working together

toward a common vision of success.

Take a deep dive into your data!

• Data-driven planning,

• goal setting, and

• using survey measures and

custom questions to capture

actionable evidence.

Our experts will:

• Review your data

• present notable highlights, and

• identify areas for further

investigation.

• Review your progress,

• assess the impact of your

implemented strategies, and

• discuss ways to help your team

adjust strategies and set

priorities for the coming year.

In this session you will:

• Explore strategies for how to

engage your whole school

community and build a culture of

consultation and collabora-tion

between all stakeholders, and

• learn how to triangulate the

perspectives of students,

teachers and parents to gain

deep insight into your school

communities' alignment.

• This session is available for

schools that use the OurSCHOOL

suite of surveys (Student, Parent,

and Teacher).

Leave the session with:

Participants will:

• Strategies and resources to

generate excitement for the

OurSCHOOL survey, and

• ideas you can use for a

successful survey

implementation.

• Learn to interpret and share

their OurSCHOOL findings,

• identify areas to celebrate,

• pinpoint opportunities for

improvement, and develop

achievable action plans

for further success.

Leave the session with:

• Knowledge and resources to

assess the impact of your

current strategies, and

• tips and resources for creating

a plan before your next survey

implementation.

* Sessions are available in both online and in-person format with up to 50 attendees. Pricing is based on session details and availability may be limited.

The Confident Learner’s program adopts a skills-based, rather than grade-based approach to student learning and instruction. It also includes a structured Literacy Pathway to provide a clear and coherent pathway to proficiency through a sequence of clearly-stated learning outcomes.

Many curricula require students to master a number of poorly-defined skills to demonstrate the achievement of curriculum objectives. Educators must identify the appropriate skills and sequence of instruction that will enable each student to demonstrate proficiency of a given outcome.

How to set challenging and visible learning goals

The OurSCHOOL Teacher Survey requires schools to set challenging and visible learning goals for students. Teachers can do this by:

transferring knowledge through planned learning opportunities;
providing quality feedback that guides students’ efforts and attention; and
supporting students to overcome obstacles to achieving their learning goals.

Customized Elementary School Surveys

MEASURES FOR GRADES 4-6

Social – Emotional Outcomes

– Social Engagement – Intellectual Engagement

– Institutional Engagement – Emotional Health

Physical Health Outcomes

– Nutrition

– Physical Fitness

Demographic

Factors

Drivers Of

Student Outcomes

– Quality Instruction – Classroom Context

– School Context – Family Context

Confident Learners Program Overview

The Confident Learners framework was developed over the course of a three year research partnership with 32 First Nations communities to develop a scalable program, informed by best evidence, and designed to meet the literacy needs of each child, school and community. To outline some of the key challenges we addressed through our research, and evidence underlying the program’s approach and design, we have developed the Confident Learners Program Overview. To receive the Overview directly in your inbox please provide us with your contact details below.

Measuring Impact: How Student Agency drives improved Outcomes

Efficient reading instruction is at the center of community transformation, and children’s future opportunities. Learn about the financial benefits of reading success, or access resources to understand the literacy needs of your community now.