IAE Member Research Series: Richard Shavelson discusses the research paper "Methodological perspectives: Standardized (summative) or contextualized (formative)"

NEWS

Dive into the fascinating world of educational evaluation with our latest video from the IAE Member Research Series!

Join Nick Burbules from the International Academy of Education as he explores Richard Shavelson, Professor of Education, Emeritus at Stanford University’s thought-provoking paper on the nuanced differences between formative and summative evaluations.

This insightful interview sheds light on the complex interplay of data, politics, and real-world challenges in assessing educational systems, with a compelling case study from Mexico. Discover how different types of evaluation questions can profoundly impact our understanding of teaching quality and student outcomes. Whether you’re an educator, a policy maker, or simply passionate about the future of education, this video is a must-watch! Tune in to unravel the layers of educational evaluation and its role in shaping better futures.

Request a demo

Make literacy your bottom line

Efficient reading instruction is at the center of community transformation, and children’s future opportunities. Learn about the financial benefits of reading success, or access resources to understand the literacy needs of your community now.

How much could grade 3 reading success save you in the next

Education system costs

What do you pay annually for reading interventions and interventionists in grades 3+ ?

How much does an additional year of schooling cost per student?

How many students take an additional year or more to complete high school?

Community costs

How many students drop out of school each year?*

EFFICIENT READING INSTRUCTION ADDS UP

* 20,000$ per student - Estimate based on Canadian Council on Learning (2009). No “drop” in the bucket: the high costs of dropping out.

The OurSCHOOL survey only uses multi-question measures, as opposed to single questions as found on most other school surveys. When developing our measures, we take steps to ensure they are the most representative of the construct under investigation. We begin with the broad construct (or domain) – for example, the concept of Anxiety – then we conduct a thorough examination of the existing literature and consult with subject matter experts.

A bank of questions is then developed that are intended to reflect the construct and are rigorously tested. We know that measures are more reliable when they include a number of well-formulated questions as this allows us to more accurately differentiate between students.

Click here for a more detailed breakdown of how are measures are constructed.

The RTI classification includes three tiers, which are based on a prediction model derived from longitudinal data that estimated the likelihood of a child being a successful reader at age 8 or 9, based on his or her EYE-TA domain scores. Children with a greater than 80% chance of becoming a successful reader are classified as having Tier 1 learning needs; those with a 50% to 80% chance of becoming a successful reader are classified as having Tier 2 learning needs; and those with less than a 50% chance are considered to have Tier 3 learning needs. The prediction model takes into account the age of the child at the time of the assessment and his or her skill level in each of the five domains. The results derived from the model weights some skills more heavily than others. Skills in the Cognitive and Language and Communication domains figure most prominently; children’s Awareness of Self and Environment, Social Skills and Approaches to Learning, and Fine Motor skills also contribute to the classification.

An RTI classification is not permanent or necessarily long term, nor is it a label assigned to a child. As children are developing their literacy skills, their progress should be monitored on a regular basis, using a variety of formal and informal tests. In practice, teachers use the domain-specific results to plan their instruction. Schools with more than one kindergarten or Grade 1 classroom can use the RTI score to ensure the prevalence of vulnerable children is evenly distributed among classrooms.

Also, children with Tier 2 or Tier 3 learning needs do not necessarily have a specific learning or intellectual disability. The classification simply indicates that a child may need extra instructional time and resources during the primary school years.

School monitoring programs and national and international studies such as PISA provide trailing indicators of student’s outcomes. The primary aim of trailing indicators is to assess changes in system performance across schools and districts, and over time.

Educators need leading indicators to inform timely school policy and instructional decisions. Leading indicator data allow you to not only focus on what is happening in your school now, but also to predict what will happen as a result (e.g. high bullying rates now will lead to high anxiety rates later). Leading indicator data are collected before learning occurs or as it is occurring. They typically employ short tests and administer surveys more frequently. The data can be used to:

provide a framework for intervention (i.e., a response-to-intervention (RTI) program);
guide school policy and practice in order to improve school climate;
stimulate discussion about school reform;
identify issues relevant to particular students or groups of students;
identify students who need extra support;
involve parents in meaningful ways; and
increase student and teacher engagement.

We enable schools to continuously monitor several leading indicators that can help guide school policy and practice and assess the effects of school reforms.

Validity refers to whether a set of survey questions is measuring what it is intended to measure.

Steps for establishing the validity of an OurSCHOOL measure

Determine a theoretical definition of the construct by reviewing literature, and by seeking expert opinion.
Identify the questions that reflect the construct through thorough discussions with experts, as well as teachers and students in focus groups, and a review of other instruments.

Data are analyzed using two psychometric techniques: factor analysis and item response theory (IRT). The analysis provides an indication of how well the questions hold together to provide a measure of a single unified construct, and whether each question contributes to the reliable measurement of the construct.

Click here to delve deeper into validity in the OurSCHOOL context.

High accuracy of reported measures is best achieved by having 100% of students participate. However, a reasonable estimate can be achieved by having as few as 30 students participate.

Click here to get more details on the data accuracy generated by the OurSCHOOL survey.

Reliability refers to the consistency of the measurement process and is one of the key criteria for a sound measure of a schooling outcome or school process.

High reliability is dependent on having multiple well-formulated questions contributing to a measure.
Reliability is usually assessed with a coefficient that ranges from 0 to 1.0, with 1.0 being perfectly reliable. Most of the measures used in our surveys have reliability coefficients ranging from 0.80 to 0.95.

Learn more about the level of reliability in the OurSCHOOL Survey.

Join the OurSCHOOL Engagement Team in one of our personalized interactive sessions. Each session was designed to reach all levels of data literacy, meaning you don’t need to be a data expert or statistician to attend. Let our team of experts guide you in finding your data story!

PRE-SURVEY

STRATEGY SESSION

DATA

DEEP DIVE

ASSESSING

YOUR IMPACT

BUILDING A

THRIVING SCHOOL

Explore topics such as:

In this year-end online debrief, the

Engagement Team will provide

you with tips and resources you

can use to:

A school is thriving when teachers,

students, parents / guardians, and

the community are working together

toward a common vision of success.

Take a deep dive into your data!

• Data-driven planning,

• goal setting, and

• using survey measures and

custom questions to capture

actionable evidence.

Our experts will:

• Review your data

• present notable highlights, and

• identify areas for further

investigation.

• Review your progress,

• assess the impact of your

implemented strategies, and

• discuss ways to help your team

adjust strategies and set

priorities for the coming year.

In this session you will:

• Explore strategies for how to

engage your whole school

community and build a culture of

consultation and collabora-tion

between all stakeholders, and

• learn how to triangulate the

perspectives of students,

teachers and parents to gain

deep insight into your school

communities' alignment.

• This session is available for

schools that use the OurSCHOOL

suite of surveys (Student, Parent,

and Teacher).

Leave the session with:

Participants will:

• Strategies and resources to

generate excitement for the

OurSCHOOL survey, and

• ideas you can use for a

successful survey

implementation.

• Learn to interpret and share

their OurSCHOOL findings,

• identify areas to celebrate,

• pinpoint opportunities for

improvement, and develop

achievable action plans

for further success.

Leave the session with:

• Knowledge and resources to

assess the impact of your

current strategies, and

• tips and resources for creating

a plan before your next survey

implementation.

* Sessions are available in both online and in-person format with up to 50 attendees. Pricing is based on session details and availability may be limited.

The OurSCHOOL Teacher Survey requires schools to set challenging and visible learning goals for students. Teachers can do this by:

transferring knowledge through planned learning opportunities;
providing quality feedback that guides students’ efforts and attention; and
supporting students to overcome obstacles to achieving their learning goals.

Confident Learners Program Overview

The Confident Learners framework was developed over the course of a three year research partnership with 32 First Nations communities to develop a scalable program, informed by best evidence, and designed to meet the literacy needs of each child, school and community. To outline some of the key challenges we addressed through our research, and evidence underlying the program’s approach and design, we have developed the Confident Learners Program Overview. To receive the Overview directly in your inbox please provide us with your contact details below.

NEWS

ARCHIVE

WEBINAR: Trauma-Sensitive Leadership: Creating Caring, Safe and Supportive Learning Environments

WEBINAR: Taking Care of Staff so… They Can Take Care of Students

WEBINAR: Enhance school climate and student success: A guide to utilizing perception data

IAE Member Research Series: Maria de Ibarrola discusses the research paper “Evaluation of teachers of basic education: Political tensions and radical oppositions”

WEBINAR: Discover research-based interventions proven to be effective in impacting staff well-being

WEBINAR: Supporting Staff Well-being – Fostering Belonging in Schools

Request a demo

Make literacy your bottom line

Efficient reading instruction is at the center of community transformation, and children’s future opportunities. Learn about the financial benefits of reading success, or access resources to understand the literacy needs of your community now.

Click here for a more detailed breakdown of how are measures are constructed.

Click here to delve deeper into validity in the OurSCHOOL context.

Click here to get more details on the data accuracy generated by the OurSCHOOL survey.

Learn more about the level of reliability in the OurSCHOOL Survey.

Confident Learners Program Overview

NEWS

ARCHIVE

Related News

Efficient reading instruction is at the center of community transformation, and children’s future opportunities. Learn about the financial benefits of reading success, or access resources to understand the literacy needs of your community now.