Author Archives: Learning Bar Staff

Validity refers to whether a set of survey questions is measuring what it is intended to measure.

Steps for establishing the validity of an OurSCHOOL measure

1. Determine a theoretical definition of the construct by reviewing literature, and by seeking expert opinion. 
2. Identify the questions that reflect the construct through thorough discussions with experts, as well as teachers and students in focus groups, and a review of other instruments.

Data are analyzed using two psychometric techniques: factor analysis and item response theory (IRT). The analysis provides an indication of how well the questions hold together to provide a measure of a single unified construct, and whether each question contributes to the reliable measurement of the construct.

High accuracy of reported measures is best achieved by having 100% of students participate.  However, a reasonable estimate can be achieved by having as few as 30 students participate.

Reliability refers to the consistency of the measurement process and is one of the key criteria for a sound measure of a schooling outcome or school process. 

• High reliability is dependent on having multiple well-formulated questions contributing to a measure.
• Reliability is usually assessed with a coefficient that ranges from 0 to 1.0, with 1.0 being perfectly reliable. Most of the measures used in our surveys have reliability coefficients ranging from 0.80 to 0.95.

The OurSCHOOL survey only uses multi-question measures, as opposed to single questions as found on most other school surveys. When developing our measures, we take steps to ensure they are the most representative of the construct under investigation. We begin with the broad construct (or domain) – for example, the concept of Anxiety – then we conduct a thorough examination of the existing literature and consult with subject matter experts. 

A bank of questions is then developed that are intended to reflect the construct and are rigorously tested. We know that measures are more reliable when they include a number of well-formulated questions as this allows us to more accurately differentiate between students.

School monitoring programs and national and international studies such as PISA provide trailing indicators of student’s outcomes. The primary aim of trailing indicators is to assess changes in system performance across schools and districts, and over time. 

Educators need leading indicators to inform timely school policy and instructional decisions.  Leading indicator data allow you to not only focus on what is happening in your school now, but also to predict what will happen as a result (e.g. high bullying rates now will lead to high anxiety rates later). Leading indicator data are collected before learning occurs or as it is occurring. They typically employ short tests and administer surveys more frequently. The data can be used to:

• provide a framework for intervention (i.e., a response-to-intervention (RTI) program);
• guide school policy and practice in order to improve school climate;
• stimulate discussion about school reform;
• identify issues relevant to particular students or groups of students;
• identify students who need extra support;
• involve parents in meaningful ways; and
• increase student and teacher engagement.

We enable schools to continuously monitor several leading indicators that can help guide school policy and practice and assess the effects of school reforms.

The RTI classification includes three tiers, which are based on a prediction model derived from longitudinal data that estimated the likelihood of a child being a successful reader at age 8 or 9, based on his or her EYE-TA domain scores. Children with a greater than 80% chance of becoming a successful reader are classified as having Tier 1 learning needs; those with a 50% to 80% chance of becoming a successful reader are classified as having Tier 2 learning needs; and those with less than a 50% chance are considered to have Tier 3 learning needs. The prediction model takes into account the age of the child at the time of the assessment and his or her skill level in each of the five domains. The results derived from the model weights some skills more heavily than others. Skills in the Cognitive and Language and Communication domains figure most prominently; children’s Awareness of Self and Environment, Social Skills and Approaches to Learning, and Fine Motor skills also contribute to the classification.

An RTI classification is not permanent or necessarily long term, nor is it a label assigned to a child. As children are developing their literacy skills, their progress should be monitored on a regular basis, using a variety of formal and informal tests. In practice, teachers use the domain-specific results to plan their instruction. Schools with more than one kindergarten or Grade 1 classroom can use the RTI score to ensure the prevalence of vulnerable children is evenly distributed among classrooms.

Also, children with Tier 2 or Tier 3 learning needs do not necessarily have a specific learning or intellectual disability. The classification simply indicates that a child may need extra instructional time and resources during the primary school years.

Join the OurSCHOOL Engagement Team in one of our personalized interactive sessions. Each session was designed to reach all levels of data literacy, meaning you don’t need to be a data expert or statistician to attend. Let our team of experts guide you in finding your data story!

 * Sessions are available in both online and in-person format with up to 50 attendees. Pricing is based on session details and availability may be limited.

The OurSCHOOL Teacher Survey requires schools to set challenging and visible learning goals for students. Teachers can do this by:

1. transferring knowledge through planned learning opportunities;
2. providing quality feedback that guides students’ efforts and attention; and
3. supporting students to overcome obstacles to achieving their learning goals.