PME 802: Program Inquiry and Evaluation

Part 1: Select and describe program context.

Focus:

The social program I would like to evaluate is the use of Freshgrade as an online digital learning portfolio to promote learner agency, and to facilitate quality, ongoing assessment of curricular objectives in our elementary school. This is not an evaluation of the computer program itself, but rather an evaluation of its use carried out by teachers, students, and parents in our learning community.

Outcomes/Goals:

Our school mission statement asserts that our goal is to foster "a collaborative community culture rich in inquiry and deep learning" (Privett, 2018). The intended outcome of Freshgrade in our school is to facilitate deeper learning for students. Success criteria include having students take more ownership of their learning, having parental involvement in student learning, finding evidence of ongoing assessment and immediate feedback for students and parents, and documentation of reliable (and diverse) evidence of learning.

This program implementation is being evaluated in order to justify the annual subscription, the time and effort being invested, and to ensure that it is serving its purpose in our school community. Ultimately, do the benefits of its use outweigh the costs of continuing the Freshgrade program at our school?

Staff:

Freshgrade is a whole-school initiative with all faculty participating in the program. There are currently 23 teachers and two administrative staff (principal and vice-principal) who are involved. Students and parents are also stakeholders who play a key role in learning portfolios.

Demographics:

Our school is a K-5 public elementary school located in a neighbourhood populated by families of middle/high socioeconomic status. The students at this school often have to wait on a list to get in, even if they live in the school catchment, as there are many children in the neighbourhood, and the school has a very strong reputation. There are 425 students at our school.

Facility/Resources:

Our school is considered an innovative learning environment with sliding glass partitions to create open learning spaces. Two classes share a class set of iPads, and therefore tools to access Freshgrade are readily available. We have access to the internet as well. Funding is centralized from the school district board office and our Parent Advisory Council subsidizes funds for additional technology.

Potential Unintended Results:

Due to the necessity of technology to carry out this program, there may be times when the internet is not accessible temporarily or items are not uploading properly to Freshgrade. This can be especially frustrating when teachers have set aside time in class or have booked the iPads to have students reflect on their learning through posts. Furthermore, parents may have trouble accessing the app or devices in order to participate in student learning. Parents have also struggled with commenting on student learning. They may be insecure about giving feedback when they are unsure about what success looks like.

Part 2: Identify purpose for evaluation and specify evaluation questions.

We have been involved with Freshgrade, a local Okanagan company, for four years. In that time, our student portfolios have evolved as we have grown and learned through experience. At this point, a more formal utilization-focused evaluation for internal management would be useful so that we can improve on our initiative with solid data and evidence to support our findings (Patton, 2014). Process use would be very valuable as well as having primary intended users involved in the evaluation itself to ensure commitment to improvement.

As educators, deep learning for students is our ultimate goal. Is Freshgrade as a platform for documenting a student's learning journey achieving the goals of deeper learning through learner agency and the inclusion of visible, quality, ongoing assessment? Below are some questions to consider when conducting this evaluation.

Key Evaluative Questions:

How well has Freshgrade worked?

Is it frequently accessed by students, teachers, and parents?
Is there sufficient evidence of student learning (artifacts, videos, observations recorded, etc.)?
Have teachers been "on board" in dedicating extra time and effort to implementing Freshgrade?
Is there consistency across the learning community in the way portfolios are organized?
Is there sufficient access to technology?
Is the technology reliable?
Do parents support and enjoy the program?

2. Has Freshgrade contributed to depth of student learning?

Are students demonstrating reflection?
Are students able to articulate learning growth?
Are students able to independently select and analyze evidence of learning?
Do students demonstrate a better understanding of teacher standards and curriculum objectives? Are students aware of criteria for success?
Are comments from parents and teachers helpful for student learning?
Is there ongoing, specific teacher feedback?

3. What are the particular features of Freshgrade's e-portfolios that make a difference in student learning?

How are portfolios organized?
Are posts showing student learning growth over time?
Are curricular standards accessible from the program?

4. Does Freshgrade address student learning needs?

Are students engaged?
Are all students able to access and utilize the technology?