For formal evaluation, review the implementation plan created in Essential Action 6.
Systematic use of data-driven discussions supports the institution and its leadership team in understanding the nuances of progress with the systemic change to mathematics pathways. The team is more likely to monitor and adjust the implementation process if the team plans for it. The implementation plan includes clear targets, data collection, and scheduled check-ins.
offers recommendations for formulating questions about mathematics pathways, collecting data to answer those questions, presenting the data to different audiences, and facilitating discussion about the data.
Make decisions about needed adjustments and implement them.
With a clear picture about where a given institution is in its implementation of mathematics pathways, the leadership team can make decisions about adjustments if the implementation is not going as planned. The following questions may assist the leadership team to reach conclusions about possible adjustments.
- What is working well and what can we learn from these successes? Do we need to modify our implementation plan?
- Do we need to provide more professional learning opportunities for faculty and staff?
- Do we need to provide more or different resources?
- Do we need to do something different to help faculty and staff manage the change?
Answers to these questions may signal to the leadership team that adjustments are necessary to enhance support for mathematics pathways implementation. For example, there may have been insufficient sections to accommodate the number of students who registered for the statistics course. The leadership team can take the actions needed to add the necessary sections; alternatively, the team might decide to share data and findings to faculty and staff to solicit their input on possible adjustments. Data may suggest that some faculty are relying on past instructional practices rather than using those recommended. In this case, the leadership team could take the information directly to faculty and staff, and ask for their guidance on how to better support them with the instructional changes.
As the team checks in periodically to assess implementation progress, it might be helpful to use a planning tool to align actions. The leadership team is charged with communicating expectations, monitoring progress, making suggestions about adjustments, and ensuring that implementation is supporting collaborative learning across the different stakeholder groups.
Keep in mind that meetings to review data and to make decisions about adjustments do not have to take a lot of time. A structured agenda with clear time allotments can help the team stay focused on the task at hand.
Monitor the effect of implementation on implementers.
Remember that major changes affect all involved, and implementation can be successful if the effect on faculty and staff is monitored. One strategy to consider is using a transition monitoring team (Bridges, 2009). Keeping abreast of the impact of a change is difficult to do, and this task can be especially challenging for those in leadership positions. A transition monitoring team comprised of members from the campus community who have influence with their colleagues can help improve communication to and from the leadership team.
Creating a transition monitoring team lets the community know that the leadership team is interested in receiving feedback. The transition monitoring team can also review information that the leadership team intends to share to identify any potential problems and can serve as another avenue to reduce miscommunication.
Keep in mind that a certain amount of anxiety and frustration is inevitable in a change process. Over-responding and not allowing people to work through their reactions can be counterproductive. Understanding the Stages of Concern referenced earlier can help determine if faculty and staff are progressing through normal stages or are getting stuck in unproductive patterns.
For cumulative evaluation, review goals created in Essential Action 5.
Determine if the goals are still correct. As implementation progresses and the context evolves, the leadership team may need to revise the goals. Reviewing the goals is essential to continuous improvement.
Collect and review data on outcomes.
A critical review of data will help you determine if you are making adequate progress toward your goals.
Create or refine the plan for the next implementation cycle.
Ideally, the team evaluates the first implementation cycle before planning for the next, but this may not be realistic given the lag in collecting outcome data and the demands of the academic calendar. It is very possible that the team will have to draft a plan for the upcoming cycle based on formative evaluation and then revise as necessary after reviewing outcome data.
Revisit the communication and engagement plan from Essential Action 3 to build broad understanding of the cumulative evaluation and future plans.
Transparency is critical to building a sense of ownership among stakeholders. It is not enough to simply disseminate the evaluation results. Faculty and staff should understand the process and how the evaluation results are used to refine goals and improve implementation. Find ways in which people can give meaningful input.
Systemic change can be challenging, and it is easy to become too focused on the work that we forget to stop and celebrate successes. Public celebrations highlight the commitment to mathematics pathways and the importance that the move toward this normative practice plays in the success of students. The leadership team can brainstorm on what should be celebrated and ways to celebrate throughout the implementation process.
Celebrating success and honoring those who have contributed to the success are an important part of sustaining individual commitment. They also cycle back to Essential Action 1 by communicating the institutional commitment to full implementation of mathematics pathways. Refer to steps in Essential Action 1 that connect the work to the overall mission and strategic plan of the institution.
What is the culture of using data at your institution? Who typically has access to data and how are data used?
How will the data available to institutional researchers and faculty be shared with the leadership team and with other stakeholders?
Are there existing processes or structures for discussing data, possibly through other initiatives such as Achieving the Dream? Is there a need to create new processes? Will faculty or staff need training on the use of data?