Portfolio

Artifact 16: Assessment Design Checklist

BACK TO PORTFOLIO HOME

Primary Alignments:

Standard 2: This assessment design checklist can be used to check if assessments and other teaching materials align with best practices for teaching understanding.

Standard 3: The theoretical foundation of the checklist are cited linked throughout the document.

Goal 1: This document is a reflection of my practice in light of readings done in the program.

Secondary Alignments:

Standard 1: Included in the assessment checklist are provisions for ensure equal access to all students, as well as a section examining ways in which the assessment being evaluated draws on the student's background.

Standard 4: This checklist went through a number of revisions. I also used it to evaluate different assessments that I created.

Standard 6: In the revision process, the assessment was shared with fellow students, both for their evaluation and also through me applying the checklist to other student's work.

Introduction and First Draft

This spring, I am fortunate enough to have the experience of living in Spain for a month. One of my goals for this time period is to become at least somewhat functional in the Spanish language. This has also proven the perfect opportunity to think about formative assessment. I came to Spain armed with (according to Memrise) about 1,000 words/phrases, so it should be a piece of cake. But going out into the world and trying to figure out how to ask where the vinegar is or how to get your parking validated is far different from regurgitating well-worn phrases. Every moment that I am around town is a formative assessment. Not only are the well-rehearsed phrases put to the test, but my ability to transfer my known words and understanding of grammatical structures to new situations is constantly being challenged. And after each interaction, there is a self-evaluation that goes on: “What did I get right? What did I miss?,” and most importantly, “What do I need to learn for next time?” In this way, my forays into the tiendas and supermercados of Calpe act as a formative assessment, testing my background knowledge and opening new pathways for further learning.

With formative assessment in mind, this week I began working on an Assessment Design Checklist. This checklist would be used to evaluate a formative assessment. There were three areas where I focused my attention:

  • Focusing on the big ideas: does the assessment address the most important and salient ideas in the topic and field for which it is written?
  • Providing actionable data: does the assessment provide the instructor with information regarding the student’s background knowledge, experience, and expectations that can help craft a better learning experience?
  • Priming and goal-setting: Does the assessment give some indication to students as to the big ideas and key skills that the unit will be covering and does it provide opportunities for the student to reflect on their personal goals for learning?

I am excited to continue reflecting on how formative assessment can be a vital part of the learning process and refining my tools for creating and evaluating assessments.

Here is a link to the Formative Assessment Checklist

Second Post and Edit of Checklist

Here is the latest version of my assessment design checklist.

A few things to note:

  • I have added more personal stories from my teaching to see how these things apply in the “real world”
  • I have added more information on feedback
  • I have added the fifth question on accessibility

The big things that I have learned in the process of doing this project:

  • Assessment, particularly formative assessment is part of the learning process
  • Assessment can be used to help students grow, not just to measure them
  • A responsive, inclusive assessment doesn’t just happen, it takes a lot of thought and reiteration
  • If we want to see our students as constructing understanding, we need to include them in the dialog as they go through assessments

Final Check List

Assessment Design Checklist 3.0

Questions:Evidence of Understanding:
Does this assessment explicitly address the most relevant skills and big ideas in the topic of the assessment and the discipline being taught in the course?An expert in the field can easily identify big ideas in the subject that the assessment is addressing. The knowledge and skills being assessed are widely agreed upon as essential to the mastery of the subject matter.Each element of the assessment has a rationale for being included that ties to the relevant skills and big ideas being assessed.
Does this assessment provide the teacher with actionable information about the student’s background and expectations for the topic? The assessment provides clear information for the teacher regarding the student’s background knowledge, misconceptions, and attitudes on the subject at hand.The assessment provides the teacher with clear information regarding which foundational skills and concepts will need to be taught and practiced.The assessment provides the teacher with information regarding the students attitudes and expectations for the class.
Does this assessment prime the student for learning within the discipline and promote student autonomy and accountability? The assessment asks questions which suggest to students the key concepts and skills needed to succeed in the subject.The assessment provides opportunities for the student to set their own goals for learning and suggest possible strategies for their achievement.
Does this assessment provide the student with clear feedback that is informative and actionable?Students are provided with rich feedback that gives them an indication of strengths, weaknesses, and paths to improvement. The feedback should tell the student:Where they are going?How are you going?Where to next?Students are given feedback in a timely mannerFeedback is presented in a way clearly identifies the criteria being used for evaluation and the major points of emphasis for the assessment. Feedback given relates directly to the learning goal that is set forth in the assessment.
Does this assessment address the primary skills being assessed and take into account the various learning styles, abilities, and disabilities of the students being assessed?Assessment is presented in a way that is accessible for all students including:English language learnersStudents with diagnosed physical disabilitiesStudents with diagnosed learning disabilitiesAssessment affords students the opportunity to engage in the assessment in a variety of ways including, but not limited to (UDL, 2011): Providing Multiple Means of RepresentationProviding Multiple Means of Action and ExpressionProviding Multiple Means of Engagement (the “why” of learning).

Checklist Annotations

Question 1: Does this assessment explicitly address the most relevant skills and big ideas in the topic of the assessment and the discipline being taught in the course?

In any endeavor, it is important to put the first things first. In the case of curriculum design this means clearly identifying the main ideas and key concepts in any field and staying focused on them. In Understanding by Design the authors warn of activities that have, “no guiding intellectual purpose or clear priorities frame(ing) the learning experience” (Wiggins & McTighe, 2005, p. 16). It is important for any assessment to be focused on these main ideas.

In this stage we are trying to identify students understanding of these ideas and in so doing we are also highlighting them to students. Curriculum has the tendency to expand, so it is essential to always keep the main ideas at the fore of any curriculum or assessment design. Today teachers have many resources when it comes to assessment and the digitization of these resources means that there is no physical limit on the length and breadth of assessments. In my experience, a student’s school year can easily end up being an endless carousel of standardized tests. With this in mind it is important to stay focused on the key understandings in any discipline. In the words of Dr. Lorrie Shepard (2001) “the content of assessments should match challenging subject matter standards and serve to instantiate what it means to know and learn in each of the disciplines” (pp. 5-6).

It is important to note that connections between ideas can sometimes be opaque so a strong background in the subject matter is necessary to identify the most salient points that need to be addressed. This need for expertise also holds true when we examine where this specific assessment fits into the broader curriculum.

Evidence of Understanding

When looking at the assessment, an expert in the field, should be able to identify the skeleton of concepts that underlies the structure of the assessment, meaning they should be able to identify the key understandings and skills being assessed. Additionally, these key skills and understandings should be tied directly to the big ideas in the field.

For example, in a unit on finding the area and perimeter of quadrilaterals, the big ideas might be that the area and perimeter of quadrilaterals may be calculated by using the lengths of the sides. Multiplying the length and width gives the area of the quadrilateral, while adding the lengths of all of the sites together will result in the perimeter. These specific relationships would tie into the bigger mathematical concept that the measurement of shapes features predictable relationships, and that these relationships can be used to calculate the unknown variables. This standard should apply to all elements of the assessment with every assessment question or task having a identifiable purpose for teaching and/or assessing a skill or understanding which ties to a widely accepted big idea in the field.

Question 2: Does this assessment provide the teacher with actionable information about the student’s background and expectations for the topic?  

The teacher is using this assessment to evaluate what the student already knows and what they need to learn about the subject matter. Additionally, this assessment gives the teacher information on the student’s experience and attitudes about the material. The teacher should be able to use the information that they get from this assessment to better craft the lesson to the student’s needs. As Shepard (2005) notes: “From a sociocultural perspective, formative assessment — like scaffolding — is a collaborative process and involves negotiation of meaning between teacher and learner about expectations and how best to improve performance” (pp. 66-67).  The information in this assessment should help the teacher adjust their instruction to account for the student’s background knowledge and also give the teacher some ideas on how to draw on the student’s motivations and interest. All too often we think of assessment as a measuring tool, when in truth assessment can be a vital tool in instruction. But to do this “assessment practices need to be so well grounded in the instructional process that the information they reveal will identify whether and how instruction should be adapted to advance students’ understandings” (Trumbull & Lash, 2013, p.2).

An example of this would be in the time that I was working in middle school math intervention, I started the daily practice of writing four problems for each student on a whiteboard, each one addressing a key mathematical operation. I could track their progress over a period of weeks and continually respond to their work. Say a student did not regroup correctly when subtracting. This concept could be quickly reviewed and the next day the student could be given subtraction with regrouping problems. Once they had mastered this, they would have to regroup across 0. (solve 900-188 =, for example). Once they could do this problem I would include problems without regrouping, as I had found once students started regrouping they would apply it to all subtraction situations. While this type of ongoing formative assessment can be difficult to manage in large groups, having information on the students current understanding and misconceptions, as well as, knowledge of their attitudes and expectations that can help teachers create better differentiated and more engaging instruction.

If we are to think of learners as constructing knowledge it is important to remember that “new knowledge must be constructed from existing knowledge is that teachers need to pay attention to the incomplete understandings, the false beliefs, and the naive renditions of concepts that learners bring with them to a given subject” ((Bransford et al., 2000, p. 10).

Evidence of Understanding

In reviewing the assessment, the teacher can easily access information regarding the student’s background knowledge, misconceptions, and attitudes to create a student dossier for that particular topic and subject. This information could also be used for grouping purposes, whether that be for scaffolding instruction or creating groups with students that have divergent strengths.

Upon reviewing the assessment the teacher has a clear idea which foundational concepts need to be reviewed or retaught, as well as some insights into what examples or frameworks are likely to resonate with students .The assessment provides the teacher with information regarding the students attitudes and expectations for the class.

Question 3: Does this assessment prime the student for learning within the discipline and promote student autonomy and accountability?

A formative assessment does not just give information to the teacher about the student, it represents an opportunity for the teacher to prime the student for learning and to help the student set personal goals and accountability standards for the lesson. “Becoming competent in a field of study means learning and internalizing the standards by which others will judge our performance” (Shepard, 2005, p. 69), so the assessment should give some indication to the student as to where the lesson is heading and get them excited about the journey. Additionally, the assessment should have the student reflect on the motivations and habits and set some short-term goals with plans for how to achieve them. In this way the assessment serves “both metacognitive and motivational purposes” (Shepard, 2005, p. 69).

As teachers we have all had the experience of trying to inspire students who have been worn-out by the education system. It could be poor past performance, behavioral issues, poor teaching, whatever the reason at a certain point all students feel unmotivated and uninspired. This presents a real challenge when we ask them to do things that are hard, uncomfortable, and feel foreign to them. This is why it is so important to get students invested in beginning to construct their understanding and develop a sense of autonomy and accountability in their learning. One of the benefits of having worked across grades K through 8, is that I have seen how the love of learning is put under stress throughout a student’s school career due to incentives, ideas about autonomy, and sense of connection with material. I have seen many early primary school teachers rely on the student’s desire to please the teacher and sense of wonder to force students to learn what the teacher deems important. As the years pass, students are less and less inclined to learn simply to please the teacher. By the time they get to middle school, they openly revolt against the teacher’s imposition of outside rules in service of students learning the teacher’s ideas. This is why, whenever possible it is valuable to promote student autonomy and construction of understanding.

Evidence of Understanding

In order to achieve these outcomes, the assessment should ask questions that suggest the key skills and big ideas in a particular topic. Asking these questions can be thought of as the first step in setting up students for the course, unit, etc. by giving the students a model or framework from which to begin constructing their understanding of the material. The assessment should also provide some mechanism for students to set their own learning goals and space for reflection on how to achieve them. Upon completion of the assessment students should be able to articulate some understanding of the coming coursework, as well as their personal learning goals.

Question 4: Does this assessment provide the student with clear feedback that is informative and actionable?

Feedback is an essential element in any effective assessment. Assessment can be interwoven into the instructional processes and serve to better the students understanding. But to do this, “the feedback needs to provide information specifically relating to the task or process of learning that fills a gap between what is understood and what is aimed to be understood (Hattie & Timperley 2007, p. 82)”. It is important that this feedback is provided to students in a timely manner and that the feedback helps students understand their understanding, what they need to work on, and ways that they can improve.  

Evidence of Understanding

Feedback should give students a clear indication as to how they are doing in terms of what they know and do well, and what they need further work on. The feedback that students get should go beyond a score or simply pass or fail, and suggest pathways for improvement and future success. As noted by Shepard (2001) traditional models of are “of little value to us in reconceptualizing assessment from a constructivist perspective, because the great majority of existing studies are based on behaviorist assumptions (p. 110).” In order to promote student construction of understanding we need to think of feedback as part of a dialog that will help the student construct meaning, rather than a reporting of student competence.

This can be done using the three-part model put forward by Hattie & Timperley (2007):

  • Where are you going? What are the major goals that you are moving towards?
  • How are you going? What indication do we have that you are moving towards this goal?
  • Where to next? Aside from simply “keep going” where is the student moving and how do they keep moving forward.

In order for the assessment data to be of use, students need to be given the information in time for them to make improvements.

Question 5: Does this assessment address the primary skills being assessed and take into account the various learning styles, abilities, and disabilities of the students being assessed?

The assessment should make allowance for students to demonstrate their understanding of key concepts without constraints. This means that the assessment should provide multiple ways for students to demonstrate understanding. Technology can help make education more accessible to students, but it is important that curriculum developers, education administrators, and teachers work together to “ensure that e-learning technologies continue to benefit rather than hamper students with all types of disabilities and that accessibility gains are maintained and built upon” (Fitchten et al., 2009, 254).

In my experience, when teachers are dismissive of accommodation it is because of a lack of resources or an assumption that certain accommodations may negate the validity of the assessment. In the first case, teachers are not alone in the creation and implementation of accessible curriculum. Most schools have teachers dedicated to meeting the needs of students whether they be English language learner coordinators, occupational therapists, or special education teachers. What is demanded is consistent dialog between the various teachers, the student, and the administration. I remember becoming very frustrated working with some Rwandan students that we had at a Michigan Title I school, when the administration insisted they sit and mimic written work, literally, copying down letters that made no sense to the student, when resources were available (chromebooks, for example) that could be utilized to help them begin catching up on the language learning skills. On a positive note, I was able to make a breakthrough with them when I visited the ELL coordinator and learned that in the students native Kinyarwanda, articles do not exist. So while a sight word like “an” or “the” has an inherent meaning to a native English speaker, they are hopelessly esoteric to my Rwandan students. As a result, a Rwandan student may be able to do very advanced writing, speaking, and comprehension but still get stuck on the occasional article. This knowledge is important and only came about through open lines of communication between multiple teachers and service providers.

As to the second objection, if the assessment has clear objectives (as defined in question 1) it should be relatively easy to discern if an accommodation negates the validity of the assessment. For example, if the purpose of an assessment is to test students ability to decode words, having a word read to the student by a computer, would undermine the value of the assessment.  In contrast, if the purpose of the assessment is to test students ability to find the main idea and supporting details in a passage of writing, having the text read to the student does not inhibit the assessment.

Accounting for all learners is essential if we are to think of education as a tool for the promotion and reflection of our democratic ideals. Or as stated by the authors of Universal Design for Learning (2011) “When curricula are designed to meet the needs of an imaginary ‘average’, they do not address the reality learner variability. They fail to provide all individuals with fair and equal opportunities to learn by excluding learners with different abilities, backgrounds, and motivations who do not meet the illusive criteria for ‘average’” (p. 4). In this diverse, rapidly adapting world, we as educators must provide rich and diverse assessments and be ready to adapt to the needs of our learners.

For more on universal design vist http://udlguidelines.cast.org/.

Evidence of Understanding

The assessment is presented in a way that is accessible for all students including: English language learners, students with diagnosed physical disabilities, and students with diagnosed learning disabilities. Things to consider when evaluating this standard are how text is presented and used, whether or not audio is used, required, or available, and what visual elements are available and if there are ways for visual elements to serve as substitutes for audio and text elements. It is also important to ensure that where text is used, it is essential to the assessment and, where appropriate, alternatives are provided.

Ways to create assessment that afford students the opportunity to engage in a variety of ways include, but are not limited to (UDL, N.D.): providing Multiple Means of representation, action and expression, means of engagement. This means getting outside of just pencil and paper tests and multiple choice questions, and giving students a variety of ways to express understanding. A useful resource for thinking of ways to develop more dynamic, rich assessments can be found in the Next Generation Science Standards 3-Dimensional Assessment. A link (stemteachingtools.com; n.d.)is attached to an article including a wide variety of assessment task formats that are applicable to practically any discipline.

References

Bransford, J., Brown, A.L. & Cocking, R. R. (Eds.), (2000),  How people learn: Brain, mind, experience and school (pp. 3-27). Washington, D.C.: National Academy Press. Retrieved from http://www.nap.edu/openbook.php?isbn=0309070368.

Fichten, C., Ferraro, V., Asuncion, J., Chwojka, C., Barile, M., Nguyen,M., Wolforth, J. (2009). Disabilities and e-Learning Problems and Solutions: An Exploratory Study. Journal of Educational Technology & Society, 12(4), 241-256. Retrieved from http://www.jstor.org.proxy1.cl.msu.edu/stable/jeductechsoci.12.4.241

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. [PDF]

National Center for Universal Design for Learning (n.d.); UDL: The UDL Guidelines; retrieved from http://udlguidelines.cast.org/

Next Generation Science Standards (n.d.) Instruction and Assessment Supports; retrieved from

Shepard, L. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14.

Shepard, L. (2005). Linking formative assessment to scaffolding. Educational Leadership, 63(3), 66-70.

Stemteachingtools.org; (n.d.) STEM Teaching Tool #30: Task Formats for 3D Assessment retrived from http://stemteachingtools.org/assets/landscapes/STEM-Teaching-Tool-30-Task-Formats-for-3D-Assessment-Design-v2.pdf

Trumbull, E. & Lash, A. (2013). Understanding formative assessment: Insights from learning theory and measurement theory. San Francisco: WestEd.

Universal Design for Learning (UDL); 2011; Guidelines: Full-Text Representation retrieved from http://www.udlcenter.org/sites/udlcenter.org/files/UDL_Guidelines_Version_2.0_(Final)_3.doc

Wiggins, G. P. & McTighe, J. (2005). Understanding by design. Alexandria, VA: Association for Supervision and Curriculum Development.

Leave a Reply

Your email address will not be published. Required fields are marked *