SBAC Math Test Not Immune To Problems Either
The problems in SBAC test questions are not limited to the language arts portion of the test. A dual degreed (Ed Phd and JD) math teacher from Missouri began looking at Grade 6 SBAC math items last year, especially those considered at DOK 3 and found problems there as well.
Teachers were sent a link to the practice test and encouraged to send questions/comments to SBAC. She figured the practice items would represent the best SBAC had to offer. What she found was many questions written in a confusing manner, but even more disconcerting, the fact that the answers for some of the questions were wrong. Math is a subject where there may be many ways to get to the answer, but there is only one correct answer. The sign of a well written question is one where it does not take a committee and a consensus process to find the answer. Unfortunately, it looks like SBAC writers took the committee approach.
Here is one example of an SBAC question that is not clear on what it is asking the students to calculate.
A girl was eating apple slices with peanut butter and raisins. She was to put 1/16 cup peanut butter on each apple slice along with 8 raisins. She has a total of 2/5 cup peanut butter and 80 raisins and is supposed to dress up apple slices in this way and eat them “until the peanut butter is all gone.” The problem asks, “What fraction of the raisins does she eat?”
Here’s the math: Divide 2/5 cup PB by 1/16 cup to get 32/5 portions of PB to be used on the apple slices (that’s 6 2/5 or 6.4 if you prefer a decimal). You should stop using raisins when you run out of PB. This allows for 6 apple slices with the full allotment of PB and then a partially complete additional slice (if you are going to use up all peanut butter as the problem requires). This is actually a good example of when you would “round up” even though the fractional part is “less than half of a whole.” The teacher wished to remain agnostic on the question of whether this is appropriately denominated “sixth grade level” but, regardless, there is no question at all but that to be “correct” following the parameters of the problem you must round up and use a 7th slice.
However, the answer that Smarter Balanced believes is correct is to round down (leaving almost half of a serving of peanut butter left over and unused.) This leaves you with 6 apple slices with 8 raisins a piece. That is 6 x 8 = 48 raisins out of 80 providing the answer of 3/5 of the raisins being eaten.
However, the SBAC answer is wrong, period. The ONLY correct solution is to use 7 apple slices to use up all the PB, so 7 x 8 = 56 raisins out of 80 = 7/10 . It was suggested that the teacher who noticed this problem write up her observations and send them to SBAC for a response.
Shelbi Cole (math head of Smarter Balanced) replied as follows:
“In response to the other inquiry, non-multiple choice questions go through a process called rubric validation where content teams decide whether there are additional responses that could be deemed correct that may not have been included in the original machine scoring rubric (that item has not gone through that process yet). Since we are still in field testing and data review mode, it is important that the items in the practice test be viewed through the appropriate lens. Some were in the pilot and so we have good information about how they function with students and that the rubrics are inclusive of all student responses that would be deemed acceptable, and others are new in the field test and we are going through all of the data review processes and rubric validations now.”
SBAC could have considered clarifying the problem’s parameters by either removing the requirement to use up all the peanut butter or changing the fractions required to equal 6 slices with no remainders. Instead, their response sounds like they are saying other answers would be OK too — in addition to theirs. Not only is math a precise subject, but we are also teaching them close reading in their ELA classes which should make students pay very close attention to the words actually in the paragraph. Ignoring that you have left over peanut butter follows neither of these rules. This is a level of sloppiness that is hard to understand or excuse. This correction does NOT need any field test review at all. The answer given is simply mathematically incorrect, period.
The teacher found similar problems, less egregious, with a few other questions. The standardized test community does not appear to value accuracy and precision the way teachers do. It is impossible to defend the deliberate confusion that would result to a sixth grader as a result of trying to do this problem correctly and many would have their answer marked wrong. Given that district accreditation and teacher evaluations (high stakes) are riding on how these answers are scored, these kinds of problems represent a major concern for school districts and teachers, as well as parents who will be given the impression that their child’s mastery of the subject is lower than it may actually be.
The average person, and this includes legislators who “just want to make sure our kids are getting the best education possible,” need to understand that the ruler they are using to measure school’s success with that goal is not only imperfect, but sometimes wildly inaccurate. In a math section that may only have 20 questions, getting only two wrong answers on a scaled score may place a child in the “below proficient” category so these kinds of sloppy errors really do matter, but apparently not to the folks in SBAC.