# SBAC Math Test Not Immune To Problems Either

The problems in SBAC test questions are not limited to the language arts portion of the test. A dual degreed (Ed Phd and JD) math teacher from Missouri began looking at Grade 6 SBAC math items last year, especially those considered at DOK 3 and found problems there as well.

Teachers were sent a link to the practice test and encouraged to send questions/comments to SBAC. She figured the practice items would represent the best SBAC had to offer. What she found was many questions written in a confusing manner, but even more disconcerting, the fact that the answers for some of the questions were wrong. Math is a subject where there may be many ways to get to the answer, but there is only one correct answer. The sign of a well written question is one where it does not take a committee and a consensus process to find the answer. Unfortunately, it looks like SBAC writers took the committee approach.

Here is one example of an SBAC question that is not clear on what it is asking the students to calculate.

A girl was eating apple slices with peanut butter and raisins. She was to put ^{1}/_{16 } cup peanut butter on each apple slice along with 8 raisins. She has a total of ^{2}/_{5} cup peanut butter and 80 raisins and is supposed to dress up apple slices in this way and eat them “until the peanut butter is all gone.” The problem asks, “What fraction of the raisins does she eat?”

Here’s the math: Divide ^{2}/_{5 } cup PB by ^{1}/_{16 } cup to get ^{32}/_{5 } portions of PB to be used on the apple slices (that’s 6 ^{2}/_{5 } or 6.4 if you prefer a decimal). You should stop using raisins when you run out of PB. This allows for 6 apple slices with the full allotment of PB and then a partially complete additional slice (if you are going to use up all peanut butter as the problem requires). This is actually a good example of when you would “round up” even though the fractional part is “less than half of a whole.” The teacher wished to remain agnostic on the question of whether this is appropriately denominated “sixth grade level” but, regardless, there is no question at all but that to be “correct” following the parameters of the problem you must round up and use a 7^{th} slice.

However, the answer that Smarter Balanced believes is correct is to round down (leaving almost half of a serving of peanut butter left over and unused.) This leaves you with 6 apple slices with 8 raisins a piece. That is 6 x 8 = 48 raisins out of 80 providing the answer of ^{3}/_{5 } of the raisins being eaten.

However, the SBAC answer is wrong, period. The ONLY correct solution is to use 7 apple slices to use up all the PB, so 7 x 8 = 56 raisins out of 80 = ^{7}/_{10 }. It was suggested that the teacher who noticed this problem write up her observations and send them to SBAC for a response.

Shelbi Cole (math head of Smarter Balanced) replied as follows:

“In response to the other inquiry, non-multiple choice questions go through a process called rubric validation where content teams decide whether there are additional responses that could be deemed correct that may not have been included in the original machine scoring rubric (that item has not gone through that process yet). Since we are still in field testing and data review mode, it is important that the items in the practice test be viewed through the appropriate lens. Some were in the pilot and so we have good information about how they function with students and that the rubrics are inclusive of all student responses that would be deemed acceptable, and others are new in the field test and we are going through all of the data review processes and rubric validations now.”

SBAC could have considered clarifying the problem’s parameters by either removing the requirement to use up all the peanut butter or changing the fractions required to equal 6 slices with no remainders. Instead, their response sounds like they are saying other answers would be OK too — in addition to theirs. Not only is math a precise subject, but we are also teaching them close reading in their ELA classes which should make students pay very close attention to the words actually in the paragraph. Ignoring that you have left over peanut butter follows neither of these rules. This is a level of sloppiness that is hard to understand or excuse. This correction does NOT need any field test review at all. The answer given is simply mathematically incorrect, period.

The teacher found similar problems, less egregious, with a few other questions. The standardized test community does not appear to value accuracy and precision the way teachers do. It is impossible to defend the deliberate confusion that would result to a sixth grader as a result of trying to do this problem correctly and many would have their answer marked wrong. Given that district accreditation and teacher evaluations (high stakes) are riding on how these answers are scored, these kinds of problems represent a major concern for school districts and teachers, as well as parents who will be given the impression that their child’s mastery of the subject is lower than it may actually be.

The average person, and this includes legislators who “just want to make sure our kids are getting the best education possible,” need to understand that the ruler they are using to measure school’s success with that goal is not only imperfect, but sometimes wildly inaccurate. In a math section that may only have 20 questions, getting only two wrong answers on a scaled score may place a child in the “below proficient” category so these kinds of sloppy errors really do matter, but apparently not to the folks in SBAC.

Related!

http://stopcommoncorenc.org/sbac-math-test-is-fatally-flawed-so-why-is-nc-still-a-member/

The confusion with this problem is the phrase “Until the peanut butter is all gone.” Since 1/16 cup of peanut butter is needed for each slice, as soon as 1/16 is not available “it is all gone” for the purpose at hand, thereby obtaining the desired answer of 6.

There was no indication in the problem that one should continue doing partial apple slices.

However, the problem could have avoided the confusion by stating “Until there is not enough peanut butter to fully dress a slice.”

I completely agree. There’s no reason for the author of the post to get all upset since the item is still in the field test stage. It would be another matter if the item had been field tested. Some people are looking for any excuse to get upset about the assessments.

The BEST ‘excuse’ to get upset about the assessments? How about that they are created/controlled/owned by private NGOs and these NGOs are not held publicly accountable in any manner? And how about this one? That they are funded by the Federal Government which is illegal? Anything to get upset about? I’d say so.

Ms. Gassel,

I read with interest your article, entitled “SBAC Math Test Not Immune to Problems Either.” While I certainly understand, and appreciate, your basic premise that the present incarnation of standardized testing contains a number of flaws, there are a few pieces of your argument that I find problematic. First, I should say that I think tests (especially multiple-choice tests) probably have always and will always contain errors, simply because reading and writing are interpretive activities–to which we each bring our own set of background lenses. In short, none of us can fully understand what is inside of another’s mind.

You write, “Math is a subject where there may be many ways to get to the answer, but there is only one correct answer. The sign of a well written question is one where it does not take a committee and a consensus process to find the answer.” I think that this statement is a bit of a distortion of the true enterprise of mathematics. Let me illustrate: What is 1 + 1? Is it 2? Are you sure? Well, if we accept the assumptions embedded within our conventional Hindu-Arabic base-10 number system with the usual arithmetic operations and properties, then yes, 1 + 1 should be 2. If we employ a different set of assumptions, however–such as using base-2 arithmetic–then 1 + 1 = 10! It is also possible to demonstrate that 1 + 1 = 0. Here’s another example from geometry: most people would say that the sum of the measures of the interior angles of a triangle is “180 degrees.” Using a different set of measurements, though, we could say that it is “pi radians,” and using different types of geometry, we can show that it is either LESS than or MORE than 180 degrees. Even more bizarre, you can logically argue–using sophisticated mathematical techniques that are crucial for our understanding of modern physics–that 1 + 2 + 3 + … = 1/12 ! Some of the best, most influential realizations in mathematics have emerged through debates among groups of mathematicians, or committees working toward a consensus process, arriving at wildly different conclusions from different sets of assumptions.

A colleague of mine also pointed out that, historically, different assumptions have been important considerations. The ancient Greeks, said my colleague, avoided the use of 0 and negative numbers, because (for them) quantities needed to represent physical measurements. In the modern era, thankfully, this assumption has not only been challenged, but also has very important consequences: if you have 3 apple slices and 10 friends, you cannot give each of your friends a full slice of an apple; however, you might be able to buy 7 more apple slices on credit (negative numbers!) and then give all of your friends a full slice.

To me, and to professional mathematicians, that is the great beauty of mathematics–that it admits as many possibilities for “truth” as the human mind can possibly imagine. The essence of mathematics involves making a convincing logical argument, once you have established and articulated a baseline set of assumptions. It is this endeavor–justifying your thinking, clearly and explicitly–that many students find challenging; yet, it is also the crucial skill that we need in an educated workforce and citizenry. Imagine if our politicians were able to draw consistent chains of reasoning from their philosophical foundations, rather than contradicting themselves at seemingly every turn!

I cannot say, for sure, whether the designers of this particular question on the SBAC practice test intended to permit multiple solutions. In reviewing the response of Ms. Cole, you admit this possibility. I also find the raisin-and-peanut-butter question problematic, but for a different reason: why would anyone ever want to impose such a strict paradigm on making a snack? The question, itself, is unrealistic–and example of what researcher Sue Gerofsky would say shows students that math word problems are a strange “genre” of writing, indeed!

You also write that “the standardized test community does not appear to value accuracy and precision the way teachers do.” This seems like a bit of an exaggeration, as well, since the entire enterprise of standardized testing is built on a desire to be as accurate and precise as possible. My difficulty with standardized testing, on the whole, is that it is extremely difficult–if not impossible–to be as accurate and precise as the industry claims it can be: standardized tests do not generally permit students to fully explain their thinking or demonstrate their nuances of understanding. Nonetheless, I actually wish that teachers would take a different stance on accuracy and precision, recognizing that not only are there multiple valid ways to arrive at a given mathematical answer, but also that, in many, many cases, there are multiple valid answers to a given question. Some of us would argue, in fact, that “well-written” questions are those that admit such possibilities and offer students opportunities to flex their thinking muscles.

I work with one such group, the American Institutes of Mathematics Math Teachers’ Circle Network (mathteacherscircle.org), whose aim is to share with teachers (and hence their students) the beauty and possibility of rich mathematical problems and authentic mathematical inquiry. The Math Teachers’ Circle Network represents a collaboration among research mathematicians, teachers, educators, and others interested in mathematics education, hoping to reform mathematics education in the U.S. It is our hope that, as a country, we can come to see mathematics as a discipline that is much less black-or-white and much more artistic, creative, exciting, and colorful.

Thank you for calling attention to these important, ongoing issues in mathematics education and for opening the possibility for dialogue!

Sincerely,

Josh Taton

B.A., Mathematics, Yale University

Ph.D. Candidate, Mathematics Education, University of Pennsylvania

Josh, you make some interesting points about the more advanced study of mathematics. Perhaps my observations were a bit too simplistic when considering this SBAC problem, but they were given in the context of an appropriate expectation for a 12 year old’s understanding of math since this was a 6th grade question. The literal point of a standardized test is to have an answer that you are shooting for 50% of the students to be able to correctly select. In an effort to make the question useful to the differentiation of student ability, they made it confusing instead of just complex. With so much riding on student scores for students, teachers and districts, I think it is a fair standard to hold the test developers to that their answer keys do not have multiple answers that could be considered correct when only one will ultimately be deemed correct and will lock in that student’s score (and that teacher’s evaluation and that district’s accreditation.) I appreciate your acknowledgement that “standardized tests do not generally permit students to fully explain their thinking or demonstrate their nuances of understanding.” A standardized test only has so much time to assess a student’s level of knowledge and, even with computer adaptive models as they currently stand, limited ability to look into a child’s thinking about the subject. That, I fear, will forever be a limitation of standardized testing and is, in fact, something better assessed by the teacher in the classroom.

I am intrigued by the Math Teacher’s Circle and admire their goals, but I wonder if you see standardized testing as ever having a role in getting us to that level of investigation and debate about mathematics.

I have yet to see any measuring devices that would enable anyone to easily and accurately portion out “1/16 of a cup” or “2/5 of a cup”!!! This disconnect between “real life” and “pure” mathematics is exactly what turns some students off to the subject.

Hear, hear!