NWEA MAP Assessments. Study connects test disengagement to student deep-rooted problems. Personalized Learning and Adaptive Assessments: Parents need consent, transparency on how student data (and meta data ) are used and shared.
Algorithms and Artificial Intelligence are rampant in education, via online “personalized learning” and online adaptive assessments. If you want a good tutorial on why these hidden algorithms are a problem, see the work of experts like Elana Zeide and Cathy O’Neil and Frank Pasquale. Algorithms are hidden; they can be wrong. Hidden data are making decisions and predictions about students via personalized learning, adaptive assessments, and massive student data collection. Students cannot opt out and they cannot see the data points collected, analyzed and shared with third parties. Congress: stop focusing solely on adults and Facebook profiling, please protect children. ______________________
NWEA MAP Growth Assessments are online, adaptive (or personalized) assessments used in many schools across America. They are interim tests that are given multiple times throughout the year. As a concerned parent, I contacted NWEA about their new SEL data collection, third party sharing, and analytics. What follows is a brief account of my nearly 7 month NWEA Q & A communication and related FOIA. Many questions remain unanswered. I invite NWEA, teachers, students, other parents to give input — as this is something that potentially affects millions of US students. If you are short on time, here is the very short ‘Top 10’ version:
- NWEA rebranded in 2017, won award for new Social Emotional Assessment tool for MAP Growth, announced new “engagement” profiling for every student, and hired new CEO, Chris Minnich, of CCSSO, the Common Core copyright co-owner.
- Detecting Social Emotional skills is a $43 Billion global market for edtech.
- NWEA maintains one of the largest repositories of student growth data in the nation. Why hasn’t NWEA signed the SIIA privacy pledge to not sell student data?
- Where is NWEA data being shared; how is it being used and analyzed? Collaborators, Researchers, Hitachi.
- Do parents know their child has an engagement profile with NWEA ? Can parents opt out of this profile? (No).
- In a study with Harvard’s Center for Education Policy Research Proving Ground Network , and a Gates funded CORE school district, NWEA has used personal data from the NWEA engagement profile and COMBINED this pii data with personal, student-level (pii) data from students’ self-reported behavior surveys. NWEA’s study concludes that test disengagement (rapid response) is “connected to risk of dropping out of school …and a lot of deep-rooted problems”.
- Who will be able to see if a student has been flagged as disengaged (or at risk for these “problem areas”) by NWEA? (ie: Will this disengagement profile or flag ever be shared with colleges, employers?) Did parents give informed consent for this NWEA study that combines and potentially re-purposes sensitive student data? In asking NWEA and FOIA of the school district, here, I could find no evidence of explicit permission or informed parent consent. Will NWEA obtain informed consent in the future and allow parents to opt out of SEL or Engagement data?
- NWEA Reading Fluency now uses Voice Recognition technology. How is THAT data being analyzed, used?
- NWEA and researchers, edtech, Congress can learn from the Facebook scandal. Children’s data should not be taken without consent, should not be repurposed, researched, and analyzed without transparency and consent. You need to do better. We deserve #GDPR for ALL.
“Nearly half of all United States districts use our assessments to gain the insights they need to elevate student and school success. 9 million+ students, 50 million testing events annually” —NWEA
There is a push to replace end of the year high stakes tests with online adaptive tests, (like NWEA), taken multiple times throughout the year. As Tom Vander Ark of Project Unicorn states, these adaptive tests provide “better, faster and cheaper data.”
“We’ve developed and maintain one of the largest repositories of student growth data in the nation.” –NWEA
The Million Dollar Question: What data are being collected and how are data being used and shared?
Per NWEA’s February 2018 post, “Starting in the July 2017 MAP Suite release, MAP Growth added a new feature that monitors the amount of time a student spends answering individual items to identify disengaged test takers.” For this profile, it doesn’t matter whether the student answers the question correctly, because this new profile measures the student’s engagement. (If the student answers questions too quickly, three times in a row, they are flagged as disengaged.) If you didn’t know about this new engagement profile, you aren’t alone; NWEA has been making many changes. Through communication with NWEA, I learned that the new engagement tool was announced to schools in a June 2017 Partner Update. NWEA rebranded itself that summer, with a new website, new partners, sending out updates about their slight name change, from Northwest Evaluation Association to NWEA, and changing the name of their tests, (ie: from MAP to MAP GROWTH). Four months after its rebranding, in October 2017 NWEA formally announced they had a new CEO – Christopher Minnich, who previously served as the Executive Director of the Council of Chief State Schools Officers (CCSSO), the group that promoted and owns the copyright to the Common Core State Standards. However, over the course of the last 7 months, I have talked to many parents and schools that use NWEA MAPS who were not aware of NWEA MAP’s engagement tracking, or how this data will be used.
This engagement metric, called Response Time Effort, is explained in an NWEA report that accompanies this November 2017 NWEA webinar hosted by EdWeek. An excerpt of the NWEA webinar states, “Student Engagement: Are Your Test Scores Valid?”
“The MAP Growth assessment is not relying on the Proctor to encourage the student. The test doesn’t pause nor does it feel any different to the student doing the assessment. Take a look at reports. What you are seeing here is the student profile report. The student profile report, how this information is captured for teachers starting November 11, which is coming right up and beginning with tests delivered this fall, the student profile will indicate if the student exhibited rapid guess behavior on a test.There will be pop-up boxes within the student timeline view of student assessment results built right into the student report. This will show two new metrics, first, the percentage of disengaged responses. The percentage of items on the test answered too quickly for a student to have applied effortful behavior to answer that question. If 30 percent or more of test questions are marked as disengaged the entire test score should be considered invalid and a retest might be appropriate for your student at this point. The second metric that is being introduced here is the estimated impact of disengagement on Response Time Effort (RTE). That shows how much RIT points higher the student might have scored if they had been fully engaged in the test process. With the RIT score of 200 estimated impact of negative 2, the student might have the potential to score 233 if they were fully engaged during testing…” –NWEA November 7, 2017 EdWeek sponsored NWEA Rapid Guessing Webinar [Emphasis added]
I asked NWEA if parents could opt their children out of this engagement (Response Time Effort) profile. NWEA said no. NWEA said if parents are concerned, they can opt children out of the test.
BigData and Big Money in Social Emotional Engagement
The NWEA engagement profile originally caught my attention when NWEA won 1st place in CASEL’s Social Emotional Assessment Challenge. You can see NWEA’s press release here and you can read a publicly posted narrative about this work from NWEA researchers, entitled, Using Student Assessment Engagement as a Measure of Student SEL and School Engagement. Importantly, also read about CASEL and the Aspen Commission whose mission is to standardize, rate, and shape students’ Social Emotional Learning skills (often referred to with acronyms SEL or SEAD). There is a big push to measure children’s emotions, or Social Emotional Skills, because non-cognitive factors can now be used as an accountability indicator under our new federal law ESSA. Even the Department of Defense is on board with Social Emotional behavior management programs. This ETS paper talks about the big 5 personality traits as they relate to academics and workforce. Not surprising, Edtech companies want a piece of this multi-billion dollar Social Emotional pie.
As this World Economic Forum report explains,
“We see great potential for improving specific social and emotional skills using select existing products in the $43 billion global ed-tech market.“
Why hasn’t NWEA signed the SIIA Privacy Pledge and does NWEA sell student data?
Given the billions to be made on student data, especially social emotional data, it is interesting and concerning that NWEA still has not signed the national SIIA privacy pledge to protect and not sell personal student data.
Letters to NWEA.
On September 28, 2017, I asked NWEA a series of questions about what they did with students’ data, in the spirit of helping other parents, see redacted letters and NWEA responses here. For example, I asked NWEA why they had not signed this student privacy pledge. My questions and NWEA’s response:
Question: Does NWEA profit from, lease, rent, sell, or trade for services student data?
Answer: NWEA does not sell personal identifiable student education records to third parties for their commercial use and do not use such data to target advertisement at students. Instead, we:
• Use personal identifiable student education records to create, generate, and deliver reports back to schools and districts, including custom reports as requested. In addition, we may use personal identifiable student education records, only with a school or district’s written permission, for statistical studies and research.
• Subject to applicable law and under separate permission by our subscribers, we may share and transfer personal identifiable student education records to third parties to evaluate educational or research programs or to conduct research studies.
• Generate aggregate data, which do not identify students but tend to reflect collective information about students.
• Deal with legal processes such as subpoenas, claims of test security breach.
Question: Along those lines, is there a reason why NWEA has not signed the SIIA student data privacy pledge to be good stewards and not sell student data? https://studentprivacypledge.org/privacy-pledge/
Answer: NWEA utilizes a data privacy addendum tailored specifically to Colorado student privacy law for our Colorado partners. NWEA does not sell personal identifiable student education records to third parties for their commercial use and do not use such data to target advertisement at students.” [Emphasis added] (Note: NWEA did not say that they do not ever sell data, nor did they say why they have not signed the SIIA pledge.)
NWEA and Engagement Data Collection, Analyzing, and Sharing
I was concerned about NWEA’s Social Emotional engagement data collection and who it was shared with, especially after reading this NWEA Master contract which stated that NWEA shared data with the well funded (many Foundations) America Achieves and OECD. (OECD is the global organization that directs the PISA standardized test which measures and ranks countries’ academic achievement.) My question and NWEA’s response:
Question: NWEA OECD shares pii student data with America Achieves; does NWEA share MAP data with America Achieves? for what purpose?
Answer: No personally identifiable student data was redisclosed to America Achieves; OECD school reports do not contain any personally identifiable student data. In addition, NWEA no longer shares data with America Achieves.
The Gates funded Data Quality Campaign is also promoting non-cognitive Social Emotional factors for accountability, seen in this report on the (also Gates funded) Core Districts, Using Social-Emotional Learning Data in the CORE Districts: Lessons Learned,
“In 2013, the CORE Districts (then consisting of 10 districts from across California) received a No Child Left Behind waiver that allowed them to use their own rigorous accountability system instead of California’s…. The CORE Districts have designed a rigorous accountability structure … [that] recognizes the importance of factors beyond academic reparedness [and] values multiple measures of student success in the social/emotional domain, as well as the critical importance of a school and district’s culture and climate.” -Data Quality Campaign [Emphasis added]
NWEA, Collaborators say test disengagement is connected to students’ deep-rooted problems
I was alarmed by the study that NWEA conducted in Santa Ana Unified School District, (which is part of the CORE Districts that the Data Quality Campaign mentions above), in collaboration with Harvard CEPR as part of their Proving Ground Network. In this study, NWEA combined a student’s disengaged status (rapid Response Time Effort on MAP assessments) with answers from a separate student survey of self reported behaviors, (surveys were given to all students in the CORE Districts), NWEA relates rapid response test disengagement to other ‘deep-rooted problems’ in a student’s life. This EdWeek article quotes a NWEA researcher saying,
“As they disengage from tests and the course material, a whole host of other things come up … attendance, suspensions, course failure … that have been connected to risk of dropping out of school,” Soland said. “What we’re really showing is lack of test engagement is a symptom around a lot of deep-rooted problems,” [NWEA researcher] Soland said.
In the narrative of their research, the NWEA researchers also discuss tying test disengagement to the CORE District survey data.
“Recent research of ours conducted in collaboration with Santa Ana Unified School District (SAUSD) shows that RTE is useful as much more than a proxy for test motivation. Our study indicates that rapid-guessing behavior is associated with low self-management scores on district administered SEL surveys (Soland, Jensen, Keys, Bi, & Wolk, 2017). … In our research, we show that RTE is associated with other behaviors that are warning signs of low academic motivation including course failures, suspensions/expulsions, and absenteeism. For example, students who rapidly guessed on 10% or more of the items on a test were absent from school an additional day, on average, compared to students who did not rapidly guess. …From a measurement perspective, self-management is a complicated construct. Rather than refer to a specific latent variable, it more frequently represents a collection of behaviors. For example, the survey used by Santa Ana Unified (and by all districts in the California Office to Reform Education or CORE) to measure self-management focuses entirely on whether students self-report exhibiting behaviors like coming to class prepared, following directions, and being able to work independently. Therefore, like other measures of self-management, rapid guessing is not exactly measuring a latent construct in the way that a growth mindset survey captures an unobservable belief that intelligence is malleable. Rather, rapid guessing is just one of several behaviors suggesting students may have trouble controlling their thoughts, behaviors, and emotions. To acknowledge this important if subtle distinction, we often refer to rapid guessing as a proxy for, rather than a measure of, self-management.”–NWEA [Emphasis added]
For background on California’s Core Districts and Social Emotional (SE) skills, (1600 schools), see this 2016 report which states, “Through a No Child Left Behind waiver granted in 2013, the CORE Districts have implemented a holistic system of accountability and continuous improvement that focuses on students’ social-emotional skills and school climate/culture alongside academic outcomes…. Core Districts began mapping the intersections between SE skills and the Common Core State Standards. …. The CORE Districts entered into a partnership with Policy Analysis for California Education (PACE), a research center at Stanford to support the districts in identifying promising practices to support students’ SE development.”–A Case Study of the CORE Districts
Parent consent for combining NWEA data with personal survey data on student behavior?
I wondered if parents and students had given informed consent to allow their NWEA engagement data to be combined with personal information shared in separate school behavior survey. I asked NWEA to provide this data sharing agreement and also to provide evidence of parent consent forms. (NWEA did not provide either. After waiting months for NWEA to provide this information, I instead requested these public records directly from Santa Ana Unified District and received documents (below) in December 2017 and January 2018. However, I was never able to obtain evidence of informed parent consent, permission.) See my questions about student and parent informed consent and NWEA response here,
Question: Would you please share with me the SEL contract, SEL parent permission forms that NWEA has used with Santa Ana schools? Please provide contract and permission forms.
Answer: It is our understanding that you have reached out directly to Santa Ana Unified School District (SAUSD) regarding this request and SAUSD has provided you with the documents. Please note that SAUSD administered SEL surveys to their students. Monitoring and improving student SEL is an initiative of the SAUSD and many other districts in California (the CORE consortium of districts). NWEA did not administer these SEL surveys. Likewise, NWEA did not request that SAUSD administer these surveys. These surveys were already being administered in the district, and the data sharing agreement established between SAUSD and NWEA allowed SAUSD to share these student-level data with NWEA. NWEA has no current plans to proceed with this type of research with your district. [Emphasis added]
Per NWEA communication,
“These surveys were already being administered in the district, and the data sharing agreement established between SAUSD and NWEA allowed SAUSD to share these student-level data with NWEA.”
The FOIA of Santa Ana Unified School District
The Communications Office of Santa Ana Unified School District was helpful and responsive to my FOIA request. However, as you can see by the conversation and documents posted, they did not provide copies of permission slips or show evidence of informed parent consent to have students’ personal information on behavioral surveys shared and analyzed by researchers at Harvard CEPR and NWEA.
The Santa Ana Unified School District received your California Public Records Act Request dated December 3, 2017, wherein you request the following:
⦁ A copy of contract and written agreement(s) between NWEA and the Santa Ana Unified School District for any NWEA student disengagement research, including research to connect frequent test disengagement to other problems in school https://drive.google.com/file/d/1JFW3aGVLT1V8CMqrlEHZU-BpLHKprV76/view?usp=sharing
⦁ A copy of Data-Sharing Agreement between Harvard’s Center for Education Policy Research Proving Ground Network and Santa Ana Unified School District https://drive.google.com/file/d/1AJDNwtEpiR-kIF_35nAT5BI-JMbIKFkO/view?usp=sharing
⦁ Any notification to parents regarding data collection and use, copies of any informed parental consent permission notification, and any notifications of ability to opt students out of NWEA test disengagement research studies
Attached you will find documents responsive to bullet points 1 and 2; we hope you are willing to accept these documents electronically. You may locate Board Policy and Administrative Regulation 6162.8 regarding research and data at: https://www.sausd.us/site/handlers/filedownload.ashx?moduleinstanceid=25350&dataid=34809&FileName=BP%206162.8.pdf
See more here at FOIA and Communication with SAUSD https://drive.google.com/file/d/1kpUVv_e9X-kL9v-zrMJyC-Wss4pJzC6f/view?usp=sharing
Hitachi Consulting Corporation is NWEA’s only Subcontractor
I asked NWEA to tell me what third parties/vendors/entities they share the data with, (who has access to student data), what data is shared and for what purpose, and to please provide me with the data sharing contracts, terms of service. NWEA responded that there is *currently* only ONE vendor who has access to NWEA MAP student data. That party is Hitachi Consulting Corporation. I had a hard time believing that only one entity has access to NWEA MAP data, so I asked multiple times, since state law requires transparency about contractors, subcontractors and student data. I specifically asked if Google Analytics or Harvard, had access to student data, because we know about NWEA’s projects with Harvard CEPR and also at least the k-2 NWEA assessment (CPAA) requires Google Analytics access. And although the current website doesn’t display research partners, we also know that NWEA partners with Universities of Portland, Delaware, Notre Dame and Connecticut, per this archived NWEA page. NWEA is also involved in Ohio Schools school climate surveys. We know data can be analyzed and shared with multiple downstream parties. Yet , when I asked NWEA: Please provide names of all subcontractors and all parties (ie: nonprofits, researchers, analytics) who have access to this data and what data elements these subcontractors receive and the use and purpose of the shared data. NWEA responded,
“NWEA relies on select vendors to perform certain services. Currently, NWEA retains Hitachi Consulting Corporation to perform software development services. …NWEA creates two granular MAP Growth reports that a school can provide to parents: (i) a Comprehensive Data File that contains the MAP Growth data for your child/children; and (ii) your child’s student profile report, which shows engagement data. These reports include your child’s MAP Growth student personally identifiable information, which NWEA shares only with our sole subcontractor, Hitachi Consulting Corporation as needed…” –NWEA
Since NWEA did not provide the Hitachi Consulting Corporation contract, terms of service, exact data elements shared “as needed”, parents can only wonder what Hitachi Consulting Corporation does with the personally identifiable student information and engagement data shared with them. (According to NWEA tax returns, NWEA paid Hitachi $3,779,614.00 as a contractor in 2016.) Does Hitachi further share this “as needed” student information with other subcontractors or agencies?
A quick search for Hitachi Consulting yields some interesting results such as Hitachi Data Systems has created a “data-gathering laboratory” that it hopes will improve the student experience and Hitachi Facial Expression Techonology- The technology which never forgets a face and Hitachi Data Systems big-data tools. Hitachi also boasts alliances with Google Cloud, Microsoft, SAP, and Oracle. (Oracle, as in the owners of this Oracle Data Directory. Really?) Why would parents possibly be concerned?
Scoring the test.
I obtained copies of my children’s student profiles from NWEA. I can find no plausible explanation as to why several questions on my children’s NWEA MAP Growth assessments were listed twice, with varying response times. (See same test question marked as incorrect and later marked as correct. There were multiple instances of this on my children’s profiles.) NWEA’s answer was that the proctor must have stopped and restarted the test. ??? My children and proctors can attest that this was not the case, but this is only my anecdotal evidence. So I will leave it at that: a question. I would be curious if other parents asked for their children’s NWEA student reports/profiles and whether they see multiple events for the same test question.
The engagement measuring tool.
A student (as I have personally witnessed with my children) may answer rapidly because they recognize the content, may instantly know the answer. (ie: The passage may be about John Locke, which the student just finished studying in class and is very familiar with, hence, she skips reading the passage and goes straight to the questions.)
A student might not care. They could be clicking anything just to get done with this “waste of time” as one straight A student told me. Is it fair or accurate to flag this student as disengaged and then tie that engagement data to “deep-rooted problems”, as in the Harvard CEPR/NWEA study?
NWEA takes time away from regular teaching. It interrupts the schedule. Students have said they really do not see the value in this test. Are they right? Does this test really have an impact on achievement? For instance, this USDoE REL study found that “The MAP program had no statistically significant impact on the reading achievement of grade 5 students as measured by the ISAT or the composite scores on the MAP reading and language use assessments (table 5.1).” While some teachers like that NWEA MAP gives immediate feedback, some teachers question its value.
While the world is freaking out about Facebook profiling users and their friends, Facebook users are learning how to download their Facebook data, people are joining the delete Facebook movement, and learning what a Shadow Profile is, I’ve been thinking about school children. Students’ personal data are tracked, profiled, profited from, and researched against their will with no consent, no ability to opt out, no ability to download their data or see their shadow profile, no way to delete their SLDS account, no way to opt out of or delete PowerSchool, iReady, Naviance, GoogleClassroom, no way to see how this information is shared and analyzed by third parties. Is there media coverage? Congressional Hearings asking to put a stop to the student datamining over reach? No. Instead, we have research and BigData organizations with ties to Harvard Proving Ground and Data Quality Campaign lobbying Congress for MORE ACCESS to student data, without parent or student consent. And we have mega funded Unicorn edtech organizations wanting to make student data MORE interoperable, easily shared, again without consent.
Americans care about Equifax breaches; Americans care about Facebook privacy scandals. We have State Attorneys General writing letters demanding Facebook privacy protections, but why aren’t these state attorneys writing letters about protecting school children from similar data collection, profiling, and sharing without consent ?
Where is the concern about predatory vendors using children?
To paraphrase European regulators regarding Facebook data collection, by collecting and profiting from data without consent, Facebook (and edtech vendors) are “forcibly extracting a valuable commodity from consumers”. In this case, the consumers are children who are compelled to attend school and are mandated to use these edtech products. Parents don’t send their children to school to profiled or for them to bankroll the edtech industry.
Parents are asking for the same data protections Americans are asking of Facebook: consent and ability to see how the data is being used.
These links explain how Facebook, Twitter, Google are each collecting data on your mental health. Facebook even has a patent for facial recognition tool called Techniques, for “emotion detection and content delivery. It would use the camera in your phone to take pictures of you as you scroll through content.” Education testing giant Pearson conducted social-psychological experiments on students without their consent or knowledge, and thousands of “free” educational apps that are improperly tracking children including familiar apps like Google Classroom, Remind, Khan Academy, Class Dojo. We know there are billions to be unbundled by sharing education data and the tech industry is pushing to make that data sharing more interoperable. Why not stop. Why not be the change?
Edtech doesn’t have to be creepy; vendors could ask for consent to collect, analyze and market. They could be transparent but they choose not to. This needs to change.
Edtech and researchers: you could bake in privacy and get parent consent before you collect, analyze and share or market student data; be transparent about the data use, even and especially meta data. Student data are not just numbers; student data represents a child’s identity, a child’s future. Please respect the basic human right of privacy, respect FIPs; respect Belmont. In human subject research, individuals always must give informed consent and participation must be voluntary. Treat American children as decently as GDPR demands you treat European children.
A few selected links:
NWEA power point https://www.edweek.org/media/171107presentation.pdf
NWEA Webinar https://webinars.on24.com/edweek/RapidGuessing
Cleveland Schools, AIR, NWEA School Climate https://ies.ed.gov/funding/grantsearch/details.asp?ID=1969
Harvard Proving Ground School Improvement and Redesign Santa Ana Unified Partners https://cepr.harvard.edu/proving-ground
NWEA GRANT CLEVELAND SCHOOL CLIMATE SURVEY https://ies.ed.gov/funding/grantsearch/details.asp?ID=1969
Harvard CEPR Ohio schools https://provingground.cepr.harvard.edu/news/ohio-district-pilot-partnership-using-proving-ground-impact-chronic-absenteeism and here http://education.ohio.gov/Topics/Data/Proving-Ground
Data sharing MOU Santa Ana School District and NWEA https://drive.google.com/file/d/1JFW3aGVLT1V8CMqrlEHZU-BpLHKprV76/view?usp=sharing
Harvard CEPR collaboration agreement with Santa Ana School District https://drive.google.com/file/d/1AJDNwtEpiR-kIF_35nAT5BI-JMbIKFkO/view?usp=sharing
NWEA: Using Student Assessment Engagement as a Measure of Student SEL and School Engagement https://www.nwea.org/content/uploads/2017/09/Using-Student-Assessment-Engagement-as-a-Measure-of-Student-SEL-and-School-Engagement.pdf
NWEA MAP Reading Fluency K-3 Assessment has implemented a new voice recognition tool. https://www.educationdive.com/news/nwea-unveils-k-3-reading-fluency-test/515364/
NWEA 990 tax forms http://990finder.foundationcenter.org/990results.aspx?action=Find&fn=northwest+evaluation&st=&ei=
NWEA LEGAL https://legal.nwea.org/supplementalterms.html
NWEA MAP VOICE RECOGNITION IN READING FLUENCY ASSESSMENTS https://www.nwea.org/map-reading-fluency/
NWEA PARENTS AND MAP GROWTH https://www.nwea.org/blog/2017/parents-map-growth-7-things-know/
NWEA Privacy and Security Policy for Student Information https://legal.nwea.org/NWEA%20Privacy%20and%20Security%20for%20PII%20Sept%2024%202014.pdf
NWEA MASTER SUBSCRIPTION AGREEMENT https://drive.google.com/file/d/10SN_B7oEDOlvbPaBIqMX079yOg-rJexv/view?usp=sharing
NWEA MASTER CONTRACT, OECD TEST FOR SCHOOLS https://legal.nwea.org/OECD%20Test%20for%20Schools%20MSA.pdf
10. Subscriber Information. Subscriber hereby permits NWEA and OECD to use information regarding its schools or district (other than PII) to perform their obligations hereunder and to include such information in Reports for public dissemination and research to be used and disclosed to internal and external researchers that have executed confidentiality agreements. This permission survives termination or expiration of this Subscription.
10.1 Reports. Subscriber hereby permits NWEA to redisclose and deliver Reports and underlying raw data with America Achieves, a non-profit education organization. America Achieves manages the Master Subscription Agreement – OECD Test for Schools Global Learning Network, a professional learning network for educators who participate in the OECD Test for Schools. America Achieves may use the Reports and raw data for research, analysis and development in the field of education, including conducting aggregate analyses of data from all participating schools in the America Achieves network and capturing this data in publicly shared reports. Reports and raw data may be used for illustrative case studies.
CORE District partnership with Standford’s Policy Analysis for California Education (PACE), “Should Non-Cognitive Skills be Included in School Accountability Systems? Preliminary Evidence from California’s CORE Districts,” Brookings Institution, March 17, 2016, http://www.brookings.edu/research/reports/2016/03/17-non-cognitive-skills-school-accountability-california-core-west
Linking the Partnership for Assessment of Readiness for College and Careers (PARCC) Assessments to NWEA MAP Growth Assessments https://www.nwea.org/state-solutions/colorado/
NWEA and AIR (REL): The Impact of the Measures of Academic Progress (MAP) Program on Student Reading Achievement, report prepared for the National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, under contract ED-06C0-0019 with Regional Educational Laboratory Midwest administered by Learning Point Associates, an affiliate of the American Institutes for Research. https://ies.ed.gov/ncee/edlabs/regions/midwest/pdf/REL_20134000.pdf
Hitachi Consulting Group https://www.hitachiconsulting.com/home.html
Hitachi Facial Expression Techonology- The technology which never forgets a face www.hitachi.eu/en/social-innovation-stories/technology/technology-which-never-forgets-face
Hitachi Data Systems has created a “data-gathering laboratory” that it hopes will improve the student experience. https://www.intheblack.com/articles/2017/09/01/data-makes-uni-campus-smarter
Hitachi Data Systems big-data tools https://www.cio.com/article/2916273/hds-bigdata-tools-should-help-it-mobile-health-workers.html?phint=newt%3Dcio_cloud_computing&phint=idg_eid%3D7bf74665618d29964ae6bd274cafec78#tk.CIONLE_nlt_cloud_computing_2015-05-04
RAND: Social and Emotional Learning Interventions Under the Every Student Succeeds Act: Evidence Review. This RAND analysis offers guidance to states and districts on how they can use funds through ESSA to support Social and Emotional Learning and to identify evidence-based SEL interventions that meet their needs. http://www.wallacefoundation.org/knowledge-center/pages/sel-interventions-under-essa-evidence-review.aspx
World Economic Forum–New Vision for Education: Fostering Social and Emotional Learning through Technology
March 2016 Prepared in collaboration with The Boston Consulting Group “We see great potential for improving specific social and emotional skills using select existing products in the $43 billion global ed-tech market.” https://mchdata.com/wp-content/uploads/2016/07/WEF_New_Vision_for_Education.pdf