#small data

- By Debora Irene Christine , Mamello Thinyane , The Conversation -

The trend of tracking health and well-being using digital technologies has permeated mainstream culture. The real-time monitoring capabilities, interactive decision-support algorithms and diagnostic testing features of digital health devices have drawn the interest of users everywhere, including the Global South.

I work with data of various sizes. Most of the time I spin up an AWS EMR instance and analyze the data in Scala. I recently read this piece by Adam Drake. It really made me to this piece here. Essentially, I have been ignoring the useful tools buried inside my Terminal inside my Mac. And most of the time I really don’t need a cluster. My local machine can do a lot.

The coolest thing I learned is the notion of a pipeline in the command line, i.e. the ability to open a file, pass data from one program to the next, including to my surprise, python via sys.stdin. I did not know this! Why didn’t you tell me about this?

Opening a pipeline is too easy in unix with “cat”:

cat filename.ext

Data with personality: See the person, the problems, and the philosophy

Schools have become massive repositories of data on families, employees, alumni. However, schools rarely reflect on the implications of how the collected data serves the interests of the school and community-at-large. While data can be used for mission-driven work, it can also perpetuate cycles of privilege and selectivity because access to data exists in power relationships controlled by a hierarchy. We seek to 1. understand how the data you collect may affect privacy, ethics, and identities in your community, 2. apply an empathetic lens to different constituents and their data sets, and 3. develop an intentional mindset about the use of data.

Written with Linda Vasu.

A slimmer version of this article will appear in the National Association of Independent Schools’ Independent School Magazine. There will be a session offered on this topic by myself, Linda, Don Buckley, and Alex Ragone at NAIS Annual Conference.

“Distinguishing the signal from the noise requires both scientific knowledge and self-knowledge: the serenity to accept the things we cannot predict, the courage to predict the things we can, and the wisdom to know the difference.”

― Nate Silver, The Signal and the Noise: Why So Many Predictions Fail — But Some Don’t

Seeing things clearly

A. Defining Data in Schools

B. Small Data to Support Students (person)

C. Using data to prepare for the future (problems)

D. Using data to support equity (philosophy)

E. Independent Schools

A. Defining Data in School

As with many buzzwords, they can lose meaning when used in association with disparate agendas and philosophies. When we hear the word “data”, it can be difficult to understand what it means and what purpose it serves. Educators can use the word to mean research, information, or plain numbers. While all those meanings might be true and easy to see parsing out the agenda and intention is more challenging. Often data is used to privilege certain ideas as insight, truth, over others to guide institutions towards certain behaviors. Even the tools that are available guide the types of data we can collect. Many of the tools currently being used by schools are products from education technology companies. Those tools are developed due to financial incentives, so they are often in service of expanding an already privileged position or perspective. A hidden agenda of free digital tools is making the data that is collected into a product to be sold elsewhere. It is perhaps naive to assume that extensive and pervasive data collection will add value and wisdom to an educational institution.

The NAIS website (Data and Analysis for School Leadership) links to its online tool developed for the use of member schools “to find actionable data related to all aspects of their school operations, including admission, enrollment, salaries, and more.” NAIS recognizes that schools have a special responsibility in the collection, handling, and use of data. As centers of learning, givers of privilege, and enviable places of employment, independent schools collect data from employees, students, and families. From applications for employment to admissions to student work and test results there are piles of data available in a school. But once this data has been collected, often by different departments, it can seem to enter a dark box. School leadership may not know the data the school collects and ways that the data can be used to make decisions. As digital tools become essential for most of the processes, the data can change hands in ways that are unforeseen. All leaders need to consider the implications of this change in how we work. While there are challenges ahead, especially in creating systems of accountability, there are also opportunities worth considering.

To use data effectively to guide policy, schools must consider the goals and agendas they wish to serve. Fortunately, schools have already declared those intentions via mission and diversity statements, posted publicly and repeated often. The thoughtful collection and use of data in discussions can bring those intentions to life in a way that has not happened until now. Useful data can serves these purposes:

In supporting students

In being equitable institutions

Preparing the school for the future

Moving in the right direction

B. Small data to support students

The word data is often used to suggest utopian silver-bullet solutions to a myriad of problems, wherein the enterprise of schooling can be lost. To use data meaningfully a school needs to be guided by its mission and its values. Instead of looking for a single solution to complex situations schools should look to data to help reveal a more complete picture. Instead of Big Data, as some would have it, we can use small data that tells the stories.

Small data can be used for instructional purposes and curriculum design to support students and track the development of the skills that teachers value. In weekly grade level meetings to discuss students of concern, faculty members often cite test grades and calculations recorded in digital gradebooks. This numerical can be meaningless unless it provides a learning portrait of the student’s capacity for critical thinking, abstract reasoning, problem solving, and the competence to write effectively using the lens and terminology of the subject domain. For example, faculty across departments notice that although students can perform well in Khan Academy type blended learning modules, taking the concepts and applying them across disciplines and outside of the classroom presents challenges. Faculty recognize that in order to support effective student learning, they need to “see the person” behind the data.

Rather than data sets referring to GPA, forward thinking teachers would like to measure performances that foster a growth mindset and social-emotional aspects related to academic progress. This means devising new forms of performative assessments to measure character strengths, social and emotional intelligence, and the cognitive skills and dispositions associated with focused engagement, working memory, entrepreneurial approaches, and the motivation to embrace challenges with efficacy and zest. In an ideal world, data sets that measure schooling as a collaborative enterprise centered on individualized progress and success, in addition to standardized aptitude and academic performance, would be used to gauge and adjust the complex relationships that support student learning and teacher instruction.

Calculating numerical grades is easy. Creating a data set that measures habits of mind is the challenge. How might we capture and record a particular student’s lightbulb moment to track consistent progress? This is the area of edtech that needs development: data collection and design that measures social and emotional intelligence and character strengths. My math colleague suggests that we need a Watson-like program to collect what he calls “soft skills” for teacher use in guiding instruction. Just as doctors use Watson to make diagnoses, teachers could make use of data sets that capture qualitative skills such as engagement, understanding, synthesis, and the excitement that comes with that aha moment of insight and discovery. To collect these character traits, we need a model designed for data capture without interrupting classroom flow, teacher training to use the data quickly and efficiently, and then Watson’s evolving AI can make diagnoses and recommendations. It is interesting to note that Watson has already developed and released a tool called Teacher Advisor for elementary school math educators. “Adaptive learning” is the edtech sector’s buzzword for the kind of instructional design that reflects the convergence of technology, neurocognitive science, analytics, and educational theory.

Instead of GPA data, let’s aim to use small data to see the gifts of the learner. Empathy is a key factor in using data creatively to support new types of assessment that record the small data and the whole person. The concept of educating the whole child for life-long learning is central to independent school missions. With this in mind, there are new models of assessment in development. The Mastery Transcript Consortium is engaged in valuable work to develop a high school transcript to reflect three core principles: no content standardization, no numericals, and a consistent format. The NAIS September issue features an article on their ongoing work to develop alternative assessments that see the person behind the data. Most important, the MTC partnership with 130 independent school is working to invent a tool that does not “distill(s) a student’s knowledge, skills, persistence, ambivalence, integrity, curiosity, collaborative abilities, and raw talent into a single letter grade.” Their focus is on data that extends beyond GPA to non-academic factors that contribute to tell the student’s whole story. This reform aims to pry open the closed system of education anchored in grades, transcripts, and big data that can obscure an authentic rendering of a student’s competencies.

Another entity, the CAE (Council for Aid to education) helps educational institutions develop innovative assessments. Their relatively new tool, the CWRA+ (College and Work Readiness Assessment) “directly measures student performance on critical-thinking and written-communication skills, such as analysis and problem solving, scientific and quantitative reasoning, critical reading and evaluation, and critiquing an argument, in addition to writing mechanics and effectiveness.” These initiatives to design metrics reflect the desire of independent schools to focus on ensuring that curriculum is purposeful and relevant through meaningful work and tasks, a manageable workload, and opportunities for student curiosity, reflection, exploration, and free time that can lead to innovation and creative problem solving.

Seeing patterns

C. Using data to prepare for the future (problems)

In my, Linda’s, attic are two large boxes of traditional, pre-digital “data,” one for each of my daughters. School transcripts, report cards, teacher comments, drawings, paper mache sculptures, cardboard cutouts, collages, textbooks, annotated books, essays, reports, poems. These artifacts/ data points are not owned by a school, a website, an app, Google or the Cloud. In the pre-digital days, a student could carry away his/her own data. There are also boxes of my own “artifacts” or data: transcripts, diplomas, awards, undergrad and grad school notebooks. These curated collections can be mined, too. They form a data trail and portrait of my own learning profile.

In the current educational landscape of digitized data, the information creates a different kind of portrait, one that is “owned” by a third party, one that can possibly be misused, misinterpreted or expropriated. Furthermore, the data offers a more narrow, numbers-only view. Can the numbers “lie,” misrepresent or distort? Yes. So how can we not use numbers when collecting the qualitative data that tells the story of a learner? How can we drive privacy in the data, and allow students to maintain rights to privacy, management, and ownership of their data? How can students take their learning profile with them as they graduate and move on to college and careers? How can we help students manage this?

These questions about privacy rights and ownership remain unanswered when data is collected. Can students ask for their digital records when they leave a school? When and how do records get deleted in a timely fashion? Who is responsible and liable for the security of data collected?

In a school there are two key aspects to data collection and data mining: 1) Data used to guide policy, and 2) data used to guide instruction. Although many independent schools appear to have entered the current era of “data-driven metrics” and “measurable outcomes,” in fact this term may be unknown to many school constituents and stakeholders. A school’s institutional history very likely reflects past policy decisions based on non-quantifiable factors such as anecdotal evidence, perceived need, the ideas of a vocal majority or minority, accreditation recommendations, and the leadership team’s experience and best instincts. However, a new generation of school leaders concerned about mission sustainability, equity, and inclusion have begun to embrace a culture of data, using data sets to make informed strategic decisions.

Yes, the quants have arrived. And they claim that data and performance metrics will bring about more “personalized learning.” Is this the byproduct of the solutions offered by the explosive proliferation of the edtech sector and its promise of exponential growth for investors? According to David Bainbridge, CEO of UK-based Knowledgemotion, as reported in a May 2017 Forbes article, “a new education world has begun with investments in edtech set to reach $252 billion globally by 2020.” And EdWeek Market Brief reports that U.S. spending on K–12 ed-tech is expected to grow to $1.83 billion by 2020, a 38 percent increase relative to 2014 (Winter 2018, vol.18, no.1).

The well-regarded Carnegie Foundation for the Advancement of Education posits: “We cannot improve at scale what we cannot measure.” Measuring school and student performance has resulted in an accountability movement and an improvement science rooted in the idea that a school is chiefly responsible for student performance. The problem: there are additional, often overlooked socio-economic factors that affect student performance because every student belongs to a connected network of family, neighborhood, community, and society at large.

Moving together

D. Using data to support equity (philosophy)

When I, Saber, take the L train to get to work in Bushwick, Brooklyn, at 7:00 am, I see many brown faces going to working class jobs that start early in morning. Many are sleeping. Many got on the train much earlier than me from farther away. If you get on the train later in the morning the faces are whiter and dressed differently, denoting professional jobs. I live in a diverse but segregated city where the richer whiter families have access to good public and private schools while many poorer, brown-er families are stuck in failing schools.

While it can be difficult and challenging to understand how we got here and continue to be here it is the essential work of being a citizen of the city and country. Schooling is at the center of the struggle to desegregate the country and end the disenfranchisement of African American and other marginalized communities. Independent schools have played a role in keeping a system of “de facto” segregation possible in an age when “de jure” segregation is not permissible. From “segregation academies” that grew in the South in response to Brown v. Board of Education to the current underrepresentation of educators and students of color, independent schools continue to contribute to maintain a system of unequal opportunity in this country. While they are small part of the education system they have an over represented in biographies of those with power in our society.

As the NAIS DASL has reported, the vast majority of the heads of school continue to be white males. Since 2000 the number of female heads has remained at low thirty percent, while heads of color remain less than ten percent. School leaders need to consider how and why this trend continues and what it means for our system. As do colleges. The whiteness of college admission officers and independent school is a hinderance to change. In a current NACAC keynote address to college admissions counselors, Shaun R. Harper (Executive Director of USC Race and Equity Center) asserted: “Your profession is 80 percent white… even whiter when we get to those who are at the top levels. It sure would be nice if a mostly white professional association and its members more powerfully, more responsibly and more loudly advocated for racial justice on behalf of those who don’t have the resources that they deserve in high schools across our nation.” Independent schools need to recognize and dismantle ways that they are complicit in maintaining white privilege, a topic long discussed since Peggy McIntosh’s landmark 1996 paper on unpacking white privilege. It is time to take action.

For students of color, public data is hard to find. In 2005 self-reporting to the NAIS, independent schools reported 21 percent students of color. African Americans accounted for 5 percent. Compare these figures with 2000 US census, which calculate that 12 percent of the population is African American. When exaimining why this trend continues, one must consider one of the hallmarks, legacy admission. Sadly, data on this is difficult to find. But we can look at college admissions and see that legacy admission is big part of the answer. The Harvard Crimson reports that 30 percent of the class of 2021 are legacy admits. Legacy students tend to be wealthier and whiter than the overall population; this places severe restriction on the odds of non-legacy admission, people of color and African Americans. Most concerning is that this number has not changed in the past five years. The New York Times finds that “some colleges have more students from the top one percent [of wealth] than the bottom sixty percent.”

As pathways to college, independent schools must make a commitment to action steps that address issues related to socioeconomic disparities. One of these steps is the thoughtful collection of data on the academic outcomes of marginalized communities. Sharing that data with a school and using it to ask questions can help a community of engaged educators attempt innovative approaches to seemingly intractable challenges. A school that is faithfully living out its mission and diversity goals uses data to understand the context of where, when, and how it exists. It needs to relate its purpose and mission to students and the larger community, by offering access and financial aid to students from every socio-economic background. Beyond that, it needs to meaningfully engage in being part of the effort to make access to opportunities more equitable for students and their families, in supporting them to learn advocacy skills to successfully and proactively navigate the independent school landscape. A thoughtful use of data can help us get there.

Privilege accumulated in service of self-preservation and maintaining hierarchies goes against the goals of a deep and engaged education. This is another area where the collection and exploration of data can help create a common body of facts that can help start difficult conversations. While data alone will not help independent schools free themselves of the awful history of segregation in this country, it should combined with the good understanding of history and commitment to taking on challenging problems at the highest level. Am intentional study and a philosophy of data will give our students a more equitable view and context and a deeper understanding of social access, opportunity, responsibility.

Being independent

D. Independent Schools

Schools have become massive repositories of data on families, employees, alumni. However, schools rarely reflect on the implications of how the collected data serves the interests of the school and community-at-large. While data can be used for mission-driven work, it can also perpetuate cycles of privilege and selectivity because access to data exists in power relationships controlled by a hierarchy rewarded for maintaining status quo. Data has nuances beyond numbers or letters; data has a personality. While data can serve a common good, it is often employed to affirm a pre-existing agenda.

Data itself is not the problem. We should be asking these questions. How is data used? What story does it tell? What story does it not tell? Whom does it serve? Who controls it? Do we need it? For example, information on academic achievement is collected and typically used for marketing and college admissions purposes. How might we purposefully design a system using such data to increase access to high-quality education for underrepresented students? Additionally, fundraising and development data are often narrowly assigned monetary values. How might we design a system to imagine new possibilities for giving? The ubiquity of digital learning has opened classrooms to surveillance by technology corporations. Instead, how might we use “small data” to improve teaching and learning? How might we design digital communities and our own homegrown networks to better reflect our values, especially in regard to privacy? All these important questions require a philosophy about data and how it flows throughout a school. Having an intention about data and its implementation serves the common good.

With transfer learning, we can take a pretrained model, which was trained on a large readily available dataset (trained on a completely different task, with the same input but different output). Then try to find layers which output reusable features. We use the output of that layer as input features to train a much smaller network that requires a smaller number of parameters. This smaller network only needs to learn the relations for your specific problem having already learnt about patterns in the data from the pretrained model. This way a model trained to detect Cats can be reused to Reproduce the work of Van Gogh

Mug of Collaborations – Life in Clay