#ibmconsulting

Smarter Balanced Assessment Consortium

The Smarter Balanced Assessment Consortium, a member-led public organization with headquarters in California, offers assessment tools to teachers in K–12 and higher education. Founded in 2010, the business creates creative, standards-aligned test assessment systems in collaboration with state education organizations. In order to assist educators in identifying learning opportunities and enhancing student learning, Smarter Balanced provides them with lessons, tools, and resources, such as formative, interim, and summative assessments.

In the constantly evolving field of education, Smarter Balanced is dedicated to progress and creativity. The objective is to investigate a systematic methodology for utilizing artificial intelligence (AI) in educational assessments in conjunction with IBM Consulting. The partnership is still in place as of early 2024.

Specifying the difficulty

Standardized exams and structured quizzes, which are common K–12 skill evaluations, are criticized for a number of equity-related reasons. AI has the revolutionary potential to improve assessment fairness across student populations, including marginalized groups, by providing individualized learning and assessment experiences when used responsibly. Therefore, defining what responsible AI adoption and governance in a school setting looks like is the main difficulty.

Educators, professionals in artificial intelligence, ethics and policy surrounding AI, and specialists in educational measurement made up the first multidisciplinary advisory group established by Smarter Balanced and IBM Consulting. The panel’s objective is to create guiding principles for integrating justice and accuracy into the application of AI to learning materials and educational measurement. Below is a summary of some of the advisory panel’s factors.

Considering human needs when designing

Organizations can create a human-centric strategy for implementing technology by utilizing design thinking frameworks. Design thinking is driven by three human-centered principles: a focus on user outcomes, restless reinvention, and team empowerment for diversity. Stakeholders’ strategic alignment and responsiveness to both functional and non-functional organizational governance requirements are enhanced by this approach. Developers and other stakeholders can generate creative solutions, prototype iteratively, and gain a thorough understanding of user demands by applying design thinking.

This methodology is critical for early risk identification and assessment during the development process, as well as for enabling the development of reliable and efficient AI models. Design thinking aids in the development of AI solutions that are mathematically sound, socially conscious, and human-centered by consistently interacting with various communities of domain experts and other stakeholders and taking their input into consideration.

Including Diversity

A varied group of subject-matter experts and thought leaders were assembled by the merged teams to form a think tank for the Smarter Balanced initiative. Experts in the domains of law and educational evaluation, as well as neurodivergent individuals, students, and those with accessibility issues, made up this group.

The think tank aims to iteratively, rather than one-time, integrate its members’ experiences, opinions, and areas of expertise into the governance framework. A fundamental tenet of IBM’s AI ethics is reflected in the strategy: artificial intelligence should supplement human intelligence, not replace it. Incorporating continuous feedback, assessment, and examination by a range of stakeholders can enhance the development of trust and facilitate fair results, ultimately resulting in an educational setting that is more comprehensive and productive.

In grade school settings, these approaches are essential for developing equitable and successful educational assessments. Building AI models that are reflective of all students requires the many perspectives, experiences, and cultural insights that diverse teams bring to the table. Because of its inclusivity, AI systems are less likely to unintentionally reinforce existing disparities or fail to take into account the particular demands of various demographic groups. This highlights another important AI ethical tenet at IBM: diversity in AI is important since it’s about math, not opinion.

Examining beliefs that are focused on the student

Determining the human values IBM want to see represented in AI models was one of the first projects that IBM Consulting and Smarter Balanced performed together. IBM arrived at a set of principles and criteria that correspond to IBM’s AI pillars, or essential characteristics for reliable AI, as this is not a novel ethical dilemma.

Explainability: The capacity to provide results and functions that don’t require technical explanation

Fairness: Handling individuals equally

Robustness: security, dependability, and ability to withstand hostile attacks

Openness: Sharing information about the use, functionality, and data of AI

Data privacy: revealing and defending users’ rights to their privacy and data

It is difficult to put these ideas into practice in any kind of organization. Even higher standards apply to an organization that evaluates pupils’ skill sets. Nonetheless, the work is valuable due to the potential advantages of AI. The second phase is now in progress and involves investigating and defining the values that will direct the use of AI to the assessment of young learners.

The teams are debating the following questions:

What morally-based guidelines are required to properly develop these skills?

Who should be in charge of operationalizing and governing them?

What guidance should practitioners providing these models with follow?

What are the essential needs, both functional and non-functional, and what is the required strength?

Investigating varying impact and levels of affect

IBM used the Layers of Effect design thinking framework for this activity. IBM Design for AI has donated numerous frameworks to the open source community Design Ethically. Stakeholders are asked to think about the primary, secondary, and tertiary implications of their experiences or goods using the Layers of Effect framework.

The intended and known impacts of the product in this case, an AI model are referred to as primary effects. One of the main functions of a social media platform, for instance, could be to link people with shared interests.

Although less deliberate, secondary impacts can swiftly gain importance among stakeholders. Using social media as an example, the platform’s value to advertisers may have a secondary effect.

Unintentional or unexpected consequences that show up gradually are known as tertiary effects. An example of this would be a social media platform’s propensity to provide more views to messages that are insulting or misleading.

The main (desired) consequence of the AI-enhanced test assessment system for this use case is a more effective, representative, and equitable tool that raises learning outcomes throughout the educational system.

Increasing efficiency and obtaining pertinent data to aid in more effective resource allocation where it is most required are possible secondary benefits.

Unintentional and possibly recognized tertiary effects exist. Stakeholders need to investigate what would constitute unintentional harm at this point.

The groups determined that there could be five types of serious harm:

Detrimental prejudice concerns that fail to take into account or assist pupils from marginalized groups who might require additional resources and viewpoints to meet their varied needs.

Problems with personally identifiable information (PII) and cybersecurity in educational systems where insufficient protocols are in place for their networks and devices.

Insufficient governance and regulations to guarantee AI models maintain their intended behaviors.

Inadequate communication regarding the planned usage of AI systems in schools with parents, students, instructors, and administrative staff. These messages ought to outline agency, like how to opt out, and safeguards against improper usage.

Restricted off-campus connectivity that could limit people’s access to technology and the use of AI that follows, especially in rural locations.

Disparate impact evaluations, which were first used in court cases, assist organizations in identifying possible biases. These evaluations look at how people from protected classes those who are vulnerable to discrimination on the basis of gender, ethnicity, religion, or other characteristics can be disproportionately impacted by policies and practices that appear to be neutral. These evaluations have shown to be useful in the formulation of employment, financing, and healthcare policies. IBM tried to take into account cohorts of students in their education use case who might, because of their circumstances, receive unequal results from tests.

The following were the categories found to be most vulnerable to possible harm:

People who experience mental health issues

People from a wider range of socioeconomic backgrounds, including those without a place to live

Individuals whose mother tongue is not English

Those with additional non-linguistic cultural factors

People with accessibility concerns or those who are neurodivergent

IBM group’s next series of exercises is to investigate ways to lessen these harms by utilizing additional design thinking frameworks, like ethical hacking. IBM will also go over the minimal specifications that companies looking to integrate AI into student assessments must meet.

Changing how the community receives social services: IBM and Microsoft’s cooperative strategy

Connect360 IBM

In a time when economic expansion and technical breakthroughs are frequently heralded as signs of success, it is critical to take a moment to consider the underlying social issues that these developments frequently ignore. and to think about how they can be applied to actually enhance humankind’s situation.

In response, government authorities, IBM Consulting, and Microsoft are working together to create a platform that will improve the provision of social services to underserved communities and bridge the gap.

Their common goal

Community challenges including violence, mental health, food insecurity, homelessness, and juvenile justice arise daily. Many government agencies are using transformative approaches to create a society where everyone gets the resources and support they need. Consider the government officials who are implementing the “Care First” approach. The goal of this strategy is to divert thousands of people from the criminal justice system and place them in helpful programs designed to meet their requirements for “re-entry into society.” These initiatives support local communities by providing housing, transportation, access to drug rehab centres, and other necessary services.

Some creative leaders are committed to improving maternal health, reducing violence, and preparing young people in transition for success. In order to integrate social services across health, education, and other crucial sectors, a larger percentage of leaders are adopting a whole-person care strategy and concentrating on the community as a whole rather than on particular groups.

Announcing Connect360 IBM

While a plethora of diverse services and initiatives are accessible to several government organisations worldwide. They are unable to provide these systems’ power to the communities and individuals who actually need it, though. Users frequently have to work with individual systems and services since the underlying data, technology, and delivery models were designed in silos.

Connect360 IBM enables the delivery of integrated social services, converts data into useful information, and encourages cooperation across agencies. The five main objectives of this solution are as follows:

Establish an electronic information exchange system to facilitate cooperation.

Increased public access to resources and services through information sharing

Enable the public to actively participate in the selection and provision of services.

Boost your ability to make decisions by integrating data and using business analytics.

Improve the area’s interoperability to access community data.

A software called Connect360 IBM allows data from different systems inside social services agencies to be effortlessly integrated. The purpose of this capability is to conform various data sources to the HSDS (Human Services Data Specification) and HL7 FHIR (Fast Healthcare Interoperability Resources) standards. This is to ensure that data is easily accessible, standardised, and protected.

In order to provide continuity and coordination of treatment for those in need, Connect360 IBM also enables interoperability of health-related data with a variety of healthcare systems by complying to the HL7 FHIR standard.

This transformation allows for easy connectivity with other apps in addition to making the data easily accessible within Connect360 IBM . Service providers can then share and update information to help those who depend on these essential services in a coordinated and efficient manner.

The amount of isolation, security, performance, scale, and dependability needed to manage sensitive workloads across a wide range of particular requirements is provided by Connect360 IBM , which is hosted on Microsoft Azure. Future improvements to Connect360 IBM will be built upon the foundation laid by IBM and Microsoft’s commitment to advancing responsible, secure, and trustworthy AI.

Changing lives using Microsoft Azure and IBM Connect360

A wide range of initiatives from several ministries, including housing, justice, mental health, social services, and health, are included in the category of social services. Their common goal is to assist people with complicated needs.

Interdepartmental cooperation is necessary to meet these objectives and improve client results. The involvement of Community-Based Organisations (CBOs) and Communication Service Providers (CSPs) via the departments’ network is another important element in attaining improved results. Departments are accountable for providing a common resource repository that their partners can use to deliver services, in addition to directly providing services.

Support for the operational model in use is provided by Connect360 IBM . The basic systems and APIs in this paradigm are provided by the government agency, and they allow access to the core systems almost instantly. Through the use of these APIs, different business applications from different departments can communicate with one another.

Every participant in this model concentrates on their primary competencies: While CSPs and CBOs can develop client-facing, service-oriented apps, the agency handles data management, sets business rules, assures compliance, and assesses performance. This strategy makes effective use of the agency’s current resources to enable the quick implementation of new healthcare and public assistance initiatives. Throughout the launch phase, the ecosystem cooperates to guarantee that clients receive services on time and without any interruptions.

“They at IBM recognise the value of efficient coordination and communication amongst government agencies,” stated Cristina Caballe Fuguet, Vice President of IBM Consulting’s Global Public Sector. Government organisations may communicate with the public, exchange information, and get input in a safe and expandable setting with Connect360 IBM. They take great pride in the way Connect360 IBM and Microsoft Azure are assisting governments everywhere in providing better services to their inhabitants and constructing more robust and resilient communities.

Case Study: How is the life of a single citizen being changed by IBM Connect 360?

Let’s look at the made-up tale of Michael, a soldier who was having difficulty adjusting to civilian life. He struggled with PTSD, mental health problems, and substance misuse before becoming imprisoned in a cycle of homelessness. How did he benefit from IBM Connect 360?

Michael created his Connect360 IBM account with the assistance of a care coordinator. After he gave details about his situation, Connect360 IBM evaluated his requirements and made recommendations that were specifically suited for him. These suggestions included services like short-term housing, drug rehab, mental health assistance, transportation, skill development, and eventually assisted him in locating employment.

Michael became more self-assured and self-sufficient with every step. He made good use of the suggested resources, took comfort in his caring network of carers, and piece by piece, he rebuilt his life. In his quest for stability, the technological answer served as more than simply a roadmap it was a continuous companion.

This story goes beyond simply one man’s journey to stability; it demonstrates how effective tools and a caring support system can actually improve a person’s life. Michael’s case serves as evidence of both the effectiveness of empathetic intervention and the prospective uses of technology in social support networks. A brighter future is not just a fantasy but a probable reality with the correct resources and assistance. Change is always possible. These results may be achieved at scale when governments, communities, Microsoft, IBM Consulting, and others work together.

Firsthand experience the transforming power

With the help of Microsoft Azure, Connect360 IBM, and the IBM Community Health user interface, this formidable solution has the ability to significantly improve people’s lives. This all-inclusive, public platform is intended to facilitate service delivery at every level. From identifying and coordinating with the service providers that can best meet an individual’s needs to accurately measuring the treatment and results received.

Angela Heise, Corporate Vice President, Worldwide Public Sector at Microsoft, stated, “IBM Connect360 ensures that every aspect of community service delivery is enhanced, fostering a more connected, efficient, and impactful system.” “They are eager to advance the solution and maintain their strategic partnership with IBM Consulting.”

This platform stands out for its ability to serve a broad spectrum of stakeholders. Community people, care coordinators, service navigators, service providers, and government officials will all find great value in its capabilities.

They advise you to personally witness the transformational potential of this solution. Make an appointment for a customised demo with your IBM Consulting and Microsoft experts to see how this solution may be adjusted to fit your particular needs and specifications.

Blockchain-Powered Solutions by Casper Labs

Casper Labs, a provider of enterprise blockchain software and services, and IBM Consulting have announced their partnership to assist clients in utilizing blockchain technology to enhance the transparency and auditability of their artificial intelligence systems. In collaboration with IBM Consulting, Casper Labs and the latter will create a new Casper Labs solution that will be built on top of blockchain technology and utilize IBM Watsonx.governance to create an extra layer of analytics and policy enforcement for managing AI training data across enterprises.

From the original model creator to the end user organization, several organizations are involved in the training, development, and deployment of generative AI models. The outputs of various organizations change as a result of new data sets being integrated or models being altered, and numerous organizations must be able to monitor, audit, and correctly identify and fix problems in addition to tracking these changes. By minimizing the possibility of unneeded data sharing across organizational boundaries and facilitating the sharing of trusted context information through metadata in the ledger indicating model changes, blockchain technology can assist organizations in sharing their knowledge.

The goal of Casper Labs’ solution is to train generative AI systems throughout organizations by monitoring and measuring highly serialized input and output data. It will be built on Casper, a tamper-resistant and highly serialized ledger, and utilizing IBM Watsonx.governance and Watsonx.ai.

Organizations can anticipate being able to better protect sensitive data stored in the solution from being accessible to external actors because of the hybrid nature of the Casper Blockchain and its permissioning system, which gives them control over who can access what data. Additionally, the solution will be designed to support version control by utilizing blockchain’s serialization capabilities. This will enable organizations to quickly and effectively roll back AI system iterations in the event that performance problems or biased outputs arise.

Casper Labs will build the solution with help from IBM Consulting’s AI governance and technology experts. The solution is anticipated to be made available to clients in beta in the first quarter of 2024, and then more widely through Casper Labs’ channels and the IBM Cloud Marketplace.

“An AI system’s efficacy is ultimately as good as an organization’s ability to govern it,” said Shyam Nagarajan, Global Partner, Blockchain and Responsible AI Leader at IBM Consulting. Businesses require solutions that reduce risk, improve explainability, and cultivate trust. IBM said that they are excited to work with Casper Labs to develop a new solution that adds a crucial layer of transparency and risk mitigation for businesses implementing AI at scale, by bringing IBM Consulting and technology to the table.

The new solution is intended to assist businesses in deploying AI responsibly at scale throughout their ecosystem of technology and service providers, including financial services, healthcare, and retail. In addition, the solution seeks to provide:   

Compliance Dashboard: A centralized dashboard designed to help organizations monitor and manage AI systems used throughout the organization in order to support their processes for adhering to ethical standards.

Tools to track the effectiveness and quality of AI systems, as well as an interface to improve the comprehensibility and transparency of AI outputs, comprise the Quality Control Toolkit.

Version control refers to the capability of “rolling back” to earlier versions of an AI system that did not exhibit problems in order to address performance or other issues.

A system for auditing AI procedures and producing thorough reports using context metadata gathered by Casper Labs’ ledger is called the Audit and Reporting System.

The CEO of Casper Labs, Mrinal Manohar, stated, “While generative AI has rightfully excited organizations for its transformative potential, its practical applications have been severely limited by an inability to monitor and react to the data feeding AI systems.” With IBM’s assistance, we’re dedicated to providing a clearer path for correcting behavior in the event of hallucinations or performance problems, as well as a better means of understanding why AI systems behave as they do. AI’s long-term potential will be determined by how well organizations comprehend, manage, and respond to ever-larger AI training data sets.

IBM Consulting and Casper Labs to Present AI Governance Solution in Davos

Mrinal Manohar and Shyam Nagarajan will present a live demonstration of this solution at The Hub in Davos on Tuesday, January 16.

Concerning Casper Labs

The industry leader in enterprise blockchain software is Casper Labs. In order to meet the scale and operational requirements of businesses, Casper Labs created the first layer-1 blockchain, which completely transparently recorded all business transactions. Casper Labs provides businesses and governments with software and services that increase revenue and drastically improve efficiency. Their goal is to construct the fundamental framework required for a completely new era of client value and commercial success.