#ibmdpk

How OPEA and IBM DPK Enable Custom ChatQnA Retrieval Augmented Generation

GenAI is changing application development and implementation with intelligent chatbots and code generation. However, organisations often struggle to match commercial AI capabilities with corporate needs. Standardisation and customization to accept domain-specific data and use cases are important GenAI system development challenges. This blog post addresses these difficulties and how the IBM Data Prep Kit (DPK) and Open Platform for Enterprise AI (OPEA) designs may help. Deploying and customizing a ChatQnA application using a retrieval augmented generation (RAG) architecture will show how OPEA and DPK work together.

The Value of Standardisation and Customization

Businesses implementing generative AI (GenAI) applications struggle to reconcile extensive customization with standardisation. Balance is needed to create scalable, effective, and business-relevant AI solutions. Companies creating GenAI apps often face these issues due to lack of standardisation:

Disparate models and technology make it hard to maintain quality and reliability across corporate divisions.

Without common pipelines and practices, expanding AI solutions across teams or regions is challenging and expensive.

Support and maintenance of a patchwork of specialist tools and models strain IT resources and increase operational overhead.

Regarding Customization

Although standardisation increases consistency, it cannot suit all corporate needs. Businesses operate in complex contexts that often span industries, regions, and regulations. Off-the-shelf, generic AI models disappoint in several ways:

AI models trained on generic datasets may perform badly when confronted with industry-specific language, procedures, or regulatory norms, such as healthcare, finance, or automotive.

AI model customization helps organisations manage supply chains, improve product quality, and tailor consumer experiences.

Data privacy and compliance: Building and training bespoke AI systems with private data keeps sensitive data in-house and meets regulatory standards.

Customization helps firms innovate, gain a competitive edge, and discover new insights by solving challenges generic solutions cannot.

How can we reconcile uniformity and customization?

OPEA Blueprints: Module AI

OPEA, an open source initiative under LF AI & Data, provides enterprise-grade GenAI system designs, including customizable RAG topologies.

Notable traits include:

Modular microservices: Equivalent, scalable components.

End-to-end workflows: GenAI paradigms for document summarisation and chatbots.

Open and vendor-neutral: Uses open source technology to avoid vendor lockage.

Flexibility in hardware and cloud: supports AI accelerators, GPUs, and CPUs in various scenarios.

The OPEA ChatQnA design provides a standard RAG-based chatbot system with API-coordinated embedding, retrieval, reranking, and inference services for easy implementation.

Simplified Data Preparation with IBM Data Prep Kit

High-quality data for AI and LLM applications requires a lot of labour and resources. IBM's Data Prep Kit (DPK), an open source, scalable toolkit, facilitates data pretreatment across data formats and corporate workloads, from ingestion and cleaning to annotation and embedding.

DPK allows:

Complete preprocessing includes ingestion, cleaning, chunking, annotation, and embedding.

Scalability: Apache Spark and Ray-compatible.

Community-driven extensibility: Open source modules are easy to customize.

Companies may quickly analyse PDFs and HTML using DPK to create structured embeddings and add them to a vector database. AI systems can respond precisely and domain-specifically.

ChatQnA OPEA/DPK deployment

The ChatQnA RAG process shows how standardised frameworks and customized data pipelines operate in AI systems. This end-to-end example illustrates how OPEA's modular design and DPK's data processing capabilities work together to absorb raw texts and produce context-aware solutions.

This example shows how enterprises may employ prebuilt components for rapid deployment while customizing embedding generation and LLM integration while maintaining consistency and flexibility. This OPEA blueprint may be used as-is or modified to meet your architecture utilising reusable pieces like data preparation, vector storage, and retrievers. DPK loads Milvus vector database records. If your use case requires it, you can design your own components.

Below, we step-by-step explain how domain-specific data processing and standardised microservices interact.

ChatQnA chatbots show OPEA and DPK working together:

DPK: Data Preparation

Accepts unprocessed documents for OCR and extraction.

Cleaning and digestion occur.

Fills vector database, embeds

OPEA—AI Application Deployment:

Uses modular microservices (inference, reranking, retrieval, embedding).

Easy to grow or replace components (e.g., databases, LLM models)

End-user communication:

Context is embedded and retrieved upon user request.

Additional background from LLM responses