Operationalizing Foundational Models in Data Engineering

Operationalizing use cases with foundational models involves complex data engineering to ensure effective deployment and integration.
Ashwin

Ashwin Swarup Adurthi, the Vice President of Data Science and Data Engineering at NimbleWork, Inc. (formerly Digité), delivered a compelling talk at the Data Engineering Summit 2024 in Bengaluru. As a seasoned expert with a strong background in information technology and services, Ashwin has demonstrated his prowess in leveraging data science algorithms to solve industry-specific challenges across various verticals. His talk, titled “Operationalizing Foundational Models Using Kairon,” delved into the intricacies of making data science and data engineering more functional and impactful. This article encapsulates the key points from Ashwin’s talk, providing insights into how foundational models are operationalized to drive business growth and technological advancement.

Introduction

Ashwin began his talk with a warm interaction, gauging the audience’s background in data engineering and their experience with large language models (LLMs) and generative AI. He humorously noted the dual role he plays as both a data scientist and a data engineer, highlighting the perpetual relevance of data engineers in operationalizing use cases.

Understanding Use Cases

Identifying Key Areas

Ashwin identified several key areas within a company where foundational models can have a significant impact. These include marketing, sales, product development, HR, IT, and finance. Each team targets specific personas, such as leads for marketing, customers for sales, and employees for HR, with foundational models enhancing their effectiveness.

Example Use Cases

  1. Fintech Applications: Ashwin highlighted the booming fintech sector, emphasizing the ease of deploying loan application bots on platforms like WhatsApp. He noted the trend towards using QR codes over apps for marketing, as they are more user-friendly and cost-effective.
  2. Healthcare Solutions: In collaboration with intensivists, Ashwin’s team developed a bot to assist ICU doctors and nurses with pre-surgery questionnaires. This bot ensures comprehensive data collection and generates summaries to prevent medical errors.
  3. Retail E-commerce: Large language models are revolutionizing e-commerce by enabling assisted shopping experiences on websites and social media platforms. For instance, a retail bot on Instagram can initiate user conversations based on their comments, enhancing engagement and sales.
  4. Internal Process Improvement: For HR, IT, and finance teams, foundational models streamline internal processes, improving communication and overall efficiency within the organization.

The Operational Pipeline

Functional Schematic

Ashwin presented a functional schematic of operationalizing use cases, from data and APIs to deployment interfaces on various channels. He emphasized data engineers’ critical role in this pipeline, particularly in handling the aggregation and feedback loops necessary for model refinement.

Complexity Levels

Ashwin broke down the operational pipeline into four levels of complexity:

  1. Information Retrieval Bots: Simple Q&A bots that retrieve information based on user queries.
  2. Action Flows: Bots that can perform specific actions like booking orders or logging tickets.
  3. Proactive Agents: Advanced bots that can initiate actions based on user behavior and data patterns.
  4. Community Building Systems: Bots that interact and share information to create a cohesive user experience.

Data Engineering’s Role

Data engineers play an increasingly vital role as the complexity of these use cases escalates. From managing the foundational models to ensuring seamless integration and data flow, their expertise is crucial for operational success.

Foundational Models and Their Limitations

Statistical Learning

Ashwin explained that large language models are powerful sequence learners, capable of identifying statistical patterns in data. These patterns range from word-level to document-level representations, enabling LLMs to understand and generate text.

Implicit Relationships

However, real-world applications often require understanding implicit relationships that LLMs struggle with:

  • Temporal Relationships: Understanding seasonal or time-based patterns, like crop cycles for farmers.
  • Relational Representations: Tracking relationships between characters or entities across a text, such as in literature.
  • Causal Relationships: Identifying cause-and-effect scenarios, which are not always statistically apparent.
  • Hierarchical Representations: Understanding structured information, like the chapters of a book.
  • Spatial Relationships: Recognizing geographical or locational data.
  • Thematic Representations: Extracting overarching themes or morals from a body of text.

Augmenting LLMs

To address these limitations, Ashwin suggested combining foundational models with traditional data engineering techniques, such as using Spark or Lucene, to extract and highlight these implicit relationships.

Generative AI Complexity Layers

Ashwin outlined four layers of generative AI complexity:

  1. Foundational Models: Basic models trained on large datasets.
  2. Fine-tuning and Prompt Engineering: Customizing models for specific tasks.
  3. Pipeline Engineering and Observability: Ensuring seamless data flow and monitoring model performance.
  4. Generative AI Applications: Advanced applications combining retrieval augmented generation with implicit relationship extraction.

Practical Application and Observability

Real-world Example

Ashwin showcased a real-world example involving user behavior tracking on a website. By visualizing user journeys as 3D graphs, his team could identify patterns and feed this data back into their models, enabling proactive engagement and improved user experiences.

Data Engineering in Action

This process, from data collection to model deployment and feedback, is where data engineering truly shines. Ashwin emphasized the importance of a robust data pipeline to handle the complexity and scale of modern AI applications.

Conclusion

Ashwin Swarup Adurthi’s talk at the Data Engineering Summit 2024 provided invaluable insights into the operationalization of foundational models in data engineering. By highlighting practical use cases, the role of data engineers, and the intricacies of handling implicit relationships, Ashwin demonstrated the transformative potential of generative AI. As companies continue to innovate and integrate these technologies, the collaborative efforts of data scientists and data engineers will be crucial in driving forward the next wave of technological advancement.

Transform your team into AI powerhouses

Targeted suite of solutions for enterprises aiming to harness the power of AI. MachineHack is your partner in building a future-ready workforce adept in artificial intelligence.

Online AI Hackathons to accelerate innovation

With MachineHack you can not only find qualified developers with hiring challenges but can also engage the developer community and your internal workforce by hosting hackathons.