What product people need to know about LangChain

In the rapidly evolving landscape of artificial intelligence, Large Language Models (LLMs) like Open AI’s ChatGPT and Google’s Bard are revolutionizing the way we engage with technology. Tech giants and SaaS companies alike are racing to harness the power of LLMs to create more intelligent and useful applications. However, the true potential of these AIs is unlocked not when they stand alone, but when they are integrated with other tools, plugins, and knowledge bases.

Enter LangChain - a cutting-edge framework designed to give superpowers to language models, and redefine the boundaries of what's possible in the world of AI.

But what is LangChain, and why is it a game-changer? Why is everyone suddenly talking about it?

The importance of LangChain cannot be overstated. As you’re about to learn in this ultimate guide, LangChain gives seemingly magical abilities to companies who want to integrate LLMs into their software. From autonomous Agents (hello Skynet), to the ability to retrieve & understand incredibly large volumes of data instantly.

As we continue to advance in the field of AI, language models are becoming increasingly integral to a wide range of apps and services we use today. From chatbots and virtual assistants to content generation and beyond, LangChain, with its ability to leverage the power of these models, is playing a pivotal role.

In this ultimate guide, we'll demystify all-the-things LangChain, diving deep into its framework, components, and real-world use cases like our very own HelpHub. And don't worry, we've got you covered even if you're not an engineer or a computer scientist. We'll break it all down, making the complex super simple and accessible.

Welcome to the ultimate guide to LangChain.

LangChain Lingo: A Crash Course in Key Concepts

The name 'LangChain' is derived from a fusion of 'Lang' (short for language) and 'Chain', symbolizing the linking (or chaining) of different elements to create advanced applications around these LLMs. There are several key concepts that form the backbone of this functionality that we’re about to cover. These concepts are not just technical jargon, but they represent the unique strategies and techniques that LangChain uses to carry out all of its magic.

Note: From here on out, when you see “model” referred to, just know that we’re referencing an LLM like OpenAI’s GPT-4 for example. When you see “prompting”, that’s just a way of saying how you “talk” to the models by asking specific questions or making certain statements to guide it.

Let’s get into the Key Concepts of LangChain:

1. Chain of Thought (CoT):
   Picture a language model that thinks out loud, providing a step-by-step breakdown of its thought process. That's Chain of Thought. It's a prompting technique that encourages the model to generate a series of intermediate reasoning steps, providing a glimpse into its cognitive process. It's like having a conversation with the model where it explains its reasoning, making it a more effective tool for tasks that require complex problem-solving.
2. Action Plan Generation:
   This is a prompting technique that uses a language model to generate actions to take. The results of these actions can then be fed back into the language model to generate a subsequent action. It's like having a dynamic conversation with the model where it suggests actions and then reacts to the outcomes of those actions. This makes the model more interactive and responsive, capable of adapting to changing circumstances and requirements.
3. ReAct:
   You've seen how LangChain can generate a sequence of actions with Action Plan Generation, and how it can provide a glimpse into its cognitive process with Chain of Thought. Now, imagine if you could have the best of both worlds. That's where ReAct comes into play. ReAct, short for "Reasoning and Action", is like a dynamic dance between thought and action. It's not just about the model suggesting an action, it's about understanding the 'why' behind the action. It's about the model taking a moment to contemplate, to reason, and then to act. This creates a more insightful dialogue with the model, making your interaction not just a series of commands and responses, but a dynamic conversation.
4. Self-ask:
   Now imagine a language model that doesn't just respond to prompts, but actively asks itself follow-up questions. That's Self-ask. It's a prompting method that encourages the model to be more proactive and inquisitive internally. These questions are not directed to users or external entities like search engines. Instead, they are used “in the mind” of the model to guide the generation of subsequent responses. It's about transforming the model from a passive responder to an active learner, making it a more effective tool for tasks that require exploration and learning or when the model doesn’t know the right answer.
5. Prompt Chaining:
   This involves combining multiple LLM calls, with the output of one step being the input of the next - thus creating the chaining effect. It's like having a relay race where the baton is passed from one runner to the next, each contributing to the overall progress of the team.
6. Memetic Proxy:
   This concept leverages the language model's ability to simulate well-known figures or situations to shape its responses. By framing the discussion in a context that the model recognizes, such as a conversation between a student and a teacher or a moral question posed to a famous philosopher, the model is encouraged to respond in a certain way. This method effectively encodes complex tasks and assumptions about the context of the question, making the model more adaptable and responsive. It's about tailoring the model's responses to fit the context of the conversation and using cultural information and well-known scenarios to guide the model's behavior.
7. Self Consistency:
   This concept makes AI responses more reliable by creating multiple solutions and choosing the most consistent one. It helps ensure the AI's answers make sense and don't contradict each other, making it a more trustworthy and reliable tool.
8. Inception (First Person Instruction):
   This is a concept that encourages the model to think a certain way by including the start of the model’s response in the prompt. It's about guiding the model's thought process, making it a more effective tool for tasks that require a specific line of reasoning or approach. For example, you might ask the model to “Tell me a joke. Make it start with “Knock, knock”. Where “Make it start with Knock Knock” is the guiding instruction you’re giving.
9. MemPrompt:
   This concept is much like a personal assistant that learns from past interactions. It keeps a record of times when it misunderstood what you were asking for, along with your feedback on how to do better. The next time you ask something similar, MemPrompt uses this 'memory' to improve its response. It's a way for the system to continuously improve and adapt to your needs, making it more reliable and effective over time.

So far, we've explored the Key Concepts that underpin LangChain, which are the theoretical foundations and innovative AI techniques that guide its operation. These concepts provide the 'why' and 'what' of LangChain - why it works the way it does and what it aims to achieve. As we transition into the Components section, we'll be shifting our focus to the 'how'.

The Power of LangChain: Understanding the Core Components that Drive the System's Capabilities

As we delve deeper into the world of LangChain, we encounter its core framework, a meticulously designed structure that brings together various elements to create a powerful tool for language model applications. This framework is built around two main ideas: Components and Use-Case Specific Chains.

Components are the building blocks of LangChain. They are the individual elements that come together to create the overall system. These components include Models, Prompts, Indexes, Memory, Chains, and Agents. Each component plays a specific role in the LangChain ecosystem, and understanding these roles is crucial for grasping the full potential of LangChain.

On the other hand, Use-Case Specific Chains are the “get started quickly” solutions that LangChain offers. Just think of what you want to build, pick the chain that most closely resembles your goal, and build from there. They are pre-built, customized chains that are designed to fit specific use cases like Personal Assistants, ChatBots, Extraction and Summarization, and more, making it a truly versatile tool.

In the following sections, we'll take a closer look at these components and use-case-specific chains, shedding light on their roles, their importance, and how they come together to make LangChain the powerful tool that it is. Buckle up, as we're about to dive deep into the heart of LangChain!

Schema: The Blueprint of LangChain

The schema in LangChain is the underlying structure that guides how data is interpreted and interacted with. It's very similar to a blueprint of a building, outlining where everything goes and how it all fits together.

As of today, the primary interface for interacting with language models is through text. This is where the concept of "text in, text out" comes into play. As an oversimplification, many models in LangChain operate on this principle. You input text, the model processes it, and then outputs text. Therefore, understanding the Schema is fundamental as it forms the basis of how LangChain structures and interprets data. It's like setting out the rules of a game before you start playing.

Models: The Engines of LangChain

We’ve already mentioned models quite a lot so far because they’re truly the engines that drive LangChain’s capabilities. However, it’s worth digging into things a little deeper to get a handle on just how integral these models are. After all, they process all the inputs and generate all the outputs - without them, LangChain wouldn’t exist.

LangChain employs three primary types of models to use and interact with:

• Large Language Models (LLMs)
• Chat Models, and
• Text Embedding Models.

Each of these models plays a unique role in the LangChain ecosystem, contributing to its versatility and power.

Large Language Models (LLMs)

LLMs, such as OpenAI’s GPT, Google’s PaLM, and META’s LLaMA, form the core of LangChain. These models are created by training on vast amounts of text data, learning patterns, and structures of language to generate meaningful output. They process language input and generate responses, enabling dynamic and interactive language applications. Each LLM has its unique strengths, making them versatile tools for a variety of tasks within the LangChain ecosystem.

Chat Models

Chat Models, on the other hand, offer a more structured approach. Although they are typically backed by a language model, the way you interact with them is much more structured. They take a list of messages as input and return a message as output. Generally, each message in the input list has two properties: 'role' and 'content'. The 'role' is usually made up of 'system', 'user', or 'assistant', and 'content' contains the text of the message. This is exactly how ChatGPT works! This defined structure allows for more nuanced and interactive conversations with the LLM and gives it a human-like chat feeling.

It’s important to note that while both LLMs and Chat Models are used for generating text, the key difference lies in the way they handle inputs and outputs. LLMs work with single strings of text, while Chat Models work with structured lists of messages, making them more suitable for interactive dialogues like back-and-forth real-time conversations.

Text Embedding Models

The third and final type of Model component that LangChain utilizes are the Text Embedding Models. Text Embedding Models are a bit like language translators, but instead of translating from one language to another, they translate from text to numbers. They take text as input and turn it into a list of numbers (or 'floats') creating a numerical representation of the original text and storing them in a database.

This conversion of text into numbers is particularly useful because it means the model can now access the text in what’s called a Vector Database (picture a 3D space where the LLM can “think and look” in all directions of the space - forward, back, up, down, left, right, in and out). This 4D database allows for a semantic-style search, where seemingly unrelated pieces of text can be compared for similarity across the vector space.

In the simplest terms, it’s a way in which an LLM can derive meaning from text without a human telling it what the meaning should be. Funnily enough, this is kind of like how our human brain works. We convert the inputs of our senses (text) into electrical signals (numerical values) and store them in our brains (Vector Databases).

Whether it’s an LLM, Chat Model, or Text Embedding Model, they all work together as the engines that power LangChain’s functionality, and understanding them is key to harnessing the full potential of the system.

In the real-world application of this, you might see them integrated into a chatbot as we did with HelpHub. Initially, we use a Chat Model to process user inputs and generate appropriate responses. In the event that HelpHub encounters a complex question, it calls upon an LLM to generate a more nuanced response. We also use, a Text Embedding Model to upload and convert the user's input (for example, their Help Docs, support resources, SOPs, etc) into a numerical format, which is stored in a Vector Database. When the chatbot needs to retrieve relevant information, it uses a search tool to efficiently scan the Vector Database and find the correct answer. Try HelpHub here and you’ll see this live in action.

Prompts: Directing the Flow of Information in LangChain

You might have already heard the term “prompts” or “prompting” being thrown around online lately because Prompts are fundamentally the entry point to interacting with LLMs. Prompts, in the simplest terms, are the questions or statements that you feed into a model. They're like the steering wheel of a car, guiding the model in the direction you want it to go. When it comes to LangChain, the use of prompts is shaped by three essential aspects, PromptTemplates, Example Selectors, and Output Parsers.

PromptTemplate

A PromptTemplate is like a recipe for creating the input to the model. It's not just about what you ask the model, but how you ask it. A PromptTemplate can guide the model to present its response in a specific way, making the output more useful for your specific needs.

Using this “recipe” format, a prompt can contain specific instructions for the language model, a question for the model, and even a set of "few-shot" examples. (Few-shot examples are essentially a small set of examples that you provide to the model with context, helping it understand the kind of response you're looking for.)

Example Selectors

Adding to this functionally, LangChain introduces Example Selectors. These tools allow you to include examples in your prompts. But instead of these examples being fixed (like the few-shot earlier), Example Selectors dynamically select them based on user input, adding an extra layer of adaptability to your prompts.

Output Parsers

Lastly, we have Output Parsers. These tools help structure the model's responses. They instruct the model on how to format its output and then transform this output into the desired format. This could be as simple as extracting a specific piece of information from the model's response, or as complex as transforming the response into a structured data table, graph, or even coding language.

Let's imagine a real-world scenario. Say you're a Product Marketer, and you want to use your LangChain-powered app to generate a list of potential features for a new product. You could use a PromptTemplate to instruct the model to generate its ideas in the form of a bulleted list. An Example Selector could dynamically select examples based on your input, adding depth and relevance to the model's responses. And an Output Parser could structure these responses into a clear, organized format that's easy to review and discuss with your team.

In a nutshell, Prompts in LangChain serve as navigational aids, directing the dialogue with the model, molding its responses, and maximizing the usefulness of its outputs. They act as both the blueprint and the safety measures that transform LangChain and LLMs into robust, flexible, and user-tailored instruments for engaging with language models.

Indexes: The Efficient Librarians of LangChain

In the vast digital library that is LangChain, Indexes serve as the knowledgeable librarian. They organize and retrieve information efficiently, working behind the scenes to ensure your interaction with LangChain is smooth and productive.

Imagine you're in a library with thousands of books, and you're looking for a specific one. Without a librarian or an index system, it would be a daunting task. Similarly, LangChain deals with a massive amount of data, and without indexes, finding specific information would be like finding a needle in a haystack.

Indexes in LangChain structure documents so that LLMs can interact with them effectively. This interaction often involves a "retrieval" step, akin to asking the librarian for a specific book. The librarian, or in this case, the index, returns the most relevant documents based on your query.

But how does this 'librarian' know which document is the most relevant? That’s where our Vector Databases come into play again. VectorDBs power the indexes, enabling them to understand and retrieve the most relevant documents. It's like having a super-smart librarian who not only knows where every book is but also understands the content of each book to give you the best recommendation.

Indexes in LangChain are not just a single entity but a system made up of several components, each playing a unique role. Let’s break down some of the specifics:

Document Loaders:

Think of Document Loaders as the delivery trucks that bring new books to our library. They are responsible for loading documents from various sources into LangChain. Whether the data comes from a local file, a database, or an online source, Document Loaders ensure that it's properly loaded into the system, ready to be indexed and retrieved when needed.

Text Splitters:

Once the books (documents) are in the library, they need to be organized in a way that makes sense. That's where Text Splitters come in. They break down large chunks of text into smaller, more manageable pieces. This is akin to dividing a long novel into chapters, making it easier to find specific sections.

The reason for this division is due to the context window of language models. Language models can only consider a certain amount of text at a time, known as the context window. By splitting the text into smaller pieces, we ensure that the language model can effectively process the information within its context window, thereby maximizing the accuracy and relevance of the results.

VectorStores:

VectorStores is just a technical name for Vector Databases as we chatted about earlier. They are the heart of the indexing system in LangChain. They are the most common type of index, which relies on embeddings. Remember when we discusses text being converted into a numerical value earlier? That’s what embedding is.

If our library analogy still holds, think of embeddings in VectorStores as the unique code assigned to each book. This code, or numerical value, represents the content of the document, making it easier to match with relevant queries.

Retrievers:

Finally, we have Retrievers, the librarians of our digital library. They are the interface used for fetching relevant documents to combine with language models. When a query comes in, the Retriever sifts through the indexes, finds the most relevant documents, and presents them for further processing by the language model.

In the real world, let's say you're a Customer Success Lead, and you're using LangChain to analyze customer feedback. The feedback is vast and varied, just like the books in a library. With the help of indexes, you can quickly retrieve specific feedback related to a particular feature of your product. This efficient retrieval of information allows you to gain insights faster and make data-driven decisions.

To wrap things up, all you need to know is that Indexes in LangChain are a powerful tool that structures and retrieves data efficiently. They are the unsung heroes that ensure your interaction with LangChain is smooth, efficient, and productive.

Memory: The Continuity Keeper of LangChain

In the world of LangChain, Memory plays a pivotal role in creating a seamless and interactive experience. Just like human memory, LangChain's memory is responsible for retaining and recalling information.

Imagine you're having a conversation with a friend. You don't start each sentence from scratch, forgetting everything that was said before. Instead, you remember the previous parts of the conversation and respond accordingly. This is similar to how memory works in LangChain.

But here's the subtle twist - LangChain's memory is divided into two types: Short Term and Long Term.

Short-Term Memory:

Short-Term Memory is like a notepad where you jot down things you need to remember for a short while. In LangChain, short-term memory keeps track of the current conversation. It's like remembering what was said a few moments ago in a conversation, which is crucial for understanding the context and responding appropriately.

Long-Term Memory:

Long-Term Memory is like a vast library of information that's been gathered over a long period. In LangChain, long-term memory is used to remember interactions between the AI and the user across different conversations. It's like remembering a friend's preferences or past experiences, which can be used to provide more personalized and relevant responses.

Indexes vs. Memory: The Catalog System vs. The Continuity Keeper

Now, you might be wondering, how is Memory different from the concept of Indexes or Vector Databases we discussed earlier? Here’s the distinction:

• Indexes and vector databases are like the library's catalog system - they help in organizing and retrieving documents efficiently. They don't remember the sequence of interactions but focus on finding the most relevant response for the current query.
• Memory, on the other hand, is like the librarian's knowledge - it helps in remembering past interactions and providing context for future conversations. In other words, understanding the context based on what has been said or asked before.

In the real world, suppose you're a Product Manager using LangChain for product development. You're in the process of gathering user feedback to improve your product. With LangChain's memory feature, the AI can remember past interactions with each user, understand the context of their feedback based on their usage history, and provide personalized responses to their suggestions or concerns. This allows you to gain deeper insights into user needs and preferences, helping you make more informed decisions about product improvements.

Chains: The Master Orchestrators of LangChain

Chains bring together all the different components discussed so far to create an interaction that generates meaningful and relevant responses from the language models. They serve as the master orchestrators, helping to create a cohesive and usable system.

You've already been introduced to several concepts that form the building blocks of Chains, including Chain of Thought, Action Plan Generation, ReAct, Self-ask, and Prompt Chaining. Now, let's dive even deeper.

A Chain is like a production line in a factory, where each component performs a specific task, and the final product is the result of all these tasks being performed in a particular sequence. There are two common types of chains, LLMChain and Index-related Chains.

LLMChain

By far, the most common type of Chain is an LLMChain. It combines a lot of the ingredients we’ve discussed so far which include:

• A PromptTemplate: Prepares the raw materials (the input)
• A Model (either an LLM or a ChatModel): Processes these materials, and
• Guardrails (optional): The quality checks that ensure the final product meets the required standards.

You can think of LLMChains as pretty much the default way you’d typically be interacting and engaging with models.

Index-related Chains

Depending on your use case, you’ll also come across Index-related Chains as well. These chains are specifically designed for interacting with Indexes, combining your own data (stored in the Indexes) with LLMs. A prime use case is querying your own documents (much like we do with HelpHub).

There are four main types of Index-related chains in LangChain, each with its unique approach to handling multiple documents. Let's break them down:

1. Stuffing: This is the simplest method, where all the related data is “stuffed” into the prompt as context to pass to the language model.
   
   Pros: Only makes a single call to the LLM. When generating text, the LLM has access to all the data at once.
   Cons: Most LLMs have a context length, and for large documents (or many documents) this will not work as it will result in a prompt larger than the context length.
   
2. Map Reduce: This method involves running an initial prompt on each chunk of data (for summarization tasks, this could be a summary of that chunk; for question-answering tasks, it could be an answer based solely on that chunk). Then a different prompt is run to combine all the initial outputs.
   
   Pros: Can scale to larger documents (and more documents). The calls to the LLM on individual documents are independent and can therefore be carried out at the same time.
   Cons: Requires many more calls to the LLM than Stuffing. Loses some information during the final combined call.
   
3. Refine: This method involves running an initial prompt on the first chunk of data, generating some output. For the remaining documents, that output is passed in, along with the next document, asking the LLM to refine the output based on the new document.
   
   Pros: Can pull in more relevant context, and may be more accurate than Map Reduce.
   Cons: Requires many more calls to the LLM than Stuffing. The calls are also not independent, meaning they cannot be carried out at the same time as Map Reduce. There are also some potential rules you need to follow on the ordering of the documents.
   
4. Map-Rerank: This method involves running an initial prompt on each chunk of data, that not only tries to complete a task but also gives a score for how certain it is in its answer. The responses are then ranked according to this score, and the highest score is returned back to you.
   
   Pros: Similar pros as Map Reduce, though requires fewer calls compared to Map Reduce too.
   Cons: Cannot combine information between documents. This means it is most useful when you expect there to be a single simple answer in a single document.

These chains offer a range of options to handle multiple documents, catering to different needs and scenarios. Whether you're dealing with a small amount of data or a large document corpus, LangChain's Index-related chains provide a flexible and efficient way to interact with your data and LLMs.

Agents: The Decision-Makers in LangChain

Ever seen Terminator? Heard of Skynet? Worried AI might take over the world? Well, buckle up, because you’re about to learn what goes on underneath the hood of these AI systems to give credibility to those Sci-Fi fantasies.

Agents are the autonomous entities in LangChain that act on behalf of users, interacting with other agents and tools within the ecosystem. They are the decision-makers, choosing which tools to leverage based on user inputs and objectives. This adaptability and on-the-fly intelligence make agents a crucial component in handling complex, multi-step tasks.

Roles and Types of Agents

The role of Agents in LangChain is like the conductors of an orchestra, coordinating the various components to create a harmonious interaction. They are responsible for interpreting user inputs, deciding which tools to use, and managing the flow of information within the system. This adaptability and intelligence make agents a crucial component in handling complex, multi-step tasks.

LangChain supports two main types of agents: Action Agents and Plan-and-Execute Agents.

• Action Agents are more conventional and good for small tasks. They are like the sprinters in a race, quick and focused on a single task
• Plan-and-Execute Agents are more suited to complex or long-running tasks. They handle the initial planning step, which helps to maintain long-term objectives and focus. They are like marathon runners, strategic and endurance-focused.

In real-world scenarios, agents can be used in various ways. For instance, a Product Manager might use an agent to analyze customer feedback and generate insights. The agent could decide to use a sentiment analysis tool to understand the overall sentiment of the feedback, and then use a summarization tool to condense the feedback into key points.

Similarly, a Designer might use an agent to generate design ideas based on user inputs. The agent could decide to use a brainstorming tool to generate a list of ideas, and then use a ranking tool to prioritize the ideas based on certain criteria.

The Future of Agents: The Sky(net) is the Limit

As LangChain continues to evolve, the role of agents is expected to become even more significant. With advancements in AI and machine learning, agents could become capable of making more complex decisions, handling more sophisticated tasks, and providing more personalized and relevant responses. So, while we're not quite at the Skynet level yet, the future of agents in LangChain is certainly exciting and full of potential.

Enjoy Video More Than Reading?

If you’re more of a visual learner and want to soak in the ultimate video series on LangChain fundamentals, you're going to want to check out Greg Kamradt's amazing LangChain Fundamentals series on YouTube:

Conclusion: The Power of LangChain

As we wrap up this guide, it's clear that LangChain is more than just a tool—it's a revolutionary framework that's transforming the way we interact with language models. It's about harnessing the power of advanced language models like GPT-4 and making them accessible, adaptable, and highly functional.

Throughout this guide, we've explored the core concepts of LangChain, from Chains and Agents to the unique prompting techniques that guide the conversation with the model. We've seen how LangChain enables the creation of complex and dynamic language applications, capable of handling a wide range of use cases.

But the true power of LangChain lies in its versatility. Whether you're a product marketer looking to generate new product features, a data analyst seeking insights from large datasets, or a developer building a chatbot, LangChain provides the tools and framework to make it happen.

For technical and non-technical product people, LangChain opens up a world of possibilities. It's about taking the complexity out of interacting with language models and making it as simple as crafting a prompt. And with the ability to chain together multiple prompts and tools, you can create nuanced and sophisticated interactions that push the boundaries of what's possible with language models.

So, as we conclude this guide, we encourage you to explore LangChain and its potential. Dive into the concepts, experiment with the tools, and see for yourself how LangChain can transform the way you interact with language models. The future of language applications is here, and it's powered by LangChain.