LlamaIndex vs LangChain vs Mirascope: An In-Depth Comparison

In the context of building Large Language Model (LLM) applications—and notably Retrieval Augmented Generation (RAG) applications—the consensus seems to be that:

• LlamaIndex excels in scenarios requiring robust data ingestion and management.
  ‍
• LangChain is suitable for chaining LLM calls and for designing autonomous agents.

In truth, the functionalities of both frameworks often overlap. For instance, LangChain offers document loader classes for data ingestion, while LlamaIndex lets you build autonomous agents.

Which framework you actually choose will depend on a lot of different factors, such as:

• What kind of application you’re building, e.g., are you building a RAG or other app requiring vectorization and storage of large amounts of data?
  ‍
• Which LLM you’re using, i.e., which framework offers better integration with the LLM you want.
  ‍
• What features you’re looking to use, e.g., LangChain offers high-level abstractions for building agents, whereas this would require more code in LlamaIndex.

But even here, decisions about which framework to use can still be based mostly on subjective preferences, personal experience, or even hearsay.

To help sort out the ambiguity, this article points out some of the similarities and differences between LlamaIndex and LangChain in terms of four key differentiators:

• How do you accomplish prompting in each framework?
  ‍
• What RAG-specific functionality is provided?
  ‍
• How easy is it to scale your solution when building production-grade LLM applications?
  ‍
• How is chaining implemented?

In addition, we’ll compare and contrast these libraries with Mirascope, our own Python toolkit for building with LLMs.

Prompting the LLM

LlamaIndex: Retrieves Indexed Information for Sophisticated Querying

Prompting in LlamaIndex is based on `QueryEngine`. This is a class that takes your input, then goes and searches its index for information related to your input, and sends both together as a single, enriched prompt to the LLM.

Central to LlamaIndex is, of course, its index of vectorized information, which is a data structure consisting of documents that are referred to as “nodes.” LlamaIndex offers several types of indices, but the most popular variant is its VectorStoreIndex that stores information as vector embeddings, to be extracted in the context of queries via semantic search.

Sending such enriched prompts to the LLM (queries plus information from the index) have the advantage of increasing LLM accuracy and reducing hallucinations.

As for the prompts themselves, LlamaIndex makes available several predefined templates for you to customize to your particular use case. Such templates are particularly suited for developing RAG applications.

In this example reproduced from their documentation, the `PromptTemplate` class is imported and can then be customized with a query string:
‍

1from llama_index.core import PromptTemplate
2
3template = (
4    "We have provided context information below. \n"
5    "---------------------\n"
6    "{context_str}"
7    "\n---------------------\n"
8    "Given this information, please answer the question: {query_str}\n"
9)
10qa_template = PromptTemplate(template)
11
12# you can create text prompt (for completion API)
13prompt = qa_template.format(context_str=..., query_str=...)
14
15# or easily convert to message prompts (for chat API)
16messages = qa_template.format_messages(context_str=..., query_str=...)

LangChain: Offers Prompt Templates for Different Use Cases

Similar to LlamaIndex, LangChain offers a number of prompt templates corresponding to different use cases:

• `PromptTemplate` is the standard prompt for many use cases.
  ‍
• `FewShotPromptTemplate` for few-shot learning.
  ‍
• `ChatPromptTemplate` for multi-role chatbot interactions.

LangChain encourages you to customize these templates, though you’re free to code your own prompts. You can also merge different LangChain templates as needed.

An example of a `FewShotPromptTemplate` from LangChain’s documentation:
‍

1from langchain_core.prompts.few_shot import FewShotPromptTemplate
2from langchain_core.prompts.prompt import PromptTemplate
3
4examples = [
5    {
6        "question": "Who lived longer, Muhammad Ali or Alan Turing?",
7        "answer": """
8Are follow up questions needed here: Yes.
9Follow up: How old was Muhammad Ali when he died?
10Intermediate answer: Muhammad Ali was 74 years old when he died.
11Follow up: How old was Alan Turing when he died?
12Intermediate answer: Alan Turing was 41 years old when he died.
13So the final answer is: Muhammad Ali
14""",
15    },
16    {
17        "question": "When was the founder of craigslist born?",
18        "answer": """
19Are follow up questions needed here: Yes.
20Follow up: Who was the founder of craigslist?
21Intermediate answer: Craigslist was founded by Craig Newmark.
22Follow up: When was Craig Newmark born?
23Intermediate answer: Craig Newmark was born on December 6, 1952.
24So the final answer is: December 6, 1952
25""",
26    },
27  ]

LangChain’s prompting templates are straightforward to use; however the framework provides neither automatic error checking (e.g., for prompt inputs), nor inline documentation for your code editor. It’s up to developers to handle errors.

When it comes to prompt versioning, LangChain offers this capability in its LangChain Hub, which is a centralized prompt repository that implements versioning as commit hashes.

Mirascope: One Prompt That Type Checks Inputs Automatically

Rather than attempt to tell you how you should formulate prompts, Mirascope provides its `BasePrompt` class for you to extend according to your use case. 

Our BasePrompt allows you to automatically generate prompt text using the methods `prompt.messages()` or `str(prompt)`, which formats `prompt_template` according to BasePrompt’s fields and properties.

The `@property` decorator lets you directly access the class with dynamically created attributes as though they were regular, static fields. This enables the template to incorporate current values from these properties when generating text with `prompt.messages()` or `str(prompt)`, using specified template variables. 

As a result, you can integrate complex calculations or conditional formatting directly into your output without manually updating the content each time the underlying data changes.

In the example below, `list` and `list[list]` are automatically formatted with `\n` and `\n\n` separators, before being stringified:
‍

1from mirascope import BasePrompt
2
3class BookRecommendationPrompt(BasePrompt):
4    prompt_template = """
5    Can you recommend some books on the following topic and genre pairs?
6    {topics_x_genres}
7    """
8
9    topics: list[str]
10    genres: list[str]
11
12    @property
13    def topics_x_genres(self) -> list[str]:
14        """Returns `topics` as a comma separated list."""
15        return [
16            f"Topic: {topic}, Genre: {genre}"
17            for topic in self.topics
18            for genre in self.genres
19        ]
20
21
22prompt = BookRecommendationPrompt(
23    topics=["history", "science"], genres=["biography", "thriller"]
24)
25print(prompt)
26#> Can you recommend some books on the following topic and genre pairs?
27#  Topic: history, Genre: biography
28#  Topic: history, Genre: thriller
29#  Topic: science, Genre: biography
30#  Topic: science, Genre: thriller

Mirascope’s `BasePrompt` class itself is an extension of Pydantic’s `BaseModel` class, which ensures that the inputs are correctly typed according to the schema defined in Pydantic’s `BaseModel`, and gracefully handled.

You can write your own custom validation using Pydantic’s `AfterValidator` class. This is helpful for cases like verifying whether data processing rules are compliant with GDPR regulations:
‍

1from enum import Enum
2from typing import Annotated, Type
3
4from pydantic import AfterValidator, BaseModel, ValidationError
5
6from mirascope.anthropic import AnthropicExtractor
7
8class ComplianceStatus(Enum):
9    COMPLIANT = "compliant"
10    NON_COMPLIANT = "non_compliant"
11
12class GDPRComplianceChecker(AnthropicExtractor[ComplianceStatus]):
13    extract_schema: Type[ComplianceStatus] = ComplianceStatus
14
15    prompt_template = """
16    Is the following data processing procedure compliant with GDPR standards?
17    {procedure_text}
18    """
19
20    procedure_text: str
21
22def validate_compliance(procedure_text: str) -> str:
23    """Check if the data processing procedure is compliant with GDPR standards."""
24    compliance_checker = GDPRComplianceChecker(procedure_text=procedure_text)
25    compliance_status = compliance_checker.extract()
26    assert compliance_status == ComplianceStatus.COMPLIANT, "Procedure is not GDPR compliant."
27    return procedure_text
28
29class DataProcessingProcedure(BaseModel):
30    text: Annotated[str, AfterValidator(validate_compliance)]
31
32class ProcedureWriter(AnthropicExtractor[DataProcessingProcedure]):
33    extract_schema: Type[DataProcessingProcedure] = DataProcessingProcedure
34
35    prompt_template = """
36    Write a detailed description of a data processing procedure that is compliant with GDPR standards.
37    """
38
39try:
40    procedure = ProcedureWriter().extract()
41    print(procedure.text)
42except ValidationError as e:
43    print(e)
44    # > 1 validation error for DataProcessingProcedure
45    # procedure_text
46    # Assertion failed, Procedure is not GDPR compliant.
47    # [type=assertion_error, input_value="The procedure text here...", input_type=str]
48    # For further information visit https://errors.pydantic.dev/2.6/v/assertion_error

Mirascope also offers a CLI with a complete local working directory for prompt versioning, providing you a space in which to rapidly iterate on prompts for optimization while tracking changes systematically, ensuring that all your fine-tuning efforts are recorded and retrievable.
‍

|
|-- mirascope.ini
|-- mirascope
|   |-- prompt_template.j2
|   |-- versions/
|   |   |-- <directory_name>/
|   |   |   |-- version.txt
|   |   |   |-- <revision_id>_<directory_name>.py
|-- prompts/

CLI commands for prompt management allow you to:

• Initialize your management workspace
  ‍
• Commit or remove prompts to or from version management (Mirascope uses sequential numbered versioning, such as 0001 -> 0002 -> 0003 etc.)
  ‍
• Check the status of prompts
  ‍
• Switch between versions of prompts

Functionality for RAG Applications

LlamaIndex: Uses Advanced Data Embedding and Retrieval

LlamaIndex is well known for its indexing, storage, and retrieval capabilities in the context of developing RAG applications, and provides a number of high-level abstractions for:

• Importing and processing different data types such as PDFs, text files, websites, databases, etc., through document loaders and nodes.
  ‍
• Creating vector embeddings (indexing) from the data previously loaded, for fast and easy retrieval.
  ‍
• Storing data in a vector store, along with metadata for enabling more context-aware search.
  ‍
• Retrieving data that’s relevant to a given query and sending these both together to the LLM for added context and accuracy.

Such abstractions let you build data pipelines for handling large volumes of data.

LlamaIndex's RAG functionality is based on its Query Engine, which queries and retrieves information from indexed data, and then sends these as a prompt enriched by retrieved data to the LLM. 

LangChain: Offers Abstractions for Indexing, Storage, and Retrieval

LangChain offers similar abstractions to that of LlamaIndex for building data pipelines for RAG applications. These abstractions correspond to two broad phases:

• An indexing phase, where data is loaded (via document loaders), split into separate chunks, and stored in a vector store and embedding model.
  ‍
• A retrieval and generation phase, where embeddings relevant to a query are retrieved, passed to the LLM, and responses generated.

Such abstractions include DocumentLoaders, text splitters, VectorStores, and embedding models.

Mirascope: Three Classes for Setting Up a RAG System

Mirascope’s abstractions for RAG emphasize ease of use and speed of development, allowing you to create pipelines for querying and retrieving information for accurate LLM outputs.

Mirascope currently offers three main classes to reduce the complexity typically associated with building RAG systems:

`BaseChunker`

We provide a class to simplify the chunking of documents into manageable pieces for efficient semantic search. In the code below, we instantiate a simple `TextChunker` but you can extend `BaseChunker` as needed:
‍

1import uuid
2from mirascope.rag.chunkers import BaseChunker
3from mirascope.rag.types import Document
4
5class TextChunker(BaseChunker):
6    """A text chunker that splits a text into chunks of a certain size and overlaps."""
7
8    chunk_size: int
9    chunk_overlap: int
10
11    def chunk(self, text: str) -> list[Document]:
12        chunks: list[Document] = []
13        start: int = 0
14        while start < len(text):
15            end: int = min(start + self.chunk_size, len(text))
16            chunks.append(Document(text=text[start:end], id=str(uuid.uuid4())))
17            start += self.chunk_size - self.chunk_overlap
18        return chunks

`BaseEmbedder`

With this class you transform text chunks into vectors with minimal setup. At the moment Mirascope supports OpenAI and Cohere embeddings, but you can extend `BaseEmbedder` to suit your particular use case.
‍

1import os
2from mirascope.openai import OpenAIEmbedder
3
4os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
5
6embedder = OpenAIEmbedder()
7response = embedder.embed(["your_message_to_embed"])

Vector Stores

Besides chunking and vectorizing information, Mirascope also provides storage via integrations with well-known solutions such as Chroma DB and Pinecone.

The code below shows usage of both `TextChunker` and the `OpenAIEmbedder` together with `ChromaVectorStore`:
‍

1# your_repo.stores.py
2from mirascope.chroma import ChromaSettings, ChromaVectorStore
3from mirascope.openai import OpenAIEmbedder
4from mirascope.rag import TextChunker
5
6class MyStore(ChromaVectorStore):
7    embedder = OpenAIEmbedder()
8    chunker = TextChunker(chunk_size=1000, chunk_overlap=200)
9    index_name = "my_index"

Mirascope provides additional methods for adding and retrieving documents to and from the vector store, as well as accessing the client store and index. You can find out more about these, as well as our integrations with LlamaIndex and other frameworks, in our documentation.

Scaling Solutions for Production-Grade LLM Applications

LlamaIndex: Offers a Wide Scope of Integrations and Data Types

As a data framework, LlamaIndex is designed to handle large datasets efficiently, provided it’s correctly set up and configured. It features literally hundreds of loaders and integrations for connecting custom data sources to LLMs.

However, it seems to us that, when it comes to building RAG applications, LlamaIndex can be rigid in certain ways. For instance, its `VectorStoreIndex` class doesn’t permit setting the embedding model post-initialization, limiting how developers can adapt LlamaIndex to different use cases or existing systems. This can hinder scalability as it complicates the integration of custom models and workflows.

As well, LlamaIndex has certain usage costs, which increase as your application scales up with larger volumes of data.

LangChain: A General Framework Covering Many Use Cases

LangChain offers functionalities covering a wide range of use cases, such that if you can think of a use case, it’s likely you can set it up in LangChain. For example, just like LlamaIndex, it similarly provides document and BigQuery loaders for data ingestion.

On the other hand, like LlamaIndex, it rigidly defines how you implement some of your use cases. As an example, it requires you to predetermine memory allocation for AI chat conversations. Developers also find that LangChain requires many dependencies, even for simple tasks.

Mirascope: Centralizes Everything Around the LLM Call to Simplify Application Development

We understand the value of a library that scales well, so we’ve designed Mirascope to be as simple and easy to use as possible.

For instance, we minimize the number of special abstractions you must learn when building LLM pipelines, and offer convenience only where this makes sense, such as for wrapping API calls. You can accomplish much with vanilla Python and OpenAI’s API, so why create complexity where you can just leverage Python’s inherent strengths?

Therefore all you need to know is vanilla Python and the Pydantic library. We also try to push design and implementation decisions onto developers to offer them maximum adaptability and control.

And unlike other frameworks, we believe that everything that can impact the quality of an LLM call (from model parameters to prompts) should live together with the call. The LLM call is therefore the central organizing unit of our code around which everything gets versioned and tested.

A prime example of this is Mirascope’s use of `call_params`, which contains all the parameters of the LLM call and typically lives inside the prompt:
‍

1import os
2
3from mirascope import OpenAICall, OpenAICallParams
4
5os.environ["OPENAI_API_KEY"] = "YOUR_OPENAI_API_KEY"
6
7class TravelItineraryPlanner(OpenAIPrompt):
8    prompt_template = "Plan a travel itinerary that includes activities in {destination}"
9
10    destination: str
11
12    call_params = OpenAICallParams(model="gpt-4", temperature=0.6)
13
14
15response = TravelItineraryPlanner(destination="Paris").call()
16print(response.content)  # prints the string content of the call

In Mirascope, prompts are essentially self-contained classes living in their own directories, making code changes affecting the call easier to trace, thereby reducing errors and making your code simpler to maintain.

Chaining LLM Calls

LlamaIndex: Chains Modules Together Using QueryPipeline

LlamaIndex offers `QueryPipeline` for chaining together different modules to orchestrate LLM application development workflows. Through `QueryPipeline`, LlamaIndex defines four use cases for chaining:

• Chaining together a prompt with an LLM call.
  ‍
• Chaining together query rewriting (prompt + natural language model) with retrieval.
  ‍
• Chaining together a full RAG query pipeline (query rewriting, retrieval, reranking, response synthesis).
  ‍
• Setting up a custom query component that allows you to integrate custom operations into a larger query processing pipeline.

Although these are useful to implement, `QueryPipeline` is a built-in abstraction that hides the lower-level details and entails a learning curve to use.

LangChain: Requires an Explicit Definition of Chains and Flows

LangChain similarly offers its own abstractions for chaining, requiring explicit definition of chains and flows via its LangChain Expression Language (LCEL). This provides a unified interface for building chains and notably implements `Runnable`, which defines common methods such as `invoke`, `batch`, and `stream`.

A typical example of a chain in LangChain is shown below, which uses `RunnablePassthrough`, an object that forwards data to where it needs to go in the chain without changes:
‍

1# Setup for temperature conversion using a query pipeline
2runnable = (
3    {"temperature_query": RunnablePassthrough()}
4    | prompt
5    | model.bind(stop="CONVERSION_COMPLETED")
6    | StrOutputParser()
7)
8print(runnable.invoke("Convert 35 degrees Celsius to Fahrenheit"))

Here, `RunnablePassthrough` is passing `temperature_query` unaltered to `prompt`, while `Runnable.bind` in this instance allows users to define a stop condition at runtime when the token or text “CONVERSION_COMPLETED” is encountered. The condition sets up the model to behave in a certain way after it’s been invoked.

Binding arguments to the model at runtime in such a way is the functional equivalent of Mirascope’s `call_params` (described above) and can be useful in situations where you want to take into account user interactions, in this case allowing users to mark a solution as being solved. 

On the other hand, such conditions need to be carefully managed in all instances of chains to avoid negative impacts on the system.

Mirascope: Leverages Python’s Syntax and Inheritance for Chaining

Mirascope chains multiple calls together using a more implicit approach than that of LlamaIndex and LangChain, relying on existing structures already existent in Python. 

In the example below, chaining is implemented through class inheritance and the use of Python’s `@cached_property`. `CarRecommender` inherits from `BrandExpert`, extending its functionality and utilizing cached properties to make API calls efficient by ensuring that methods like `brand_expert` make a single call even when accessed multiple times.
‍

1import os
2from functools import cached_property
3
4from mirascope.openai import OpenAICall, OpenAICallParams
5
6os.environ["OPENAI_API_KEY"] = "YOUR_API_KEY"
7
8
9class BrandExpert(OpenAICall):
10    prompt_template = "Name an expert who is really good with {vehicle_type} vehicles"
11
12    vehicle_type: str
13
14    call_params = OpenAICallParams(model="gpt-4-turbo")
15
16
17class CarRecommender(BrandExpert):
18    prompt_template = """
19    SYSTEM:
20    Imagine that you are the expert {brand_expert}.
21    Your task is to recommend cars that you, {brand_expert}, would be excited to suggest.
22
23    USER:
24    Recommend a {vehicle_type} car that fits the following criteria: {criteria}.
25    """
26
27    criteria: str
28    
29    call_params = OpenAICallParams(model="gpt-4")
30
31    @cached_property  # !!! so multiple access doesn't make multiple calls
32    def brand_expert(self) -> str:
33        """Uses `BrandExpert` to select the expert based on the vehicle type."""
34        return BrandExpert(vehicle_type=self.vehicle_type).call().content
35
36response = CarRecommender(vehicle_type="electric", criteria="best range and safety features").call()
37print(response.content)
38# > Certainly! Here's a great electric car with the best range and safety features: ...

Making only a single call in this way keeps the application’s response times quick and reduces both expensive API calls and the load on the underlying systems.

As well, relying on Pythonic structures you already know keeps your code readable and maintainable, even as your chains and specific needs grow in complexity.

Mirascope Allows You to Easily Leverage the Strengths of External Libraries

Mirascope’s building block approach to developing generative AI applications means you can easily pick and choose useful pieces from other libraries when you need them.

Just import the functionality you want—and nothing more—from libraries like LangChain and LlamaIndex to get the benefit of what those libraries do best. For instance, Mirascope makes it easy to import LlamaIndex’s `VectorStoreIndex` for document storage in the context of RAG, and to further inject information retrieved from this vector store into your LLM calls.

Our approach allows you to use and combine the best available open-source tools for any specific task in your LLM app development workflows.

Want to learn more? You can find more Mirascope code samples on both our documentation site and on GitHub.