> ## Documentation Index
> Fetch the complete documentation index at: https://docs.moorcheh.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# LlamaIndex Integration

> Moorcheh Vector Store integration with LlamaIndex for advanced document processing and retrieval

## What is LlamaIndex Integration?

The Moorcheh Vector Store integration with LlamaIndex provides a powerful way to build advanced document processing and retrieval systems. This integration combines Moorcheh's lightning-fast semantic search with LlamaIndex's sophisticated document processing capabilities.

## Key Features

<CardGroup cols={2}>
  <Card title="Document Processing" icon="file">
    Advanced document loading, chunking, and processing with LlamaIndex
  </Card>

  <Card title="Vector Storage" icon="database">
    Efficient vector storage and retrieval using Moorcheh's MIB technology
  </Card>

  <Card title="Query Engine" icon="magnifying-glass">
    Powerful query engine with metadata filtering and advanced search
  </Card>

  <Card title="AI Integration" icon="brain">
    Seamless integration with various AI models for generative answers
  </Card>
</CardGroup>

## Installation

Install the required packages:

```bash theme={null}
pip install llama_index
pip install moorcheh_sdk
```

## Quick Start

### Import Required Libraries

```python theme={null}
import logging
import sys
import os
from moorcheh_sdk import MoorchehClient
from IPython.display import Markdown, display
from typing import Any, Callable, Dict, List, Optional, cast
from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
    Settings,
)
from llama_index.core.base.embeddings.base_sparse import BaseSparseEmbedding
from llama_index.core.bridge.pydantic import PrivateAttr
from llama_index.core.schema import BaseNode, MetadataMode, TextNode
from llama_index.core.vector_stores.types import (
    BasePydanticVectorStore,
    MetadataFilters,
    VectorStoreQuery,
    VectorStoreQueryMode,
    VectorStoreQueryResult,
)
from llama_index.core.vector_stores.utils import (
    DEFAULT_TEXT_KEY,
    legacy_metadata_dict_to_node,
    metadata_dict_to_node,
    node_to_metadata_dict,
)
from llama_index.core.vector_stores.types import (
    MetadataFilter,
    MetadataFilters,
    FilterOperator,
    FilterCondition,
)
```

### Configure Logging

```python theme={null}
# Logging Setup
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
```

### Load Moorcheh API Key

```python theme={null}
# Set the values of the API Keys in your Environment Variables
from google.colab import userdata

api_key = os.environ["MOORCHEH_API_KEY"] = userdata.get("MOORCHEH_API_KEY")

if "MOORCHEH_API_KEY" not in os.environ:
    raise EnvironmentError(f"Environment variable MOORCHEH_API_KEY is not set")
```

## Document Processing

### Load and Chunk Documents

```python theme={null}
# Load Documents
documents = SimpleDirectoryReader("./documents").load_data()

# Set chunk size and overlap
Settings.chunk_size = 1024
Settings.chunk_overlap = 20
```

<Info>
  Adjust `chunk_size` and `chunk_overlap` based on your document type and requirements. Larger chunks provide more context but may reduce precision.
</Info>

## Vector Store Setup

### Initialize Vector Store

```python theme={null}
# Initialize the Moorcheh Vector Store
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="llamaindex_moorcheh",
    namespace_type="text",
    vector_dimension=None,
    add_sparse_vector=False,
    batch_size=100,
)
```

### Configuration Parameters

<AccordionGroup>
  <Accordion title="Namespace Configuration">
    * **namespace**: Unique identifier for your vector store
    * **namespace\_type**: "text" for text-based or "vector" for vector-based namespaces
    * **vector\_dimension**: Dimension of vectors (None for text-based namespaces)
  </Accordion>

  <Accordion title="Performance Settings">
    * **batch\_size**: Number of documents to process in each batch
    * **add\_sparse\_vector**: Whether to include sparse vectors for hybrid search
  </Accordion>
</AccordionGroup>

### Create Vector Store Index

```python theme={null}
# Create a Vector Store Index using the Vector Store and given Documents
storage_context = StorageContext.from_defaults(vector_store=vector_store)
index = VectorStoreIndex.from_documents(
    documents, storage_context=storage_context
)
```

## Querying the Vector Store

### Basic Query

```python theme={null}
# Generate Response
query_engine = index.as_query_engine()
response = vector_store.generate_answer(
    query="Which company has had the highest revenue in 2025 and why?"
)

display(Markdown(f"<b>{response}</b>"))
print(f"\n\n================================\n\n{response}\n\n================================\n\n")
```

### AI-Powered Query

```python theme={null}
# Get generative answer using AI model
moorcheh_response = vector_store.get_generative_answer(
    query="Which company has had the highest revenue in 2025 and why?",
    ai_model="anthropic.claude-sonnet-4-6",
)

print(f"\n\n================================\n\n{moorcheh_response}\n\n================================\n\n")
```

## Advanced Features

### Metadata Filtering

Filter documents based on metadata:

```python theme={null}
# Filters for Metadata
filter = MetadataFilters(
    filters=[
        MetadataFilter(
            key="file_path",
            value="insert the file path to the document here",
            operator=FilterOperator.EQ,
        )
    ],
    condition=FilterCondition.AND,
)
```

### Supported Filter Operators

<CardGroup cols={2}>
  <Card title="EQ">Equal to</Card>
  <Card title="NEQ">Not equal to</Card>
  <Card title="GT">Greater than</Card>
  <Card title="GTE">Greater than or equal</Card>
  <Card title="LT">Less than</Card>
  <Card title="LTE">Less than or equal</Card>
  <Card title="IN">In list</Card>
  <Card title="NIN">Not in list</Card>
</CardGroup>

### Multiple Filters

```python theme={null}
# Multiple metadata filters
filters = MetadataFilters(
    filters=[
        MetadataFilter(
            key="file_type",
            value="pdf",
            operator=FilterOperator.EQ,
        ),
        MetadataFilter(
            key="date",
            value="2024-01-01",
            operator=FilterOperator.GTE,
        )
    ],
    condition=FilterCondition.AND,
)
```

## Configuration Options

### Chunking Strategy

Customize how documents are chunked:

```python theme={null}
# Custom chunking settings
Settings.chunk_size = 512  # Smaller chunks for more precise retrieval
Settings.chunk_overlap = 50  # Higher overlap for better context preservation
```

### Batch Processing

Optimize batch size for your use case:

```python theme={null}
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="llamaindex_moorcheh",
    namespace_type="text",
    batch_size=50,  # Smaller batches for memory-constrained environments
)
```

### Sparse Vectors

Enable hybrid search with sparse vectors:

```python theme={null}
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="llamaindex_moorcheh",
    namespace_type="text",
    add_sparse_vector=True,  # Enable sparse vectors for hybrid search
)
```

## AI Models

### Supported Models

| Model ID                               | Name                 | Provider  | Description                                                                         |
| -------------------------------------- | -------------------- | --------- | ----------------------------------------------------------------------------------- |
| anthropic.claude-sonnet-4-6            | Claude Sonnet 4.6    | Anthropic | Fast flagship: coding, tools, long docs and RAG (\~1M context).                     |
| anthropic.claude-opus-4-6-v1           | Claude Opus 4.6      | Anthropic | Deepest reasoning and hardest tasks; pick when quality matters most (\~1M context). |
| meta.llama4-maverick-17b-instruct-v1:0 | Llama 4 Maverick 17B | Meta      | Long context, summarization, function calling, fine-tuning friendly.                |
| amazon.nova-pro-v1:0                   | Amazon Nova Pro      | Amazon    | Chat, math, and structured answers for AWS-style workloads.                         |
| deepseek.r1-v1:0                       | DeepSeek R1          | DeepSeek  | Step-by-step reasoning; math, logic, and technical explanations.                    |
| deepseek.v3.2                          | DeepSeek V3.2        | DeepSeek  | Efficient general Q\&A, multilingual, everyday RAG (\~164K context).                |
| openai.gpt-oss-120b-1:0                | OpenAI GPT OSS 120B  | OpenAI    | Large generalist: research-style answers and long-form writing.                     |
| qwen.qwen3-32b-v1:0                    | Qwen 3 32B           | Qwen      | Code and bilingual (EN/ZH) tasks in a smaller footprint.                            |
| qwen.qwen3-next-80b-a3b                | Qwen3 Next 80B A3B   | Qwen      | MoE model for long chats, docs, and code at scale (\~256K context).                 |

### Model Selection

Choose the best model for your use case:

```python theme={null}
# For general purpose queries
response = vector_store.get_generative_answer(
    query="Your question here",
    ai_model="anthropic.claude-sonnet-4-6",
)

# For advanced reasoning and agentic search
response = vector_store.get_generative_answer(
    query="Complex analytical question",
    ai_model="anthropic.claude-opus-4-6-v1",
)
```

## Best Practices

### Document Preparation

<Steps>
  <Step title="Organize Documents">
    Place documents in a structured directory for easy loading
  </Step>

  <Step title="Set Appropriate Chunk Size">
    Balance between context and precision (512-1024 characters)
  </Step>

  <Step title="Add Rich Metadata">
    Include file paths, dates, and other relevant metadata
  </Step>

  <Step title="Test Different Configurations">
    Experiment with different chunk sizes and overlap values
  </Step>
</Steps>

### Performance Optimization

* Use appropriate batch sizes based on your memory constraints
* Enable sparse vectors for hybrid search when needed
* Monitor API usage and rate limits
* Cache frequently accessed results

### Error Handling

```python theme={null}
try:
    response = vector_store.get_generative_answer(
        query="Your question",
        ai_model="anthropic.claude-sonnet-4-6",
    )
    print(response)
except Exception as e:
    print(f"Error generating answer: {e}")
```

## Troubleshooting

### Common Issues

<AccordionGroup>
  <Accordion title="API Key Issues">
    * Ensure MOORCHEH\_API\_KEY is set in environment variables
    * Check API key permissions and validity
    * Verify the key has access to the specified namespace
  </Accordion>

  <Accordion title="Document Loading Problems">
    * Check document file paths and formats
    * Ensure documents are readable and not corrupted
    * Verify directory structure for SimpleDirectoryReader
  </Accordion>

  <Accordion title="Query Performance">
    * Adjust chunk size and overlap settings
    * Use metadata filters to narrow down results
    * Consider enabling sparse vectors for better search
  </Accordion>

  <Accordion title="Memory Issues">
    * Reduce batch size for large document collections
    * Process documents in smaller batches
    * Monitor memory usage during indexing
  </Accordion>
</AccordionGroup>

### Debug Mode

Enable detailed logging for troubleshooting:

```python theme={null}
# Enable debug logging
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))
```

## Advanced Usage

### Custom Embeddings

```python theme={null}
from llama_index.embeddings import OpenAIEmbeddings

# Use custom embeddings
embeddings = OpenAIEmbeddings()
Settings.embed_model = embeddings
```

### Custom Vector Store

```python theme={null}
class CustomMoorchehVectorStore(MoorchehVectorStore):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        # Add custom initialization logic
```

### Hybrid Search

```python theme={null}
# Enable hybrid search with sparse vectors
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="hybrid_search",
    namespace_type="text",
    add_sparse_vector=True,
)
```

## Examples

### Financial Document Analysis

```python theme={null}
# Load financial documents
documents = SimpleDirectoryReader("./financial_reports").load_data()

# Configure for financial analysis
Settings.chunk_size = 1024
Settings.chunk_overlap = 100

# Create vector store
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="financial_analysis",
    namespace_type="text",
)

# Query financial data
response = vector_store.get_generative_answer(
    query="What were the key financial metrics for Q4 2024?",
    ai_model="anthropic.claude-sonnet-4-6",
)
```

### Legal Document Search

```python theme={null}
# Load legal documents
documents = SimpleDirectoryReader("./legal_docs").load_data()

# Configure for legal documents
Settings.chunk_size = 2048  # Larger chunks for legal context
Settings.chunk_overlap = 200

# Create vector store with metadata filtering
vector_store = MoorchehVectorStore(
    api_key=api_key,
    namespace="legal_documents",
    namespace_type="text",
)

# Query with metadata filters
filters = MetadataFilters(
    filters=[
        MetadataFilter(
            key="document_type",
            value="contract",
            operator=FilterOperator.EQ,
        )
    ]
)
```

## Further Resources

<CardGroup cols={2}>
  <Card title="LlamaIndex Documentation" icon="book" href="https://docs.llamaindex.ai/">
    Complete LlamaIndex documentation and guides
  </Card>

  <Card title="Moorcheh SDK" icon="code" href="https://github.com/moorcheh-ai/moorcheh-python-sdk">
    Moorcheh Python SDK repository
  </Card>

  <Card title="Examples Repository" icon="github" href="https://github.com/moorcheh-ai/moorcheh-examples">
    Practical examples and tutorials
  </Card>

  <Card title="Community Support" icon="users" href="https://discord.gg/llamaindex">
    Join the LlamaIndex community
  </Card>
</CardGroup>

## Support

Need help with the LlamaIndex integration?

<Card title="Get Support" icon="question-mark-circle" href="mailto:support@moorcheh.ai">
  Contact our support team for assistance
</Card>