Introduction to Memvid

Memvid is a groundbreaking solution designed to revolutionize AI memory management by encoding text data into videos. This innovative approach allows for lightning-fast semantic search across millions of text chunks, achieving sub-second retrieval times. Unlike traditional vector databases that require extensive RAM and storage, Memvid compresses your knowledge base into compact video files while ensuring instant access to any piece of information.

With a total of 83 files and 8640 lines of code, Memvid is a moderate-sized project that packs a powerful punch in terms of functionality and efficiency.

Key Features of Memvid

• 🎥 Video-as-Database: Store millions of text chunks in a single MP4 file.
• 🔍 Semantic Search: Find relevant content using natural language queries.
• 💬 Built-in Chat: Conversational interface with context-aware responses.
• 📚 PDF Support: Direct import and indexing of PDF documents.
• 🚀 Fast Retrieval: Sub-second search across massive datasets.
• 💾 Efficient Storage: 10x compression compared to traditional databases.
• 🔌 Pluggable LLMs: Works with OpenAI, Anthropic, or local models.
• 🌐 Offline-First: No internet required after video generation.
• 🔧 Simple API: Get started with just 3 lines of code.

Why Choose Memvid?

Memvid stands out due to its game-changing innovation in AI memory management:

• Video as Database: Store millions of text chunks in a single MP4 file.
• Instant Retrieval: Achieve sub-second semantic search across massive datasets.
• 10x Storage Efficiency: Video compression significantly reduces memory footprint.
• Zero Infrastructure: No database servers needed; just files you can copy anywhere.
• Offline-First: Works completely offline once videos are generated.

Installation Guide

Getting started with Memvid is straightforward. Follow these steps to install it:

Quick Install

pip install memvid

For PDF Support

pip install memvid PyPDF2

Recommended Setup (Virtual Environment)

# Create a new project directory
mkdir my-memvid-project
cd my-memvid-project

# Create virtual environment
python -m venv venv

# Activate it
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate

# Install memvid
pip install memvid

# For PDF support:
pip install PyPDF2

Usage Examples

Memvid offers a simple API that allows you to create video memories from text chunks or documents. Here are some basic usage examples:

Basic Usage

from memvid import MemvidEncoder, MemvidChat

# Create video memory from text chunks
chunks = ["Important fact 1", "Important fact 2", "Historical event details"]
encoder = MemvidEncoder()
encoder.add_chunks(chunks)
encoder.build_video("memory.mp4", "memory_index.json")

# Chat with your memory
chat = MemvidChat("memory.mp4", "memory_index.json")
chat.start_session()
response = chat.chat("What do you know about historical events?")
print(response)

Building Memory from Documents

from memvid import MemvidEncoder
import os

# Load documents
encoder = MemvidEncoder(chunk_size=512, overlap=50)

# Add text files
for file in os.listdir("documents"):
    with open(f"documents/{file}", "r") as f:
        encoder.add_text(f.read(), metadata={"source": file})

# Build optimized video
encoder.build_video(
    "knowledge_base.mp4",
    "knowledge_index.json",
    fps=30,  # Higher FPS = more chunks per second
    frame_size=512  # Larger frames = more data per frame
)

Advanced Configuration

For users looking to customize their experience, Memvid offers advanced configuration options:

Custom Embeddings

from sentence_transformers import SentenceTransformer

# Use custom embedding model
custom_model = SentenceTransformer('sentence-transformers/all-mpnet-base-v2')
encoder = MemvidEncoder(embedding_model=custom_model)

Video Optimization

# For maximum compression
encoder.build_video(
    "compressed.mp4",
    "index.json",
    fps=60,  # More frames per second
    frame_size=256,  # Smaller frames
    video_codec='h265',  # Better compression
    crf=28  # Compression quality (lower = better quality)
)

Community and Contribution

Memvid is an open-source project that thrives on community contributions. If you’re interested in contributing, please follow our Contributing Guidelines. We welcome developers of all skill levels to help improve the project.

License Information

Memvid is licensed under the MIT License. This allows you to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the software, provided that the original copyright notice and permission notice are included in all copies or substantial portions of the software.

Conclusion

Memvid is a powerful tool for anyone looking to enhance their AI memory management capabilities. With its unique approach to storing and retrieving information, it opens up new possibilities for developers, educators, and researchers alike. Ready to revolutionize your AI memory management? Install Memvid and start building today!

Resources

• GitHub Repository
• PyPI Package
• Documentation

FAQ