Skip to content

feat: add LlamaIndex integration#672

Merged
nicoloboschi merged 7 commits into
mainfrom
feat/llamaindex-integration
Mar 30, 2026
Merged

feat: add LlamaIndex integration#672
nicoloboschi merged 7 commits into
mainfrom
feat/llamaindex-integration

Conversation

@DK09876

@DK09876 DK09876 commented Mar 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add hindsight-llamaindex package: BaseToolSpec subclass + create_hindsight_tools() factory giving LlamaIndex agents persistent memory via retain/recall/reflect
  • Full test suite (34 unit tests passing)
  • Docs page at hindsight-docs/docs/sdks/integrations/llamaindex.md
  • Blog post at hindsight-docs/blog/2026-03-23-llamaindex-memory.md
  • Entry added to integrations.json

Test plan

  • uv run pytest tests/ -v — 34 passed, 3 skipped (manual)
  • uv run ruff check . && uv run ruff format --check . — clean
  • E2E test with real Hindsight instance via test_manual.py
  • Verify docs page renders correctly
  • Verify blog post renders correctly

🤖 Generated with Claude Code

DK09876 and others added 4 commits March 24, 2026 06:56
Add hindsight-llamaindex package providing persistent memory tools for
LlamaIndex agents via the native BaseToolSpec pattern. Includes retain,
recall, and reflect tools, a convenience factory, global config, full
test suite, docs page, blog post, and integrations.json entry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix ReActAgent API: from_tools() → constructor, chat() → await run()
- Add create_bank step to all quickstart examples
- Add production patterns section to docs (tags, error handling, bank lifecycle)
- Add memory scoping recommendation to README
- Add when-not-to-use section to blog post
- Add LlamaIndex compatibility tests (agent acceptance, FunctionTool.call)
- Fix self-hosted auth wording in cookbook notebook

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Use await client.acreate_bank() instead of sync create_bank() to
  avoid "event loop already running" errors in notebooks and async contexts
- Wrap plain Python examples in async def main() + asyncio.run(main())
  so they are copy-paste runnable as scripts
- Add Jupyter notebook tip to docs showing top-level await pattern
- Bank lifecycle example in docs now uses async acreate_bank

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
HindsightToolSpec now provides both sync and async tool implementations
using LlamaIndex's (sync_fn, async_fn) tuple pattern in spec_functions.
Async agents (ReActAgent, etc.) use aretain/arecall/areflect natively,
avoiding the "Timeout context manager should be used inside a task"
error that occurred when sync _run_async() was called from within an
active event loop.

- Add aretain_memory, arecall_memory, areflect_on_memory async methods
- Extract shared kwargs builders (_retain_kwargs, _recall_kwargs, etc.)
- spec_functions now uses tuples: [("retain_memory", "aretain_memory"), ...]
- Tests verify tools have both sync fn and async fn set
- Notebook verified end-to-end with nbclient against local Hindsight

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@DK09876
DK09876 marked this pull request as ready for review March 25, 2026 00:23
The blog post will be pulled in separately from its own PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

@nicoloboschi nicoloboschi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Overall the PR is well-structured — clean separation of concerns (tools.py, config.py, _client.py, errors.py), comprehensive tests (34 unit tests), and good docs. A few things worth discussing:

Package structure doesn't follow LlamaIndex conventions

The PR uses hindsight_llamaindex/ as a standalone package. The standard LlamaIndex community integration uses namespace packages:

# PR has:
hindsight-integrations/llamaindex/hindsight_llamaindex/

# LlamaIndex convention:
llama-index-tools-hindsight/llama_index/tools/hindsight/base.py

With LlamaIndex, llama_index/ and llama_index/tools/ must NOT have __init__.py (PEP 420 implicit namespace packages). Main class goes in base.py, not tools.py.

If the package is ever submitted to LlamaHub or users expect from llama_index.tools.hindsight import HindsightToolSpec, it won't work. Fine as a standalone package but breaks the LlamaIndex ecosystem path.

Suggestion: Either restructure now, or document explicitly that this is standalone (not LlamaHub) and plan migration later.

Missing BaseMemory implementation

The PR only implements BaseToolSpec (agent-driven retain/recall/reflect). LlamaIndex also has BaseMemory which provides automatic memory — get() enriches prompts with recalled memories transparently, put() auto-retains messages.

The Mem0 integration (llama-index-memory-mem0) does both: BaseMemory for automatic recall/retain + tools for explicit agent control. This would align better with how Claude Code 0.3.x works — recall injected automatically into UserPromptSubmit hooks, retain on Stop events.

Consistency gaps with other Hindsight integrations

Feature Claude Code 0.3.x CrewAI This PR
Auto-recall (inject into prompt) ✅ (hook) ✅ (Storage.search) ❌ (tool only)
Auto-retain (on conversation end) ✅ (hook) ✅ (Storage.save) ❌ (tool only)
Bank mission setup
document_id generation ✅ (session+timestamp) N/A Static only
async: true retain
context source label ✅ ("claude-code")
Error handling Graceful (logs, continues) Graceful Raises HindsightError

Specific items:

  • No document_id auto-generation — Claude Code generates {session_id}-{timestamp} for upsert/grouping. This only supports a static retain_document_id.
  • No bank mission management — other integrations call set_bank_mission() on first use so the memory engine has context for fact extraction.
  • No context param on retain — Claude Code passes "claude-code" as source label.
  • Retain doesn't use async: true — other integrations use async retain for non-blocking storage. Valid for tool-based usage (agent expects confirmation) but worth noting.

Minor items

  • config.py — default URL is production (https://api.hindsight.vectorize.io) while CrewAI defaults to http://localhost:8888. Should be consistent across integrations.
  • _client.py — hardcoded timeout: 30.0. Claude Code uses different timeouts per operation (retain: 15s, recall: 10s, health: 5s).
  • Blog post referenced in PR description but not in the diff.

DK09876 and others added 2 commits March 27, 2026 15:52
…ion, graceful errors

- Add `retain_context` param (default: "llamaindex") as source label on retain ops
- Auto-generate `document_id` as `{session_id}-{timestamp_ms}` when not provided
- Add `retain_async` param (default: True) for non-blocking retain processing
- Add `mission` param for automatic bank creation/management on first use
- Change error handling from raising HindsightError to graceful log + return message
- Add per-operation timeout constants in _client.py
- Add `context` and `mission` fields to config.py and configure()
- Update docs: document as standalone package (not LlamaHub), new params, patterns
- Tests: 51 passing (up from 34), covering all new features

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ntation

Tools package (llama-index-tools-hindsight):
- Restructured from hindsight_llamaindex/ to llama_index/tools/hindsight/
- Import: from llama_index.tools.hindsight import HindsightToolSpec
- Follows PEP 420 implicit namespace package convention
- Removed retain_async param (client.retain() doesn't support async_processing)

Memory package (llama-index-memory-hindsight):
- New package: llama_index/memory/hindsight/
- HindsightMemory(BaseMemory) for automatic memory
- put() auto-retains user/assistant messages to Hindsight
- get(input) auto-recalls relevant memories, prepends as system message
- Graceful error handling, bank mission management, document_id generation
- 28 unit tests passing

Both packages follow LlamaIndex community conventions for future LlamaHub submission.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@nicoloboschi
nicoloboschi merged commit 2d787c4 into main Mar 30, 2026
34 of 37 checks passed
nicoloboschi added a commit that referenced this pull request Mar 30, 2026
The LlamaIndex integration doc was added to docs/ (next version) in
#672 but not to versioned_docs/version-0.4/, causing a broken link
on the /integrations page which resolves to the latest version.
nicoloboschi added a commit that referenced this pull request Mar 30, 2026
* Convert codex tool_choice test to pytest style

Follow-up to #734: replace unittest.TestCase + manual sys.path
manipulation with idiomatic pytest + @pytest.mark.asyncio,
matching the rest of the test suite.

* Fix test_hierarchical_fields_categorization for new configurable fields

Update expected count from 20 to 21 and add assertions for fields
added by recent PRs: retain_default_strategy, retain_strategies,
max_observations_per_scope, reflect_source_facts_max_tokens,
llm_gemini_safety_settings, mcp_enabled_tools.

* Add LlamaIndex doc to v0.4 versioned docs and sidebars

The LlamaIndex integration doc was added to docs/ (next version) in
#672 but not to versioned_docs/version-0.4/, causing a broken link
on the /integrations page which resolves to the latest version.

* Regenerate docs skill references

Run generate-docs-skill.sh to pick up new integration pages
(codex, llamaindex) and updated configuration docs.

* Add Codex integration doc to v0.4 versioned docs and sidebar

Same issue as LlamaIndex: doc was added to docs/ (next) but not
versioned_docs/version-0.4/, causing broken link on /integrations.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants