Define package structure for integrations

[psschwei]

Summary

Decide and implement the package structure for integrations - whether they live in the core package as extras or as separate packages.

Problem / Opportunity

The package structure affects how users install and use integrations. There are tradeoffs between:

• Extras (mellea[langchain]): Simpler for users, single package to manage
• Separate packages (mellea-langchain): Cleaner dependency isolation, independent versioning

This decision impacts all integration tasks and should be made early.

User Story / Use Case

As a mellea user, I want a clear, simple way to install the integrations I need so that I can get started quickly without unnecessary dependencies.

Proposed Solution (High-level)

Implement the chosen structure in:

• Updated pyproject.toml with extras or separate package configs
• mellea/integrations/__init__.py with lazy imports
• Import guards for optional dependencies
• Documentation for installation

Recommendation: Start with extras (mellea[langchain], mellea[dspy], etc.) for simplicity. Consider separate packages later if versioning becomes complex.

Scope

In scope:

• pyproject.toml configuration for chosen structure
• Lazy import system for optional dependencies
• Helpful error messages when dependencies are missing
• Installation documentation

Out of scope:

• Separate package repositories (if separate packages chosen)
• Publishing automation
• Version compatibility matrices

Acceptance Criteria

• pip install mellea[langchain] works (or equivalent for chosen structure)
• Import errors are helpful when dependencies are missing
• No core functionality broken when extras are not installed

[psschwei]

@akihikokuroda 's comment from another issue: #434 (comment)

[ajbozarth]

#453 is partial a sub-task of this

[akihikokuroda]

Can this be closed? https://github.com/generative-computing/mellea-contribs/blob/main/README.md

[ajbozarth]

For my sanity I had Claude double check I it is actually complete and it gave the following summary, so this IS good to close.

---

The package structure question raised in this issue has been addressed through work done across several PRs:

1. Core extras — pyproject.toml now defines extras for backends (hf, litellm,
   watsonx) and features (tools, telemetry, docling, granite_retriever), with
   try/except ImportError guards throughout the codebase providing helpful error messages
   when optional deps are missing.

2. Separate contribs repo — Framework integrations (LangChain, DSPy, CrewAI) were moved
   to mellea-contribs, which is
   the "separate packages" path the issue described.

3. Sub-task Clean up and streamline pyproject.toml #453 (pyproject.toml cleanup) is closed.

All acceptance criteria appear met. Closing.

[ajbozarth]

The following PRs collectively addressed this issue:

• feat: separate out remaining dependencies and improve tests #789 (feat: separate out remaining dependencies and improve tests) — @jakelorocco — most
  directly addressed this: split out extras, dependency isolation, and helpful import error
  messages (e.g. pip install "mellea[cli]")
• feat: add tests for mellea optional dependencies #724 (feat: add tests for mellea optional dependencies) — @jakelorocco
• fix: guarding optional imports for hooks #627 (fix: guarding optional imports for hooks) — @psschwei
• chore: improve pyproject.toml readability #590 (chore: improve pyproject.toml readability) — @ajbozarth
• fix: add missing dependencies #715 (fix: add missing dependencies) — @psschwei

Framework integrations (LangChain, DSPy, CrewAI) landing in mellea-contribs as separate
packages also resolves the "separate packages vs extras" question from the issue description.