Port the context-attribution intrinsic from the granite-common library into mellea. This follows up on granite-common-related PRs #571 and #609. Context attribution identifies which sentences in the context (including previous conversation turns and any documents) were most important to the LLM when generating each sentence in its response.
Changes:
- Extend sentence boundary marking to previous conversation turns with shared index with documents
- Extend DecodeSentences to previous conversation turns and multi-source decoding
- Add context-attribution catalog entry and update repo to
ibm-granite/granitelib-core-r1.0
- Add context-attribution test data and formatter tests
Port the context-attribution intrinsic from the granite-common library into mellea. This follows up on granite-common-related PRs #571 and #609. Context attribution identifies which sentences in the context (including previous conversation turns and any documents) were most important to the LLM when generating each sentence in its response.
Changes:
ibm-granite/granitelib-core-r1.0