refactor: replace filteredServerCache with generic syncutil.TTLCache

[Copilot]

internal/server/routed.go contained a ~70-line hand-rolled filteredServerCache that duplicated the get-or-create pattern already present in internal/syncutil/cache.go, while adding TTL eviction and LRU-cap that the generic helper lacked.

Changes

• internal/syncutil/ttl_cache.go — New generic TTLCache[K comparable, V any] with lazy TTL eviction and LRU-cap eviction on every GetOrCreate call. Includes an injectable clock (newTTLCacheWithClock) for deterministic testing.

• internal/server/routed.go — Drops filteredServerCache, filteredServerEntry, newFilteredServerCache, and the getOrCreate method. Replaced with syncutil.NewTTLCache[string, *sdk.Server]:

  // Before: ~70-line hand-rolled struct
  serverCache := newFilteredServerCache(sessionTimeout)
  return serverCache.getOrCreate(backendID, sessionID, func() *sdk.Server { ... })
  
  // After: generic TTLCache
  serverCache := syncutil.NewTTLCache[string, *sdk.Server](sessionTimeout, filteredServerCacheMaxSize)
  cacheKey := fmt.Sprintf("%s/%s", backendID, sessionID)
  return serverCache.GetOrCreate(cacheKey, func() *sdk.Server { ... })

• internal/server/session_util.go — truncateCacheKeyForLog moved here alongside truncateSessionID, its natural home.

• internal/syncutil/ttl_cache_test.go — 9 tests covering cache hit/miss, TTL eviction (fake clock + real-time), LRU ordering, last-used refresh on hit, concurrent access, and maxSize=1 edge case.

• internal/server/routed_test.go — TestFilteredServerCache_MaxSize and TestFilteredServerCache_TTLEviction removed; equivalent coverage migrated to syncutil/ttl_cache_test.go.

[Copilot]

Pull request overview

This PR refactors routed-mode filtered server caching by removing the bespoke filteredServerCache from internal/server/routed.go and replacing it with a new reusable generic syncutil.TTLCache that provides lazy TTL eviction plus an LRU size cap. It also relocates truncateCacheKeyForLog into internal/server/session_util.go and migrates cache behavior tests into the new syncutil test suite.

Changes:

• Added internal/syncutil/ttl_cache.go: a generic TTLCache[K,V] with TTL + LRU eviction on GetOrCreate, plus injectable clock for deterministic tests.
• Updated internal/server/routed.go to use syncutil.NewTTLCache[string, *sdk.Server] instead of the removed hand-rolled cache.
• Added internal/syncutil/ttl_cache_test.go and removed the routed-mode cache tests that are now covered in syncutil.

Show a summary per file

File	Description
internal/syncutil/ttl_cache.go	Introduces the new generic TTL+LRU cache implementation used by routed mode.
internal/syncutil/ttl_cache_test.go	Adds unit tests for TTLCache behavior (TTL eviction, LRU eviction, concurrency, edge cases).
internal/server/session_util.go	Moves truncateCacheKeyForLog alongside other session/log-safety helpers.
internal/server/routed.go	Replaces the removed filteredServerCache with syncutil.TTLCache for per-(backend,session) server reuse.
internal/server/routed_test.go	Removes cache tests that are now covered by ttl_cache_test.go.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 5/5 changed files
• Comments generated: 3

[lpcox]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Merged main into the branch. The only conflict was a modify/delete on internal/server/session_util.go — main had deleted it (moving truncateSessionID into session.go), while our branch had added truncateCacheKeyForLog there. Resolved by moving truncateCacheKeyForLog into session.go alongside truncateSessionID and deleting session_util.go. Build and tests pass.