[data-1743] MCP Improvement

[Zhimin-arya]

Goal of this PR

1. Reduce MCP “tool discovery / handshake” overhead in code-suggestion by switching to static tool definitions.
2. Improve phenotype search usability/perf by returning a small topK result set (semantic search) instead of “all”.
3. Keep/standardize timing logs around cohort tools
4. Add a batch of cohort JSON templates under the MCP server data folder for tool fetch_templates_for_cohort_generation.

Review file ↔ function mapping

1) Code-suggestion: static MCP tools (core perf improvement)

• functions/code-suggestion/src/mcp/staticTools.ts

  • Defines createStaticMcpTools(token, datasetId) and callMcpTool(...).
  • Directly calls MCP JSON-RPC tools/call without getTools() / handshake.

• functions/code-suggestion/src/code-suggestion/services.ts

  • getChatResponse(...) now uses createStaticMcpTools(...) (instead of initializing an MCP client/manager + getTools()).
  • Adds timing logs around tool+agent creation and stream start.

• functions/code-suggestion/src/utils/utils.ts

  • Removes initMcpManager(...) helper because the manager path is no longer needed in runtime.

• functions/code-suggestion/deno.json

  • Updates import mappings so code-suggestion can import the new static tools module cleanly.

• functions/code-suggestion/src/mcp/client.ts

  • Kept as a backup client; exposes createMcpClient(...) helper (manager logic moved here).

• Removed: functions/code-suggestion/src/mcp/mcpManager.ts

  • Manager removed because static tools make it unnecessary for the main runtime path.

2) Code-suggestion: prompting / Strategus gating

• functions/code-suggestion/src/code-suggestion/prompts.ts
  • Adds isStrategusRelated(...) so Strategus-specific system instructions are only injected when the user query looks Strategus-related.
  • Intended to reduce irrelevant prompt bloat for non-Strategus questions.

3) MCP server: phenotype search behavior (topK + semantic search)

• Key behavior: server supports searchTerm, useSemanticSearch, topK.
• Critical fix: client-side static tool schema includes these args, so queries like “bronchiolitis” no longer behave like “(all)”.

Primary review focus:

• The search_phenotype_library tool schema in functions/code-suggestion/src/mcp/staticTools.ts matches server-side expectation.

4) MCP server: cohort tool timing logs

• functions/mcp-server/src/tools/cohort-management.tools.ts
  • Keeps/adds per-tool START/END + “time used” logging for cohort CRUD/list tools.

5) Data-only bulk additions

• functions/mcp-server/data/Cohorts.csv

  • Phenotype/cohort index used by phenotype search.

• functions/mcp-server/data/cohorts/*.json (1100+ files)

  • Template cohort definitions; intended as static data.
  • Reviewers can skim/skip unless validating data packaging.

Results

1. Code-suggestion side: Replace client.getTools() with createStaticMcpTools() definitions which send direct tools/call to the MCP server, bypassing the 3 handshake/cold starts of MCP server, setup improve from ~5.5s to ~2.1s
   |---Method-----|------Get MCP tool list-----|---Agent init time---|---Total time----|

2. MCP-server side: added semantic search with embedding cache for tool call on search_phenotype_library
   embedding cache on the server side. Semantic search is slower per-request (641-954ms vs 70-116ms) but returns 5 items instead of 1100, drastically reducing LLM processing time downstream from ~5s to ~1s.

3. Current time consumption, e.g. query: phenotypeid of diabetes:
   code-suggestion cold start (~2s) --> Maybe trex runtime overhead (~1-2s) --> agent initialization (~2s, LLM decides tools) --> MCP tool calls (For each tool call: args streaming ~ 1s, mcp-server cold start ~1s, tool execution time ~0.1-1s) --> LLM inference (~0.5-1s)

4. Prompt and instruction files were updated to align with stateless mechanism and current UI.

Merge Checklist

Please cross check this list if additions / modifications needs to be done on top of your core changes and tick them off. Reviewer can as well glance through and help the developer if something is missed out.

• Automated Tests (Jasmine integration tests, Unit tests, and/or Performance tests)
• Updated Manual tests / Demo Config
• Documentation (Application guide, Admin guide, Markdown, Readme and/or Wiki)
• Verified that local development environment is working with latest changes (integrated with latest develop branch)
• following best practices in code review doc

[Copilot]

Pull request overview

This PR improves MCP tool initialization latency in the code-suggestion function by switching from dynamic getTools() discovery (handshake) to locally defined static tool schemas, and it extends MCP-server data/config by adding cohort definitions and linting configuration.

Changes:

• Code-suggestion: add createStaticMcpTools() and use it instead of MCP handshake-based tool discovery.
• MCP-server: add a large set of new cohort definition JSON files used by the server.
• MCP-server: introduce ESLint configuration/ignores.

Reviewed changes

Copilot reviewed 57 out of 1122 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
functions/mcp-server/data/cohorts/1048.json	Adds a new cohort definition JSON.
functions/mcp-server/data/cohorts/1047.json	Adds a new cohort definition JSON with inclusion rules.
functions/mcp-server/data/cohorts/1046.json	Adds a new cohort definition JSON with inclusion rules.
functions/mcp-server/data/cohorts/1045.json	Adds a new cohort definition JSON (trimethoprim).
functions/mcp-server/data/cohorts/1042.json	Adds a new cohort definition JSON (IL23 inhibitors).
functions/mcp-server/data/cohorts/1041.json	Adds a new cohort definition JSON (JAK inhibitor).
functions/mcp-server/data/cohorts/1040.json	Adds a new cohort definition JSON (TNF inhibitors).
functions/mcp-server/data/cohorts/1039.json	Adds a new cohort definition JSON (DPP4 inhibitors).
functions/mcp-server/data/cohorts/1038.json	Adds a new cohort definition JSON (GLP-1 receptor agents).
functions/mcp-server/data/cohorts/1037.json	Adds a new cohort definition JSON (SGLT2 inhibitors).
functions/mcp-server/data/cohorts/1036.json	Adds a new cohort definition JSON (beta blockers).
functions/mcp-server/data/cohorts/1035.json	Adds a new cohort definition JSON (thiazide diuretics).
functions/mcp-server/data/cohorts/1034.json	Adds a new cohort definition JSON (RSV infection) with clean window rule.
functions/mcp-server/data/cohorts/1031.json	Adds a new cohort definition JSON (coronary artery disease).
functions/mcp-server/data/cohorts/1030.json	Adds a new cohort definition JSON (prostate cancer) with demographics gate.
functions/mcp-server/data/cohorts/1029.json	Adds a new cohort definition JSON (mantle cell lymphoma) using standard + non-standard concepts.
functions/mcp-server/data/cohorts/1028.json	Adds a new cohort definition JSON (primary/malignant urothelial bladder cancer) with exclusions.
functions/mcp-server/data/cohorts/1027.json	Adds a new cohort definition JSON (CLL) with exclusions.
functions/mcp-server/data/cohorts/1026.json	Adds a new cohort definition JSON (multiple sclerosis).
functions/mcp-server/data/cohorts/1025.json	Adds a new cohort definition JSON (ADHD condition/procedure + meds).
functions/mcp-server/data/cohorts/1024.json	Adds a new cohort definition JSON (anhedonia + MDD + visits) with correlated criteria.
functions/mcp-server/data/cohorts/1021.json	Adds a new cohort definition JSON (myasthenia gravis + pyridostigmine + inpatient/ER).
functions/mcp-server/data/cohorts/1019.json	Adds a new cohort definition JSON (HDFN / pregnancy endpoints) with censoring/collapse.
functions/mcp-server/data/cohorts/1018.json	Adds a new cohort definition JSON (warm autoimmune hemolytic anemia) with exclusions.
functions/mcp-server/data/cohorts/1017.json	Adds a new cohort definition JSON (neonatal thrombocytopenia) with age constraint.
functions/mcp-server/data/cohorts/1016.json	Adds a new cohort definition JSON (polyarticular JIA) with start date + age constraint.
functions/mcp-server/data/cohorts/1013.json	Adds a new cohort definition JSON (ankylosing spondylitis) with juvenile exclusions.
functions/mcp-server/data/cohorts/1012.json	Adds a new cohort definition JSON (Waldenstrom macroglobulinemia).
functions/mcp-server/data/cohorts/1011.json	Adds a new cohort definition JSON (marginal zone lymphoma).
functions/mcp-server/data/cohorts/1007.json	Adds a new cohort definition JSON (epilepsy/seizures) with pregnancy/newborn/alcohol exclusions.
functions/mcp-server/data/cohorts/1006.json	Adds a new cohort definition JSON ([LEGEND HTN] vomiting) with end strategy/collapse.
functions/mcp-server/data/cohorts/1005.json	Adds a new cohort definition JSON ([LEGEND HTN] VTE) with end strategy/collapse.
functions/mcp-server/data/cohorts/1004.json	Adds a new cohort definition JSON (thyroid neoplasm).
functions/mcp-server/data/cohorts/1003.json	Adds a new cohort definition JSON (kidney cancer).
functions/mcp-server/data/cohorts/1001.json	Adds a new cohort definition JSON (edema) with end strategy/collapse.
functions/mcp-server/data/cohorts/1000.json	Adds a new cohort definition JSON ([LEGEND HTN] nausea) with end strategy/collapse.
functions/mcp-server/data/cohorts/100.json	Adds a new cohort definition JSON (Alzheimer’s disease) with end strategy/collapse.
functions/mcp-server/data/cohorts/10.json	Adds a new cohort definition JSON (nausea/vomiting) with end strategy.
functions/mcp-server/.eslintrc	Adds ESLint configuration for the server workspace.
functions/mcp-server/.eslintignore	Adds ignore patterns for ESLint runs in the server workspace.
functions/code-suggestion/src/utils/utils.ts	Removes MCP manager initialization helper.
functions/code-suggestion/src/mcp/staticTools.ts	Adds static MCP tool definitions to avoid MCP handshake latency.
functions/code-suggestion/src/mcp/mcpManager.ts	Removes MCPManager singleton (handshake-based client initialization).
functions/code-suggestion/src/mcp/client.ts	Removes eager getTools() call; adds helper to construct an MCP client.
functions/code-suggestion/src/code-suggestion/services.ts	Switches agent tool wiring to static MCP tools and adds timing logs.
functions/code-suggestion/src/code-suggestion/prompts.ts	Makes Strategus prompting conditional based on keywords.
functions/code-suggestion/deno.json	Updates import map to point to new static tools module and adds tools dependency.