Skip to content

chore: run official sweep to add evals#558

Merged
cquil11 merged 14 commits intomainfrom
evals-sweep
Jan 26, 2026
Merged

chore: run official sweep to add evals#558
cquil11 merged 14 commits intomainfrom
evals-sweep

Conversation

@Oseltamivir
Copy link
Collaborator

For sweep to generate data for frontend

@cquil11 cquil11 changed the title Evals sweep chore: run official sweep to add evals Jan 26, 2026
@functionstackx functionstackx marked this pull request as draft January 26, 2026 18:12
Copy link
Contributor

@functionstackx functionstackx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this PR to enable evals. Gaint PRs are inherently hard to ship & cause an massive wave of jobs that spike CI queue times.

divide this PR up into multiple small PRs

for example:
PR # 1: "enablement eval for gptoss hopper class & mi3x0 class"
PR # 2: "enablement eval for deepseek hopper class & mi3x0 class"
PR # 3: "enablement eval for deepseek blackwell class & mi355x class"
etc

@cquil11 cquil11 marked this pull request as ready for review January 26, 2026 18:25
@claude
Copy link
Contributor

claude bot commented Jan 26, 2026

Claude finished @Oseltamivir's task in 44s —— View job


Review of PR #558

  • Gather context and understand PR changes
  • Check master config validation rule
  • Review perf-changelog.yaml changes
  • Provide review summary

LGTM - no blocking issues found

This PR adds a single perf-changelog.yaml entry with evals-only: true to enable evals for 24 single-node configurations (NVIDIA B200/H100/H200 and AMD MI300X/MI325X/MI355X). The changes are clean:

Note: Reviewer @functionstackx has requested splitting this into smaller PRs. That's a process/workflow decision outside my scope to enforce.

Copy link
Collaborator

@cquil11 cquil11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cquil11 cquil11 merged commit 5c6e194 into main Jan 26, 2026
15 of 43 checks passed
@cquil11 cquil11 deleted the evals-sweep branch January 26, 2026 23:12
@cquil11 cquil11 restored the evals-sweep branch January 27, 2026 21:41
@cquil11 cquil11 mentioned this pull request Jan 27, 2026
nlevin-ui added a commit that referenced this pull request Jan 28, 2026
Keep all entries:
- H200 dynamo-trt entry (this PR)
- Evals-only entry (PR #558)
- B300 dynamo-trt entry (PR #585)
cquil11 added a commit that referenced this pull request Jan 29, 2026
* remove assign

* initial

* update perf

* fix perf changelog

* trigger test sweep

* trigger test sweep pt 2

* rebase for evals only

* Update perf-changelog.yaml

* remove newline

* update perf changelog

---------

Co-authored-by: Cam Quilici <cjquilici@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

3 participants