Project
Changelog
Release notes for @m4trix/evals.
Product updates for @m4trix/evals, generated from conventional commits in the monorepo. Scopes in commit messages route entries to the Agents, Evals, or Tracing changelogs.
Last regenerated 2026-10-14 (UTC).
Unreleased · latest evals v0.34.3
Bug fixes
-
override vulnerable transitive dependencies to clear high-severity audit findings (
evals) (f5b079d)pnpm audit(scripts/audit-deployed-packages.ts) reported high/critical advisories reachable from published packages, which blocks the release pipeline. Pin patched versions via pnpm overrides, staying within each dependency's current major line:- brace-expansion@5 >=5.0.11, nanoid@3 >=3.3.18, ws@8 >=8.21.0, js-yaml@4 >=4.3.2 (version-scoped so other majors are untouched)
- browserslist >=4.28.7, engine.io >=6.6.10, postcss >=8.5.18, @grpc/grpc-js >=1.13.6
- raise existing floors: axios >=1.20.0, form-data >=4.0.6, socket.io-parser >=4.2.7
- trace-viewer: vite ^6.4.3; vitest's auto-installed vite peer now resolves to the same patched 6.4.3 instead of 7.3.3
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
v0.34.3
Bug fixes
-
upgrade dependencies to clear high-severity audit findings (
evals) (3316b84)- bump effect, langchain, and socket.io in core; effect in evals
- upgrade vitest 4 and tsup 8 across packages; turbo at root
- add pnpm overrides for transitive advisories (axios, protobufjs, etc.)
- align example apps and website on patched next releases
- fix ui vitest 4 test options API in AiCursor spec
v0.29.0
New features
- introduced test case name in meta (
evals) (a7959dc) - introduced experiment name (
evals) (c2bf0d7) - adjustment of dataset and test case naming (
evals) (f941bd7) - introduction of ci mode (
evals) (3aa1ec1) - introduction of tags (
evals) (3b82060) - new naming convention (
evals) (24f2dd9) - new run structure with run configs (
evals) (65e46db)
v0.16.0
New features
-
integrate json-diff for enhanced diff logging and evaluation (
evals) (0050228)- Replaced 'diff' dependency with 'json-diff' for improved diff functionality.
- Added logging capabilities to evaluators, allowing for detailed message capture during test execution.
- Updated evaluator interface to include a new 'log' method for logging messages and objects.
- Enhanced RunDetailsView and RunView to display log messages alongside diffs, improving clarity in test case results.
- Introduced options for customizing diff output, including sorting and key exclusion.
- Added tests for new diff and logging functionalities to ensure reliability.
-
enhance RunDetailsView and RunView for better score and metric display (
evals) (5bf9416)- Updated RunDetailsView to conditionally render scores and metrics, improving clarity in test case results.
- Modified RunView to aggregate and display evaluator scores and metrics, ensuring a more informative output.
- Introduced handling for cases with no scores, enhancing user experience in the CLI.
- Improved overall structure and readability of score and metric presentation in the CLI.
-
update dependencies and enhance CLI functionality (
evals) (2d29850)- Added '@pppp606/ink-chart' and 'diff' as dependencies.
- Removed 'json-diff' dependency in favor of 'diff'.
- Introduced overview row count management in the CLI for better navigation.
- Enhanced DatasetsView to display run scores and trends using LineGraph.
- Updated score aggregation to include standard deviation for better insights.
- Improved overall CLI experience with refined navigation and display logic.
v0.8.0
New features
-
update version to 0.7.0, add output handling in evaluator, and enhance test case structure (#27) (
evals) (eaf9d8f)- Bump package version to 0.7.0 in package.json
- Introduce output handling in Evaluator and TestCase classes
- Update README and examples to reflect new output features
- Refactor evaluator tests to accommodate new input structure
- Add output schema and output handling in test cases for better validation
-
first version (#26) (
evals) (9146b35)-
feat(evals): first version
-
fix: for tests
-