Skip to content

test: add unit tests for assertions#176

Merged
matzew merged 1 commit into
mcpchecker:mainfrom
Cali0707:assert-unit-tests
Feb 4, 2026
Merged

test: add unit tests for assertions#176
matzew merged 1 commit into
mcpchecker:mainfrom
Cali0707:assert-unit-tests

Conversation

@Cali0707

@Cali0707 Cali0707 commented Feb 4, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Contributor

Part of #146
Fixes #147

This PR adds tests for all the assertions within pkg/eval/assert.go

Summary by CodeRabbit

Release Notes

  • Tests
    • Enhanced test coverage for the evaluation framework to improve reliability and maintainability.

Note: This release contains internal testing improvements with no user-facing changes.

@Cali0707
test: add unit tests for assertions
0af6080 
Signed-off-by: Calum Murray <cmurray@redhat.com>
@Cali0707
Cali0707 requested a review from a team February 4, 2026 16:28
@coderabbitai

coderabbitai Bot commented Feb 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

A comprehensive unit test suite for the evaluation framework has been added to pkg/eval/assert_test.go, covering all 11+ assertion types with table-driven tests across multiple scenarios including edge cases, pattern matching, and error handling.

Changes

Cohort / File(s) Summary
Assertion Test Suite
pkg/eval/assert_test.go		 Comprehensive table-driven test suite for all assertion evaluators (ToolsUsedEvaluator, ToolsNotUsedEvaluator, MinToolCallsEvaluator, MaxToolCallsEvaluator, ResourcesReadEvaluator, ResourcesNotReadEvaluator, PromptsUsedEvaluator, PromptsNotUsedEvaluator, CallOrderEvaluator, NoDuplicateCallsEvaluator, RequireAnyEvaluator) with scenarios covering exact matches, pattern-based matching, server/tool scoping, edge cases (nil, empty), and error/reason propagation. Includes tests for result aggregation, matching helpers (matchesToolAssertion, matchesResourceAssertion, matchesPromptAssertion), and composite evaluator construction.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Hops through assertions with glee,
Tests for eleven kinds, you see!
Table-driven rows, edge cases glow,
Coverage blooms where doubts would grow!
A warren of tests, all pristine and bright, ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1
❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'test: add unit tests for assertions' clearly and concisely describes the main change: adding unit tests for assertion functionality in the codebase.
Linked Issues check ✅ Passed The PR adds comprehensive unit tests covering all 11+ assertion types (toolsUsed, toolsNotUsed, minToolCalls, maxToolCalls, resourcesRead, resourcesNotRead, promptsUsed, promptsNotUsed, callOrder, noDuplicateCalls, requireAny) with edge cases and multiple scenario tables as required by issue #147.
Out of Scope Changes check ✅ Passed All changes are scoped to adding unit tests for pkg/eval/assert.go as specified in the linked issue; no unrelated modifications to production code or other test files are present.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

matzew
matzew approved these changes Feb 4, 2026
View reviewed changes

@matzew matzew left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@matzew
matzew merged commit 6e573d6 into mcpchecker:main Feb 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@matzew matzew matzew approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add unit tests for pkg/eval/assert.go

2 participants

@Cali0707 @matzew