benchmark result #40

Activity

1. dhcode-cpp commented on Feb 19, 2025

   

   dhcode-cpp

   on Feb 19, 2025

   Owner

   More actions

   As far as I know, using the model trained by X-R1 based on Qwen2.5-7B-math, MATH500 can reach 53.4%, while the base model is only 49.8%.

   Evaluation code：

   https://github.com/dhcode-cpp/X-R1/tree/main?tab=readme-ov-file#benchmark-evaluation
2. glennccc commented on Feb 20, 2025

   

   glennccc

   on Feb 20, 2025

   More actions

   As far as I know, using the model trained by X-R1 based on Qwen2.5-7B-math, MATH500 can reach 53.4%, while the base model is only 49.8%.

   Evaluation code：

   https://github.com/dhcode-cpp/X-R1/tree/main?tab=readme-ov-file#benchmark-evaluation

   Why I use this benchmark.py to eval DeepSeek-R1-Distill-Qwen-32B, the eval acc is only 0.32?
3. ViviqwerAsd commented on Feb 20, 2025

   

   ViviqwerAsd

   on Feb 20, 2025

   Author

   More actions

   hmm that's a good grade!! I also wanna know about gsm8k!
4. dhcode-cpp

   added

   enhancementNew feature or requestNew feature or request

   on Feb 20, 2025
5. dhcode-cpp commented on Feb 21, 2025

   

   dhcode-cpp

   on Feb 21, 2025

   Owner

   More actions

   As far as I know, using the model trained by X-R1 based on Qwen2.5-7B-math, MATH500 can reach 53.4%, while the base model is only 49.8%.
   Evaluation code：
   https://github.com/dhcode-cpp/X-R1/tree/main?tab=readme-ov-file#benchmark-evaluation

   Why I use this benchmark.py to eval DeepSeek-R1-Distill-Qwen-32B, the eval acc is only 0.32?

   Please confirm the consistency of the conversation template.

   X-R1/src/x_r1/benchmark.py

   Line 20 in dd6b0e3

   from grpo import SYSTEM_PROMPT