BSR project page

BSR project page

The page is _pages/bsr.html, published by Jekyll at /projects/bsr/. Its styles and interactions live in assets/css/bsr.css and assets/js/bsr.js.

Current sources

The published content is based on SWE/SWE (32).pdf, supplied on October 4,

  1. The older shared LaTeX sources have not been synchronized with that PDF and must not be used to overwrite the current results.
  • paper.pdf is an unchanged copy of SWE/SWE (32).pdf.
  • framework.pdf is an unchanged copy of SWE/framework.pdf. framework.png is a high-resolution web render with unused white margins removed. The rewards in this figure are illustrative, not experiment results.
  • The accessible main results table reproduces Table 1 and the supplied SWE/Screenshot 2026-10-04 at 1.49.23 PM.png (whose actual filename includes a narrow no-break space before PM). It includes SWE-bench Verified and SWE-bench-Live Lite. Every numeric entry agrees between the PDF and screenshot.
  • The additional Terminal-Bench 2.1 table reproduces Table 8.
  • Behavior statistics and the ablations follow Tables 2-5 and 9. In Table 3, deadzone without softness has 44.0% Pass@3, replacing the old 44.8% value.
  • critic_score_mean_14b_ema_270.png, critic_score_mean_32b_ema_270.png, 14b_avg_270_step.png, and 32b_avg_270_step.png remain renders of the matching SWE/figs/ PDFs. Their displayed average-accuracy values agree with the latest paper. Mean critic score here is the execution-verifier score.
  • The interactive opening diagram is a separate HTML/CSS illustration with centered rewards bounded by the main setting’s amplitude of 0.15. Its A/B/C labels describe that illustration, independently of the framework figure.

Reporting consistency

Verified BSR Pass@3 is 48.0% for 14B/400 steps and 50.6% for 32B/270 steps in Table 1. The 32B appendix curve (Figure 5) also shows 52.2% at another checkpoint; this is not the Table 1 value paired with 35.5% average accuracy. Do not independently maximize each metric when reproducing a reported row. The previous Max accuracy column is absent from the new Table 1 and has been removed from the page.

The current paper no longer lists Andrew Zhao’s affiliation as Mistral AI. Judge-quality wording follows Appendix B.5 (Kimi K2.5 agreement), and training cost follows Appendix B.1 (19.86% additional wall-clock time in the measured 14B setting; no BSR judging at evaluation).

The raw SWE directory is excluded from Jekyll output; publishable assets are copied here intentionally. Update the linked paper, results, figures, and metadata together when a new manuscript is supplied. Homepage release/link visibility is managed separately in _data/home_projects.json and _data/home_publications.json.