What does the interval let you say?
Kitebench Research compares two training recipes on one benchmark. Partial credit is allowed. The mean score is on a 0–100 percentage scale.
The question
Before testing, the lab set an equivalence margin of ±1 percentage point. A claim within that tolerance requires the entire pre-specified 90% interval to lie strictly inside the margin at the fixed final analysis. The first 100 items are for an exploratory read. The final analysis uses all 400 planned items.
| Items | Use | |
|---|---|---|
| Interim | 100 | Exploratory only |
| Final | 400 | Apply the equivalence rule |
The interval estimates are supplied. You do not need to calculate them.
“Explain what the results support under the lab’s preplanned rule.”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: What does the small estimate tell you?
The first 100 items give an interim difference of +0.2 percentage points for the new recipe.
What would you do here? Open the move you’d make.
Claim equivalence because +0.2 is between −1 and +1
Tempting, but it misses something. The estimate alone does not give the interval or the final result.
Report the interim estimate and check its uncertainty
The strong move. This describes the evidence at the stage it belongs to.
Change the tolerance to ±0.2 after seeing the estimate
Tempting, but it misses something. The lab fixed the margin before testing.
See how a strong candidate says it
“The interim estimate is inside the ±1-point margin. I would check the interval and keep the claim for the fixed final analysis.”
Why it works: The rule concerns the full interval at the planned final analysis. The point estimate is one value.
Step 2: Does the interim interval fit?
The interim 90% interval is [−0.8, +1.2] percentage points.
What would you do here? Open the move you’d make.
Say the interval fits because its lower bound is above −1
Tempting, but it misses something. The upper bound also has to be strictly inside +1.
Keep the planned final analysis and report the interim uncertainty
The strong move. This follows both the margin and the fixed-sample rule.
Stop at 100 and use the interval to prove a practical difference
Tempting, but it misses something. The interval spans differences inside and outside the practical margin.
See how a strong candidate says it
“The interim interval extends to +1.2 points, which is 0.2 beyond the upper margin. It would miss the interval rule. This check is exploratory and the plan calls for all 400 items.”
Why it works: The interim read can identify uncertainty. The final rule applies at the fixed analysis.
Step 3: Does the final interval fit?
At the fixed final analysis of all 400 items, the estimate remains +0.2 points. The 90% interval is [−0.4, +0.8] points.
What would you do here? Open the move you’d make.
Report equivalence within one point for this benchmark
The strong move. The entire final interval is within the planned margin.
Reject the final result because the exploratory interval crossed +1
Tempting, but it misses something. The study assigns the claim to the fixed final analysis.
Report that the two recipes have exactly equal scores
Tempting, but it misses something. The estimate and interval support performance within a tolerance, not equality.
See how a strong candidate says it
“The final estimate is still +0.2 points, and the final interval is [−0.4, +0.8]. Both endpoints lie strictly inside [−1, +1]. The interval is 1.2 points wide. The final comparison meets the planned equivalence rule.”
Why it works: The final analysis uses all 400 planned items. The tighter interval satisfies the pre-specified condition.
Step 4: What claim can the paper make?
What would you do here? Open the move you’d make.
State the tolerance and scope with the estimate and interval
The strong move. The reader can compare the evidence with the rule.
Say the recipes will perform the same on other tasks
Tempting, but it misses something. This study has evidence from one benchmark.
Report only the point estimate because it stayed constant
Tempting, but it misses something. The change in interval precision is what allowed the final decision.
See how a strong candidate says it
“Under the preplanned rule, the recipes perform within one percentage point of each other on this benchmark. I would give the estimate, the final 90% interval, the margin and the sample, and limit the claim to these settings.”
Why it works: The wording identifies the comparison, the precision and the scope of the decision.
The answer, in short
- Decision
- Report performance within the preplanned one-point margin for this benchmark.
- Because
- The fixed final 90% interval lies strictly inside the equivalence bounds.
- What would change it
- A final interval that reaches or crosses either bound would leave the equivalence claim open.
Common traps
- Use the interval and fixed analysis rule together.
- State the tolerance and benchmark with the result.
Check your answer
- I distinguished the estimate from the interval.
- I kept the interim read exploratory.
- I applied the rule at the fixed final sample.
- I scoped the paper’s claim.
Last reviewed Nov 11, 2026