Unit 3 · Topic 2.6
Competing Function Model Validation
Compare models using residual patterns, prediction error, context, and the reasonableness of extrapolation. Develop the idea through symbolic, numerical, graphical, and contextual representations.
Learning Goals
- Construct linear, quadratic, and exponential regressions.
- Interpret residuals as observed minus predicted.
- Use residual patterns and context to validate model choice.
1. Essential Structure
Compare models using residual patterns, prediction error, context, and the reasonableness of extrapolation.
Read the formula together with its domain, units, starting input, and the interval length over which change is measured.
2. Core Ideas
- A positive residual means the model underpredicts; a negative residual means it overpredicts.
- A useful residual plot is an unstructured band around zero.
- A high \(R^2\) alone does not prove the model family is appropriate.
- Interpolation generally requires fewer assumptions than extrapolation.
3. Graph and Representation
Use graphical evidence together with an algebraic or numerical reason. A graphing window alone does not establish domain, end behavior, or model validity.
4. Original Worked Example
Two models fit the same data; Model A has smaller random residuals while Model B shows a curved residual pattern. Choose the stronger model.
Answer: Model A is better supported because its errors are smaller and show less structure.
Check the result in the original representation and explain its meaning, including units when the quantities are contextual.
5. AP Reasoning Workflow
- Plot the original data.
- Fit plausible model families.
- Plot residuals against the same inputs.
- Compare error size and pattern.
- Choose a model and state limitations.
A strong AP response shows the mathematical evidence first and then states a precise conclusion.
6. Extended Example and Application
If exponential residuals are small early but increasingly positive later, the model systematically underpredicts late outputs. A changed process or different family should be considered.
7. Technology and Validation
Record regression equations and residual evidence; a calculator screenshot is not a complete rationale.
When technology is used, record the model or result, preserve sufficient precision, and explain why the output answers the question.
8. Build Plausible Competing Models
A data set may look nearly linear over a short interval even when its underlying pattern is quadratic or exponential. Begin with the scatterplot and contextual mechanism, then fit reasonable candidates with technology.
| Candidate | Characteristic change | Contextual clue |
|---|---|---|
| Linear \(a+bx\) | Approximately constant differences | A fixed amount is added per input unit |
| Quadratic \(ax^2+bx+c\) | Approximately constant second differences | A rate itself changes by a nearly fixed amount |
| Exponential \(ab^x\) | Approximately constant ratios | A fixed percentage or factor acts per input unit |
Regression supplies coefficients for each candidate; it does not make the final decision. Record the equations with enough digits to reproduce predictions, and keep the input and output units attached to every interpretation.
9. Calculate and Interpret Residuals
For an observed value \(y_i\) and model prediction \(\hat y_i\), the AP residual convention is
| Input \(x\) | Observed \(y\) | Predicted \(\hat y=11.8(1.59)^x\) | Residual \(y-\hat y\) |
|---|---|---|---|
| 0 | 12 | 11.80 | 0.20 |
| 1 | 18 | 18.76 | -0.76 |
| 2 | 29 | 29.83 | -0.83 |
| 3 | 47 | 47.43 | -0.43 |
| 4 | 76 | 75.42 | 0.58 |
A positive residual means the observed value is above the prediction, so the model underestimates. A negative residual means the prediction is too high, so the model overestimates. The residual has the same units as the output variable.
10. Read the Residual Plot
A residual plot places input values on the horizontal axis and residuals on the vertical axis. The horizontal line \(e=0\) represents perfect predictions.
“No pattern” does not mean every residual equals zero. It means the signs and sizes do not follow a systematic trend that the model should have captured.
11. Diagnose Common Residual Patterns
| Residual pattern | What it suggests | Next step |
|---|---|---|
| Curved arc or U-shape | The function family misses curvature | Compare quadratic or exponential candidates |
| Residuals steadily rise | The model increasingly underestimates | Recheck trend, variables, and domain |
| Residuals steadily fall | The model increasingly overestimates | Test a different rate of change |
| Widening funnel | Error size grows with input | Report changing uncertainty; reconsider scale or model |
| Separated clusters | An omitted category may matter | Investigate groups or contextual changes |
| Random band around zero | No obvious structure remains | Keep the model as a viable candidate |
A residual pattern identifies a weakness; it does not by itself prove one particular replacement model. Return to the original scatterplot and context before refitting.
12. Compare More Than One Fit Statistic
Numerical summaries help compare error size, but they compress information that a residual plot displays. Two common summaries are
Smaller values mean smaller in-sample errors when the models use the same data and output units. The coefficient of determination, \(R^2\), measures the fraction of observed variation accounted for by a fitted model, but a high \(R^2\) does not guarantee that the residuals are patternless or that the model makes sense in context.
| Evidence | Useful question | Limitation |
|---|---|---|
| Scatterplot | Does the curve follow the overall data shape? | Small systematic errors can be hard to see |
| Residual plot | Is unexplained structure left behind? | Judgment can be difficult with very few points |
| SSE or MAE | How large are the fitted errors? | Does not show where errors occur |
| \(R^2\) | How much variation is accounted for? | Does not validate function family or context |
The strongest AP justification combines residual behavior, relative error size, and a contextual reason.
13. Let Context Influence the Choice
Mathematically similar fits can have different practical consequences. The preferred model depends partly on how its errors will be used.
- Inventory: underestimating demand may cause shortages, so examine intervals with negative prediction consequences carefully.
- Structural safety: underestimating a load may be less acceptable than a modest overestimate.
- Medication: either direction may be consequential, so both magnitude and sign require attention.
- Long-term population: an exponential fit may match early data but become implausible when resources are limited.
Also compare domains and end behavior. A quadratic model can eventually turn, a positive-slope linear model grows by equal amounts, and an exponential growth model accelerates without bound. Similar interpolation does not imply similar extrapolation.
14. Complete Model-Selection Example
The outputs \(12,18,29,47,76\) are observed at inputs \(0,1,2,3,4\). Technology produces a linear model \(L(x)=5+15.7x\) and an exponential model \(E(x)=11.8(1.59)^x\).
- Compare raw change: differences \(6,11,18,29\) increase, while successive ratios are approximately \(1.50,1.61,1.62,1.62\).
- Inspect the linear residuals: \(7.0,-2.7,-7.4,-5.1,8.2\) form a U-shaped pattern.
- Inspect the exponential residuals: \(0.20,-0.76,-0.83,-0.43,0.58\) are much smaller and do not show the same strong curvature.
- Use context: if the output describes repeated proportional growth, the exponential mechanism is also reasonable.
- Conclude: choose \(E\) for interpolation across the observed input interval, while treating distant extrapolation cautiously.
Complete justification: The exponential model is better supported because its residuals remain close to zero without the pronounced U-shape of the linear residuals, and its nearly constant multiplicative change matches the stated process.
If the context instead imposed a strict capacity or a future policy change, neither candidate's long-term extrapolation would automatically be valid.
15. Common Errors
- Do not select a model solely because its graph visually passes near most points.
- Automatically selecting the model with the most parameters.
- Ignoring a curved residual pattern because individual errors seem small.
- Giving a numerical result without a domain check, units, or interpretation.
Key Takeaways
- Compare models using residual patterns, prediction error, context, and the reasonableness of extrapolation.
- Core relationship: \(\operatorname{SSE}=\sum( y_i-\hat y_i)^2\)
- Error check: Do not select a model solely because its graph visually passes near most points.
- Interpret residual \(-4.7\) when output is measured in grams.
- Compare a random residual band with a U-shaped pattern.
- Give one reason a strong in-sample fit may fail in long-term extrapolation.