Validation report

Every statistic in QM Bench is calculated by our own engine. To make sure the numbers are right, an automated test compares them with independent calculations in Python (NumPy 2.5.4, SciPy 1.18.1) and with published reference values such as the AIAG MSA example and the ISO 2859-1 sampling tables.

Checks

144

Failures

Last run

2026-10-24

A check passes when the relative difference is within the stated tolerance (absolute for values below 1). Tolerances of 1e-9 to 1e-12 mean the results agree to the last printed digit. Looser tolerances are used only where the reference itself is rounded, for example published tables with 3 or 4 digits, or SciPy's numerical optimisers.

The test script validation/validate.py is part of the project and runs before every release. See also the methods.

Special functions

Check	QM Bench	Reference	Source	Result
Normal CDF(-8)	6.220961e-16	6.220961e-16	SciPy	Pass
Normal CDF(-3)	0.001349898032	0.001349898032	SciPy	Pass
Normal CDF(-1)	0.1586552539	0.1586552539	SciPy	Pass
Normal CDF(0)	0.5	0.5	SciPy	Pass
Normal CDF(1.5)	0.9331927987	0.9331927987	SciPy	Pass
Normal CDF(4.5)	0.9999966023	0.9999966023	SciPy	Pass
Normal quantile(1e-12)	-7.034483825	-7.034483825	SciPy	Pass
Normal quantile(0.001)	-3.090232306	-3.090232306	SciPy	Pass
Normal quantile(0.2)	-0.8416212336	-0.8416212336	SciPy	Pass
Normal quantile(0.5)	1.391458e-16	0	SciPy	Pass
Normal quantile(0.975)	1.959963985	1.959963985	SciPy	Pass
F survival(2.5; 9, 18)	0.0468258929	0.0468258929	SciPy	Pass
F survival(0.5; 2, 20)	0.6139132535	0.6139132535	SciPy	Pass
F survival(7; 4, 30)	4.182142e-4	4.182142e-4	SciPy	Pass
ln Gamma(0.5)	0.5723649429	0.5723649429	SciPy	Pass
ln Gamma(2.5)	0.2846828705	0.2846828705	SciPy	Pass
ln Gamma(7.3)	7.147892523	7.147892523	SciPy	Pass
Incomplete gamma P(2.5, 1.2)	0.2085258794	0.2085258794	SciPy	Pass
Incomplete gamma P(0.7, 3)	0.9744473877	0.9744473877	SciPy	Pass
Incomplete gamma P(10, 8)	0.2833757413	0.2833757413	SciPy	Pass
Digamma(0.3)	-3.502524222	-3.502524222	SciPy	Pass
Digamma(2)	0.4227843351	0.4227843351	SciPy	Pass
Digamma(11.5)	2.39823913	2.39823913	SciPy	Pass
Trigamma(0.3)	12.24536455	12.24536455	SciPy	Pass
Trigamma(2)	0.6449340667	0.6449340668	SciPy	Pass
Trigamma(11.5)	0.09084666127	0.09084666127	SciPy	Pass
t two-sided p(2.1, df 7)	0.07387119621	0.07387119621	SciPy	Pass
t two-sided p(0.4, df 30)	0.6919905165	0.6919905165	SciPy	Pass
t two-sided p(4, df 3)	0.02800845601	0.02800845601	SciPy	Pass
t quantile(0.975, df 10)	2.228138852	2.228138852	SciPy	Pass
t quantile(0.95, df 4)	2.131846786	2.131846786	SciPy	Pass
Binomial CDF(2; 20, 0.05)	0.9245163262	0.9245163262	SciPy	Pass
Binomial CDF(0; 80, 0.01)	0.4475232138	0.4475232138	SciPy	Pass
Binomial CDF(7; 125, 0.04)	0.8709262398	0.8709262398	SciPy	Pass
Clopper-Pearson lower(3/50)	0.01254858784	0.01254858784	SciPy	Pass
Clopper-Pearson upper(3/50)	0.1654819466	0.1654819466	SciPy	Pass
Clopper-Pearson lower(0/20)	0	0	SciPy	Pass
Clopper-Pearson upper(0/20)	0.168433471	0.168433471	SciPy	Pass
Clopper-Pearson lower(20/20)	0.831566529	0.831566529	SciPy	Pass
Clopper-Pearson upper(20/20)	1	1	SciPy	Pass

Control chart constants

Check	QM Bench	Reference	Source	Result
A2 (n=5)	0.5768017139	0.577	AIAG SPC manual table	Pass
D3 (n=5)	0	0	AIAG SPC manual table	Pass
D4 (n=5)	2.114359415	2.114	AIAG SPC manual table	Pass
D4 (n=2)	3.268617021	3.267	AIAG SPC manual table	Pass
c4 (n=5)	0.939985603	0.94	AIAG SPC manual table	Pass
c4 (n=10)	0.9726592741	0.9727	AIAG SPC manual table	Pass
A3 (n=5)	1.427299293	1.427	AIAG SPC manual table	Pass
B4 (n=5)	2.088997869	2.089	AIAG SPC manual table	Pass

Capability, X-bar R, n=5

Check	QM Bench	Reference	Source	Result
Sigma within	0.04236170823	0.04236170823	SciPy	Pass
Sigma overall	0.05502596475	0.05502596475	SciPy	Pass
Cp	1.770466847	1.770466847	SciPy	Pass
Cpk	1.758417181	1.758417181	SciPy	Pass
Pp	1.362992913	1.362992913	SciPy	Pass
Ppk	1.353716484	1.353716484	SciPy	Pass
Expected PPM overall	43.62363867	43.62363867	SciPy	Pass
Cpk 95 % CI lower (Bissell)	1.551767034	1.551767034	SciPy	Pass
Anderson-Darling A²	2.264621384	2.264621384	SciPy	Pass
Cpm	1.253725774	1.253725774	SciPy	Pass
UCL X-bar	10.08030286	10.0803224	AIAG constants	Pass
UCL R	0.2083348811	0.2082994667	AIAG constants	Pass
Rule 1 signals (subgroup 18 shifted)	true	true	Test design	Pass

Capability, I-MR

Check	QM Bench	Reference	Source	Result
Sigma within	1.132422767	1.132422767	SciPy	Pass
Sigma overall	1.161174623	1.161174623	SciPy	Pass
Cp	0.06622968221	0.06622968221	SciPy	Pass
Cpk	-11.6923549	-11.6923549	SciPy	Pass
Pp	0.06458976845	0.06458976845	SciPy	Pass
Ppk	-11.40284039	-11.40284039	SciPy	Pass
Expected PPM overall	1.000000e+6	1.000000e+6	SciPy	Pass

Capability, X-bar S, n=12

Check	QM Bench	Reference	Source	Result
Sigma within	0.8138262149	0.8138262149	SciPy	Pass
Sigma overall	0.8067265971	0.8067265971	SciPy	Pass
Cp	0.09215726727	0.09215726727	SciPy	Pass
Cpk	-36.76128987	-36.76128987	SciPy	Pass
Pp	0.09296829963	0.09296829963	SciPy	Pass
Ppk	-37.08480853	-37.08480853	SciPy	Pass
Expected PPM overall	1.000000e+6	1.000000e+6	SciPy	Pass

Capability, Box-Cox

Check	QM Bench	Reference	Source	Result
Box-Cox lambda (MLE)	-0.3447324251	-0.3447324623	SciPy	Pass

Capability, excluded subgroup

Check	QM Bench	Reference	Source	Result
Sigma within without subgroup 18	0.04311382572	0.04311382572	SciPy	Pass
Cpk without subgroup 18	1.678134645	1.678134645	SciPy	Pass
Ppk without subgroup 18	1.669920464	1.669920464	SciPy	Pass

Capability, limits from first 15 subgroups

Check	QM Bench	Reference	Source	Result
Centre line	10.02088	10.02088	SciPy	Pass
UCL X-bar	10.07294981	10.07294981	Formula	Pass

Capability, variable subgroup sizes

Check	QM Bench	Reference	Source	Result
Pooled sigma within	0.0528326577	0.0528326577	SciPy	Pass
Grand mean (weighted)	10.01439091	10.01439091	SciPy	Pass
UCL of subgroup 1	10.07909744	10.07909744	SciPy	Pass

Gage R&R, ANOVA (AIAG example)

Check	QM Bench	Reference	Source	Result
Interaction p-value	0.9741064043	0.9741064043	SciPy	Pass
%Study variation GRR	27.86065088	27.86065088	SciPy	Pass
ndc	4	4	SciPy	Pass

Gage R&R, average and range method (AIAG example)

Check	QM Bench	Reference	Source	Result
EV (repeatability)	0.2018566667	0.20186	AIAG MSA 4th ed.	Pass
AV (reproducibility)	0.2296670291	0.22967	AIAG MSA 4th ed.	Pass
GRR	0.3057663457	0.30577	AIAG MSA 4th ed.	Pass
%GRR	26.67804984	26.68	AIAG MSA 4th ed.	Pass

Distribution fit, normal

Check	QM Bench	Reference	Source	Result
Mean	1.12048625	1.12048625	SciPy	Pass
Std dev	0.6887088455	0.6887088455	SciPy	Pass
Anderson-Darling A²	1.067341363	1.067341363	SciPy	Pass

Distribution fit, lognormal

Check	QM Bench	Reference	Source	Result
Scale (sigma of ln x)	0.7155723286	0.7155723286	SciPy	Pass
Location (mean of ln x)	-0.1011847813	-0.1011847813	SciPy	Pass
99.865 % quantile	7.733145811	7.733145811	SciPy	Pass

Distribution fit, Weibull

Check	QM Bench	Reference	Source	Result
Shape (MLE)	1.705451142	1.705451142	SciPy	Pass
Scale (MLE)	1.258042873	1.258042873	SciPy	Pass
Shape vs scipy.stats.weibull_min.fit	1.705451142	1.705443581	SciPy	Pass
0.135 % quantile	0.02613508732	0.02613508732	SciPy	Pass
Anderson-Darling A²	0.216703657	0.216703657	SciPy	Pass

Distribution fit, gamma

Check	QM Bench	Reference	Source	Result
Shape (MLE)	2.480098936	2.480098936	SciPy	Pass
Scale (MLE)	0.4517909483	0.4517909483	SciPy	Pass
Shape vs scipy.stats.gamma.fit	2.480098936	2.480098936	SciPy	Pass
0.135 % quantile	0.0523277232	0.0523277232	SciPy	Pass
50 % quantile	0.9740210577	0.9740210577	SciPy	Pass
99.865 % quantile	4.45986098	4.45986098	SciPy	Pass
CDF(0.3)	0.07013524105	0.07013524106	SciPy	Pass
CDF(1.0)	0.5156893896	0.5156893896	SciPy	Pass
CDF(2.5)	0.9511164553	0.9511164553	SciPy	Pass

Distribution fit, exponential

Check	QM Bench	Reference	Source	Result
Scale	1.12048625	1.12048625	SciPy	Pass

Type 1 gauge study

Check	QM Bench	Reference	Source	Result
Cg	3.486207954	3.486207954	SciPy	Pass
Cgk	2.443134534	2.443134534	SciPy	Pass
Bias t-test p-value	3.700409e-27	3.700409e-27	SciPy	Pass

Linearity and bias

Check	QM Bench	Reference	Source	Result
Slope	0.001811666667	0.001811666667	SciPy	Pass
Intercept	-0.009636666667	-0.009636666667	SciPy	Pass
R²	0.1839140985	0.1839140985	SciPy	Pass
SE of slope	5.011004e-4	5.011004e-4	SciPy	Pass
p-value of slope	6.289731e-4	6.289731e-4	SciPy	Pass
SE of intercept	0.003323923817	0.003323923817	SciPy	Pass
Average bias	0.001233333333	0.001233333333	SciPy	Pass

Attribute agreement

Check	QM Bench	Reference	Source	Result
Fleiss' kappa	0.05221818182	0.05221818182	Formula (Fleiss 1971)	Pass
Cohen's kappa	0.8279816514	0.8279816514	Formula (Cohen 1960)	Pass

Attribute control charts

Check	QM Bench	Reference	Source	Result
p chart centre line	0.052875935	0.052875935	Formula	Pass
p chart UCL (sample 1)	0.0992039906	0.0992039906	Formula	Pass
u chart centre line	1.394444444	1.394444444	Formula	Pass
u chart UCL (sample 1)	2.462577906	2.462577906	Formula	Pass

EWMA and CUSUM

Check	QM Bench	Reference	Source	Result
EWMA last value	0.9575672951	0.9575672951	Formula (Montgomery)	Pass
EWMA UCL at point 30	0.9999992338	0.9999992338	Formula (Montgomery)	Pass
CUSUM C+ last value	5.2258	5.2258	Formula (Montgomery)	Pass

AQL sampling plans (ISO 2859-1, normal inspection)

Check	QM Bench	Reference	Source	Result
Lot 1000, level II, AQL 0.65: J n=80 Ac 1 Re 2	J n=80 Ac 1 Re 2	J n=80 Ac 1 Re 2	ISO 2859-1 Table 2-A	Pass
Lot 1000, level II, AQL 1.0: J n=80 Ac 2 Re 3	J n=80 Ac 2 Re 3	J n=80 Ac 2 Re 3	ISO 2859-1 Table 2-A	Pass
Lot 1000, level II, AQL 2.5: J n=80 Ac 5 Re 6	J n=80 Ac 5 Re 6	J n=80 Ac 5 Re 6	ISO 2859-1 Table 2-A	Pass
Lot 500, level II, AQL 1.0: H n=50 Ac 1 Re 2	H n=50 Ac 1 Re 2	H n=50 Ac 1 Re 2	ISO 2859-1 Table 2-A	Pass
Lot 5000, level II, AQL 0.65: L n=200 Ac 3 Re 4	L n=200 Ac 3 Re 4	L n=200 Ac 3 Re 4	ISO 2859-1 Table 2-A	Pass
Lot 100, level II, AQL 1.0: E n=13 Ac 0 Re 1	E n=13 Ac 0 Re 1	E n=13 Ac 0 Re 1	ISO 2859-1 Table 2-A	Pass
Lot 3000, level II, AQL 4.0: K n=125 Ac 10 Re 11	K n=125 Ac 10 Re 11	K n=125 Ac 10 Re 11	ISO 2859-1 Table 2-A	Pass
Lot 10000, level III, AQL 0.4: M n=315 Ac 3 Re 4	M n=315 Ac 3 Re 4	M n=315 Ac 3 Re 4	ISO 2859-1 Table 2-A	Pass
Lot 200, level II, AQL 0.1: K n=125 Ac 0 Re 1	K n=125 Ac 0 Re 1	K n=125 Ac 0 Re 1	ISO 2859-1 Table 2-A	Pass
Lot 50, level S-2, AQL 4.0: B n=3 Ac 0 Re 1	B n=3 Ac 0 Re 1	B n=3 Ac 0 Re 1	ISO 2859-1 Table 2-A	Pass

Cpk, PPM and sigma level

Check	QM Bench	Reference	Source	Result
PPM at Cpk 1.33 (centred)	66.07329526	66.07329526	SciPy	Pass
Cpk from 63.34 PPM	1.333336426	1.333336426	SciPy	Pass
Six Sigma long-term PPM	3.397673125	3.4	Six Sigma convention (3.4 DPMO)	Pass