Statistical Testing: User Requirements Document
|
Prepared by:
Purpose of the document
This document collects the specific requirements concerning Goodness-of-Fit testing.
Scope of the software
The Statistical Testing project will provide tools for the statistical comparison of distributions; the system will be usable in the context of other software projects, like, for instance, the Geant4 Test & Analysis Project.
Product perspective
This software product is meant to be a component of a software system for statistical tools for HEP data analysis. Since it adopts a component-based architecture, this software can be of interest also in a wider context.
General capabilities
- The user shall be able to compare binned distributions.
- The user shall be able to compare unbinned distributions.
- The user shall be able to compare binned and unbinned distributions, simply converting one kind into the other.
- The user shall be able to choose among a variety of goodness-of-fit tests.
General constraints
It will be based on AIDA interfaces, without any dependency on any specific analysis tool.
User characteristics
Users of the product are espected to be:
Operational environment
The software will operate in the same environment as the development of the Geant4 Toolkit.
Assumptions and dependencies
The system will depend on AIDA interfaces
UR 1.1 The user shall be able to compare one-dimensional binned distributions by means of chi-squared test.
UR 1.2 The user shall be able to compare one-dimensional binned distributions by means of approximated Anderson-Darling test.
UR 1.3 The user shall be able to compare one-dimensional binned distributions by means of exact Anderson-Darling test.
UR 1.4 The user shall be able to compare one-dimensional binned distributions by means of Fisz-Cramer-von Mises test.
UR 1.5 The user shall be able to compare one-dimensional binned distributions by means of Tiku test.
UR 1.6 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov-Smirnov test with Goodman'approximation.
UR 1.7 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov-Smirnov test.
UR 1.8 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov-Smirnov test with the Anderson-Darling weighting function.
UR 1.9 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov-Smirnov test with the Buning weighting function.
UR 1.10 The user shall be able to compare one-dimensional unbinned distributions by means of approximated Anderson-Darling test.
UR 1.11 The user shall be able to compare one-dimensional unbinned distributions by means of exact Anderson-Darling test.
UR 1.12 The user shall be able to compare one-dimensional unbinned distributions by means of Fisz-Cramer-von Mises test.
UR 1.13 The user shall be able to compare one-dimensional unbinned distributions by means of Fisz-Cramer-von Mises test with the Buning weighting function.
UR 1.14 The user shall be able to compare one-dimensional unbinned distributions by means of Kuiper test.
UR 1.15 The user shall be able to compare one-dimensional unbinned distributions by means of Tiku test.
UR 1.16 The user shall be able to compare one-dimensional unbinned distributions by means of Watson test.
UR 1.17 The user shall be able to compare curves by means of chi-squared test.
UR 1.18 The user shall be able to select the type of test.
UR 1.19 The user shall be able to compare a list of histograms.
UR 1.20 The user will be told that the test he chooses is not appropriate to the distribution.
Note: Requirements for other bayesian tests are currently under consideration.
UR 2.1 The user shall be able to compare one-dimensional binned distributions by means of chi-squared test.
UR 2.2 The user shall be able to compare one-dimensional binned distributions by means of Kolmogorov test with Conover modification.
UR 2.3 The user shall be able to compare one-dimensional binned distributions by means of Anderson-Darling test.
UR 2.4 The user shall be able to compare one-dimensional binned distributions by means of Cramer-von Mises test.
UR 2.5 The user shall be able to compare one-dimensional binned distributions by means of Tiku test.
UR 2.6 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov test with Goodman'approximation.
UR 2.7 The user shall be able to compare one-dimensional unbinned distributions by means of Kolmogorov test.
UR 2.8 The user shall be able to compare one-dimensional unbinned distributions by means of Anderson-Darling test.
UR 2.9 The user shall be able to compare one-dimensional unbinned distributions by means of Cramer-von Mises test.
UR 2.10 The user shall be able to compare one-dimensional unbinned distributions by means of Kuiper test.
UR 2.11 The user shall be able to compare one-dimensional unbinned distributions by means of Tiku test.
UR 2.12 The user shall be able to compare one-dimensional unbinned distributions by means of Lilliefors test.
UR 2.13 The user shall be able to compare one-dimensional unbinned distributions by means of Watson test.
UR 2.14 The user shall be able to compare curves by means of chi-squared test.
UR 2.15 The user shall be able to select the type of test.
UR 2.16 The user shall be able to compare a list of histograms.
UR 2.17 The user will be told that the test he chooses is not appropriate to the distribution.
Note: Requirements for other bayesian tests are currently under consideration.
UR 3.1 The user shall be able to compare k one-dimensional binned distributions by means of approximated Anderson-Darling test.
UR 3.2 The user shall be able to compare k one-dimensional binned distributions by means of exact Anderson-Darling test.
UR 3.3 The user shall be able to compare k one-dimensional unbinned distributions by means of approximated Anderson-Darling test.
UR 3.4 The user shall be able to compare k one-dimensional unbinned distributions by means of exact Anderson-Darling test.
UR 4.1 The user shall be able to compare two-dimensional binned distributions by means of chi-squared test.
UR 4.2 The user shall be able to compare two-dimensional binned distributions by means of Kolmogorov-Smirnov test.
UR 4.3 The user shall be able to compare two-dimensional unbinned distributions by means of Kolmogorov-Smirnov test.
UR 4.4 The user shall be able to compare two-dimensional unbinned distributions by means of Anderson-Darling test.
UR 4.5 The user shall be able to compare two-dimensional unbinned distributions by means of the energy test.
UR 5.1 The user shall be able to compare two-dimensional binned distributions by means of chi-squared test.
UR 5.2 The user shall be able to compare two-dimensional unbinned distributions by means of Kolmogorov test.
UR 5.3 The user shall be able to compare two-dimensional unbinned distributions by means of Anderson-Darling test.
UR 6.1 The user shall be able to compare multi-dimensional binned distributions by means of chi-squared test.
UR 6.2 The user shall be able to compare multi-dimensional unbinned distributions by means of Kolmogorov-Smirnov test.
UR 6.3 The user shall be able to compare multi-dimensional binned distributions by means of the energy test.
UR 7.1 The user shall be able to compare a binned and a unbinned distribution, converting the unbinned distribution in a binned one.
UR 7.2 The user shall be able to compare a binned and a unbinned distribution, converting the binned distribution in an unbinned one.
UR 8.1 The user shall be able to choose a confidence level.
UR 8.2 The user shall be able to calculate the exact probability of the comparison.
UR 9.1 The user shall be able to handle mono- and many-dimensional distributions.
UR 9.2 The user shall be able to transform a many-dimensional distribution into a lower-dimensional one.
UR 9.3 The user shall be able to select an interval of the original distribution to which the test is applied.
UR 9.4 The user shall be able to select a single channel of the original distribution to which the test is applied.
UR 9.5 The user shall be able to select a range of values of the original distribution to which the test is applied.
UR 9.6 The user shall be able to calculate the 3-standard deviation value of data belonging to a normal distribution to which the test is applied.
UR 9.7 The user shall be able to delete data out of 3-standard deviation interval.
UR 9.8 The user shall be able to select an integer number of standard deviations for each data.
UR 9.9 The user shall be able to use a toy Monte Carlo.
UR 10.1 The user shall be able to handle statistical experimental errors.
UR 10.2 The user shall be able to handle systematic experimental errors.
UR 11.1 The user shall be able to plot the original distributions.
UR 11.2 The user shall be able to plot the normalised distributions.
UR 11.3 The user shall be able to plot the cumulative distributions.
UR 11.4 The user shall be able to plot the result of the test.
UR A.1 The code shall be in ISO C++.
UR A.2 The system shall be based on AIDA interfaces.
UR A.3 The system shall not depend on any concrete analysis system.
UR A.4 The system should work on Geant4 supported platforms.
History |
||
09 Jan 2003 | S. Donadio, B. Mascialino, MG Pia | Created |
28 May 2003 | S. Donadio, B. Mascialino | Added UR1.10, UR4.6, UR4.7, UR4.8 |
14 March 2004 | S. Donadio, B. Mascialino, P.Viarengo | Added UR for Tiku algorithm |
29 March 2005 | B. Mascialino | Added UR for 1D, 2D, ND two-sample, one-sample and k-sample problems |
Last modified 24 March 2004 - Barbara Mascialino