GoF Integration and
Systems Tests
The following tests, at the user layer level of the architecture, verify the
integration of the elements in the Comparison package and the correct
functioning of the GoF component as a whole:
Test |
Purpose |
testHisto1D.cpp |
Comparison of 1-D histograms (binned distributions) |
testCloud1D.cpp |
Comparison of 1-D clouds (unbinned distributions) |
testDPS.cpp |
Comparison of sets of points |
testEverything.cpp |
System test |
In order to test the implementation of any statistical test
we use the following strategy, based of Monte Carlo trials.
First, we check that the basic properties of any statistical test
are satisfied:
- the p-value must be 0 when the two samples we are
comparing do not overlap
- the p-value must be 1 when the two samples we are
comparing are identical
- on average, the more the two samples we are comparing
overlap, the higher must be the corresponding p-value
- (not rigorously valid in the case of binned distributions)
the p-value should not change for any monotonic
transformation of the data of the two samples we are comparing
- the p-value must not depend on the order of the two samples
If the statistical test fulfills all the above requirements,
then we perform a large number of pseudo-experiments, each consisting
in drawing randomly two samples from the same parent distribution.
Then we compare the p-values returned by the statistical test with
the ones calculated directly from the distribution of distances,
using the definition of p-value, which is the probability to get a
distance greater than or equal to the one observed, when two samples
are drawn from the same parent distribution.
There are some subtleties which must be taken into consideration when
this procedure is applied: we invite to read this paper
(published in the Proceeding of the PhyStat2003 Conference)
for these aspects and for any further detail of the system testing
procedure.
Last modified 03 March 2006 - Maria
Grazia Pia