|
Test and Analysis Project |
Prepared by: Andreas Pfeiffer (CERN), Maria Grazia Pia (INFN Genova)
Introduction
This document collects the specific requirements concerning the Geant4 Test & Analysis project.
The Test & Analysis project will provide tools for statistical testing of Geant4 physics and related quantities, both at unit and system level. It addresses primarily the areas of physics validation and regression testing, by means of comparison between simulation results and reference data.
The software product is meant primarily as a tool in the context of the Geant4 Collaboration. It may be of interest also in a wider context, such as testing and validation of Geant4 by users, or in the LCG (LHC Computing Grid) environment.
The user shall be able to produce histograms of relevant physics quantities.
The user shall be able to compare simulated and reference data.
The user shall be able to perform analysis (e.g. fitting) on the data.
Geant4 Test and Analysis will be compatible for usage with the Geant4 Toolkit. It will be based on AIDA interfaces, without any dependency on specific analysis tool or AIDA.[1]
Users of the product are expected to be:
· Geant4 developers, as part of the testing process in the development and maintenance of the Geant4 Toolkit;
· Coordinators of Geant4 Working Groups (especially in the physics domain), as part of the testing process at the package or subsystem level, especially in the context of integration;
· Geant4 System Testing Team, as part of the system Testing process.
In addition, other potential users may be:
· Users of the Geant4 Toolkit, wishing to verify the results of their applications with respect to reference data or their own experimental results;
· Other projects with similar requirements, or using the Geant4 Toolkit (such as, for instance, the LHC Computing Grid project).
The software will operate in the same environment as the development of the Geant4 Toolkit.
This document relies on the assumption that the Geant4 Test and Analysis Project will be supported and adequately funded.
https://www.ge.infn.it/geant4/analysis/TandA/
Specific Requirements
UR 1.1 The user shall be able to compare a histogram with reference data.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: Geant4 electromagnetic Working Groups
Clarity: Clear
Verifiability: To be verified
Note: especially relevant to physics validation w.r.t. experimental data
UR 1.2 The user shall be able to compare a histogram with another reference histogram, both in a batch system and interactively.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: Geant4 electromagnetic Working Groups
Clarity: Clear
Verifiability: To be verified
Note: especially relevant to regression testing
UR 1.3 The user shall be able to compare a histogram with a function.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: Geant4 electromagnetic Working Groups
Clarity: Clear
Verifiability: To be verified
Note: especially relevant to physics validation w.r.t. theoretical predictions, or to comparison w.r.t. the result of a fit.
UR 1.4 The user should be able to calculate a confidence level as a result of the test, both in a batch system and interactively.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: STT coordinator
Clarity: Clear
Verifiability: To be verified
UR 1.5 The user shall be able to select a confidence limit for each test.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: Anaphe team
Clarity: Clear
Verifiability: To be verified
UR 1.6 The user shall be able to select a confidence limit for each test, w.r.t. which to compare the results.
Need: Useful
Clarity: Clear
Verifiability: To be verified
UR 2.1 The user should be able to produce default histograms of some typical physics quantities, and to compare them to some reference published data or theoretical functions, or to his/her own data (e.g. cross section of a Geant4 process).
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: Geant4 electromagnetic Working Groups
Clarity: T.b.d.
Verifiability: To be verified
Note: this requirement applies also to external users, such as generic Geant4 end-users in experiments.
UR 2.2 The reference to the data used for a comparison test should be accessible to the user.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: LowE e.m. WG
Clarity: T.b.d.
Verifiability: To be verified
UR 3.1 The test submission and the collection of results should be automated.
Need: Essential
Priority: T.b.d.
Stability: T.b.d.
Source: Geant4 LowE Working Group coordinator, G. Cosmo (for STT)
Clarity: Clear
Verifiability: To be verified
UR 3.2 It should be possible to activate tests selectively.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: Standard e.m. WG
Clarity: Clear
Verifiability: To be verified
UR 3.3 It should be possible to define a correlation between physics changes in Geant4 and tests to run.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: STT
Clarity: T.b.d.
Verifiability: To be verified
UR A.1 Statistical testing tools, such as the Kolmogorov-Smirnov test, should be used.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: Geant4 LowE electromagnetic Working Group, J. Apostolakis
Clarity: Clear
Verifiability: To be verified
UR B.1 The system should be based on AIDA interfaces.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: Geant4 TSB
Clarity: Clear
Verifiability: To be verified
UR B.2 The system should run on Geant4 supported platforms.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: STT
Clarity: Clear
Verifiability: To be verified
UR B.3 The system should not require additional licences w.r.t. what required for Geant4 development.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: TSB
Clarity: Clear
Verifiability: To be verified
UR B.4 The system should not introduce additional dependencies in Geant4.
Need: Essential
Priority: T.b.d.
Stability: Stable
Source: TSB
Clarity: Clear
Verifiability: To be verified
UR B.5 The data should be stored in HBOOK format.
Need: Useful
Priority: T.b.d.
Stability: T.b.d.
Source: Standard e.m. WG
Clarity: Clear
Verifiability: To be verified
Document Status
Document Title: |
Geant4 Test & Analysis URD |
||
Issue: |
Revision: |
Date: |
Reason for change: |
1 |
0 |
2 June 2002 |
Created |
1 |
1 |
18 September 2002 |
Reformatted |
1 |
2 |
26 September 2002 |
Corrections by G. Cosmo |
[1] This constraint obviously concerns the batch application of the software system; interactive applications will necessarily use the interactive tool of a specific AIDA implementation.