For different purposes, computer hardware and software is often tested on certain benchmarks. Although being sometimes controversially discussed, such benchmarks set (at least) well defined environments to compare otherwise incomparable technologies, algorithms, and implementations.
Benchmark suites for symbolic computations are not as well established as for other areas of computer science. This is probably due to the fact that there are not yet well agreed upon aims and technologies of such a benchmarking. However, during the last years efforts towards systematic benchmark collections for symbolic computations were intensified.
Following the trend of the development of Computer Algebra software, we can classify these efforts roughly into two categories:
For further qualification of these efforts it would be of great benefit to unify the different benchmark approaches and to systematically collect the existing special and general benchmark data such that they are electronically available in a more or less uniform way. This would provide the community with an electronic repository of certified inputs and results that could be addressed and extended during further development. The SYMBOLICDATA project is set out to realize this.
However, the aims mentioned above do not reach far enough: symbolic computations often lead to voluminous data as input, output or intermediate results. Therefore, such a project has not only to collect benchmark data but also to develop tools to generate, store, manipulate, present and maintain it.
Consequently, the SYMBOLICDATA project has the following three goals:
In the first development stage of the project we concentrated on the general design principles of the tools and the data collection, thereby trying to achieve a balance between the necessary flexibility/extensibility on the one hand, and simplicity/practicability on the other.
A first application of our tools and concepts was realized on collections of data from two areas of Computer Algebra: Polynomial System Solving and Geometry Theorem Proving.
Further applications of our tools and concepts to collect data from other areas of symbolic computation are intended. For this, we seek the cooperation of persons and groups that have related data collections at their disposal and are willing to spend some effort to enter these data into the SYMBOLICDATA data base and provide the respective add-ons to already existing tools.
The SYMBOLICDATA project grew out of the special session on benchmarking at the 1998 ISSAC conference in Rostock which was organized by H. Kredel. Since then, the project has steadily developed from ideas to implementations and data collections and back. At the beginning of 1999, the authors joint forces with the symbolic computation groups of the University of Paris VI (J. C. Faugere, D. Lazard), of Ecole Polytechnique (J. Marchand, M. Giusti), and of the University of Saarbrücken (M. Dengel, W. Decker). Furthermore, the project was incooperated into the benchmarking activities of the Fachgruppe Computeralgebra of the Deutsche Mathematiker Vereinigung.
In this paper we report about the current state of the SYMBOLICDATA project. Based on the general design of SYMBOLICDATA which is outlined in section 2, we describe in section 3 how the above mentioned goals were realized. These concepts are illustrated in section 4 by two examples of data collections from different areas of Computer Algebra. Section 5 gives an overview of what deliverables the SYMBOLICDATA project has produced so far which is finally followed by some concluding remarks in section 6.