About the Benchmark

Description of the project.

Abstract

Understanding complex dynamical systems begins with identifying their topological structures, which expose the organization of the systems. This requires robust structural inference methods that can deduce structure from observed behavior. However, existing methods are often domain-specific and lack a standardized, objective comparison framework. We address this gap by benchmarking 13 structural inference methods from various disciplines on simulations representing two types of dynamics and 11 interaction graph models, supplemented by a biological experimental dataset to mirror real-world application. We evaluated the methods for accuracy, scalability, robustness, and sensitivity to graph properties. Our findings indicate that deep learning methods excel with multi-dimensional data, while classical statistics and information theory based approaches are notably accurate and robust. Additionally, performance correlates positively with the graph’s average shortest path length. This benchmark should aid researchers in selecting suitable methods for their specific needs and stimulate further methodological innovation.

License

This work is licensed under a CC BY 4.0 license.

Acknowledgement

Will update this section before NeurIPS 2024.

Update Plan

  • We will document the benchmark with ‘Read the Docs’.

  • We will update the page “Teams” with more details after the publish of the paper.

  • Now we are searching for possible solutions to include trajectories generated by graphs of 150, 200, and 250 nodes, as well as the trajectories with Gaussian noise of the remaining 10 graph types. As the total size would exceed 7.8 TB, we are still working on it.