Froese, Vincent, Hertrich, Christoph ORCID: 0000-0001-5646-8567 and Niedermeier, Rolf (2022) The computational complexity of ReLU network training parameterized by data dimensionality. Journal of Artificial Intelligence Research, 74. pp. 1775-1790. ISSN 1076-9757
Text (The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality)
- Published Version
Download (350kB) |
Abstract
Understanding the computational complexity of training simple neural networks with rectified linear units (ReLUs) has recently been a subject of intensive research. Closing gaps and complementing results from the literature, we present several results on the parameterized complexity of training two-layer ReLU networks with respect to various loss functions. After a brief discussion of other parameters, we focus on analyzing the influence of the dimension d of the training data on the computational complexity. We provide running time lower bounds in terms of W[1]-hardness for parameter d and prove that known brute-force strategies are essentially optimal (assuming the Exponential Time Hypothesis). In comparison with previous work, our results hold for a broad(er) range of loss functions, including `p-loss for all p ∈ [0, ∞]. In particular, we improve a known polynomial-time algorithm for constant d and convex loss functions to a more general class of loss functions, matching our running time lower bounds also in these cases.
Item Type: | Article |
---|---|
Additional Information: | © 2022 AI Access Foundation. |
Divisions: | Mathematics |
Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
Date Deposited: | 13 Oct 2022 11:24 |
Last Modified: | 12 Oct 2024 18:51 |
URI: | http://eprints.lse.ac.uk/id/eprint/116972 |
Actions (login required)
View Item |