Öppna denna publikation i ny flik eller fönster >>Visa övriga...
2022 (Engelska)Ingår i: IMA Journal of Numerical Analysis, ISSN 0272-4979, E-ISSN 1464-3642, Vol. 43, nr 5, s. 2585-2632Artikel i tidskrift (Refereegranskat) Published
Abstract [en]
Estimates of the generalization error are proved for a residual neural network with L random Fourier features layers z¯+1 = ¯z + ReK k=1 b¯k eiωkz¯ + ReK k=1 c¯k eiω k·x. An optimal distribution for the frequencies (ωk, ω k) of the random Fourier features eiωkz¯ and eiω k·x is derived. This derivation is based on the corresponding generalization error for the approximation of the function values f(x). The generalization error turns out to be smaller than the estimate ˆf 2 L1(Rd) /(KL) of the generalization error for random Fourier features, with one hidden layer and the same total number of nodes KL, in the case of the L∞-norm of f is much less than the L1-norm of its Fourier transform ˆf . This understanding of an optimal distribution for random features is used to construct a new training method for a deep residual network. Promising performance of the proposed new algorithm is demonstrated in computational experiments.
Ort, förlag, år, upplaga, sidor
Oxford University Press (OUP), 2022
Nyckelord
residual network, deep random feature networks, supervised learning, error estimates, layer-by-layer algorithm
Nationell ämneskategori
Beräkningsmatematik
Identifikatorer
urn:nbn:se:kth:diva-336842 (URN)10.1093/imanum/drac049 (DOI)000853541200001 ()2-s2.0-85174497733 (Scopus ID)
Anmärkning
QC 20250513
2023-09-212023-09-212025-05-13Bibliografiskt granskad