Back شبكة عصبونية التفافية Arabic ائوریشیملی عصبی شبکه‌سی AZB Xarxa neuronal convolutiva Catalan Convolutional Neural Network German Red neuronal convolucional Spanish Konvolutsiooniline närvivõrk ET شبکه عصبی پیچشی FA Réseau neuronal convolutif French רשת קונבולוציה HE Rete neurale convoluzionale Italian

Convolutional neural network

For other uses, see CNN (disambiguation).
Part of a series on
Machine learning
and data mining
Paradigms
Problems
Clustering
Dimensionality reduction
Structured prediction
Anomaly detection
Artificial neural network
Reinforcement learning
Learning with humans
Model diagnostics
Mathematical foundations
Machine-learning venues
Related articles

Convolutional neural network (CNN) is a regularized type of feed-forward neural network that learns feature engineering by itself via filters (or kernel) optimization. Vanishing gradients and exploding gradients, seen during backpropagation in earlier neural networks, are prevented by using regularized weights over fewer connections.[1][2] For example, for each neuron in the fully-connected layer, 10,000 weights would be required for processing an image sized 100 × 100 pixels. However, applying cascaded convolution (or cross-correlation) kernels,[3][4] only 25 neurons are required to process 5x5-sized tiles.[5][6] Higher-layer features are extracted from wider context windows, compared to lower-layer features.

They have applications in:

CNNs are also known as Shift Invariant or Space Invariant Artificial Neural Networks (SIANN), based on the shared-weight architecture of the convolution kernels or filters that slide along input features and provide translation-equivariant responses known as feature maps.[12][13] Counter-intuitively, most convolutional neural networks are not invariant to translation, due to the downsampling operation they apply to the input.[14]

Feed-forward neural networks are usually fully connected networks, that is, each neuron in one layer is connected to all neurons in the next layer. The "full connectivity" of these networks make them prone to overfitting data. Typical ways of regularization, or preventing overfitting, include: penalizing parameters during training (such as weight decay) or trimming connectivity (skipped connections, dropout, etc.) Robust datasets also increases the probability that CNNs will learn the generalized principles that characterize a given dataset rather than the biases of a poorly-populated set.[15]

Convolutional networks were inspired by biological processes[16][17][18][19] in that the connectivity pattern between neurons resembles the organization of the animal visual cortex. Individual cortical neurons respond to stimuli only in a restricted region of the visual field known as the receptive field. The receptive fields of different neurons partially overlap such that they cover the entire visual field.

CNNs use relatively little pre-processing compared to other image classification algorithms. This means that the network learns to optimize the filters (or kernels) through automated learning, whereas in traditional algorithms these filters are hand-engineered. This independence from prior knowledge and human intervention in feature extraction is a major advantage.[to whom?]

  1. ^ Venkatesan, Ragav; Li, Baoxin (2017-10-23). Convolutional Neural Networks in Visual Computing: A Concise Guide. CRC Press. ISBN 978-1-351-65032-8. Archived from the original on 2023-10-16. Retrieved 2020-12-13.
  2. ^ Balas, Valentina E.; Kumar, Raghvendra; Srivastava, Rajshree (2019-11-19). Recent Trends and Advances in Artificial Intelligence and Internet of Things. Springer Nature. ISBN 978-3-030-32644-9. Archived from the original on 2023-10-16. Retrieved 2020-12-13.
  3. ^ Zhang, Yingjie; Soon, Hong Geok; Ye, Dongsen; Fuh, Jerry Ying Hsi; Zhu, Kunpeng (September 2020). "Powder-Bed Fusion Process Monitoring by Machine Vision With Hybrid Convolutional Neural Networks". IEEE Transactions on Industrial Informatics. 16 (9): 5769–5779. doi:10.1109/TII.2019.2956078. ISSN 1941-0050. S2CID 213010088. Archived from the original on 2023-07-31. Retrieved 2023-08-12.
  4. ^ Chervyakov, N.I.; Lyakhov, P.A.; Deryabin, M.A.; Nagornov, N.N.; Valueva, M.V.; Valuev, G.V. (September 2020). "Residue Number System-Based Solution for Reducing the Hardware Cost of a Convolutional Neural Network". Neurocomputing. 407: 439–453. doi:10.1016/j.neucom.2020.04.018. S2CID 219470398. Archived from the original on 2023-06-29. Retrieved 2023-08-12. Convolutional neural networks represent deep learning architectures that are currently used in a wide range of applications, including computer vision, speech recognition, malware dedection, time series analysis in finance, and many others.
  5. ^ Habibi, Aghdam, Hamed (2017-05-30). Guide to convolutional neural networks : a practical application to traffic-sign detection and classification. Heravi, Elnaz Jahani. Cham, Switzerland. ISBN 9783319575490. OCLC 987790957.{{cite book}}: CS1 maint: location missing publisher (link) CS1 maint: multiple names: authors list (link)
  6. ^ Atlas, Homma, and Marks. "An Artificial Neural Network for Spatio-Temporal Bipolar Patterns: Application to Phoneme Classification" (PDF). Neural Information Processing Systems (NIPS 1987). 1. Archived (PDF) from the original on 2021-04-14.{{cite journal}}: CS1 maint: multiple names: authors list (link)
  7. ^ Valueva, M.V.; Nagornov, N.N.; Lyakhov, P.A.; Valuev, G.V.; Chervyakov, N.I. (2020). "Application of the residue number system to reduce hardware costs of the convolutional neural network implementation". Mathematics and Computers in Simulation. 177. Elsevier BV: 232–243. doi:10.1016/j.matcom.2020.04.031. ISSN 0378-4754. S2CID 218955622. Convolutional neural networks are a promising tool for solving the problem of pattern recognition.
  8. ^ van den Oord, Aaron; Dieleman, Sander; Schrauwen, Benjamin (2013-01-01). Burges, C. J. C.; Bottou, L.; Welling, M.; Ghahramani, Z.; Weinberger, K. Q. (eds.). Deep content-based music recommendation (PDF). Curran Associates, Inc. pp. 2643–2651. Archived (PDF) from the original on 2022-03-07. Retrieved 2022-03-31.
  9. ^ Collobert, Ronan; Weston, Jason (2008-01-01). "A unified architecture for natural language processing". Proceedings of the 25th international conference on Machine learning - ICML '08. New York, NY, US: ACM. pp. 160–167. doi:10.1145/1390156.1390177. ISBN 978-1-60558-205-4. S2CID 2617020.
  10. ^ Avilov, Oleksii; Rimbert, Sebastien; Popov, Anton; Bougrain, Laurent (July 2020). "Deep Learning Techniques to Improve Intraoperative Awareness Detection from Electroencephalographic Signals". 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (PDF). Vol. 2020. Montreal, QC, Canada: IEEE. pp. 142–145. doi:10.1109/EMBC44109.2020.9176228. ISBN 978-1-7281-1990-8. PMID 33017950. S2CID 221386616. Archived (PDF) from the original on 2022-05-19. Retrieved 2023-07-21.
  11. ^ Tsantekidis, Avraam; Passalis, Nikolaos; Tefas, Anastasios; Kanniainen, Juho; Gabbouj, Moncef; Iosifidis, Alexandros (July 2017). "Forecasting Stock Prices from the Limit Order Book Using Convolutional Neural Networks". 2017 IEEE 19th Conference on Business Informatics (CBI). Thessaloniki, Greece: IEEE. pp. 7–12. doi:10.1109/CBI.2017.23. ISBN 978-1-5386-3035-8. S2CID 4950757.
  12. ^ Zhang, Wei (1988). "Shift-invariant pattern recognition neural network and its optical architecture". Proceedings of Annual Conference of the Japan Society of Applied Physics. Archived from the original on 2020-06-23. Retrieved 2020-06-22.
  13. ^ Zhang, Wei (1990). "Parallel distributed processing model with local space-invariant interconnections and its optical architecture". Applied Optics. 29 (32): 4790–7. Bibcode:1990ApOpt..29.4790Z. doi:10.1364/AO.29.004790. PMID 20577468. Archived from the original on 2017-02-06. Retrieved 2016-09-22.
  14. ^ Mouton, Coenraad; Myburgh, Johannes C.; Davel, Marelie H. (2020). "Stride and Translation Invariance in CNNs". In Gerber, Aurona (ed.). Artificial Intelligence Research. Communications in Computer and Information Science. Vol. 1342. Cham: Springer International Publishing. pp. 267–281. arXiv:2103.10097. doi:10.1007/978-3-030-66151-9_17. ISBN 978-3-030-66151-9. S2CID 232269854. Archived from the original on 2021-06-27. Retrieved 2021-03-26.
  15. ^ Kurtzman, Thomas (August 20, 2019). "Hidden bias in the DUD-E dataset leads to misleading performance of deep learning in structure-based virtual screening". PLOS ONE. 14 (8): e0220113. Bibcode:2019PLoSO..1420113C. doi:10.1371/journal.pone.0220113. PMC 6701836. PMID 31430292.
  16. ^ Cite error: The named reference fukuneoscholar was invoked but never defined (see the help page).
  17. ^ Cite error: The named reference hubelwiesel1968 was invoked but never defined (see the help page).
  18. ^ Cite error: The named reference intro was invoked but never defined (see the help page).
  19. ^ Matusugu, Masakazu; Katsuhiko Mori; Yusuke Mitari; Yuji Kaneda (2003). "Subject independent facial expression recognition with robust face detection using a convolutional neural network" (PDF). Neural Networks. 16 (5): 555–559. doi:10.1016/S0893-6080(03)00115-1. PMID 12850007. Archived (PDF) from the original on 13 December 2013. Retrieved 17 November 2013.