FPGA Implementation of Hand-written Number Recognition Based on CNN

Daniele Giardino, Marco Matta, Francesca Silvestri, Sergio Spanò, Valerio Trobiani

Abstract


Convolutional Neural Networks (CNNs) are the state-of-the-art in computer vision for different purposes such as image and video classification, recommender systems and natural language processing. The connectivity pattern between CNNs neurons is inspired by the structure of the animal visual cortex. In order to allow the processing, they are realized with multiple parallel 2-dimensional FIR filters that convolve the input signal with the learned feature maps.  For this reason, a CNN implementation requires highly parallel computations that cannot be achieved using traditional general-purpose processors, which is why they benefit from a very significant speed-up when mapped and run on Field Programmable Gate Arrays (FPGAs). This is because FPGAs offer the capability to design full customizable hardware architectures, providing high flexibility and the availability of hundreds to thousands of on-chip Digital Signal Processing (DSP) blocks. This paper presents an FPGA implementation of a hand-written number recognition system based on CNN. The system has been characterized in terms of classification accuracy, area, speed, and power consumption. The neural network was implemented on a Xilinx XC7A100T FPGA, and it uses 29.69% of Slice LUTs, 4.42% of slice registers and 52.50% block RAMs. We designed the system using a 9-bit representation that allows for avoiding the use of DSP. For this reason, multipliers are implemented using LUTs. The proposed architecture can be easily scaled on different FPGA devices thank its regularity. CNN can reach a classification accuracy of 90%.


Keywords


machine learning; FPGA; accelerator; CNN.

Full Text:

PDF

References


G. Lo Sciuto, G. Susi, G. Cammarata e G. Capizzi: A spiking neural network-based model for anaerobic digestion process, in IEEE 23rd Int. Symp. on power electronics, electrical drives, automation and motion (SPEEDAM), 2016.

S. Brusca, G. Capizzi, G. Lo Sciuto e G. Susi: A new design methodology to predict wind farm energy production by means of a spiking neural network based-system, Int. Journal of Numerical Modelling: Electronic Networks, Devices and Fields, vol. 7, 2017.

Scarpato, N., Pieroni, A., Di Nunzio, L., Fallucchi, F.: E-health-IoT universe: A review 2017 International Journal on Advanced Science, Engineering and Information Technology, 7 (6), pp. 2328-2336

I. Dalmasso, I. Galletti, R. Giuliano, F. Mazzenga, “WiMAX Networks for Emergency Management Based on UAVsâ€, IEEE – AESS European Conference on Satellite Telecommunications. (IEEE ESTEL 2012), Rome, Italy, Oct. 2012, p. 1 - 6.

Pieroni, A., Scarpato, N., Di Nunzio, L., Fallucchi, F., Raso, M.

Smarter City: Smart energy grid based on Blockchain technology (2018) International Journal on Advanced Science, Engineering and Information Technology, 8 (1), pp. 298-306.

Hidra Amnur Customer Relationship Management and Machine Learning Technology for Identifying the Customer 2017 JOIV: International Journal on Informatics Visualization Vol 1, No 1

Salah, R.E.E, Zakaria, L.Q. A comparative review of machine learning for Arabic named entity recognition International Journal on Advanced Science, Engineering and Information Technology Volume 7, Issue 2, 2017, Pages 511-518

Giuliano, R., Mazzenga, F., Neri, A., Vegni, A.M., “Security access protocols in IoT capillary networksâ€, IEEE Internet of Things Journal, Vol. 4, Is. 3, Jun. 2017, p.645-657.

Guadagni, F., Zanzotto, F.M., Scarpato, N., Rullo, A., Riondino, S., Ferroni, P., Roselli, M. RISK: A random optimization interactive system based on kernel learning for predicting breast cancer disease progression (2017) Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10208 LNCS, pp. 189-196.

Ferroni, P., Zanzotto, F.M., Scarpato, N., Riondino, S., Guadagni, F., Roselli, M. Validation of a machine learning approach for venous thromboembolism risk prediction in oncology (2017) Disease Markers, 2017, art. no. 8781379

Fallucchi, F., Zanzotto, F.M. Inductive probabilistic taxonomy learning using singular value decomposition 2011 Natural Language Engineering

Fallucchi, F., Zanzotto F.M. Singular value decomposition for feature selection in taxonomy learning 2009 International Conference Recent Advances in Natural Language Processing, RANLP

Pazienza, M.T., Scarpato, N., Stellato, A., Turbati, A. Semantic Turkey: A browser-integrated environment for knowledge acquisition and management (2012) Semantic Web, 3 (3), pp. 279-292.

Ferroni, P., Zanzotto, F.M., Scarpato, N., Riondino, S., Nanni, U., Roselli, M., Guadagni, F.Risk Assessment for Venous Thromboembolism in Chemotherapy-Treated Ambulatory Cancer Patients (2016) Medical Decision Making, 37 (2), pp. 234-242.

Cardarilli, G.C., Cristini, A., Di Nunzio, L., Re, M., Salerno, M., Susi, G.: Spiking neural networks based on LIF with latency: Simulation and synchronization effects (2013) Asilomar Conference on Signals, Systems and Computers, pp. 1838-1842.

Khanal, G.M., Acciarito, S., Cardarilli, G.C., Chakraborty, A., Di Nunzio, L., Fazzolari, R., Cristini, A., Re, M., Susi, G.: Synaptic behaviour in ZnO-rGO composites thin film memristor 2017 Electronics Letters, 53 (5), pp. 296-298.

Cardarilli, G.C., Di Nunzio, L., Re, M., Nannarelli, A. ADAPTO: Full-adder based reconfigurable architecture for bit level operations (2008) Proceedings - IEEE International Symposium on Circuits and Systems, art. no. 4542197, pp. 3434-3437.

Khanal, G.M., Cardarilli, G., Chakraborty, A., Acciarito, S., Mulla, M.Y., Di Nunzio, L., Fazzolari, R., Re, M. A ZnO-rGO composite thin film discrete memristor (2016) IEEE International Conference on Semiconductor Electronics, Proceedings, ICSE, 2016-September, art. no. 7573608, pp. 129-132

Acciarito, S., Cardarilli, G.C., Cristini, A., Nunzio, L.D., Fazzolari, R., Khanal, G.M., Re, M., Susi, G.: Hardware design of LIF with Latency neuron model with memristive STDP synapses 2017 Integration, the VLSI Journal, 59, pp. 81-89.

Acciarito, S., Cristini, A., Di Nunzio, L., Khanal, G.M., Susi, G.: An aVLSI driving circuit for memristor-based STDP, 2016 12th Conference on Ph.D. Research in Microelectronics and Electronics, PRIME 2016,

Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Pontarelli, S., Re, M., Salsano, A., Implementation of the AES algorithm using a Reconfigurable Functional Unit, ISSCS 2011 - International Symposium on Signals, Circuits and Systems, Proceedings, art. no. 5978668, pp. 97-100.

Cardarilli, G.C., Di Nunzio, L., Re, M. Arithmetic/logic blocks for fine-grained reconfigurable units (2009) Proceedings - IEEE International Symposium on Circuits and Systems, art. no. 5118184, pp. 2001-2004

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.

Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Re, M., Silvestri, F. and Spanò, S. Energy consumption saving in embedded microprocessors using hardware accelerators, Telkomnika (Telecommunication Computing Electronics and Control), vol. 16, no. 3, pp. 1019-1026, 2018.

Gian Carlo Cardarilli, Luca Di Nunzio, Rocco Fazzolari, Daniele Giardino, Marco Matta, Marco Re, Francesca Silvestri and Sergio Spanò Efficient Ensemble Machine Learning implementation on FPGA using Partial Reconfiguration Lecture Notes in Electrical Engineering 2019 ARTICLE IN PRESS

Daniele Giardino, Marco Matta, Marco Re, Francesca Silvestri and Sergio Spanò : IP Generator Tool for Efficient Hardware Acceleration of Self-Organizing Lecture Notes in Electrical Engineering 2019 ARTICLE IN PRESS

Khanal, G., Acciarito, S., Cardarilli, G.C., Chakraborty, A., Di Nunzio, L., Fazzolari, R., Cristini, A., Susi, G., Re, M. ZnO-rGO composite thin film resistive switching device: Emulating biological synapse behavior (2017) Lecture Notes in Electrical Engineering, 429, pp. 117-123

Li, S., Wen, W., Wang, Y., Han, S., Chen, Y., Li, H.H An FPGA design framework for CNN sparsification and acceleration

(2017) Proceedings - IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines, FCCM 2017, art. no. 7966642, p. 28.

Huang, C., Ni, S., Chen, G. A layer-based structured design of CNN on FPGA (2018) Proceedings of International Conference on ASIC, 2017-October, pp. 1037-1040.

Simonetta, A., Paoletti, M.C. Designing digital circuits in multi-valued logic (2018) International Journal on Advanced Science, Engineering and Information Technology, 8 (4), pp. 1166-1172

Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Re, M., Lee, R.B. Integration of butterfly and inverse butterfly nets in embedded processors: Effects on power saving (2012) Conference Record - Asilomar Conference on Signals, Systems and Computers, art. no. 6489268, pp. 1457-1459

Silvestri, F., Acciarito, S., Cardarilli, G.C., Khanal, G.M., Di Nunzio, L., Fazzolari, R., Re, M. FPGA implementation of a low-power QRS extractor (2019) Lecture Notes in Electrical Engineering, 512, pp. 9-15.

Francesca, S., Carlo, C.G., Luca, D.N., Rocco, F., Marco, R. Comparison of low-complexity algorithms for real-time QRS detection using standard ECG database (2018) International Journal on Advanced Science, Engineering and Information Technology, 8 (2), pp. 307-314

Acciarito, S., Cardarilli, G.C., Di Nunzio, L., Fazzolari, R., Re, M. A wireless sensor node based on microbial fuel cell (2017) Lecture Notes in Electrical Engineering, 409, pp. 143-150

Iazeolla, G., Pieroni, A Power management of server farms

(2014) Applied Mechanics and Materials, 492, pp. 453-459.

Cardarilli, G.C., Di Nunzio, L., Massimi, F., Fazzolari, R., De Petris, C., Augugliaro, G., Mennuti, C. A wireless sensor node for acoustic emission non-destructive testing (2019) Lecture Notes in Electrical Engineering, 512, pp. 1-7




DOI: http://dx.doi.org/10.18517/ijaseit.9.1.6948

Refbacks

  • There are currently no refbacks.



Published by INSIGHT - Indonesian Society for Knowledge and Human Development