Integrating Deep Learning Algorithms to Overcome Challenges in Big Data Analytics ( etc.) (Z-Library)

📄 File Format: PDF

💾 File Size: 15.5 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

INTEGRATING DEEP LEARNING ALGORITHMS TO OVERCOME CHALLENGES IN BIG DATA ANALYTICS

📄 Page 3

Green Engineering and Technology: Concepts and Applications Series Editors: Brujo Kishore Mishra, GIET University, India and Raghvendra Kumar, LNCT College, India The environment is an important issue these days for the whole world. Different stra- tegies and technologies are used to save the environment. Technology is the application of knowledge to practical requirements. Green technologies encompass various aspects of technology that help us reduce the human impact on the environment and creates ways of sustainable development. This book series will enlighten the green technology in different ways, aspects, and methods. This technology helps people to understand the use of different resources to fulfill needs and demands. Some points will be discussed as the combination of involuntary approaches, government incentives, and a comprehen- sive regulatory framework that will encourage the diffusion of green technology; under- developed countries and developing states of small islands require unique support and measure in order to promote green technologies. Machine Learning and Analytics in Healthcare Systems Principles and Applications Edited by Himani Bansal, Balamurugan Balusamy, T. Poongodi, and Firoz Khan KP Convergence of Blockchain Technology and E-Business: Concepts, Applications, and Case Studies Edited by D. Sumathi, T. Poongodi, Bansal Himani, Balamurugan Balusamy, and Firoz Khan K P Big Data Analysis for Green Computing Concepts and Applications Edited by Rohit Sharma, Dilip Kumar Sharma, Dhowmya Bhatt, and Binh Thai Pham Handbook of Sustainable Development Through Green Engineering and Technology Edited by Vikram Bali, Rajni Mohana, Ahmed Elngar, Sunil Kumar Chawla, and Gurpreet Singh Integrating Deep Learning Algorithms to Overcome Challenges in Big Data Analytics Edited by R. Sujatha, S. L. Aarthy, and R. Vettriselvan For more information about this series, please visit: https://www.routledge.com/Green- Engineering-and-Technology-Concepts-and-Applications/book-series/CRCGETCA

📄 Page 4

INTEGRATING DEEP LEARNING ALGORITHMS TO OVERCOME CHALLENGES IN BIG DATA ANALYTICS Edited by R. Sujatha, S. L. Aarthy, and R. Vettriselvan

📄 Page 5

First edition published 2022 by CRC Press 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742 and by CRC Press 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN © 2022 Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, LLC Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyrighted material has not been acknowledged, please write and let us know so that we may rectify it in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC please contact mpkbookspermissions@tandf.co.uk Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Sujatha, R. (Computer science professor), editor. | Aarthy, S. L., editor. | Vettriselvan, R., editor. Title: Integrating deep learning algorithms to overcome challenges in big data analytics / edited by R. Sujatha, S.L. Aarthy, R. Vettriselvan. Description: Boca Raton : CRC Press, 2022. | Series: Green engineering and technology: concepts and applications | Includes bibliographical references and index. Identifiers: LCCN 2021016673 (print) | LCCN 2021016674 (ebook) | ISBN 9780367466633 (hbk) | ISBN 9781032104461 (pbk) | ISBN 9781003038450 (ebk) Subjects: LCSH: Machine learning‐‐Industrial applications. | Big data. | Artificial intelligence‐‐Industrial applications. | Algorithms. Classification: LCC Q325.5 .I475 2022 (print) | LCC Q325.5 (ebook) | DDC 006.3/1‐‐dc23 LC record available at https://lccn.loc.gov/2021016673 LC ebook record available at https://lccn.loc.gov/2021016674 ISBN: 978-0-367-46663-3 (hbk) ISBN: 978-1-032-10446-1 (pbk) ISBN: 978-1-003-03845-0 (ebk) DOI: 10.1201/9781003038450 Typeset in Times by MPS Limited, Dehradun

📄 Page 6

Contents Preface......................................................................................................................vii Editors .......................................................................................................................ix Contributors ..............................................................................................................xi Chapter 1 A Study on Big Data and Artificial Intelligence Techniques in Agricultural Sector..........................................................................1 D. Helen and C. Padmapriya Chapter 2 Deep Learning Models for Object Detection in Self-Driven Cars ....................................................................................................17 Anisha M. Lal and D. Aju Chapter 3 Deep Learning for Analyzing the Data on Object Detection and Recognition.................................................................................39 N. Anandh and S. Prabu Chapter 4 Emerging Applications of Deep Learning........................................57 S. Karthi, M. Kalaiyarasi, P. Latha, M. Parthiban, and P. Anbumani Chapter 5 Emerging Trend and Research Issues in Deep Learning with Cloud Computing......................................................................73 S. Karthi, D. Deepa, and M. Kalaiyarasi Chapter 6 An Investigation of Deep Learning ..................................................87 S. Karthi, P. Kasthurirengan, M. Kalaiyarasi, D. Deepa, and M. Sangeetha Chapter 7 A Study and Comparative Analysis of Various Use Cases of NLP Using Sequential Transfer Learning Techniques..............101 R. Mangayarkarasio, C. Vanmathi, Rachit Jain, and Priyansh Agarwal Chapter 8 Deep Learning for Medical Dataset Classification Based on Convolutional Neural Networks .....................................................121 S. Nathiya and R. Sujatha v

📄 Page 7

Chapter 9 Deep Learning in Medical Image Classification............................139 A. Suganya and S. L. Aarthy Chapter 10 A Comparative Review of the Role of Deep Learning in Medical Image Processing ..............................................................171 Erapaneni Gayatri and S. L. Aarthy Index......................................................................................................................195 vi Contents

📄 Page 8

Preface Data science revolves around two giants: Big Data analytics and Deep Learning. Both public and private sectors accumulate huge data of specific domains that hold useful information. Big Data analytics is the process of mining and extracting patterns from hefty data sets for prediction and, in turn, to make a beneficial decision. Yet, it is too challenging a procedure because of the non-uniformity of the data; because streaming data comes at a gushing speed; because the input source is highly distributed; and obviously because clarity is missing in input data. In 1957, Rosenblatt worked on the principle that makes a machine learn and classify the human way. This system requires two basic things, namely, the ability to recognize the complex system, and the ability to perform which requires voluminous amounts of data and, in turn, to provide information. This is the basis for Deep Learning, which works as the basis of neural networks with a number of layers of nodes between entry and exit. It is a well-known fact that data is expanding at a greater rate and by 2020 it will reach 40 zettabytes of data. Evidently, it is going to be challenging to handle and retrieve useful information from this huge data set. The optimization of Deep Learning algorithms over Big Data to accommodate the challenges is a great area of research. The chapters of this book act as a great support to address this issue in several areas. A thorough analysis of Big Data along with Deep Learning is mandatory to ensure flawless analysis or classifi- cation over uncategorized data. The research pertaining to addressing these challenges of Deep Learning over Big Data is in a preliminary and evolutionary phase. It is inevitable that it will provide insight into the advent of Deep Learning in all the fields and it is going to be a potential research area in data science that is growing at an extraordinary rate. Irrespective of the domain, Big Data analytics, along with the Deep Learning part of Machine Learning, makes the study interesting. IBM has predicted that the demand for data scientists will soar by 2020. It is clear that all target audiences will benefit by building a framework-based solution, which caters to each based on the requirements and provides the best platform irrespective of the domain. It helps in taking effective decisions with available huge generated data along with effective algorithms to solve the problem. For instance, in healthcare, various stakeholders are patients, medical practitioners, hospital operators, pharma and clinical researchers, and healthcare insurers. Various tasks from data integration, searching, processing, Machine Learning, streamed data processing, and visual data analytics are various tools that make a highly user-friendly healthcare environment. This book will bring forth, in a precise and concise manner, the details about all the right and relevant technologies and tools to simplify and streamline the formation of Big Data, along with Deep Learning System architects and designers, the data coders, statisticians, business people, researchers, and others. R. Sujatha S. L. Aarthy R. Vettriselvan vii

📄 Page 9

(This page has no text content)

📄 Page 10

Editors R. Sujatha is an Associate Professor at the School of Information Technology and Engineering, Vellore Institute of Technology. She earned a Ph.D. in data mining at Vellore Institute of Technology in 2017, an ME in computer science at Anna University in 2009, with the ninth rank in university, a Master’s degree in Financial Management at Pondicherry University in 2005, and a BE in computer science at Madras University in 2001. Dr. Sujatha has 17 years of teaching experience. She has organized and attended a number of workshops and faculty development programs. Dr. Sujatha is actively involved in the growth of Vellore Institute through various committees at both the academic and administrative levels and guides undergraduate, postgraduate, and doctoral students. She gives technical talks and acts as an advisory, editorial member, and technical committee member in various symposiums. Dr. Sujatha authored Software Project Management for College Students as well as research articles in high-impact journals. The Institution of Green Engineers awarded her the IGEN Women Achiever in 2021 in the category of future computing. Dr. Sujatha is interested in exploring different places to learn about various cultures and people. Her areas of interest include data mining, Machine Learning, software engineering, soft computing, Big Data, Deep Learning, and blockchain. S. L. Aarthy is an Associate Professor at the School of Information Technology and Engineering, Vellore Institute of Technology. She earned a Ph.D. degree in medical image processing at Vellore Institute of Technology in 2018, an ME in computer science at Anna University in 2010, and a BE in computer science at Anna University in 2007. Dr. Aarthy has 11 years of teaching experience. Her research interest includes image processing, soft computing, and data mining. She has published approximately 20 papers in reputed journals, and she guides undergraduate, postgraduate, and doctoral students. She is a life member of CSI and IEEE, as well as various institute committees. R. Vettriselvan is an Assistant Professor at AMET Business School, AMET (Deemed to be University), Chennai. He earned a BA in economics at Madurai Kamaraj University, an MBA at Anna University, an MPhil in research and development, and a Ph.D. at Gandhigram Rural Institute–Deemed University. Dr. Vettriselvan received an ICSSR Doctoral Fellowship, ICSSR, New Delhi; a GRI Fellowship; and a Post-Graduate Diploma in personnel management and industrial relations at Alagappa University, Karaikudi. He specializes in human ix

📄 Page 11

resource management and marketing. Dr. Vettriselvan has published 5 books and 65 research articles in SCOPUS, UGC, referred international, national peer-reviewed journals, and conference volumes. He has presented more than 60 research articles at national and international conferences in India, Zambia, Malawi, and the United States. He received a travel grant award in 2015 from the Population Association of America, USA, to present a research article in California. Dr. Vettriselvan has received recognition for “best paper”, “best paper presenter”, “best young faculty”, “bright educator”, “best academician of the year (male)”, and “most promising educator in higher education” across India. He is an editorial and review board member for a number of peer-reviewed journals. He is guiding three Ph.D. research scholars and has guided 30 MBA and 86 undergraduate projects. He has experience in NBA and NAAC acceptance documentation processes. He has 6 years of experience as a manager in human resources, NEST Abroad Studies Academy Private Limited, Madurai; lecturer and head of the department, School of Commerce and Management, DMI–St. Eugene University, Zambia; assistant professor, AMET Business School, AMET University, India; and lecturer at the School of Commerce and Management and Coordinator, Department of Research and Publication, DMI–St. John the Baptist University. He has been a review member for the National Council for Higher Education, Malawi (February 2020 to June 2020) to review the accreditation process. x Editors

📄 Page 12

Contributors S. L. Aarthy School of Information Technology and Engineering Vellore Institute of Technology Vellore, India D. Aju School of Computing Science and Engineering Vellore Institute of Technology Vellore, India N. Anand School of Computing Science and Engineering Vellore Institute of Technology Vellore, India P. Anbumani Department of Computer Science and Engineering V.S.B. Engineering College Karur, India Vanmathi C School of Information Technology and Engineering Vellore Institute of Technology Vellore, India D. Deepa School of Computing Science and Engineering Vellore Institute of Technology Vellore, India Erapaneni Gayatri School of Information Technology and Engineering Vellore Institute of Technology Vellore, India D. Helen AMET University Chennai, India S. Karthi Department of Computer Science and Engineering V.S.B. Engineering College Karur, India P. Kasthurirengan Department of Computer Science and Engineering V.S.B. Engineering College Karur, India Anish M. Lal School of Computing Science and Engineering Vellore Institute of Technology Vellore, India P. Latha Department of Computer Science and Engineering V.S.B. Engineering College Karur, India R. Mangayarkarasi School of Information Technology and Engineering Vellore Institute of Technology Vellore, India S. Nathiya School of Information Technology and Engineering Vellore Institute of Technology Vellore, India xi

📄 Page 13

C. Padmapriya Rose Mary College of Arts and Science Tirunelveli, India M. Parthiban Department of Information Technology V.S.B. Engineering College Karur, India S. Prabu School of Computing Science and Engineering Vellore Institute of Technology Vellore, India M. Sangeetha Department of Computer Science and Engineering V.S.B. Engineering College Karur, India A. Suganya School of Information Technology and Engineering Vellore Institute of Technology Vellore, India R. Sujatha School of Information Technology and Engineering Vellore Institute of Technology Vellore, India xii Contributors

📄 Page 14

1 A Study on Big Data and Artificial Intelligence Techniques in Agricultural Sector D. Helen and C. Padmapriya CONTENTS 1.1 Introduction.......................................................................................................2 1.1.1 The Life Cycle of Agriculture .............................................................2 1.2 The Role of Big Data in the Agricultural Sector ...........................................3 1.2.1 Overall Characteristics of Big Data Applicable to the Agricultural Sector ....................................................................................................4 1.2.2 The Processing Steps of Big Data in Agriculture...............................4 1.3 Some Cases of the Use of Big Data on Farm ................................................4 1.3.1 To Evade Food Scarcity of the Growing Population .........................4 1.3.2 Managing Pesticides and Farm Equipment........................................5 1.3.3 Supply Chain Management..................................................................5 1.3.4 Yield Prediction and Risk Management..............................................5 1.4 Challenges Faced by Farmers versus AI Solutions ........................................5 1.4.1 Forecasting Weather Conditions..........................................................6 1.4.2 Decision-Making ..................................................................................6 1.4.3 Diagnosing Defects in Soil and Weed Detection ...............................6 1.4.4 Nutrition Treatment..............................................................................6 1.5 AI Techniques in Agricultural Sector .............................................................6 1.5.1 Machine Learning.................................................................................6 1.5.1.1 Supervised Learning ..............................................................7 1.5.1.2 Unsupervised Learning..........................................................7 1.5.2 Neural Networks...................................................................................8 1.5.2.1 Working Process of Neural Network....................................8 1.5.3 The Expert System...............................................................................9 1.5.3.1 Components of the Expert System .......................................9 1.5.3.2 The Working Process of the Expert System ......................10 1.5.4 The Decision Tree..............................................................................10 1.5.4.1 Working Steps of the Decision Tree ..................................11 1.5.5 Support Vector Machine ....................................................................11 DOI: 10.1201/9781003038450-1 1

📄 Page 15

1.5.6 Random Forest ...................................................................................12 1.5.6.1 Working Steps of an RF .....................................................12 1.6 Application of AI in Agriculture ...................................................................13 1.6.1 Image Recognition .............................................................................13 1.6.2 Disease Detection ...............................................................................13 1.6.3 Field Management..............................................................................13 1.6.4 Driverless Tractor...............................................................................13 1.6.5 Weather Forecasting...........................................................................14 1.6.6 AI Agricultural Bots...........................................................................14 1.6.7 Reduction of Pesticide Usage ............................................................14 1.7 Advantages of Using AI in Agriculture ........................................................14 1.8 Conclusion ......................................................................................................15 References................................................................................................................15 1.1 INTRODUCTION Agriculture plays a vital role in the overall development of a country’s economy. Agriculture is the major source of livelihood (Guruprasad et al., 2019). To ensure the financial development of a country, it is necessary to monitor and estimate crop production (Shah & Shah, 2019). The main aim of the country is to increase crop yield using minimal resources (Kumar et al., 2015). The yield prediction is most important for universal food production. The accurate crop prediction and the timely report reinforce the overall food security. The crop yield prediction helps the government in planning for manufacturing, supply, and utilization of the food. The major issue for agricultural development is an accurate yield prediction for the number of crops involved in the production. Big Data and Artificial Intelligence (AI) is an emerging technology in the agricultural sector, which automates the agricultural process. In the agricultural field, AI techniques are applied in three major areas: (1) Artificial Robots, which harvest crops faster and in high volume; (2) Deep Learning and Computer Vision techniques, which monitor the health of crops and soil; and (3) Predictive Analysis Method, which predicts environmental changes such as temperature, rainfall, etc. AI techniques work efficiently in complex relationships between Input and Output variables (Jain et al., 2017). AI techniques depend on the semi-parametric and non-parametric structures, and the justification is based on accurate prediction (Breiman, 2001). Machine Learning, Artificial Neural Networks (Fortin et al., 2011; Liu et al., 2001), Regression Trees, and Support Vector Machines (Jaikla et al., 2008) are the popular AI techniques used for crop yield prediction. 1.1.1 THE LIFE CYCLE OF AGRICULTURE Soil Preparation: This is the first stage of farming where farmers sow the seeds in the soil. This process involves breaking up huge soil clumps and removing wreckage such as sticks, rocks, and roots. Fertilizers and organic matter are added according to the type of crop. 2 Integrating Deep Learning Algorithms to Overcome Challenges

📄 Page 16

Sowing: At this stage, climate conditions play an important role. The distance be- tween two seeds and the depth for planting seeds is necessary while sowing the seeds. Adding Manures and Fertilizers: Soil fertility is an important factor that helps farmers to grow nutritious and healthy crops. Fertilizers are chemical substances with the composition of nitrogen, phosphorus, and potassium that are added to the soil to increase crop productivity. Crop yield can be increased by adding manure and fertilizers. Irrigation: Humidity and soil moisture can be maintained at this stage. Watering the crops plays an important role here. Underwatering and overwatering can damage the growth of crops. Weeding: Weeds are the unwanted plants that grow along with the main crops. Weeding plays a necessary role in agriculture because the presence of weeds decreases crop yield, reduces crop quality, and increases production cost. Harvesting: In this phase, ripe crops are collected from the fields. A lot of laborers are needed during this activity. Harvesting also includes post-harvest handling such as cleaning, sorting, packing, and cooling. Threshing: This is the process of removing grains from the straw and chaff. This operation can be carried out manually or through machines. Threshing can be done by three methods that include rubbing, impact, and stripping. Storage: Foodgrains obtained after harvesting should be dried to remove moisture. The products are stored in such a way as to guarantee food security. Grains are stored in silos. This phase also includes the packing and transportation of crops (Figure 1.1). 1.2 THE ROLE OF BIG DATA IN THE AGRICULTURAL SECTOR Big Data plays a prominent role in the agricultural sector. Big Data is a combination of technology and analytics that can collect a huge amount of both structured and unstructured data. It compiles and processes these data effectively to assist in decision-making (Sonka, 2014). To process these large amounts of data, advanced tools are required. Real-time data analytics and automated processing are done with Big Data. In order to implement Big Data successfully, many techniques are used, Soil Preparation Sowing Adding Fertilizers & Manuers Irrigation Weeding Harvesting threshing Storage FIGURE 1.1 The life cycle of agriculture. A Study on Big Data and AI Techniques 3

📄 Page 17

such as predictive analytics, machine learning, time series analysis, classification and clustering, data mining, regression analytics, etc. More advancement in the technology of Big Data can establish a smart agri- cultural system. Agriculture is rapidly moving from traditional methods to these modern tools and technology. A farming process would be simpler and better with the help of Big Data. Big Data can solve complex problems in agricultural systems. Massive data are collected through various kinds of control devices, drones, sa- tellites, and sensors. These data are analyzed and used to plan for better crop production. Farmers can face critical problems regarding decision-making (Sonka, 2014). Using Big Data analytics, farmers can make predictions and appropriate decisions with the data drawn from the preceding years’ rainfall and climate con- ditions to avoid crop failure. Big Data not only creates smart farming but also influences the supply and marketing chain. Thus, Big Data helps the farmers make appropriate decisions, such as when to irrigate the field, as well as weather and crop health predictions. As there are labor shortages in agriculture, Big Data analysis can reduce the need for physical manpower. The end result of Big Data is to give better results at the right time from the gathered data. This advancement in tech- nology increases crop production and the economic condition of the country. 1.2.1 OVERALL CHARACTERISTICS OF BIG DATA APPLICABLE TO THE AGRICULTURAL SECTOR • Volume: Huge amount of data is stored. Massive data are collected through various kinds of control devices, drones, satellites, and sensors. • Velocity: Real-time analysis and decision-making are carried out using Big Data to predict weather forecasting, fertilizer requirements, pest infestation, water availability, nutritional status of the soil, and also can send alerts in emerging situations at an optimal time from the data which are collected. • Variety: A variety of file formats with different sizes from various devices is collected. The collected data may be text, images, audio, or videos. Appropriate decisions are carried out by connecting all these data points. 1.2.2 THE PROCESSING STEPS OF BIG DATA IN AGRICULTURE 1. A huge amount of data is combined and analyzed. 2. Real-time delivery of information is stored in mobile devices and smart equipment. 3. Data from weather conditions, soil conditions, GPS mapping, fertilizer/pes- ticide use, water resources, and field characteristics are analyzed. 4. Appropriate decisions are carried out by connecting all these data points. 1.3 SOME CASES OF THE USE OF BIG DATA ON FARM 1.3.1 TO EVADE FOOD SCARCITY OF THE GROWING POPULATION A growing population is one of the most common problems of the world. The government takes many steps to solve the problems faced by overpopulation. 4 Integrating Deep Learning Algorithms to Overcome Challenges

📄 Page 18

One of the foremost challenges is feeding the growing population. This can be achieved by increasing the yield of present farmlands. Farmers can face critical problems regarding decision-making. Using Big Data analytics, they can conclude predictions and make appropriate decisions with the help of data drawn from the preceding years’ rainfall and climate conditions to avoid crop failure. Thus, the farmers can increase the yield of crop production using these advanced technologies. 1.3.2 MANAGING PESTICIDES AND FARM EQUIPMENT Pesticides play an important role in agriculture. Pesticides include chemicals or natural products, which are used to get rid of different pest organisms such as insects, weeds, plant diseases, rodents, and mollusks. Farmers can manage and use the right amount of pesticides according to the soil type using Big Data. This can avoid massive chemicals in food production. Big Data increases the profit for the farmers as the crops are not getting destroyed by insects and weeds. Farming equipment is integrated with sensors so that the farmers can manage tractor availability, fuel refill alerts, and service due dates. 1.3.3 SUPPLY CHAIN MANAGEMENT Every year, food produced for human feeding is lost or wasted. The world struggles to bridge the gap between supply and demand. To solve these problems, food de- livery cycles from producers to markets must be minimized. Big Data can achieve supply chain improvement effectively by tracking and regulating the delivery truck routes. Farmers and all stakeholders are flexible in choosing their business partners and modern technology for the sake of production and development. Multistakeholders can easily collaborate and function together effectively in an agroecosystem using Big Data. It promotes transparent operations in the agricultural sector. The import and export of agricultural products are monitored using digital technology. Big Data helps farmers and distributors to enhance fleet management. 1.3.4 YIELD PREDICTION AND RISK MANAGEMENT Mathematical models are used to analyze data regarding weather conditions, yield, and chemicals. Thus, Big Data helps the farmers to make appropriate de- cisions, such as when to irrigate the field, as well as weather and crop health predictions. This modern technology also helps the farmers to plant their seeds with appropriate distance and depth. The historic yield records help the farmers in making appropriate decisions to overcome risk management. 1.4 CHALLENGES FACED BY FARMERS VERSUS AI SOLUTIONS The traditional methods followed by farmers are not able to satisfy the increasing demand of agricultural products. Some of the challenges faced by farmers are described in the following subsections (Dwivedy, 2011). A Study on Big Data and AI Techniques 5

📄 Page 19

1.4.1 FORECASTING WEATHER CONDITIONS Weather conditions play a major role in the agricultural life cycle. The tremendous increase in deforestation and pollution affects climate change. Hence, improper climate prediction leads to crop loss. Farmers face challenging situations for making proper decisions about sowing seeds and harvesting. The Regression Analysis will help farmers to forecast the weather conditions. This helps the farmers in making proper decisions about sowing the seeds and harvesting. 1.4.2 DECISION-MAKING Decision-making plays a vital role in the agricultural sector. The predictive analysis helps the farmers to make the correct decision about choosing the right seed for the right area in the following ways: • The accurate decision for sowing • Predicting the healthy crop yield • Recommendation of fertilizer depending on the plant situation 1.4.3 DIAGNOSING DEFECTS IN SOIL AND WEED DETECTION Insects, weeds, and diseases are the biological factors that affect the yield of crops. Controlling them is a major challenge for farmers. Anomaly Analysis helps to identify the defects in soil, and Image Classification techniques help to identify the weeds on the farm. 1.4.4 NUTRITION TREATMENT Nutrients such as nitrogen, phosphorous, and potassium found in soil are very es- sential for the growth of crops. The absence of nutrients can lead to poor-quality crops. Nutrients are very important for increasing crop yield. The Anomaly Detection Mechanism uses an unsupervised learning method to find an appropriate mineral level for growing healthier plants faster. 1.5 AI TECHNIQUES IN AGRICULTURAL SECTOR 1.5.1 MACHINE LEARNING Machine Learning (ML) is a subfield of Artificial Intelligence (AI). The ML is a tool that turns data into knowledge. The ML algorithms help the system learn from the dataset and accurately predict the result without being programmed explicitly (Okori & Obua, 2011). The ML algorithms receive the input dataset and apply statistical methods to forecast a result (Shastry et al., 2016). The ML methods can automatically discover hidden patterns within complex data. The hidden patterns can be used to forecast the future of the decision-making process. The process of the ML algorithm includes data collection, data pre-processing, training, evaluation, and tune (Snehal & Sandeep, 2014). The ML technique 6 Integrating Deep Learning Algorithms to Overcome Challenges

📄 Page 20

can be used in various applications such as recognizing anomalies, pattern re- cognition, prediction, neural networks, and so on. Machine Learning is a model of an automated data-processing algorithm. The Machine Learning algorithm models are: • Supervised Machine Learning: The outputs are labeled, and the inputs are mapped to subsequent outputs. • Unsupervised Machine Learning: The inputs are unlabeled, and the algo- rithms must discover patterns. • Reinforcement Machine Learning: Similar to a supervised ML, but as an alternative for a labeled output, there are rewards that are given to the algorithm. 1.5.1.1 Supervised Learning Supervised Learning is a machine learning task that builds an algorithm to learn and map the input for the desired output. The Supervised Learning system learns the labeled training data set and predicts the output of unseen data. In the Supervised Learning process, collected datasets are labeled as training data. The training data contains the pair of input data (vector) and preferred output data (supervisory signal). In Supervised Learning, the algorithm uses the input variable (X) and an output variable (Y) to learn and map the function of the input to the output. Y = f(X) The mapping function maps the input data (X) that predicts the output variable (Y) for that dataset (Figure 1.2). The key benefits of a Supervised Learning algorithm are as follows: • With the Supervised Learning algorithm, knowledge can be used to predict the output of new unnoticed data. • The knowledge gained by the algorithm can be used for optimizing performance. • Supervised Learning algorithms are more suitable for real-world problems. 1.5.1.2 Unsupervised Learning The Unsupervised Learning algorithms are trained using an unlabeled dataset and the models act on information without any guidance. Unsupervised Learning is a self- learning process where the model can discover the unknown patterns from the dataset. FIGURE 1.2 Supervised algorithm learning phase. A Study on Big Data and AI Techniques 7