Cloud Computing Solutions

List of Figures xvii List of Tables xix Foreword xxi Preface xxiii Acknowledgments xxv Acronyms xxvii Part I: Cloud Computing Architecture 1 1 Basics of Cloud Computing 3 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 1.1 Evolution of Cloud Computing 4 1.2 Cluster Computing 7 1.2.1 The Architecture of Cluster Computing Environment 7 1.2.2 Components of Computer Cluster 8 1.3 Grid Computing 9 1.3.1 Grid-Related Technologies 10 1.3.2 Levels of Deployment 11 1.3.3 Architecture of Grid Computing Environment 13 1.4 Mobile Computing 16 1.4.1 Characteristics of Mobile Computing 17 1.4.2 Characteristics of Mobile Networks 17 1.5 Summary 18 Exercises 18 References 19 2 Introduction to Cloud Computing 21 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 2.1 Definition of Cloud Computing 22 2.2 Characteristics of Cloud 22 2.2.1 Elasticity and Scalability 23 2.2.2 Metered and Billing of Service 23 2.2.3 Self-Service Allocation of Resources 23 2.2.4 Application Programming Interface (APIs) 24 2.2.5 Efficiency Measurement Service 24 2.2.6 Device and Location Interdependency 24 2.2.7 Customization 24 2.2.8 Security 25 2.3 Cloud Computing Environment 25 2.3.1 Access to Supporting Business Agility 25 2.3.2 Minimizing Investment Expenditures 25 2.3.3 Public Cloud Computing Environment 25 2.3.4 Private Cloud Computing Environment 26 2.3.5 Hybrid Cloud Computing Environment 27 2.3.6 Community Cloud Computing Environment 27 2.4 Cloud Services 28 2.4.1 Resources as a Service (RaaS) 28 2.4.2 Infrastructure as a Service (IaaS) 28 2.4.3 Platform as a Service (PaaS) 29 2.4.4 Software as a Service (SaaS) 30 2.4.5 Network as a Service (NaaS) 31 2.4.6 Desktop as a Service (DaaS/VDI) 32 2.4.7 Recovery as a Service (DRaaS) 32 2.5 Security Paradigms and Issues of Cloud Computing 32 2.6 Major Cloud Service Providers 33 2.6.1 IaaS CSPs 33 2.6.2 PaaS CSPs 35 2.6.3 SaaS CSPs 35 2.7 Summary 35 Exercises 36 References 37 3 Architectural Framework for Cloud Computing 39 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 3.1 Challenges of Cloud Computing Environment 40 3.2 Architectural Framework for Cloud Computing 41 3.2.1 Service-Oriented Architecture (SOA) 41 3.2.2 SOA Characterization 42 3.2.3 Life Cycle of Services in SOA 43 3.2.4 Integrating SOA and the Cloud 45 3.2.5 Cloud Architecture 46 3.3 Architectural Workflow and Co-ordination of Multiple Activities 49 3.3.1 Characteristics of Workflow 50 3.3.2 Need for Workflow 50 3.4 Examples of Workflow Tools 52 3.5 Summary 53 Exercises 53 References 54 4 Virtualization Environment in Cloud Computing 57 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 4.1 Introduction 58 4.1.1 Need of Virtualization in Cloud Computing Environment 58 4.1.2 Virtualization versus Traditional Approach 58 4.2 Virtualization and Virtual Machine 59 4.2.1 Advantages of Virtualization Technique in Cloud Computing Environment 60 4.2.2 Category of Virtual Machine 61 4.3 Virtualization Model for Cloud Computing 64 4.3.1 Distributed Resources of Physical Hosts 65 4.3.2 Hypervisor Monitoring Environment (HME) 65 4.3.3 Platform Service 66 4.3.4 Software Service 66 4.3.5 Broker Service 67 4.3.6 Business Service 67 4.4 Categorization of Guest OS Virtualization Techniques 68 4.4.1 Full Virtualization 68 4.4.2 Paravirtualization 69 4.4.3 Hardware-Assisted Virtualization 70 4.5 Mapping Technique of Virtual Machine to Physical Machine in a Private Cloud 71 4.6 Drawbacks of Virtualization 72 4.7 Summary 73 Exercises 74 References 75 5 Classification of Virtualization Environment 77 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 5.1 Introduction 78 5.2 Classification 78 5.2.1 Scheduling-Based Environment 79 5.2.2 Load Distribution-Based Environment 80 5.2.3 Energy-Aware-Based Environment 81 5.2.4 Operational-Based Environment 82 5.2.5 Distribution Pattern-Based Environment 85 5.2.6 Transaction-Based Environment 86 5.3 Summary 87 Exercises 87 References 88 Part II: Cloud Computing Data Storage 91 6 An Approach to Live Migration of Virtual Machines in Cloud Computing Environment 93 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 6.1 Introduction 94 6.2 Need of Live Migration of Virtual Machine 94 6.3 Advantages of Live Migration 94 6.4 A Design Approach to Live Migration 95 6.4.1 Live Migration Process 97 6.5 Security Issues 99 6.5.1 Possible Attacks 99 6.5.2 Solutions 100 6.6 Summary 100 Exercises 100 References 101 7 Reliability Issues in Cloud Computing Environment 103 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 7.1 Introduction 104 7.1.1 Research Problem Statement 105 7.1.2 Research Aim 105 7.1.3 Research Question 105 7.2 Literature Review 106 7.2.1 Cloud Service Models 107 7.2.2 Elements of Reliable Cloud Computing 108 7.2.3 Cloud Computing Gaps and Concerns 109 7.2.4 Trends in Cloud Computing 110 7.3 Reliability Issues in Cloud Computing Research 111 7.3.1 Research Methodology 111 7.3.2 Research Strategy 111 7.3.3 Data Collection 112 7.3.4 Sampling 112 7.3.5 Data Analysis and Findings 112 7.4 Findings 114 7.4.1 Lack of Effort to Address Reliability and Availability Issues 114 7.4.2 Performance Issues 115 7.4.3 Privacy Issues 115 7.5 Summary 115 Exercises 116 References 116 8 Cloud Database 123 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 8.1 Introduction 124 8.2 Non-Relational Data Models 124 8.2.1 Transactions in Cloud Databases 125 8.2.2 Advantages of Cloud Database 125 8.3 Heterogeneous Databases in DaaS 126 8.3.1 Relational and Non-Relational Database 126 8.3.2 Centralized and Distributed Database 126 8.3.3 Structured and Unstructured Database 127 8.3.4 Infrastructure-Based and Infrastructureless Databases 127 8.3.5 SQL-Based and NoSQL-Based Databases 128 8.4 Study of a Document-Oriented Cloud Database - MongoDB 129 8.4.1 Data Model 129 8.4.2 Replication 129 8.4.3 Sharding 130 8.4.4 Architecture 131 8.4.5 Consistency 132 8.4.6 Failure Handling 132 8.5 CAP Theorem for Cloud Database Transaction 132 8.6 Issues in Live Migration of Databases in Cloud 133 8.7 Cloud Database Classification Based on Transaction Processing 134 8.7.1 ACID-Based Cloud Database 134 8.7.2 NoACID-Based Cloud Database 135 8.8 Commercially Available Cloud Database Platform 136 8.8.1 Amazon Web Services 137 8.8.2 Microsoft Windows Azure 138 8.8.3 Google App Engine 138 8.9 Summary 138 Exercises 138 References 140 9 Cloud-Based Data Storage 143 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 9.1 Relevant Hadoop Tools 144 9.2 Hadoop Distributed File System (HDFS) 145 9.2.1 HDFS Architecture 145 9.2.2 Data Read Process in HDFS 146 9.2.3 Data Write Process in HDFS 147 9.2.4 Authority Management of HDFS 148 9.2.5 Limitations of HDFS 148 9.3 Data Mining Challenges with Big Data 149 9.4 MapReduce 151 9.4.1 MapReduce Architecture 151 9.4.2 MapReduce Procedure 152 9.4.3 Limitations of MapReduce 153 9.5 Next Generation of MapReduce: YARN 154 9.5.1 YARN Compared to MapReduce 1.0 (MRv1) 155 9.5.2 YARN and MapReduce 2.0 (MRv2) 156 9.5.3 YARN Architecture 156 9.5.4 Advantages of YARN 159 9.6 Classification of Data Mining Systems 160 9.6.1 Classification According to Kind of Databases Mined 160 9.6.2 Classification According to Kind of Knowledge Mined 160 9.6.3 Classification According to Kind of Techniques Utilized 161 9.6.4 Classification According to the Applications Adapted 161 9.7 Summary 162 Exercises 162 References 163 10 Auditing Concept in Cloud Computing 165 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 10.1 Introduction 166 10.2 Data Security in Cloud Computing Environment 166 10.2.1 Characteristics of a Secure Cloud Computing Environment 166 10.2.2 Need for Auditing in Cloud Computing Environment 167 10.2.3 Auditing Background Within Third-Party Service Provider 167 10.3 Cloud Auditing Outsourcing Life Cycle Phases 167 10.4 Auditing Classification 168 10.5 Auditing Service 169 10.5.1 How Third-Party Service Provider is Enabling Auditing Service 171 10.5.2 Auditing Process Analysis 171 10.5.3 Privacy and Integrity 174 10.5.4 Cloud-Auditing Architecture Analysis 176 10.6 Summary 177 Exercises 178 References 178 Part III: Cloud Computing Implementation, Security and Applications 181 11 Security Paradigms in Cloud Computing 183 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 11.1 Security Paradigms and Issues 184 11.2 Cloud Security Challenges 185 11.3 Cloud Economics 187 11.4 Security of Big Data in Cloud 187 11.4.1 The Biggest Risk: Data Breach 188 11.4.2 Data Loss 188 11.4.3 Account or Service Traffic Hijacking 189 11.4.4 Insecure Interfaces and APIs 189 11.4.5 Denial of Service 190 11.4.6 Malicious Insiders 190 11.4.7 Abuse of Cloud Users 190 11.4.8 Inadequate Due Diligence 191 11.4.9 Vulnerabilities in Shared Technology 191 11.5 Security as a Service in Cloud 191 11.6 Summary 194 Exercises 194 References 195 12 Privacy Preservation Issues in Cloud Computing 197 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 12.1 Privacy Issues in Cloud Storage 198 12.1.1 Encryption Methods 198 12.1.2 Access Control Mechanisms 199 12.1.3 Query Integrity/Keyword Searches 200 12.1.4 Auditability Schemes 200 12.2 Privacy and Security 201 12.2.1 Performance Unpredictability, Latency and Reliability 202 12.2.2 Portability and Interoperability 203 12.2.3 Data Breach Through Fiber-Optic Networks 204 12.2.4 Data Storage over IP Networks 204 12.2.5 Data Storage and Security in Cloud 205 12.3 Threats to Security in Cloud Computing 208 12.3.1 Basic Security 208 12.3.2 Network-Level Security 209 12.3.3 Application-Level Security 211 12.4 Security Issues in Cloud Deployment Models 215 12.4.1 Security Issues in a Public Cloud 215 12.4.2 Security Issues in a Private Cloud 216 12.5 Ensuring Security Against Various Types of Attacks 217 12.6 Survey of Privacy Preservation Using Fuzzy Set and Genetic Algorithm 219 12.6.1 Fuzzy-Based Approach for Privacy-Preserving Publication of Data 219 12.6.2 Privacy-Preserving Fuzzy Association Rules Hiding in Quantitative Data 220 12.6.3 A Rough Computing-Based Performance Evaluation Approach for Educational Institutions 222 12.6.4 A New Method for Preserving Privacy in Quantitative Association Rules Using Genetic Algorithm 223 12.6.5 Privacy Preserving in Association Rules Using a Genetic Algorithm 224 12.7 Summary 225 Exercises 225 References 226 13 Applications of Wireless Sensor Network in Cloud 233 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 13.1 Introduction 234 13.2 Architectural Issues of Combining Cloud Computing and Wireless Sensor Networks 234 13.3 Sensor Network Overview 235 13.3.1 Terminology 235 13.3.2 Routing Protocols in WSNs 236 13.4 Application Scenarios 237 13.4.1 Military Use 237 13.4.2 Weather Forecasting 237 13.4.3 Healthcare 238 13.4.4 Transport Monitoring 238 13.5 Summary 238 Exercises 239 References 239 14 Applications of Mobile Cloud Computing 243 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 14.1 What is Mobile Cloud Computing? 244 14.2 The Architecture of Mobile Cloud Computing 245 14.3 Characteristics of Mobile Cloud Computing 245 14.4 Advantages of Mobile Cloud Computing 246 14.5 Mobile Cloud Applications 248 14.5.1 Mobile Commerce 248 14.5.2 Mobile Learning 249 14.5.3 Mobile Healthcare 249 14.5.4 Mobile Gaming 250 14.5.5 Mobile Social Network 251 14.5.6 Multimedia Sharing 252 14.6 Summary 252 Exercises 252 References 253 15 Big Data in Cloud Computing 257 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 15.1 Introduction to Big Data 258 15.2 Big Data vs. Cloud Computing 259 15.3 Big Data and the Cloud 261 15.4 Cloud Computing to Support Big Data 262 15.4.1 Cloud Storage for Big Data Storage 262 15.4.2 Cloud Computing for Big Data Processing 262 15.4.3 Cloud Computing for Big Data Analytics 263 15.4.4 Cloud Computing for Big Data Sharing and Remote Collaboration 263 15.5 Opportunities and Challenges 263 15.5.1 Pros of Putting Big Data in the Cloud 263 15.5.2 Potential Challenges of Big Data in the Cloud 264 15.6 Summary 265 Exercises 265 References 266 Part IV: Cloud Computing Simulator Tools 269 16 CloudSim: A Simulator for Cloud Computing Environment 271 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 16.1 Introduction 272 16.2 Main Features 272 16.3 CloudSim Architecture 273 16.3.1 Modeling the Cloud 274 16.3.2 Modeling the VM Allocation 275 16.3.3 Modeling the Cloud Market 276 16.3.4 Modeling the Network Behavior 276 16.3.5 Modeling a Federation of Clouds 276 16.3.6 Modeling Dynamic Workloads 277 16.3.7 Modeling Data Center Power Consumption 278 16.3.8 Modeling Dynamic Entities Creation 278 16.4 Design and Implementation of CloudSim 279 16.5 Setting up Development Environments 282 16.6 How to Use CloudSim with Eclipse 282 References 285 17 OpenFaaS 287 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 17.1 Introduction 288 17.2 OpenFaaS Architecture 288 17.3 OpenFaaS Installation 289 17.3.1 Development Environment with Docker Swarm 290 17.3.2 Multi-Node Cluster with Docker Swarm 291 17.3.3 Production Environment with Kubernetes 293 17.3.4 Installing OpenFaaS Using Helm 297 17.3.5 Install OpenShift 298 17.4 Considerations 300 17.5 Operation of OpenFaaS 300 17.5.1 Setup and Configuration of the Open FaaS Command Line Tool 300 17.5.2 OpenFaaS Store 301 17.5.3 Management and Usage of Functions 301 17.5.4 Development of Functions 302 17.5.5 Working with Docker Registries 302 17.5.6 Web UI 303 References 303 18 OpenNebula 305 Prasant Kumar Pattnaik, Dac-Nhuong Le, Souvik Pal 18.1 Project Goal and Environment 306 18.2 Set Up Masternode with Frontend 306 18.2.1 Install Components 306 18.2.2 Starting the Frontend 306 18.3 Set Up Worker Node with KVM 307 18.3.1 Install Components 307 18.3.2 Establish an SSH Communication Pipeline between Master and Worker 308 18.3.3 Network Configuration 308 18.4 Register Worker Node 308 18.5 Deploy VM 309 References 311 19 OpenStack 313 Dac-Nhuong Le, Souvik Pal, Prasant Kumar Pattnaik 19.1 OpenStack 314 19.2 Terminologies in OpenStack 314 19.3 OpenStack Architecture 315 19.3.1 Compute (Nova) 316 19.3.2 Networking (Neuron) 316 19.3.3 Image 316 19.3.4 Object Storage (Swift) 316 19.3.5 Block Storage (Cinder) 316 19.4 Logical Architecture 317 19.5 OpenStack Installation Guide 318 19.5.1 Hardware Requirements 318 19.5.2 Networking Requirements 319 19.6 OpenStack Work 321 References 322 20 Eucalyptus 325 Souvik Pal, Dac-Nhuong Le, Prasant Kumar Pattnaik 20.1 Introduction to Eucalyptus 326 20.1.1 Eucalyptus Overview 326 20.1.2 Eucalyptus Architecture 326 20.1.3 Eucalyptus Components 327 20.2 Eucalyptus Installation 328 20.2.1 System Requirements 329 20.2.2 Services Placement 330 20.2.3 Eucalyptus Features 331 20.2.4 Networking Modes 332 20.2.5 Install Repositories 332 20.3 Configure Eucalyptus 335 20.4 Amazon Web Services Compatibility 337 References 337 Glossary 339 Authors 365