Detailed, practical guidance for implementing IoT cellular network connectivity solutions for software developers and electrical engineers, and project managers.
Focusing exclusively on using cellular connectivity for IoT devices, Cellular IoT presents a flexible approach, using algorithms and software designs, to drastically reduce the complexity of interacting with a wide variety of Cellular Communication Modules (CCMs) which lie at the heart of cellular modems. Written in an accessible style, this book is one of the first to cover all practical aspects of cellular network connectivity, from network and SIM selection through to custom algorithms for detecting and recovering from a wide variety of connectivity problems, and an innovative approach to reliably manage AT commands in modern cellular modems.
This book explains the factors related to establishing and maintaining cellular connectivity including
geography and topology, population density, SIM card (and connectivity provider) selection, antenna choice and placement, and CCM selection. The book also provides detailed examples and troubleshooting advice, showing how to transfer data using low-level sockets and also using a high-level protocol (HTTP), creating a brief, temporary connection for a primitive IoT device to send a small amount of data, and also establishing and maintaining a continuous cellular connection with full Internet access on powerful IoT devices running Linux.
Written by an author with considerable professional expertise and experience with cellular connectivity, Cellular IoT includes information on:
Platforms, tools, and debugging, covering tool-chain selection, computing/OS platforms, programming language choices, and running IoT connectivity code in a debugger Cellular network basics, covering base stations, range, cell towers, tracking areas and paging, frequency and modulation, bandwidth and latency, frequency bands, and SIM cards Similarities and differences across CCMs, frequency,
band, Radio Access Technology (RAT), protocol and data representation, selection and consequences Low-level communication protocols including transmission control protocol (TCP), user datagram protocol (UDP), point-to-point protocol (PPP), and custom hybrids for cellular IoT Full coverage, for the first time, of SMS, GNSS (available in most CCMs), obtaining precise time, and utilizing the power saving functionality of Extended Discontinuous Reception (eDRX) and Power Saving Mode (PSM) available in NB-IoT, LTE Cat M and LTE Cat 1 bis CCMs Entirely new and innovative software approach, Command Independent Processing (CIP), to systematically manage and execute AT commands across families of CCMs and integrating standardized (3GPP) AT commands with vendor specific ones to achieve greater software portability across CCMs.
Cellular IoT is an essential resource for software developers, hardware engineers, and project managers seeking to avoid connectivity pitfalls and be better able to diagnose and resolve newly encountered challenges in the field while drastically reducing the time required to produce reliable, IoT connectivity solutions.
Table of Contents Chapter 1 – Introduction 1.1 Wireless Connectivity Alternatives 1.2 Goals 1.3 The Fundamental Problem 1.4 Audience 1.5 Recommended Reading 1.6 Can One Size Fit All? 1.7 Hardware History 1.8 On-the-Move Connectivity Problems 1.9 Reference Implementations 1.10 Reference Microcontroller/OS Platform 1.11 Reference Cellular Communication Modules (CCMs) Family 1.12 A Few Words on Advice, Practices and Efficiency 1.12.1 Best Practice or Good Practice 1.12.2 Efficiency Is a Large Umbrella 1.12.2.1 Spatial and Temporal Efficiency 1.12.2.2 Data Efficiency 1.12.2.3 Developmental Efficiency 1.13 3G, 4G, 5G, 6G Chapter 2 - Platforms, Tools and Debugging 2.1 Importance of Tool-Chain Selection 2.2 An Expanded View of the Tool Chain 2.3 Computing/OS Platforms 2.4 Programming Language Choices 2.5 Running the Same Code on Development Computer and IoT Device 2.6 Running IoT Connectivity Code in a Debugger Chapter 3 - Cellular Network Basics 3.1 Standards 3.2 What do Cellular Networks do? 3.3 Are Cellular Networks Wireless? 3.4 What is a Cell? What is a Sector? 3.5 Omnidirectional Cellular Coverage 3.6 Cell Towers 3.7 How are Cellular Networks Identified? 3.8 How Are IoT Devices Identified 3.9 eNodeB IDs, and Cell IDs 3.10 Tracking Areas and Paging 3.11 Frequency and Modulation 3.11.1 Modulation 3.11.1.1 Radio Telegraphy 3.11.1.2 Amplitude Modulation (AM Radio) 3.11.1.3 Frequency Modulation (FM Radio) 3.11.1.4 Phase Modulation 3.12 Spectral Efficiency 3.13 Error Detection 3.13.1 Luhn Algorithm 3.14 Error Correction 3.15 LTE Modulation Techniques 3.15.1 Binary Phase Shift Keying (BPSK) 3.15.2 Quadrature Phase Shift Keying (QPSK) 3.15.3 Quadrature Amplitude Modulation (QAM) 3.16 Bandwidth and Latency 3.17 Range 3.18 Frequency Bands 3.18.1 Frequency Affects Range 3.19 Radio Access Technologies (RAT) and Categories 3.20 SIM Cards 3.21 What Happens When a Cellular Modem Switches On? 3.21.1 Network Selection, Cell Selection, Camping, and Cell Reselection 3.21.2 Network Registration 3.22 Handoff (also called Handover) 3.22.1 Maintaining Connectivity 3.22.2 Load Balancing 3.23 Sharing the Air 3.24 Timing Advance 3.24.1 Why is Timing Advance Useful? 3.24.2 How Accurate are Distance Estimates Using Timing Advance? 3.24.3 Timing Advance Band Depth and Maximum Range 3.25 Expressing Power Chapter 4 - SIM Card Basics 4.1 Mobile Virtual Network Operators (MVNOs) 4.2 Size 4.3 Native Versus Multi-SIMs or MNO Versus MVNO 4.4 Home Versus Roaming Access 4.5 SIM Factors Affecting Price and Coverage 4.5.1 How Much Do SIM Cards Cost? 4.5.2 Is there a Monthly Activation Fee? 4.5.3 Are There Fees for Activating and/or Deactivating SIM Cards? 4.5.4 How Much Does Data Cost 4.5.5 Is the Monthly Data FPooled”? 4.5.6 Are There Fees for Deactivated (but not Terminated) SIMS Cards? 4.5.7 Is There a Not-Yet-Activated Fee? 4.6 Text Messages (SMS) 4.7 Usage Limits 4.8 Firewalls 4.9 Replacing SIMs and/or Network Providers 4.10 Access Point Name (APN) Chapter 5 - Verify Cellular Connectivity 5.1 Preparation 5.1.1 Adequate Power 5.1.2 Activated SIM Card 5.1.3 Base Station in Range 5.1.4 SIM Card Authorization 5.1.5 Band Configuration 5.1.6 RAT Configuration 5.1.7 Automatic Registration 5.2 Try to Auto-Register 5.3 What Can Go Wrong? 5.3.1 Operating System Interference 5.3.2 Communicating With Modem 5.3.3 Malformed AT commands 5.3.4 Parsing Responses to AT Commands 5.3.5 Timing Problems 5.3.6 Unset or Incorrect Access Point Name (APN) 5.4 Modem Configuration for Auto-Registration Chapter 6 - Let’s Move Some Data 6.1 Low-level Sockets or High Level Protocols 6.2 Verify ServerServer is Running 6.3 Verify EchoServer is Running 6.4 USB or UART? 6.5 AT Commands — a Troubled Past 6.6 Unsolicited Response Codes (URCs) 6.7 A Handy Modem Program 6.8 AT Commands Manuals 6.9 Communicating with the Cellular Modem 6.10 Getting EchoServer Information from ServerServer 6.11 Bouncing Data off EchoServer 6.12 No Problems is Bad Luck Chapter 7 - Cellular Connectivity Regions 7.1 How Geography, Topology and Population Density Affect Connectivity 7.1.1 Geography and Topology 7.1.2 Population Density 7.2 Region Categories 7.2.1 Rural 7.2.2 Rural Town 7.2.3 Flat Farmland/Flat Arid 7.2.4 Mountainous 7.2.5 Suburban 7.2.6 Dense City 7.2.7 Interstate Highway 7.2.8 Uninhabited Chapter 8 - Cellular Communication Modules (CCMs) 8.1 CCM Worldwide Market Share 8.2 Frequency Band Usage 8.3 Protocol Implementation 8.4 Similarities and Differences Across CCMs 8.4.1 Single or Dual AT Command Channels 8.4.2 Different AT command Sets 8.4.3 Different Response Times for Similar or Identical Commands 8.4.4 Differing Response Formats 8.4.5 Differing Responses For Compound Statements 8.4.6 Different Timing Requirements 8.4.7 AT Command are not Thread-Safe 8.4.8 Support For Different Protocols 8.5 Consider the Whole CCM Family 8.6 CCM Firmware Bugs 8.7 CCMs, Are a Lot Like Sensors: Imprecise and not Entirely Reliable Chapter 9 - AT Commands (a New Approach) 9.1 Purpose of AT Commands 9.2 Problems of AT Commands 9.2.1 Maximum Response Time for an AT Command 9.3 Traditional Solution to Executing AT Commands and Extracting Responses 9.4 Command Independent Processing (CIP) 9.4.1 The Central Observation Underlying CIP 9.4.2 Fundamental Elements of CIP 9.4.2.1 AtParams 9.4.2.2 AtCommand 9.4.3 AT Command in CIP 9.4.3.1 Step 1 – Define a Name for a Command 9.4.3.2 Step 2 - Create a Set of Parameters for Each Command 9.4.3.3 Step 3 - Store the Command Name and AtParams Object in a Map 9.4.3.4 Step 4 - Create a Command Object 9.4.3.5 Step 5 - Pass Arguments to the Command Object (if necessary) 9.4.3.6 Step 6 - Perform the Command 9.4.3.7 Step 7 – Verify Success or Failure 9.4.3.8 Step 8 - Extract Response Information 9.4.3.9 AT Commands with Parameters 9.4.3.10 Timing Out 9.4.4 Using CIP Across CCM Families and Across Manufacturers Chapter 10 - CIP Design and Details 10.1 Pseudocode Conventions 10.1.1 Identifier Names 10.1.2 Angle Brackets 10.1.3 Constructors 10.1.4 Dot Operator 10.1.5 Unified Modeling Language (UML) 10.2 A Note on Objected-Orientation and Threads 10.3 AT Command Basics 10.3.1 Echoing 10.3.2 Enable/Disable Response Codes 10.3.3 Short or Long Response Codes 10.3.4 Line Terminators 10.3.5 Housekeeping 10.4 Categories of Responses to AT Commands 10.4.1 OK_ONLY 10.4.2 TEXT_OK 10.4.3 AFTER_COLON 10.4.3 OK_PLUS_AFTER_COLON 10.4.5 MULTI_RECEIVE 10.4.6 MULTI_SEND 10.4.7 MULTI_AFTER_COLON 10.5 Details of Command Independent Processing (CIP) 10.5.1 AtStep Purpose 10.5.2 AtStep Attributes 10.6 A “Factory Method” for Creating AtCommand Objects 10.7 Performing AT Commands 10.7.1 Why AT Commands Fail 10.7.2 Timing Out 10.7.3 Details of the Execute Method 10.7.4 Response Length 10.7.5 Hardware Timing 10.7.6 Combining Parameter Settings - Method Chaining 10.7.7 Assessing Success, and Multiple Tries 10.7.8 Multi-Line AT Commands - AtStep 10.7.9 A Second Example With a Regular Expressions 10.7.10 Integrating AtStep Into the execute Methods 10.8 AT Commands For Multiple Modems 10.8.1 The Simplest Case 10.8.2 Connectors 10.13.2.1 All Connectors Are Also Threads 10.13.2.2 Connectors Are Created Using a Factory Method 10.13.2.3 Custom AT Commands are Added In Static Blocks of Connectors 10.13.2.4 Where to Override Methods or Parts of Methods 10.8.3 An Asymmetrical Case – AtParamsNoOp Chapter 11 - Geographical Coverage, Signal Strength and Quality 11.1 Radio Technologies (RATs) 11.2 Cellular Network Coverage Maps 11.3 Signal Strength and Quality: RSSI, RSRP, RSRQ, SINR 11.3.1 RSSI and RSRP 11.3.2 RSRQ 11.3.3 SINR 11.3.4 Using Modem For to Report Signal Strength and Quality 11.4 Antenna Selection and Performance 11.4.1 Antenna Size 11.4.2 Passive Versus Active Antennas 11.4.3 Antenna Connectors 11.4.4 Antenna Placement 11.5 Antenna Testing 11.6 Geography and Signal Strength Must Be Considered Together Chapter 12 - Network Selection and Registration 12.1 Network Registration 12.2 Radio Access Technology (RAT) 12.3 Network Frequency Band Selection 12.4 PLMN Selection 12.4.1 Manual PLMN Selection 12.4.2 Automatic PLMN Selection 12.5 How to Create Your Own User Preference List 12.5.1 Reading the UPL and OPL 12.5.2 Modifying the UPL 12.6 Once a PLMN is Auto-Selected, Is it Always Selected? 12.7 Forcing the CCM Back to the PLMN Preference List 12.8 A Mysterious PLMN Selection Behavior 12.9 Troubleshooting Registration Problems 12.9.1 New Modem, Never Registered 12.9.2 Old Modem, Previously Registered 12.10 Anomalous Behavior Chapter 13 - Communication Protocols: TCP, UDP, PPP and Hybrids 13.1 Internet Protocol (IP) 13.2 Transmission Control Protocol (TCP) 13.3 Considering Data Consumption 13.4 User Datagram Protocol (UDP) 13.5 TCP Pros and Cons 13.6 Point-to-Point Protocol (PPP) 13.7 AT Commands for Data Transfer are Completely Unstandardized 13.8 PPP on Linux 13.8.1 Debugging PPP 13.9 Alternatives to PPP Chapter 14 - Thin Air 14.1 A Most Dramatic Case 14.1.1 Watching the Server 14.1.2 Packets Not Getting to the Server 14.2 What Was Going On? Thin Air 14.3 Why Did Thin Air Persist Over Hundreds of Miles? 14.4 How to Detect Thin Air 14.5 What to Do About Thin Air 14.6 Minimizing the Size of a Thin Air Region 14.7 A Hybrid UDP Protocol For Detecting Thin Air 14.8 Reducing (or Eliminating) Thin Air by PLMN or Band Selection 14.8.1 The Most Direct Approach 14.9 Putting the Hybrid Protocol to a Second Use Chapter 15 - Time and Location (GNSS) 15.1 Clarifying Terminology 15.2 Time 15.3 Location 15.4 Obtaining Time information 15.4.1 Real-Time Clock (RTC) 15.4.2 Cellular Modem 15.4.2.1 Additional Configuration 15.4.2.2 Local Time or UTC 15.4.2.3 Daylight Saving Time 15.4.2.4 Using Modem to Read the Clock 15.4.3 Get Time From a GNSS Receiver 15.4.4 Get Time From a Server 15.5 Sources of Location Information 15.6 Pros and Cons of CCM’s GNSS Receiver Versus Stand-Alone GNSS Receiver 15.7 Cold Start, Warm Start, Hot Start 15.8 Assisted GPS (A-GPS) 15.9 GNSS Antenna Selection 15.10 GNSS Receiver Placement 15.11 GNSS Accuracy and Precision 15.11.1 Improving Accuracy 15.12 NMEA Sentences 15.12.1 Using Modem to Read GNSS Sentences 15.13 Three Ways to Obtain Location Information 15.13.1 Simple AT Command Request for Location 15.13.2 Read Streaming Data From gpsd Daemon 15.13.2.1 For a Stand-Alone GNSS Receiver 15.13.2.2 For a CCM’s GNSS Receiver 15.13.3 Read Streaming Data Directly From CCM’s GNSS receiver 15.14 Understanding gpsd JSON Output 15.15 Writing Software To Capture and Process gpsd Output 15.16 GNSS Data Streamed From a CCM 15.17 NMEA 0183 15.17.1 Talker Sentence Format 15.17.1.2 RMC Sentence Format 15.17.1.3 GSV Sentence Format 15.17.2 NMEA Checksums 15.17.3 CCM GNSS Receivers Only Stream Some NMEA Sentences 15.18 Some Additional gpsd Utilities 15.19 Setting System Time From Cellular Network, GNSS Time, or Server Chapter 16 - Establishing and Maintaining a Cellular Connection 16.1 Modem Selection 16.2 Foundational Tasks 16.2.1 State 1 - Detecting CCM 16.2.1.1 Using Modem to Detect a CCM 16.2.2 State 2a - Initializing CMM 16.2.2.1 Viewing Modem’s Initializations 16.2.3 State 2b - Waiting to Retry 16.2.4 State 3 - Set MNO 16.2.5 State 4 - Checking Registration Status 16.2.5.1 Using Modem to Check Registration Status 16.2.6 State 5 - Connecting 16.2.7 State 6 - Manage Connection Chapter 17 - Sending and Receiving Text Messages (SMS) 17.1 Why Send/Receive Text Messages? 17.1.1 Need to “Push” Information to an IoT Device 17.1.2 Serverless IoT Devices That Interact With End-Users 17.2 Cost of Text Messaging via Cellular Modem 17.3 Application-to-Person (A2P) Messaging is Often Regulated 17.4 Overview of Sending/Receiving Text Messages 17.5 Sending Text Messages 17.5.1 Set the Message Format 17.5.2 Set Parameters for Sending 17.5.3 Specify the Destination Phone Number and the Text to Send 17.5.4 What if Sending an SM Fails? 17.5.5 Using Modem to Send a Text Message 17.6 Receiving and Reading a Text Messages 17.6.1 Configure the CCM 17.6.1.1 Set the Message Format 17.6.1.2 Configure SMS Storage Pitfall: Don’t Forget to Align Memory Blocks 17.6.1.3 Check for a Received Text Messages 17.6.1.4 Using Modem to List Text Messages 17.6.1.5 Delete a Text Messages 17.6.1.6 Using Modem to Delete a Text Message 17.7 SMS with Constrained Devices 17.7.1 Set the Message Format 17.7.2 Set Parameters for Writing to mem-2 17.7.3 Specify the Destination Phone Number and Text to Store 17.7.4 Send a Text Message Already Stored in mem-2 17.7.5 Verifying a Text Message Was Sent From mem-2 17.8 Integrating SMS into CIP Chapter 18 - Power Saving Modes and Techniques 18.1 What are Low-Power CCMs (LP-CCMs) 18.2 Plenty of Power, Most of the Time 18.3 Low-Power IoT Devices 18.3.1 Microcontroller Energy Consumption 18.3.2 Temperature Sensor Energy Consumption 18.4 Battery Capacity 18.5 Transmitter Power 18.6 Legacy (GSM) Power Consumption 18.7 Cellular Modem Energy Consumption 18.7.1 Additional Energy Consumption 18.8 Network Registration States – RRC_CONNECTED and RRC_IDLE 18.8.1 RRC_CONNECTED (without DRX) 18.8.1.1 Scenario 1 – Sending a Location Packet 18.8.1.2 Scenario 2 – Fetching an Over the Air Update 18.8.2 RRC_IDLE (without DRX) 18.8.3 Discontinuous Reception (DRX) 18.8.3.1 Discontinuous Reception in RRC_IDLE (iDRX) 18.8.3.2 Discontinuous Reception in RRC_CONNECTED (cDRX) 18.8.4 Registration Characteristics Summary 18.9 Latency 18.10 Using Low-Power CCM – Cat M and NB-IoT and Cat 1 bis 18.11 Power Saving Mode (PSM) 18.11.1 How to Enable PSM 18.11.1.1 Using Modem to Enter PSM 18.11.2 Verifying PSM and Possible Problems or Surprises 18.11.2.1 Using Modem to Check PSM Status 18.11.3 Actual PSM Cycle Length 18.11.4 Exiting PSM 18.11.4.1 Using Modem to Exit PSM 18.11.5 Sending Data From PSM Inactive 18.11.6 PSM Effectiveness 18.11.7 Integrating PSM into CIP 18.12 Extended Discontinuous Reception (eDRX) 18.12.1 How to Enable eDRX 18.12.1.1 Using Modem to Enable eDRX 18.12.2 Verifying eDRX Cycle Length 18.12.2.1 Using Modem to Check eDRX Status 18.12.3 Disabling eDRX 18.12.3.1 Using Modem to Disable eDRX 18.12.4 Integrating eDRX into CIP 18.13 When to Use PSM, eDRX or Both 18.14 Don't Trust the Numbers Appendix A - A UML Primer A.1 Assumptions A.2 UML Syntax A.3 Visibility (private, protected, public) A.4 Attribute/Parameter/Method Names and Types A.5 Class Attributes and Methods A.6 Aggregation A.7 Multiplicities A.8 Inheritance A.9 Interfaces A.10 Hidden Attributes A.11 Layout Closing Notes Appendix B - 3GPP AT Commands Used in This Book Appendix C - The Modem Utility Glossary
Matthew A. Brenner, President, Singular IoT, VA, USA. Matt manufactures electronic equipment for vehicle tracking and a line of cellular modems and has developed specialized software and hardware tools and custom communication protocols for investigating cellular connectivity anomalies. He has vast experience teaching computer science and software engineering at every level. He offers consulting services to help companies achieve the best cellular connectivity for their IoT devices.
