Streamlining Your Research Laboratory with Python

Contents Dedication. 1 Contents. 2 Preface. 10 Chapter 1 : Introduction. 11 Section 1.1 : Python Implementations. 11 Section 1.2 : Installing the Python Toolkit. 12 Section 1.3 : Python 3 vs. Python 2. 13 Section 1.4 : Python Package Index. 13 Section 1.5 : Programming Editors. 14 Section 1.6 : Notebook Editors. 15 Section 1.7 : Using the Jupyter Notebook Interface. 17 Section 1.8 : JupyterLite. 18 Section 1.9 : Things Change. 20 Section 1.10 : Key Takeaways. 20 Chapter 2 : Language Basics. 21 Section 2.1 : Python Interactive Console. 21 Section 2.2 : Data Types. 22 Section 2.3 : Variables and Literals. 24 Section 2.4 : Strings. 26 2.4.1 : Simple Strings. 26 2.4.2 : Multi-line Strings. 26 2.4.3 : Escape Characters in a String. 26 2.4.4 : Raw Strings. 28 2.4.5 : Formatted Strings. 28 2.4.6 : Strings as Objects. 30 2.4.7 : Characters and Encodings. 31 Section 2.5 : Expressions using Operators. 32 2.5.1 : Arithmetic Operators. 33 2.5.2 : Assignment Operators. 35 2.5.3 : Comparison Operators. 36 2.5.4 : Boolean Operators. 37 2.5.5 : Chaining Comparisons. 39 2.5.6 : Comparing Floating Point Numbers. 39 Section 2.6 : Functions and How to Use Them.. 40 2.6.1 : Invoking Functions. 40 2.6.2 : Built-in Functions. 41 2.6.3 : The math Module for Additional Mathematical Functions. 42 2.6.4 : The random Module for Pseudo-random Number Generation. 44 2.6.5 : The time and datetime Modules for Handling Dates and Times. 47 2.6.6 : The sys Module for System Interactions. 50 2.6.7 : Scope and Namespace. 51 Section 2.7 : Your First Python Program.. 52 Section 2.8 : Key Takeaways. 53 Chapter 3 : Data Structures. 55 Section 3.1 : Lists. 55 3.1.1 : Introducing Lists. 55 3.1.2 : Global Functions that Operate on Lists. 55 3.1.3 : Accessing List Elements. 56 3.1.4 : Slicing Lists. 57 3.1.5 : Lists Operators. 59 3.1.6 : Lists as Objects. 61 Section 3.2 : Tuples. 65 3.2.1 : Introducing Tuples. 65 Section 3.3 : Dictionaries. 66 3.3.1 : Introducing Dictionaries. 66 3.3.2 : Global Functions that Operate on Dictionaries. 67 3.3.3 : Accessing Dictionary Items. 67 3.3.4 : Dictionary Operators. 68 3.3.5 : Dictionaries as Objects. 69 Section 3.4 : Sets. 71 3.4.1 : Introducing Sets. 71 3.4.2 : Global Functions that Operate on Sets. 71 3.4.3 : Accessing Set Elements. 72 3.4.4 : Set Operators. 72 3.4.5 : Sets as Objects. 74 Section 3.5 : Destructuring Assignment. 75 Section 3.6 : Key Takeaways. 76 Chapter 4 : Controlling the Flow of a Program.. 77 Section 4.1 : Conditional Execution. 77 4.1.1 : If-statements. 77 4.1.2 : if-else Statements. 78 4.1.3 : If-elif-else Statements. 80 4.1.4 : If-statement Strategies. 82 4.1.5 : Truthy and Falsy Values. 85 4.1.6 : Conditional Expressions. 86 Section 4.2 : Repeated Execution. 88 4.2.1 : While-statements. 88 4.2.2 : For-statements. 93 4.2.3 : For-statements with Range. 95 4.2.4 : Break and Continue. 96 4.2.5 : Comprehensions. 98 Section 4.3 : Key Takeaways. 99 Chapter 5 : Custom Functions and Exceptions. 101 Section 5.1 : Defining Custom Functions. 101 Section 5.2 : Arguments and Parameters. 106 Section 5.3 : Names and Scope. 109 5.3.1 : Local vs. Global 109 5.3.2 : Built-in and Nonlocal Scope. 111 Section 5.4 : Scope vs Namespace. 113 Section 5.5 : Organizing your Code with Modules. 114 Section 5.6 : Decorators. 116 Section 5.7 : How Things Go Wrong. 118 Section 5.8 : Python Exceptions. 119 Section 5.9 : Handling Exceptions. 120 Section 5.10 : Raising Your Own Exceptions. 123 Section 5.11 : Key Takeaways. 126 Chapter 6 : Regular Expressions. 128 Section 6.1 : Matching Literal Text. 128 Section 6.2 : Alternation. 129 Section 6.3 : Defining and Matching Character Classes. 129 Section 6.4 : Metaclasses. 130 Section 6.5 : Pattern Sequences. 130 Section 6.6 : Repeating Patterns with Quantifiers. 130 Section 6.7 : Anchors. 132 Section 6.8 : Capturing Groups. 133 Section 6.9 : Regular Expressions in Python. 135 Section 6.10 : Project - A Formula Mass Calculator. 136 Section 6.11 : Key Takeaways. 143 Chapter 7 : Working with Data. 144 Section 7.1 : A File System Primer. 144 Section 7.2 : Text Files. 145 Section 7.3 : Reading and Writing Text Files. 147 Section 7.4 : Working with Comma-Separated Values (CSV) Files. 153 Section 7.5 : The csv Module. 157 Section 7.6 : Reading and Writing Excel Spreadsheet. 158 7.6.1 : openpyxl Workbook Object. 159 7.6.2 : openpyxl Worksheet Object. 160 7.6.3 : openpyxl Cell Object. 161 Section 7.7 : Project – Generate a Random Sample Layout in a Spreadsheet 161 Section 7.8 : Project – Forecast Monthly Sample Processing. 163 Section 7.9 : Managing the File System.. 168 7.9.1 : The Path Object. 168 7.9.2 : Path Properties. 169 7.9.3 : Path Attributes. 170 7.9.4 : Operating a Path. 170 7.9.5 : Combining Paths. 172 7.9.6 : The shutil Module for High-level File Operations. 172 Section 7.10 : Walking a File System Tree. 173 Section 7.11 : Project – Find Duplicate Files. 174 Section 7.12 : Working with Zip Files. 176 7.12.1 : ZipFile Object. 177 7.12.2 : zipfile.Path Object. 177 7.12.3 : Creating Zip Archives. 178 Section 7.13 : Working with Standard Data Formats. 179 7.13.1 : JSON - JavaScript Object Notation. 179 7.13.2 : json Python Module. 180 7.13.3 : XML - Extensible Markup Language. 182 7.13.4 : Python XML modules. 183 7.13.5 : Other Standard Data Formats. 189 Section 7.14 : Key Takeaways. 189 Chapter 8 : Web Resources. 191 Section 8.1 : TCP/IP Networks – What You Need to Know.. 191 8.1.1 : Internet Protocol (IP). 191 8.1.2 : Transmission Control Protocol (TCP). 192 8.1.3 : Connections and Ports. 192 8.1.4 : Application-layer Protocols. 192 8.1.5 : IPv4 vs. IPv6 Addresses. 193 8.1.6 : Proxy Servers. 193 Section 8.2 : Introduction to Hypertext Transfer Protocol (HTTP). 194 8.2.1 : The Uniform Resource Locator (URL). 195 8.2.2 : Anatomy of an HTTP Request. 196 8.2.3 : Anatomy of an HTTP Response. 198 Section 8.3 : Web Services and the Python Requests Module. 199 8.3.1 : HTTP GET Requests and the Response Object. 200 8.3.2 : HTTP POST Requests. 201 8.3.3 : Binary Responses. 202 8.3.4 : Customizing the Request Object. 205 8.3.5 : Verifying Certificates and Encryption. 205 8.3.6 : Other Request Module Options. 206 Section 8.4 : Project – Print Weather Forecast for a Location. 207 8.4.1 : National Weather Service API Web Service. 207 8.4.2 : Getting Forecast URL from Geolocation. 209 8.4.3 : Loading and Processing Forecast Data. 209 8.4.4 : Completed Program to Generate Temperature Forecast. 210 Section 8.5 : Project – Scraping HTML Page Content. 213 Section 8.6 : Key Takeaways. 218 Chapter 9 : Data Analysis and Visualization. 220 Section 9.1 : JupyterLab. 220 Section 9.2 : Scientific Plotting with Matplotlib. 222 9.2.1 : The pyplot Submodule. 223 9.2.2 : The pyplot.plot() function. 223 9.2.3 : Customizing a Plot. 224 9.2.4 : Multiple Curves on a Single Plot. 225 9.2.5 : Additional Plot Types. 227 9.2.6 : Multiple Axes on a Single Figure. 227 9.2.7 : Other Useful Functions. 229 9.2.8 : Project – Plotting Weather Forecast. 229 9.2.9 : Project – A Custom Microplate Heat Map. 231 9.2.10 : Other Scientific Plotting Libraries. 235 Section 9.3 : NumPy – Numerical Python. 236 9.3.1 : Creating ndarray Objects. 236 9.3.2 : Working with ndarray Objects. 236 9.3.3 : Accessing and Updating ndarray Elements. 237 9.3.4 : Broadcasting. 239 Section 9.4 : pandas DataFrame. 240 9.4.1 : Creating and Inspecting DataFrames. 240 9.4.2 : Filtering DataFrames. 243 9.4.3 : Project – A Screening Experiment. 246 Section 9.5 : SciPy – A Library for Mathematics, Science, and Engineering. 254 9.5.1 : Descriptive Statistics with SciPy. 254 9.5.2 : Hypothesis Testing. 255 9.5.3 : Project – Running Hypothesis Tests on Two Samples. 256 9.5.4 : Project – Comparing Liquid Handler Syringe Performance. 258 9.5.5 : Linear Regression. 260 9.5.6 : Fitting Nonlinear Models to Data. 261 9.5.7 : Project – Four-Parameter Logistic Regression. 264 Section 9.6 : Key Takeaways. 269 Chapter 10 : Report Generation. 271 Section 10.1 : BytesIO Object. 271 Section 10.2 : Generating Reports in Microsoft Word. 272 10.2.1 : Document Object. 273 10.2.2 : Paragraph Object. 274 10.2.3 : Run Object. 275 10.2.4 : Picture and InlineShape Objects. 276 10.2.5 : Table Object. 277 10.2.6 : Project – Generate a Complete Word Report. 279 Section 10.3 : Generating Microsoft PowerPoint Presentations. 282 10.3.1 : Presentation Object. 283 10.3.2 : Slide Objects. 284 10.3.3 : SlideShapes Object. 284 10.3.4 : Length Objects. 285 10.3.5 : Table Object. 287 10.3.6 : Project – Generate a PowerPoint Document with Figures and Tables. 289 Section 10.4 : Generating PDF File Reports. 293 10.4.1 : ReportLab PDF Generation Process. 294 10.4.2 : Creating a Canvas Object. 294 10.4.3 : Setting Canvas Styles. 294 10.4.4 : Managing Text Blocks with PDFTextObjects. 297 10.4.5 : Canvas State Stack. 299 10.4.6 : Drawing Images. 300 10.4.7 : PLATYPUS for Page Layout. 302 10.4.8 : Project – Generate a Complete PDF Report. 303 Section 10.5 : Sending E-mail Programmatically. 306 10.5.1 : Simple Mail Transfer Protocol 306 10.5.2 : SMTP Mail Server. 307 10.5.3 : Send a Simple Email Message. 307 10.5.4 : Sending Email Messages over a Secure Connection. 310 10.5.5 : Building an Email Message with Attachments. 311 Section 10.6 : Serving Results with an HTTP Server. 314 Section 10.7 : Key Takeaways. 315 Chapter 11 : Control and Automation. 317 Section 11.1 : Concurrency in Python. 317 Section 11.2 : Asynchronous Execution. 319 Section 11.3 : Concurrent Programs with AsyncIO.. 320 Section 11.4 : Asynchronous Instrument Control and Coordination. 324 11.4.1 : Project – Integrated Laboratory System Control and Coordination. 324 Section 11.5 : Communicating over a Serial Port. 332 11.5.1 : Reading Barcodes from a Serial Port. 334 11.5.2 : Project – Scanning Sample Tasks into a Running Controller. 339 Section 11.6 : Execute Remote Commands over HTTP. 341 11.6.1 : A Basic HTTP Server with aiohttp. 341 11.6.2 : Routing an HTTP Request to a Custom Python Function. 343 Section 11.7 : Persistent Network Connections using a WebSocket. 347 11.7.1 : A User Interface for an Asynchronous Networked Programs. 348 11.7.2 : WebSocketResponse and FileResponse Objects. 348 11.7.3 : Project – A Browser-Based WebSocket Message Broadcaster. 349 11.7.4 : Project – A Browser UI to Schedule Samples for Analysis. 357 Section 11.8 : Responding to File System Changes. 361 11.8.1 : Watching a Directory for Changes with watchfiles. 361 11.8.2 : File System Monitoring Options. 363 Section 11.9 : Executing Tasks on a Schedule. 365 11.9.1 : sched Module. 365 11.9.2 : Project – Taking and Sending Images on a Schedule. 367 Section 11.10 : Key Takeaways. 371 Postface. 373 References. 374 Appendix A: ASCII American Standard Code for Information Interchange. 377 Index. 378