Learn how to manage a modern data stack and get the most out of data in your organization!
Thanks to the emergence of new technologies and the explosion of data in recent years, we need new practices for managing and getting value out of data. In the modern, data driven competitive landscape the best guess approach-reading blog posts here and there and patching together data practices without any real visibility-is no longer going to hack it. The Informed Company provides definitive direction on how best to leverage the modern data stack, including cloud computing, columnar storage, cloud ETL tools, and cloud BI tools. You'll learn how to work with Agile methods and set up processes that's right for your company to use your data as a key weapon for your success . . . You'll discover best practices for every stage, from querying production databases at a small startup all the way to setting up data marts for different business lines of an enterprise.
In their work at Chartio, authors Fowler and David have learned that most businesspeople are almost completely self-taught when it comes to data. If they are using resources, those resources are outdated, so they're missing out on the latest cloud technologies and advances in data analytics. This book will firm up your understanding of data and bring you into the present with knowledge around what works and what doesn't.
Discover the data stack strategies that are working for today's successful small, medium, and enterprise companies Learn the different Agile stages of data organization, and the right one for your team Learn how to maintain Data Lakes and Data Warehouses for effective, accessible data storage Gain the knowledge you need to architect Data Warehouses and Data Marts Understand your business's level of data sophistication and the steps you can take to get to level up your data
The Informed Company is the definitive data book for anyone who wants to work faster and more nimbly, armed with actionable decision-making data.
By:
Dave Fowler,
Matthew C. David
Imprint: John Wiley & Sons Inc
Country of Publication: United States
Dimensions:
Height: 231mm,
Width: 185mm,
Spine: 13mm
Weight: 499g
ISBN: 9781119748007
ISBN 10: 1119748003
Pages: 256
Publication Date: 08 October 2021
Audience:
General/trade
,
ELT Advanced
Format: Paperback
Publisher's Status: Active
About This Book xiii Foreword xxi Introduction xxv Stage 1 Source (aka Siloed Data) 1 Chapter 1 Starting with Source Data 3 Common Options for Analyzing Source Data 4 Chapter 2 The Need to Replicate Source Data 11 Replicate Sources 12 Create Read-Only Access 14 Chapter 3 Source Data Best Practices 15 Keep a Complexity Wiki Page 15 Snippet Dictionary 16 Use a BI Product 17 Double Check Results 18 Keep Short Dashboards 19 Design Before Building 20 Stage 2 Data Lake (aka Data Combined) 23 Chapter 4 Why Build a Data Lake? 25 What Is a Data Lake? 26 Reasons to Build a Data Lake Summarized 27 Chapter 5 Choosing an Engine for the Data Lake 33 Modern Columnar Warehouse Engines 35 Modern Warehouse Engine Products 38 Database Engines 41 Recommendation 42 Chapter 6 Extract and Load (EL) Data 45 ETL versus ELT 46 EL/ETL Vendors 48 Extract Options 49 Load Options 51 Multiple Schemas 52 Other Extract and Load Routes 53 Chapter 7 Data Lake Security 55 Access in Central Place 56 Permission Tiers 57 Chapter 8 Data Lake Maintenance 59 Why SQL? 60 Data Sources 61 Performance 64 Upgrade Snippets to Views 68 Stage 3 Data Warehouse (aka the Single Source of Truth) 69 Chapter 9 The Power of Layers and Views 75 Make Readable Views 77 Layer Views on Views 78 Start with a Single View 81 Chapter 10 Staging Schemas 83 Orient to the Schemas 84 Pick a Table and Clean It 85 Other Staging Modeling Considerations 98 Building on Top of Staging Schemas 106 Chapter 11 Model Data with dbt 111 Version Control 111 Modularity and Reusability 112 Package Management 112 Organizing Files 113 Macros 113 Incremental Tables 114 Testing 115 Chapter 12 Deploy Modeling Code 119 Branch Using Version Control Software 119 Commit Message 120 Test Locally 120 Code Review 121 Schedule Runs 122 Chapter 13 Implementing the Data Warehouse 123 Manage Dependencies 124 Combine Tables Within Schemas 126 Combine Tables Across Schemas 128 Keep the Grain Consistent 130 Create Business Metrics 131 Keeping Accurate History 133 Chapter 14 Managing Data Access 135 How to Secure Sensitive Data in the Data Warehouse 137 How to Secure Sensitive Data in a BI Tool 140 Chapter 15 Maintaining the Source of Truth 143 Track New Metrics 144 Deprecate Old Metrics 147 Deprecate Old Schemas 149 Resolve Conflicting Numbers 150 Handling Ongoing Requests and Ongoing Feedback 151 Updating Modeling Code 152 Manage Access 153 Tuning to Optimize 156 Code Review All Modeling 157 Maintenance Checklist 158 Stage 4 Data Marts (aka Data Democratized) 161 Chapter 16 Data Mart Implementation 167 Views on the Data Warehouse 167 Segment Tables 168 Access Update 169 Chapter 17 Data Mart Maintenance 171 Educate Team 172 Identifies Issues 172 Identify New Needs 176 Help Track Success 176 Chapter 18 Modern versus Traditional Data Stacks: What's Changed? 177 What's Changed? 177 Chapter 19 Row-versus Column-Oriented Database 181 Row-Oriented Databases 182 Column-Oriented Databases 184 Summary 190 Chapter 20 Style Guide Example 191 Simplify 192 Clean 194 Naming Conventions 195 Share It 197 Chapter 21 Building an SST Example 199 First Attempt-Same Tables with Prefixes 199 Second Attempt-Operational Schema (Source Agnostic) 205 Third Attempt-Application Separate, Other Sources Smashed 207 Less Planning, More Implementing 209 Acknowledgments and Contributions 211 Index 213
MATT DAVID is the Product Marketing Manager for Platform Data at Atlassian. He formerly worked at Chartio as the Head of Data and before that at Udacity as Product Lead for the School of Data Science. DAVE FOWLER is Head of Analytics and Visualization at Atlassian and Founder of Chartio. He has worked in business intelligence for over ten years. His professional focus is on enabling anyone and everyone to explore and understand their data.
