Trust Rules To Validate Your Data, AgileData Engineering Pattern #3

Content provided by AgileData. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by AgileData or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://podcastplayer.com/legal.

AgileData »
Trust Rules to validate your data, AgileData Engineering Pattern #3 - Episode #69

23h ago 12:01

MP3•Episode home

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Trust Rules to validate data.

The Trust Rules pattern provides automated data validation to ensure all incoming data is fit for purpose and trustworthy. It bakes in essential checks such as unique business keys and business effective dates, which run automatically upon data load or table refresh. Users can also define custom validation rules for specific columns. Results are collected, persisted, and surfaced via applications or alerts, with the system optimising validation for cost and speed through smart partitioning and clustered columns.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/167964917/pattern-name
Discover more AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you want to join us on the next podcast, get in touch over at https://agiledata.io/podcasts/#contact

Or if you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Buy the Green Book now!

Simply Magical Data

69 episodes

#Tech #AgileData