The problem with data flow diagrams is they only convey how data flows, not how well data flows. To fix that, I’d like to present a new type of diagram called a Data State Transition Diagram.
This is a diagram that helps you simplify and optimize data flows. The idea is based on an analysis technique conceived by Chris Wensel of Cascading that he described in an article called “The distance between simple and complex”. It is one of the best articles about simplifying data processing I’ve ever read.
Chris suggests data goes through 4 logical types of transitions as it flows in any system, and these state transitions can each be scored. The core data transitions he proposes are:
- Model Transitions – the conversion of data from one meta model to another (ie from xml to csv, or normalised to denormalised)
- Value Transitions – the creation of new data via value added business rules
- Phase Transitions – the reading or writing of data between dynamic memory and static persistent storage
- Location Transitions – the transfer of data between locations.
Through assigning costs to data transitions, he calculates the complexity cost of a data flow. Reducing the cost = reducing the complexity. He expertly observes that overly complex flows have early redundant steps which generate downstream compensation actions. In these cases the simplification is to remove the early redundant action, as well as the compensatory ones correcting for it downstream.
This idea is really big: Chris has produced a framework for scoring data flow complexity as a way to find simplifications. Chris says it perfectly:
Simplicity is about subtracting the obvious, and adding the meaningful.
My own contribution has been to turn this analysis into a drawing. I propose it because knowing how to simplify a data flow is not enough. You need to sell the idea, and to do that you need to communicate the simplification ideas to decision makers effectively.
A data state transition diagram has four swimlanes representing the four types of transition. Each step in the flow is a numbered box describing the transition going on, and these are joined by arrows denoting the sequence of states the data goes through.
This diagramming approach results in a visual representation that shows where your system is spending its energies, and how you can simplify and reduce the costs of the flows while achieving the same value.
Below is an example diagram clearly highlighting a poor data flow design. The original pdf version is available through this link: DataStateTransitionAnalysisExample
Discover how to simplify your data flows using Data State Transition Diagrams
Assessing the usefulness of the diagram:
In my workplace, the people who fund simplification programs want to “see” the opportunities for simplification before committing funds to them.
These are mostly programme managers who ultimately decide how to invest resources to achieve a business goal. As such, they aren’t necessarily all that technical, and they don’t have a lot of time or desire to study hundreds of pages of detailed technical analysis.
But they may have time to glance at a good diagram. And after all, it’s pretty pictures that sell ideas, right?
bytefreq 