What is Dataflow? Part 3: Doing the Practical
Apologies for the delay in getting this next section up - past few weeks have been super busy and then, hilariously, I was ill last week.
Read Part 1 here.
Read Part 2 here.
In Part 2 I wrote about how important diagrams have been throughout history. Understanding the 'big picture' has been important for every triumph of engineering. From bridges to skyscrapers to oil rigs and wind turbines, all of these have had diagrams backed by international standards which enabled them to be built.
The digital world hasn't quite managed that yet. In the other posts I've tried to drill home the point that modern digital businesses are often extremely siloed, communication and documentation isn't there and there is a lack of a common language between 'Business' and 'IT'.
This lack of understanding means organisations do not understand how data flows through their business and their supply chain.
It's the understanding of dataflow that's important here because it enables organisations to focus on optimising, securing and maintaining flows across an organisation rather than siloed teams patching things up where they can and not understand the upstream and downstream impact on the business.
Method and Layers
Going to preface this by saying that this may come across as complete common sense, and to some extent you'll be completely correct!
This is an example of how to create a very basic dataflow. But I will first start with understanding all of the People, Processes and Technology that I use to post on Tumblr.
So I start with six layers:
Ownership
Business Process
Application
System
Hardware
Infrastructure
What is important to remember here is that you do not have to be a specialist in every single layer.
A Business Analyst will feel much more at home in the Business Process Layer, while an Infrastructure Manager will be much more knowledgeable about the Infrastructure layer.
The important thing is that this Business & IT Diagram allows them to communicate more efficiently.
Let's Build a Dataflow!
In this example - There's an 'AyeforScotland' Element (the rectangle!) at the top. I'm the owner of everything below that element. The black lines are 'connections' representing the connectivity between the different elements.
Following the example, I'm responsible for' managing my blog 'Blog Management' which breaks down into smaller processes: Draft posts, schedule posts, answer anon abuse, and reblog shitposts.
Coming down to the Application Layer (red) - You can see that I draft and schedule posts using Tumblr Desktop and I'm using Firefox Web Browser for that.
But for answering anon abuse and reblogging shitposts, I'm using the Tumblr App.
In the Systems layer you can see I'm using Windows 11 on my PC (Hardware) and I'm using iOS on my iPhone.
Both my PC and iPhone connect to my BT Router.
And that's it for this Business & IT Diagram. I've shown clearly how I'm responsible for the processes and how I use the technology to perform those processes. I don't necessarily need to show everything on a single diagram because it would lose clarity.
This next Business & IT Diagram is much smaller, and establishes the relationships and dependencies on Tumblr to provide the service. And that's because we're complying with the laws and rules of a methodology.
In this diagram (probably need to zoom in to see it) I'm at the top left as 'AyeforScotland' and my 'BT Router' is spatially below me. Following the rules and laws of the method, that maintains the relationship that I have with the BT Router, I own it.
But I don't own the small 'Internet' that's next to it horizontally. I've simplified the concept of the internet for this example.
There's also two owners - 'Automattic' which owns and operates 'Tumblr' below it, with Tumblr being responsible for the 'Provision of Tumblr Services'.
Now naturally 'provision of Tumblr services' will break out into loads of sub-processes. Tumblr could map out their entire organisation (and if they need a hand, they can DM me!) But for this dataflow it's not really required.
Now both diagrams above are not dataflows. But close your eyes for a second and you can visualise what they are.
But because we've created our two diagrams, we understand the connectivity and using the software we can create the dataflow.
Now again, this is very basic. But when you put things into a dataflow context, you can put this down in front of a wide range of people from different business disciplines and they can start to optimise how the business works.
Here's a much larger Dataflow example, that you won't be able to read because it exceeds A0 printing size, but it should convey the scale.
If any of the connections or elements fail along this dataflow - The dataflow stops.
This costs organisations time and/or money.
So understanding dataflows allows IT people to articulate to business people "Hi boss, if this server goes down it will bring down this dataflow and cost the business $10,000 an hour" - Suddenly it's in a language they understand.
It helps with strategic decision-making, it helps with communication, it helps document how things *actually* work as opposed to how people think they work, and once you switch to thinking in terms of 'dataflow' it's hard to stop.
Conclusion
I can't wait to answer all the questions on the back of this.
Also one area I didn't go into is that each of the elements (rectangles) can also hold data (Financial data, Technical Specs, Risk & Cybersecurity metrics, Governance documentation etc).
It's also really easy to get started with it. You can start in any of the layers based on your area of work.
