Share Next Entry
Dataflow, Introduction

So, I think I’ll start this by talking about dataflow techniques in general, and also my Clojure implementation of the concept.

“Dataflow,” as a term gets used all over the place in Computer Science. In general, it refers to any algorithm that has:

  1. Data elements with non-trivial dependencies, and
  2. Some automated procedure to cascade updates.

 A concrete example will help.

Say you were working on an application to track your customer’s nutritional intake. The model would include their daily ranges of calories, fats, and other nutritional measures. These would be assigned by dietitians based on rather complex formulae based on the user’s height, weight, sex, and so on.

Alright, so the user would enter what she ate during the day. In a conventional application, your code when then loop over these meals, adding up each nutritional (in some sort of aggregator object), and then compare these values to daily limits. Simple, right?

Now, imagine there were changes to the application. Perhaps the formulae change. Perhaps new criteria are added. Specific clients might wish certain formulae to apply in certain limited cases. Over several years, the simple architecture would start to grow complex. It might reach the case that a simple question like, “What are the calorie limits for diabetics from customer X?” gets very difficult to answer just from inspecting the code. The idea that the dietitians could directly examine the code seem preposterous.

A dataflow model works like this. For each requirement you enter an equation, ideally using a domain specific language that hides away the complex code. Into this model you could plug the user’s meals, and the dataflow engine would “auto-magically” cascade the values across the model. You write no loops, and the code looks more-or-less like the equations that appear in the dietitian’s text books. Done correctly, they can directly examine the code to find errors. This makes the model highly tractable. This is priceless.

Next: some dataflow algorithms.



Log in

No account? Create an account