Do you know of an data workflow management tool that is a) open?
We've just released Drake into open source. And it seems like exactly what you're looking for. Introducing Drake, a kind of make for data Not all features you mention have been implemented yet, but all are spec'ed out in the designdoc: Drake: the data processing workflow tool.