Most elegant way to implement Pipe and Filter Pattern
Solution 1
There is a draft in the upcoming C++14 standard that covers this area:
C++ Pipelines - ISO/IEC JTC1 SC22 WG21 N3534 = 2013-03-15
And here is an implementation for it:
code.google.com/p/google-concurrency-library/source/browse/include/pipeline.h
Solution 2
What you are describing is some sort of streaming architecture or pipeline architecture. Standard C++ doesn't have anything this specific, but it provides you, as the library author, the necessary primitives in the language and standard library in order to build out such an architecture. Simply create classes and interfaces as per usual object-oriented programming and then instantiate them and connect them in a pipeline as per your needs.
For example you may have a Source
interface and a Sink
interface - and a Filter
abstract class that implements both Source
and Sink
, as well as a Pipe
class that implements both Source
and Sink
and just passes the data straight through. This is just one of many ways to name and organize such a framework.
benjist
Updated on June 04, 2022Comments
-
benjist almost 2 years
I want to create a Pipe and Filter based data handler that should manipulate incoming data sets like so, but not neccessarily limited to:
source pipe(could be a data set from a db) <-sink-source-> filter(add an additional field) <-sink-source-> filter(manipulate some more data / remove ie nullify data set)
I have an idea how such architecture will look like in C/C++. But given all the goodies that come with C++11's functional aspects, I hope this task can be done in an elegant way so that it is able to:
- easily scale and use it in a multithreaded environment (e.g. by performing filter tasks as lambda funcions, thus probably avoiding at least some thread handling)
- easily add and remove filters at runtime
- handle input streams lazily
-
benjist almost 11 yearsYou say "standard C++ doesn't have anything this specific", which is correct. However, I already know how a common implementation would look like in C++ (like in your example). But was interested in something
-
Andrew Tomazos almost 11 years@benjist: The generality of my answer matches the generality of your question. Your question would do better if you asked something more specific. "What is the most elegant way to implement design pattern X?" is overly broad and subjective.
-
benjist almost 11 yearsSorry, my previous comment was shortened. You're correct that the question is a bit general. But my question still is if there isn't a more elegant way towards the three requirements/wishes that I stated now with C++11. For an example, wouldn't it be possible to have a vector of filters and use std::transform to apply these filters on the data? Couldn't it make parallel tasks easier? Etc.
-
Andrew Tomazos almost 11 yearsIf you are interested in parallel execution of this form then take a look at the Intel Threading Building Blocks library.
-
benjist almost 11 yearsNice. Thanks a lot. That looks promising indeed.
-
malat almost 9 years@AndrewTomazos could you give more details on
Source
,Sink
andFilter
API ? Esp. the type of data that can be passed from one to the other ?