hadoop : 1 map multiple reducers with each reducer having different functionality? possible?
11,959
Solution 1
I really wanted to answer this for you but it's already been asked. Hadoop one Map and multiple Reduce
Solution 2
No it is not possible. But you can implement your own reducer that will calculate all the stuff for you and output it.
You can make an custom writable for that.
Author by
daydreamer
Hello Viewer, Some of the places to see my work are BonsaiiLabs My Website
Updated on June 05, 2022Comments
-
daydreamer almost 2 years
Here is an example:
Is it possible to have same mapper run against multiple reducers at the same time? like
map output : {1:[1,2,3,4,5,4,3,2], 4:[5,4,6,7,8,9,5,3,3,2], 3:[1,5,4,3,5,6,7,8,9,1], so on} reducer1 : sum of all numbers reducer2 : average of all numbers reducer3 : mode of all numbers act on the the same key like reducer1 output: {1:sum of values, 2:sum of values, and so on} reducer2 output: {1:avg of values, 2: avg of values and so on} reducer3 output: {1:mode of values, 2: mode of values, and so on}
and so on..Please let me know.