hadoop : 1 map multiple reducers with each reducer having different functionality? possible?

11,959

Solution 1

I really wanted to answer this for you but it's already been asked. Hadoop one Map and multiple Reduce

Solution 2

No it is not possible. But you can implement your own reducer that will calculate all the stuff for you and output it.

You can make an custom writable for that.

Share:
11,959
daydreamer
Author by

daydreamer

Hello Viewer, Some of the places to see my work are BonsaiiLabs My Website

Updated on June 05, 2022

Comments

  • daydreamer
    daydreamer almost 2 years

    Here is an example:

    Is it possible to have same mapper run against multiple reducers at the same time? like

    map output : {1:[1,2,3,4,5,4,3,2], 4:[5,4,6,7,8,9,5,3,3,2], 3:[1,5,4,3,5,6,7,8,9,1], so on}  
    
    reducer1 : sum of all numbers  
    reducer2 : average of all numbers  
    reducer3 : mode of all numbers   
    
    act on the the same key like  
    
    reducer1 output: {1:sum of values, 2:sum of values, and so on}  
    reducer2 output: {1:avg of values, 2: avg of values and so on}
    reducer3 output: {1:mode of values, 2: mode of values, and so on}  
    

    and so on..Please let me know.