Pivot table subtotals in Pandas

Question

I have the following data:

Employee    Account Currency    Amount  Location
Test 2      Basic   USD         3000    Airport
Test 2      Net     USD         2000    Airport
Test 1      Basic   USD         4000    Town
Test 1      Net     USD         3000    Town
Test 3      Basic   GBP         5000    Town
Test 3      Net     GBP         4000    Town

I can manage to pivot by doing the following:

import pandas as pd
table = pd.pivot_table(df, values=['Amount'], index=['Location', 'Employee'], columns=['Account', 'Currency'], fill_value=0, aggfunc=np.sum, dropna=True)

Output:

                      Amount                  
Account            Basic         Net      
Currency             GBP   USD   GBP   USD
Location Employee                         
Airport  Test 2        0  3000     0  2000
Town     Test 1        0  4000     0  3000
         Test 3     5000     0  4000     0

How can I achieve subtotal by location and then a final grand total at the bottom. Desired output:

                  Amount                  
Account            Basic         Net      
Currency             GBP   USD   GBP   USD
Location Employee                         
Airport  Test 2        0  3000     0  2000
Airport  Total            3000     0  2000  
Town     Test 1        0  4000     0  3000
         Test 3     5000     0  4000     0
Town Total          5000  4000  4000  3000
Grand Total         5000  7000  4000  5000

I tried following the following. But it does not give the desired output. Thank you.

Solution 1

your pivot table

table = pd.pivot_table(df, values=['Amount'],
                       index=['Location', 'Employee'],
                       columns=['Account', 'Currency'],
                       fill_value=0, aggfunc=np.sum, dropna=True, )
print(table)

                  Amount                  
Account            Basic         Net      
Currency             GBP   USD   GBP   USD
Location Employee                         
Airport  Test 2        0  3000     0  2000
Town     Test 1        0  4000     0  3000
         Test 3     5000     0  4000     0

---

pandas.concat

pd.concat([
    d.append(d.sum().rename((k, 'Total')))
    for k, d in table.groupby(level=0)
]).append(table.sum().rename(('Grand', 'Total')))


                  Amount                  
Account            Basic         Net      
Currency             GBP   USD   GBP   USD
Location Employee                         
Airport  2             0  3000     0  2000
         Total         0  3000     0  2000
Town     1             0  4000     0  3000
         3          5000     0  4000     0
         Total      5000  4000  4000  3000
Grand    Total      5000  7000  4000  5000

---

Old Answer

for posterity

build sub totals

tab_tots = table.groupby(level='Location').sum()
tab_tots.index = [tab_tots.index, ['Total'] * len(tab_tots)]
print(tab_tots)

               Amount                  
Account         Basic         Net      
Currency          GBP   USD   GBP   USD
Location                               
Airport  Total      0  3000     0  2000
Town     Total   5000  4000  4000  3000

---

all together

pd.concat(
    [table, tab_tots]
).sort_index().append(
    table.sum().rename(('Grand', 'Total'))
)

Solution 2

Here is a two-liner that should work. The loc method allows for subsetting the rows by their indexes, since there is a multiIndex, I feed loc a tuple for the row insertion points on the left hand side. Using 'Town' without the tuple, pulls all the corresponding levels of the index.

In the second line, I have to drop the last row of the DataFrame from the sum and I do this using its shape attribute.

In[1]:
table.loc[('Town Total', ''),:] = table.loc['Town'].sum()
table.loc[('Grand Total', ''),:] = table.iloc[:(table.shape[0]-1), :].sum()

In[2]:
table

Out[2]: 
                     Amount                  
Account               Basic         Net      
Currency                GBP   USD   GBP   USD
Location    Employee                         
Airport     2             0  3000     0  2000
Town        1             0  4000     0  3000
            3          5000     0  4000     0
Town Total             5000  4000  4000  3000
Grand Total            5000  7000  4000  5000

Comments

[Clayton Tosatti]

if i have more indexes on rows how can i do that?

[Kyle]

How can you do the same but assure that "Total" for each "Location" is the bottom row? In this case it works because "Test" is before "Total", but I don't see how this would work if for instance it was "Zest" instead.