UWP ObservableCollection sorting and grouping

Question

In UWP apps, how can you group and sort an ObservableCollection and keep all the live notification goodness?

In most simple UWP examples I've seen, there is generally a ViewModel that exposes an ObservableCollection which is then bound to a ListView in the View. When items are added or removed from the ObservableCollection, the ListView automatically reflects the changes by reacting to the INotifyCollectionChanged notifications. This all works fine in the case of an unsorted or ungrouped ObservableCollection, but if the collection needs to be sorted or grouped, there seems to be no readily apparent way to preserve the update notifications. What's more, changing the sort or group order on the fly seems to throw up significant implementation issues.

++

Take a scenario where you have an existing datacache backend that exposes an ObservableCollection of very simple class Contact.

public class Contact
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string State { get; set; }
}

This ObservableCollection changes over time, and we want to present a realtime grouped and sorted list in the view that updates in response to changes in the datacache. We also want to give the user the option to switch the grouping between LastName and State on the fly.

++

In a WPF world, this is relatively trivial. We can create a simple ViewModel referencing the datacache that presents the cache's Contacts collection as-is.

public class WpfViewModel 
{
    public WpfViewModel()
    {
        _cache = GetCache();
    }

    Cache _cache;

    public ObservableCollection<Contact> Contacts
    {
        get { return _cache.Contacts; }
    }
}

Then we can bind this to a view where we implement a CollectionViewSource and Sort and Group definitions as XAML resources.

<Window .....
   xmlns:scm="clr-namespace:System.ComponentModel;assembly=WindowsBase">

   <Window.DataContext>
      <local:WpfViewModel />
   </Window.DataContext>

    <Window.Resources>
        <CollectionViewSource x:Key="cvs" Source="{Binding Contacts}" />
        <PropertyGroupDescription x:Key="stategroup" PropertyName="State" />
        <PropertyGroupDescription x:Key="initialgroup" PropertyName="LastName[0]" />
        <scm:SortDescription x:Key="statesort" PropertyName="State" Direction="Ascending" />
        <scm:SortDescription x:Key="lastsort" PropertyName="LastName" Direction="Ascending" />
        <scm:SortDescription x:Key="firstsort" PropertyName="FirstName" Direction="Ascending" />
    </Window.Resources>

    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="*" />
            <RowDefinition Height="Auto" />
        </Grid.RowDefinitions>

        <ListView ItemsSource="{Binding Source={StaticResource cvs}}">
            <ListView.ItemTemplate>
                <DataTemplate>
                    <Grid>
                        <Grid.ColumnDefinitions>
                            <ColumnDefinition Width="100" />
                            <ColumnDefinition Width="100" />
                            <ColumnDefinition Width="*" />
                        </Grid.ColumnDefinitions>
                        <TextBlock Text="{Binding LastName}" />
                        <TextBlock Text="{Binding FirstName}" Grid.Column="1" />
                        <TextBlock Text="{Binding State}" Grid.Column="2" />
                    </Grid>
                </DataTemplate>
            </ListView.ItemTemplate>
            <ListView.GroupStyle>
                <GroupStyle>
                    <GroupStyle.HeaderTemplate>
                        <DataTemplate>
                            <Grid Background="Gainsboro">
                                <TextBlock FontWeight="Bold" 
                                           FontSize="14" 
                                           Margin="10,2"
                                           Text="{Binding Name}"/>
                            </Grid>
                        </DataTemplate>
                    </GroupStyle.HeaderTemplate>
                </GroupStyle>
            </ListView.GroupStyle>
        </ListView>

        <StackPanel Orientation="Horizontal" Grid.Row="1">
            <Button Content="Group By Initial" Click="InitialGroupClick" />
            <Button Content="Group By State" Click="StateGroupClick" />
        </StackPanel>

    </Grid>
</Window>

Then when the user clicks on the GroupBy buttons at the bottom of the window we can we can group and sort on the fly in code-behind.

private void InitialGroupClick(object sender, RoutedEventArgs e)
{
     var cvs = FindResource("cvs") as CollectionViewSource;
     var initialGroup = (PropertyGroupDescription)FindResource("initialgroup");
     var firstSort = (SortDescription)FindResource("firstsort");
     var lastSort = (SortDescription)FindResource("lastsort");

     using (cvs.DeferRefresh())
     {
         cvs.GroupDescriptions.Clear();
         cvs.SortDescriptions.Clear();
         cvs.GroupDescriptions.Add(initialGroup);
         cvs.SortDescriptions.Add(lastSort);
         cvs.SortDescriptions.Add(firstSort);
     }
}

private void StateGroupClick(object sender, RoutedEventArgs e)
{
     var cvs = FindResource("cvs") as CollectionViewSource;
     var stateGroup = (PropertyGroupDescription)FindResource("stategroup");
     var stateSort = (SortDescription)FindResource("statesort");
     var lastSort = (SortDescription)FindResource("lastsort");
     var firstSort = (SortDescription)FindResource("firstsort");

     using (cvs.DeferRefresh())
     {
         cvs.GroupDescriptions.Clear();
         cvs.SortDescriptions.Clear();
         cvs.GroupDescriptions.Add(stateGroup);
         cvs.SortDescriptions.Add(stateSort);
         cvs.SortDescriptions.Add(lastSort);
         cvs.SortDescriptions.Add(firstSort);
     }
}

This all works fine, and the items are updated automatically as the data cache collection changes. The Listview grouping and selection remains unaffected by collection changes, and the new contact items are correctly grouped.The grouping can be swapped between State and LastName initial by user at runtime.

++

In the UWP world, the CollectionViewSource no longer has the GroupDescriptions and SortDescriptions collections, and sorting/grouping need to be carried out at the ViewModel level. The closest approach to a workable solution I've found is along the lines of Microsoft's sample package at

https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/XamlListView

and this article

http://motzcod.es/post/94643411707/enhancing-xamarinforms-listview-with-grouping

where the ViewModel groups the ObservableCollection using Linq and presents it to the view as an ObservableCollection of grouped items

public ObservableCollection<GroupInfoList> GroupedContacts
{
    ObservableCollection<GroupInfoList> groups = new ObservableCollection<GroupInfoList>();

    var query = from item in _cache.Contacts
                group item by item.LastName[0] into g
                orderby g.Key
                select new { GroupName = g.Key, Items = g };

    foreach (var g in query)
    {
         GroupInfoList info = new GroupInfoList();
         info.Key = g.GroupName;
         foreach (var item in g.Items)
         {
             info.Add(item);
         }
         groups.Add(info);
    }

    return groups;
}

where GroupInfoList is defined as

public class GroupInfoList : List<object>
{
   public object Key { get; set; }
}

This does at least get us a grouped collection displayed in the View, but updates to the datacache collection are no longer reflected in real time. We could capture the datacache's CollectionChanged event and use it in the viewmodel to refresh the GroupedContacts collection, but this creates a new collection for every change in the datacache, causing the ListView to flicker and reset selection etc which is clearly suboptimal.

Also swapping the grouping on the fly would seem to require a completely seperate ObservableCollection of grouped items for each grouping scenario, and for the ListView's ItemSource binding to be swapped at runtime.

The rest of what I've seen of the UWP environment seems extremely useful, so I'm surprised to find something as vital as grouping and sorting lists throwing up obstacles...

Anyone know how to do this properly?

Solution 1

I've started putting together a library called GroupedObservableCollection that does something along these lines for one of my apps.

One of the key problems that I needed to solve was the refreshing of the original list that was used to create the group, i.e. I didn't want a user searching with slightly different criteria to cause the whole list to be refreshed, just the differences.

In its current form it probably won't answer all your sorting questions right now, but it might be a good starting point for others.

Solution 2

Best effort so far uses following helper class ObservableGroupingCollection

public class ObservableGroupingCollection<K, T> where K : IComparable
{
    public ObservableGroupingCollection(ObservableCollection<T> collection)
    {
        _rootCollection = collection;
        _rootCollection.CollectionChanged += _rootCollection_CollectionChanged;
    }

    ObservableCollection<T> _rootCollection;
    private void _rootCollection_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e)
    {
        HandleCollectionChanged(e);
    }

    ObservableCollection<Grouping<K, T>> _items;
    public ObservableCollection<Grouping<K, T>> Items
    {
        get { return _items; }
    }

    IComparer<T> _sortOrder;
    Func<T, K> _groupFunction;

    public void ArrangeItems(IComparer<T> sortorder, Func<T, K> group)
    {
        _sortOrder = sortorder;
        _groupFunction = group;

        var temp = _rootCollection
            .OrderBy(i => i, _sortOrder)
            .GroupBy(_groupFunction)
            .ToList()
            .Select(g => new Grouping<K, T>(g.Key, g));

        _items = new ObservableCollection<Grouping<K, T>>(temp);

    }

    private void HandleCollectionChanged(NotifyCollectionChangedEventArgs e)
    {
        if (e.Action == NotifyCollectionChangedAction.Add)
        {
            var item = (T)(e.NewItems[0]);
            var value = _groupFunction.Invoke(item);

            // find matching group if exists
            var existingGroup = _items.FirstOrDefault(g => g.Key.Equals(value));

            if (existingGroup == null)
            {
                var newlist = new List<T>();
                newlist.Add(item);

                // find first group where Key is greater than this key
                var insertBefore = _items.FirstOrDefault(g => ((g.Key).CompareTo(value)) > 0);
                if (insertBefore == null)
                {
                    // not found - add new group to end of list
                    _items.Add(new Grouping<K, T>(value, newlist));
                }
                else
                {
                    // insert new group at this index
                    _items.Insert(_items.IndexOf(insertBefore), new Grouping<K, T>(value, newlist));
                }
            }
            else
            {
                // find index to insert new item in existing group
                int index = existingGroup.ToList().BinarySearch(item, _sortOrder);
                if (index < 0)
                {
                    existingGroup.Insert(~index, item);
                }
            }
        }
        else if (e.Action == NotifyCollectionChangedAction.Remove)
        {
            var item = (T)(e.OldItems[0]);
            var value = _groupFunction.Invoke(item);

            var existingGroup = _items.FirstOrDefault(g => g.Key.Equals(value));

            if (existingGroup != null)
            {
                // find existing item and remove
                var targetIndex = existingGroup.IndexOf(item);
                existingGroup.RemoveAt(targetIndex);

                // remove group if zero items
                if (existingGroup.Count == 0)
                {
                    _items.Remove(existingGroup);
                }
            }
        }

    }
}

where the generic Grouping class (which itself exposes an ObservableCollection) comes from this article

http://motzcod.es/post/94643411707/enhancing-xamarinforms-listview-with-grouping

To make working demo:-

From new UWP Blank application, add the above ObservableGroupingCollection class. Then add another class file in same namespace and add all following classes

// Data models

public class Contact
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string State { get; set; }
}

public class DataPool
{
    public static string GenerateFirstName(Random random)
    {
        List<string> names = new List<string>() { "Lilly", "Mukhtar", "Sophie", "Femke", "Abdul-Rafi", "Mariana", "Aarif", "Sara", "Ibadah", "Fakhr", "Ilene", "Sardar", "Hanna", "Julie", "Iain", "Natalia", "Henrik", "Rasa", "Quentin", "Gadi", "Pernille", "Ishtar", "Jimmy", "Justine", "Lale", "Elize", "Randy", "Roshanara", "Rajab", "Marcus", "Mark", "Alima", "Francisco", "Thaqib", "Andreas", "Marianna", "Amalie", "Rodney", "Dena", "Amar", "Anna", "Nasreen", "Reema", "Tomas", "Filipa", "Frank", "Bari'ah", "Parvaiz", "Jibran", "Tomas", "Elli", "Carlos", "Diego", "Henrik", "Aruna", "Vahid", "Eliana", "Roxanne", "Amanda", "Ingrid", "Wesley", "Malika", "Basim", "Eisa", "Alina", "Andreas", "Deeba", "Diya", "Parveen", "Bakr", "Celine", "Daniel", "Mattheus", "Edmee", "Hedda", "Maria", "Maja", "Alhasan", "Alina", "Hedda", "Vanja", "Robin", "Victor", "Aaftab", "Guilherme", "Maria", "Kai", "Sabien", "Abdel", "Jason", "Bahaar", "Vasco", "Jibran", "Parsa", "Catalina", "Fouad", "Colette", "John", "Fred", "James", "Harry", "Ben", "Steven", "Philip", "Dougal", "Jasper", "Elliott", "Charles", "Gerty", "Sarah", "Sonya", "Svetlana", "Dita", "Karen", "Christine", "Angela", "Heather", "Spence", "Graham", "David", "Bernie", "Darren", "Lester", "Vince", "Colin", "Bernhard", "Dieter", "Norman", "William", "Nigel", "Nick", "Nikki", "Trent", "Devon", "Steven", "Eric", "Derek", "Raymond", "Craig" };
        return names[random.Next(0, names.Count)];
    }
    public static string GenerateLastName(Random random)
    {
        List<string> lastnames = new List<string>() { "Carlson", "Attia", "Quincey", "Hollenberg", "Khoury", "Araujo", "Hakimi", "Seegers", "Abadi", "Krommenhoek", "Siavashi", "Kvistad", "Vanderslik", "Fernandes", "Dehmas", "Sheibani", "Laamers", "Batlouni", "Lyngvær", "Oveisi", "Veenhuizen", "Gardenier", "Siavashi", "Mutlu", "Karzai", "Mousavi", "Natsheh", "Nevland", "Lægreid", "Bishara", "Cunha", "Hotaki", "Kyvik", "Cardoso", "Pilskog", "Pennekamp", "Nuijten", "Bettar", "Borsboom", "Skistad", "Asef", "Sayegh", "Sousa", "Miyamoto", "Medeiros", "Kregel", "Shamoun", "Behzadi", "Kuzbari", "Ferreira", "Barros", "Fernandes", "Xuan", "Formosa", "Nolette", "Shahrestaani", "Correla", "Amiri", "Sousa", "Fretheim", "Van", "Hamade", "Baba", "Mustafa", "Bishara", "Formo", "Hemmati", "Nader", "Hatami", "Natsheh", "Langen", "Maloof", "Patel", "Berger", "Ostrem", "Bardsen", "Kramer", "Bekken", "Salcedo", "Holter", "Nader", "Bettar", "Georgsen", "Cuninho", "Zardooz", "Araujo", "Batalha", "Antunes", "Vanderhoorn", "Srivastava", "Trotter", "Siavashi", "Montes", "Sherzai", "Vanderschans", "Neves", "Sarraf", "Kuiters", "Hestoe", "Cornwall", "Paisley", "Cooper", "Jakoby", "Smith", "Davies", "Jonas", "Bowers", "Fernandez", "Perez", "Black", "White", "Keller", "Hernandes", "Clinton", "Merryweather", "Freeman", "Anguillar", "Goodman", "Hardcastle", "Emmott", "Kirkby", "Thatcher", "Jamieson", "Spender", "Harte", "Pinkman", "Winterman", "Knight", "Taylor", "Wentworth", "Manners", "Walker", "McPherson", "Elder", "McDonald", "Macintosh", "Decker", "Takahashi", "Wagoner" };
        return lastnames[random.Next(0, lastnames.Count)];
    }
    public static string GenerateState(Random random)
    {
        List<string> states = new List<string>() { "Alabama", "Alaska", "Arizona", "Arkansas", "California", "Colorado", "Connecticut", "Delaware", "District Of Columbia", "Florida", "Georgia", "Hawaii", "Idaho", "Illinois", "Indiana", "Iowa", "Kansas", "Kentucky", "Louisiana", "Maine", "Maryland", "Massachusetts", "Michigan", "Minnesota", "Mississippi", "Missouri", "Montana", "Nebraska", "Nevada", "New Hampshire", "New Jersey", "New Mexico", "New York", "North Carolina", "North Dakota", "Ohio", "Oklahoma", "Oregon", "Pennsylvania", "Rhode Island", "South Carolina", "South Dakota", "Tennessee", "Texas", "Utah", "Vermont", "Virginia", "Washington", "West Virginia", "Wisconsin", "Wyoming" };
        return states[random.Next(0, states.Count)];
    }
}

public class Cache
{
    public Cache()
    {
        InitializeCacheData();
        SimulateLiveChanges(new TimeSpan(0, 0, 1));
    }

    public ObservableCollection<Contact> Contacts { get; set; }

    private static Random rnd = new Random();

    private void InitializeCacheData()
    {
        Contacts = new ObservableCollection<Contact>();

        var i = 0;
        while (i < 5)
        {
            Contacts.Add(new Contact()
            {
                FirstName = DataPool.GenerateFirstName(rnd),
                LastName = DataPool.GenerateLastName(rnd),
                State = DataPool.GenerateState(rnd)
            });

            i++;
        }
    }

    private async void SimulateLiveChanges(TimeSpan MyInterval)
    {
        double MyIntervalSeconds = MyInterval.TotalSeconds;
        while (true)
        {
            await Task.Delay(MyInterval);

            //int addOrRemove = rnd.Next(1, 10);
            //if (addOrRemove > 3)
            //{
            // add item
            Contacts.Add(new Contact()
            {
                FirstName = DataPool.GenerateFirstName(rnd),
                LastName = DataPool.GenerateLastName(rnd),
                State = DataPool.GenerateState(rnd)
            });
            //}
            //else
            //{
            //    // remove random item
            //    if (Contacts.Count > 0)
            //    {
            //        Contacts.RemoveAt(rnd.Next(0, Contacts.Count - 1));
            //    }
            //}
        }
    }

}

// ViewModel

public class ViewModel : BaseViewModel
{       
    public ViewModel()
    {
        _groupingCollection = new ObservableGroupingCollection<string, Contact>(new Cache().Contacts);
        _groupingCollection.ArrangeItems(new StateSorter(), (x => x.State));
        NotifyPropertyChanged("GroupedContacts");

    }

    ObservableGroupingCollection<string, Contact> _groupingCollection;
    public ObservableCollection<Grouping<string, Contact>> GroupedContacts
    {
        get
        {
            return _groupingCollection.Items;
        }
    }

    // swap grouping commands

    private ICommand _groupByStateCommand;
    public ICommand GroupByStateCommand
    {
        get
        {
            if (_groupByStateCommand == null)
            {
                _groupByStateCommand = new RelayCommand(
                    param => GroupByState(),
                    param => true);
            }
            return _groupByStateCommand;
        }
    }
    private void GroupByState()
    {
        _groupingCollection.ArrangeItems(new StateSorter(), (x => x.State));
        NotifyPropertyChanged("GroupedContacts");
    }

    private ICommand _groupByNameCommand;
    public ICommand GroupByNameCommand
    {
        get
        {
            if (_groupByNameCommand == null)
            {
                _groupByNameCommand = new RelayCommand(
                    param => GroupByName(),
                    param => true);
            }
            return _groupByNameCommand;
        }
    }
    private void GroupByName()
    {
        _groupingCollection.ArrangeItems(new NameSorter(), (x => x.LastName.First().ToString()));
        NotifyPropertyChanged("GroupedContacts");
    }

}

// View Model helpers

public class BaseViewModel : INotifyPropertyChanged
{
    public event PropertyChangedEventHandler PropertyChanged;

    protected virtual void NotifyPropertyChanged([CallerMemberName] string propertyName = null)
    {
        if (PropertyChanged != null)
        {
            PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
        }
    }
}

public class RelayCommand : ICommand
{
    readonly Action<object> _execute;
    readonly Predicate<object> _canExecute;

    public RelayCommand(Action<object> execute)
        : this(execute, null)
    {

    }

    public RelayCommand(Action<object> execute, Predicate<object> canExecute)
    {
        if (execute == null)
            throw new ArgumentNullException("execute");

        _execute = execute;
        _canExecute = canExecute;

    }

    public bool CanExecute(object parameter)
    {
        return _canExecute == null ? true : _canExecute(parameter);
    }

    public event EventHandler CanExecuteChanged
    {
        add { } 
        remove { } 
    }

    public void Execute(object parameter)
    {
        _execute(parameter);
    }

}

// Sorter classes

public class NameSorter : Comparer<Contact>
{
    public override int Compare(Contact x, Contact y)
    {
        int result = x.LastName.First().CompareTo(y.LastName.First());

        if (result != 0)
        {
            return result;
        }
        else
        {
            result = x.LastName.CompareTo(y.LastName);

            if (result != 0)
            {
                return result;
            }
            else
            {
                return x.FirstName.CompareTo(y.FirstName);
            }
        }
    }
}

public class StateSorter : Comparer<Contact>
{
    public override int Compare(Contact x, Contact y)
    {
        int result = x.State.CompareTo(y.State);

        if (result != 0)
        {
            return result;
        }
        else
        {
            result = x.LastName.CompareTo(y.LastName);

            if (result != 0)
            {
                return result;
            }
            else
            {
                return x.FirstName.CompareTo(y.FirstName);
            }
        }
    }
}

// Grouping class 
// credit
// http://motzcod.es/post/94643411707/enhancing-xamarinforms-listview-with-grouping

public class Grouping<K, T> : ObservableCollection<T>
{
    public K Key { get; private set; }

    public Grouping(K key, IEnumerable<T> items)
    {
        Key = key;
        foreach (var item in items)
        {
            this.Items.Add(item);
        }
    }
}

Finally, edit MainPage as follows

  <Page.DataContext>
        <local:ViewModel />
    </Page.DataContext>

    <Page.Resources>
        <CollectionViewSource 
            x:Key="cvs" 
            Source="{Binding GroupedContacts}" 
            IsSourceGrouped="True" />
    </Page.Resources>

    <Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">

        <Grid.RowDefinitions>
            <RowDefinition Height="*" />
            <RowDefinition Height="Auto" />
        </Grid.RowDefinitions>

        <ListView ItemsSource="{Binding Source={StaticResource cvs}}"
                  x:Name="targetListBox">
            <ListView.ItemTemplate>
                <DataTemplate>
                    <Grid>
                        <Grid.ColumnDefinitions>
                            <ColumnDefinition Width="100" />
                            <ColumnDefinition Width="100" />
                            <ColumnDefinition Width="*" />
                        </Grid.ColumnDefinitions>

                        <TextBlock Text="{Binding LastName}" />
                        <TextBlock Text="{Binding FirstName}" Grid.Column="1" />
                        <TextBlock Text="{Binding State}" Grid.Column="2" HorizontalAlignment="Right" />
                    </Grid>
                </DataTemplate>
            </ListView.ItemTemplate>
            <ListView.GroupStyle>
                <GroupStyle>
                    <GroupStyle.HeaderTemplate>
                        <DataTemplate>
                            <Grid Background="Gainsboro">
                                <TextBlock FontWeight="Bold" 
                                           FontSize="14" 
                                           Margin="10,2"
                                           Text="{Binding Key}"/>
                            </Grid>
                        </DataTemplate>
                    </GroupStyle.HeaderTemplate>
                </GroupStyle>
            </ListView.GroupStyle>
        </ListView>

        <StackPanel Orientation="Horizontal" Grid.Row="1">
            <Button Content="Group By Initial" Command="{Binding GroupByNameCommand}" />
            <Button Content="Group By State" Command="{Binding GroupByStateCommand}" />
        </StackPanel>
    </Grid>

HandleCollectionChanged method only handles Add/Remove so far and will break down if NotifyCollectionChangedEventArgs parameter contains multiple items (Existing ObservableCollection class only notifies changes one at a time)

So it works okay but it all feels kind of hacky.

Suggestions for improvement highly welcome.

Comments

[CularBytes]

Have you made changes since publishing this? Any improvements?

[CularBytes]

And how do you apply virtualization, so make sure more items are loaded when the gridview or listview reaches the bottom of the list.