Splitting a list or collection into chunks

Question

I have a program that will read the contents of a file into some sort of list or array. This list/array could contain any number of items. I need to split it into smaller groups, say of 50 items each and then do some processing on each item in each group.

List<string> stuffFromFile = new List<string>();

while ((line = fileReader.ReadLine()) != null)
      {
           stuffFromFile.Add(line);
      }

I have been looking through some examples online about how to chunk stuff but to be honest, I don't really understand the examples and some of them seem overly complex. I just need something simple that will chunk/split/break the original list of items into groups of 50 and then let me iterate through each item in each group until the processing is complete.

The total number of items read in will most likely not be a number that I can divide evenly by 50 so most likely the last group may contain less than 50 items, but would still need to be processed just like the rest.

Can anyone help here? It sounds like it should be simple but I don't really know how to do it. I've seen example about using LINQ but I don't understand it either.

Solution 1

Here's an extension method that will work with any list and any size chunks.

public static List<List<T>> SplitList<T>(this List<T> me, int size = 50)
{
    var list = new List<List<T>>();
    for (int i = 0; i < me.Count; i += size)
        list.Add(me.GetRange(i, Math.Min(size, me.Count - i)));
    return list;
} 

Use it like this:

List<List<string>> chunksOf50 = stuffFromFile.SplitList();

Solution 2

List<string> stuffFromFile = new List<string>() { "1", "2", "3", "4" }; //contents

while (stuffFromFile.Count > 0)
{
    List<string> newChunk = stuffFromFile.Take(50).ToList(); //Take up to 50 elements
    stuffFromFile.RemoveRange(0, newChunk.Count); // Remove the elements you took
}

Comments

[Malte R]

I think this might be what you are looking for: stackoverflow.com/questions/11463734/…

[paparazzo]

What is wrong with just read 50, process, repeat?

[imlokesh]

why do you want to split into groups when the end goal is to process each item?

[irldev]

The system processing the data can only accept stuff in small groups.

[irldev]

How do I iterate through each item in that chunksOf50 list?

[Zer0]

@irldev foreach (List<string> chunk in chunksOf50)

[Lee Harrison]

for you, something simple like nested foreach loops. foreach(List<string> listOfStrings in chunksOf50){ foreach(string chunk in listOfStrings){//do stuff}};

[irldev]

Thanks, this seems to be working.