Powershell 3.0 - Workflows - Limit number of parallel execution

Question

I am cloning VMs on ESX server from template. Simplified code looks like this:

Workflow Create-VM {
  $List = 1..500
  foreach -parallel ($Elem in $List)
  {
      # Create VM ...
      # Configure created VM ..
  }
}

Create-VM

Parallel execution is really helpful. Unfortunately in this case doesn't work pretty well. Too many parallel request are generated. I need to limit number of parallel execution to smaller number (for example 4).

I was trying to change local Session Configuration (SessionThrottleLimit, MaxSessionsPerWorkflow, MaxRunningWorkflows) http://technet.microsoft.com/en-us/library/hh849862.aspx.

$WWE = New-PSWorkflowExecutionOption  -SessionThrottleLimit 4
Set-PSSessionConfiguration -Name microsoft.powershell.workflow `
   -SessionTypeOption $WWE 
Get-PSSessionConfiguration microsoft.powershell.workflow | 
fl SessionThrottleLimit

Question

• Which parameter (or combination) of Session Configuration should I change, in order to limit the number of parallel execution to 4?
• Is there some other method how can I achieve that (For example: different way to execute workflow ...)?

Solution 1

There is an option to limit the number of parallel processes in a foreach-parallel loop using -throttlelimit N. It's great for reducing the parallelism, but if you try a high number the system may still limit you to 5, depending on all your software versions (YAY! Microsoft consistency). I know the question is old, but since it came up on Google without a decent answer, I thought I'd chime in.

Workflow Create-VM {
  $List = 1..500
  foreach -parallel -throttlelimit 4 ($Elem in $List)
  {
      # Create VM ...
      # Configure created VM ..
  }
}

Create-VM

Solution 2

A trivial solution is to divide the list into smaller chunks and use that as input for parallel foreach. Like so,

Workflow Create-VM {
  $List = 1..500
  # Calclulate indexes to get elements 0,3; 4,7; 8,11 and so on
  # Use the .. operator to extract those elements from $list and pass
  # 'em to foreach -parallel processing part
  for($i=0;$i -le $List.Count-4; $i+=4) { 
    foreach -parallel ($Elem in $list[$i..($i+3)]) {
      # Create VM ...
      # Configure created VM ..
    } 
  }
}

Solution 3

Just wanted to add this detail, ThrottleLimit switch mentioned above is available in Powershell v4.0, its not available in v3.0. We have a mix of 2.0 and 3.0 servers

Comments

[Cody Konior]

v3 is particularly bad, because even though it has a limit of 5 workflows, it won't reuse any of them until they have all completed. So it really runs in batches of 5 only. AFAIK this same limitation doesn't exist in v4. tl;dr probably don't use workflows in v3.