How to create LINQ Expression Tree to select an anonymous type

Question

I would like to generate the following select statement dynamically using expression trees:

var v = from c in Countries
        where c.City == "London"
        select new {c.Name, c.Population};

I have worked out how to generate

var v = from c in Countries
        where c.City == "London"
        select new {c.Name};

but I cannot seem to find a constructor/overload that will let me specify multiple properties in my select lambda.

Solution 1

This can be done, as mentioned, with the help of Reflection Emit and a helper class I've included below. The code below is a work in progress, so take it for what it's worth... 'it works on my box'. The SelectDynamic method class should be tossed in a static extension method class.

As expected, you won't get any Intellisense since the type isn't created until runtime. Works good on late-bound data controls.

public static IQueryable SelectDynamic(this IQueryable source, IEnumerable<string> fieldNames)
{
    Dictionary<string, PropertyInfo> sourceProperties = fieldNames.ToDictionary(name => name, name => source.ElementType.GetProperty(name));
    Type dynamicType = LinqRuntimeTypeBuilder.GetDynamicType(sourceProperties.Values);

    ParameterExpression sourceItem = Expression.Parameter(source.ElementType, "t");
    IEnumerable<MemberBinding> bindings = dynamicType.GetFields().Select(p => Expression.Bind(p, Expression.Property(sourceItem, sourceProperties[p.Name]))).OfType<MemberBinding>();

    Expression selector = Expression.Lambda(Expression.MemberInit(
        Expression.New(dynamicType.GetConstructor(Type.EmptyTypes)), bindings), sourceItem);

    return source.Provider.CreateQuery(Expression.Call(typeof(Queryable), "Select", new Type[] { source.ElementType, dynamicType },
                 Expression.Constant(source), selector));
}



public static class LinqRuntimeTypeBuilder
{
    private static readonly ILog log = LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
    private static AssemblyName assemblyName = new AssemblyName() { Name = "DynamicLinqTypes" };
    private static ModuleBuilder moduleBuilder = null;
    private static Dictionary<string, Type> builtTypes = new Dictionary<string, Type>();

    static LinqRuntimeTypeBuilder()
    {
        moduleBuilder = Thread.GetDomain().DefineDynamicAssembly(assemblyName, AssemblyBuilderAccess.Run).DefineDynamicModule(assemblyName.Name);
    }

    private static string GetTypeKey(Dictionary<string, Type> fields)
    {
        //TODO: optimize the type caching -- if fields are simply reordered, that doesn't mean that they're actually different types, so this needs to be smarter
        string key = string.Empty;
        foreach (var field in fields)
            key += field.Key + ";" + field.Value.Name + ";";

        return key;
    }

    public static Type GetDynamicType(Dictionary<string, Type> fields)
    {
        if (null == fields)
            throw new ArgumentNullException("fields");
        if (0 == fields.Count)
            throw new ArgumentOutOfRangeException("fields", "fields must have at least 1 field definition");

        try
        {
            Monitor.Enter(builtTypes);
            string className = GetTypeKey(fields);

            if (builtTypes.ContainsKey(className))
                return builtTypes[className];

            TypeBuilder typeBuilder = moduleBuilder.DefineType(className, TypeAttributes.Public | TypeAttributes.Class | TypeAttributes.Serializable);

            foreach (var field in fields)                    
                typeBuilder.DefineField(field.Key, field.Value, FieldAttributes.Public);

            builtTypes[className] = typeBuilder.CreateType();

            return builtTypes[className];
        }
        catch (Exception ex)
        {
            log.Error(ex);
        }
        finally
        {
            Monitor.Exit(builtTypes);
        }

        return null;
    }


    private static string GetTypeKey(IEnumerable<PropertyInfo> fields)
    {
        return GetTypeKey(fields.ToDictionary(f => f.Name, f => f.PropertyType));
    }

    public static Type GetDynamicType(IEnumerable<PropertyInfo> fields)
    {
        return GetDynamicType(fields.ToDictionary(f => f.Name, f => f.PropertyType));
    }
}

Solution 2

The accepted answer is very useful, but I needed something a little closer to a real anonymous type.

A real anonymous type has read-only properties, a constructor for filling in all of the values, an implementation of Equals/GetHashCode for comparing the values of each property, and an implementation ToString that includes the name/value of each property. (See https://msdn.microsoft.com/en-us/library/bb397696.aspx for a full description of anonymous types.)

Based on that definition of anonymous classes, I put a class that generates dynamic anonymous types on github at https://github.com/dotlattice/LatticeUtils/blob/master/LatticeUtils/AnonymousTypeUtils.cs. The project also contains some unit tests to make sure the fake anonymous types behave like real ones.

Here's a very basic example of how to use it:

AnonymousTypeUtils.CreateObject(new Dictionary<string, object>
{
    { "a", 1 },
    { "b", 2 }
});

Also, another note: I found that when using a dynamic anonymous type with Entity Framework, the constructor must be called with the "members" parameter set. For example:

Expression.New(
    constructor: anonymousType.GetConstructors().Single(), 
    arguments: propertyExpressions,
    members: anonymousType.GetProperties().Cast<MemberInfo>().ToArray()
); 

If you used one of the versions of Expression.New that does not include the "members" parameter, Entity Framework would not recognize it as the constructor of an anonymous type. So I assume that means a real anonymous type's constructor expression would include that "members" information.

Solution 3

Maybe a bit late but may help to someone.

You Can generate dynamic select by call DynamicSelectGenerator in select from an entity.

public static Func<T, T> DynamicSelectGenerator<T>()
            {
                // get Properties of the T
                var fields = typeof(T).GetProperties().Select(propertyInfo => propertyInfo.Name).ToArray();

            // input parameter "o"
            var xParameter = Expression.Parameter(typeof(T), "o");

            // new statement "new Data()"
            var xNew = Expression.New(typeof(T));

            // create initializers
            var bindings = fields.Select(o => o.Trim())
                .Select(o =>
                {

                    // property "Field1"
                    var mi = typeof(T).GetProperty(o);

                    // original value "o.Field1"
                    var xOriginal = Expression.Property(xParameter, mi);

                    // set value "Field1 = o.Field1"
                    return Expression.Bind(mi, xOriginal);
                }
            );

            // initialization "new Data { Field1 = o.Field1, Field2 = o.Field2 }"
            var xInit = Expression.MemberInit(xNew, bindings);

            // expression "o => new Data { Field1 = o.Field1, Field2 = o.Field2 }"
            var lambda = Expression.Lambda<Func<T, T>>(xInit, xParameter);

            // compile to Func<Data, Data>
            return lambda.Compile();
        }

And use by this code:

var result = dbContextInstancs.EntityClass.Select(DynamicSelectGenerator<EntityClass>());

Solution 4

You could use the IQueryable-Extensions here, which is an implemantation of the solution described by "Ethan J. Brown":

https://github.com/thiscode/DynamicSelectExtensions

The Extension builds dynamically an anonymous type.

Then you can do this:

var YourDynamicListOfFields = new List<string>(

    "field1",
    "field2",
    [...]

)
var query = query.SelectPartially(YourDynamicListOfFields);

Solution 5

I don't believe that you will be able to achieve this. Although when you do select new { c.Name, c.Population } it seems like you're not creating a class you actually are. If you have a look at the compiled output in Reflector or the raw IL you will be able to see this.

You'll have a class which would look something like this:

[CompilerGenerated]
private class <>c__Class {
  public string Name { get; set; }
  public int Population { get; set; }
}

(Ok, I cleaned it up a touch, since a property is really just a get_Name() and set_Name(name) method set anyway)

What you're trying to do is proper dynamic class creation, something which wont be available until .NET 4.0 comes out (and even then I'm not really sure if it'll be able to achieve what you want).

You're best solution would be to define the different anonymous classes and then have some kind of logical check to determine which one to create, and to create it you can use the object System.Linq.Expressions.NewExpression.

But, it may be (in theory at least) possible to do it, if you're getting really hard-core about the underlying LINQ provider. If you are writing your own LINQ provider you can detect if the currently-parsed expression is a Select, then you determine the CompilerGenerated class, reflect for its constructor and create.

Defiantly not a simple task, but it would be how LINQ to SQL, LINQ to XML, etc all do it.

View more solutions

Comments

[Aaron Powell]

I believe that will only work with LINQ to SQL, not another LINQ provider though

[Adrian Grigore]

I believe the framework only works with IQueryable, not with IEnumerable.

[Inferis]

Good summary. Pity there's no way to generate an expression to generate a new type. Although, I can imagine that opens up a big can of worms. :)

[CheapReference]

I will check how the extensions in System.Linq.Dynamic work, I am guessing there must be a way to do this if this class can do it.

[mCasamento]

wonderful, didn't know creating a type at runtime was that easy!thanks!

[Thulasiram]

i tryed u r code by it is giving error how to implement the above code in entity framework using datacontext?

[Davut Gürbüz]

Good but "Cannot serialize interface System.Linq.IQueryable"

[Akash Kava]

You can put OrderBy in your //TODO for optimization and its done.

[Lei Yang]

@Ethan J. Brown, could you tell me how to modify your code if the source is IEnumerable instead of IQueryable? Thanks!

[Connell]

I've been using this (well, similar) and have been getting an Unable to create a constant value of type error. I've fixed this by replacing Expression.Constant(source) with source.Expression on the last line. Hope this helps someone :)

[mesut]

@Ethan J. Brown, Thanks for your great answer, How can I use this code with nested types, for example x=> new { a=1, b = new { ba=1, bb=2} };

[oscilatingcretin]

I like this type builder, but calling .GetProperties() on the new type returns an empty array. To get those, is one required to go with the more complex solution at this question? stackoverflow.com/a/3862241/701346

[Mabakay]

Expression.New(typeof(T)) It will not work if T is one of entity mapped types.