How to implement a rule engine?

Question

I have a db table that stores the following:

RuleID  objectProperty ComparisonOperator  TargetValue
1       age            'greater_than'             15
2       username       'equal'             'some_name'
3       tags           'hasAtLeastOne'     'some_tag some_tag2'

Now say I have a collection of these rules:

List<Rule> rules = db.GetRules();

Now I have an instance of a user also:

User user = db.GetUser(....);

How would I loop through these rules, and apply the logic and perform the comparisons etc?

if(user.age > 15)

if(user.username == "some_name")

Since the object's property like 'age' or 'user_name' is stored in the table, along with the comparison operater 'great_than' and 'equal', how could I possible do this?

C# is a statically typed language, so not sure how to go forward.

Solution 1

This snippet compiles the Rules into fast executable code (using Expression trees) and does not need any complicated switch statements:

(Edit : full working example with generic method)

public Func<User, bool> CompileRule(Rule r)
{
    var paramUser = Expression.Parameter(typeof(User));
    Expression expr = BuildExpr(r, paramUser);
    // build a lambda function User->bool and compile it
    return Expression.Lambda<Func<User, bool>>(expr, paramUser).Compile();
}

You can then write:

List<Rule> rules = new List<Rule> {
    new Rule ("Age", "GreaterThan", "21"),
    new Rule ( "Name", "Equal", "John"),
    new Rule ( "Tags", "Contains", "C#" )
};

// compile the rules once
var compiledRules = rules.Select(r => CompileRule(r)).ToList();

public bool MatchesAllRules(User user)
{
    return compiledRules.All(rule => rule(user));
}

Here is the implementation of BuildExpr:

Expression BuildExpr(Rule r, ParameterExpression param)
{
    var left = MemberExpression.Property(param, r.MemberName);
    var tProp = typeof(User).GetProperty(r.MemberName).PropertyType;
    ExpressionType tBinary;
    // is the operator a known .NET operator?
    if (ExpressionType.TryParse(r.Operator, out tBinary)) {
        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
        // use a binary operation, e.g. 'Equal' -> 'u.Age == 21'
        return Expression.MakeBinary(tBinary, left, right);
    } else {
        var method = tProp.GetMethod(r.Operator);
        var tParam = method.GetParameters()[0].ParameterType;
        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));
        // use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'
        return Expression.Call(left, method, right);
    }
}

Note that I used 'GreaterThan' instead of 'greater_than' etc. - this is because 'GreaterThan' is the .NET name for the operator, therefore we don't need any extra mapping.

If you need custom names you can build a very simple dictionary and just translate all operators before compiling the rules:

var nameMap = new Dictionary<string, string> {
    { "greater_than", "GreaterThan" },
    { "hasAtLeastOne", "Contains" }
};

The code uses the type User for simplicity. You can replace User with a generic type T to have a generic Rule compiler for any types of objects. Also, the code should handle errors, like unknown operator name.

Note that generating code on the fly was possible even before the Expression trees API was introduced, using Reflection.Emit. The method LambdaExpression.Compile() uses Reflection.Emit under the covers (you can see this using ILSpy).

Solution 2

Here is some code that compiles as is and does the job. Basically use two dictionaries, one containing a mapping from operator names to boolean functions, and another containing a map from the property names of the User type to PropertyInfos used to invoke the property getter (if public). You pass the User instance, and the three values from your table to the static Apply method.

class User
{
    public int Age { get; set; }
    public string UserName { get; set; }
}

class Operator
{
    private static Dictionary<string, Func<object, object, bool>> s_operators;
    private static Dictionary<string, PropertyInfo> s_properties;
    static Operator()
    {
        s_operators = new Dictionary<string, Func<object, object, bool>>();
        s_operators["greater_than"] = new Func<object, object, bool>(s_opGreaterThan);
        s_operators["equal"] = new Func<object, object, bool>(s_opEqual);

        s_properties = typeof(User).GetProperties().ToDictionary(propInfo => propInfo.Name);
    }

    public static bool Apply(User user, string op, string prop, object target)
    {
        return s_operators[op](GetPropValue(user, prop), target);
    }

    private static object GetPropValue(User user, string prop)
    {
        PropertyInfo propInfo = s_properties[prop];
        return propInfo.GetGetMethod(false).Invoke(user, null);
    }

    #region Operators

    static bool s_opGreaterThan(object o1, object o2)
    {
        if (o1 == null || o2 == null || o1.GetType() != o2.GetType() || !(o1 is IComparable))
            return false;
        return (o1 as IComparable).CompareTo(o2) > 0;
    }

    static bool s_opEqual(object o1, object o2)
    {
        return o1 == o2;
    }

    //etc.

    #endregion

    public static void Main(string[] args)
    {
        User user = new User() { Age = 16, UserName = "John" };
        Console.WriteLine(Operator.Apply(user, "greater_than", "Age", 15));
        Console.WriteLine(Operator.Apply(user, "greater_than", "Age", 17));
        Console.WriteLine(Operator.Apply(user, "equal", "UserName", "John"));
        Console.WriteLine(Operator.Apply(user, "equal", "UserName", "Bob"));
    }
}

Solution 3

I built a rule engine that takes a different approach than you outlined in your question, but I think you will find it to be much more flexible than your current approach.

Your current approach seems to be focused on a single entity, "User", and your persistent rules identify "propertyname", "operator" and "value". My pattern, instead stores the C# code for a predicate (Func<T, bool>) in an "Expression" column in my database. In the current design, using code generation I am querying the "rules" from my database and compiling an assembly with "Rule" types, each with a "Test" method. Here is the signature for the interface that is implemented each Rule:

public interface IDataRule<TEntity> 
{
    /// <summary>
    /// Evaluates the validity of a rule given an instance of an entity
    /// </summary>
    /// <param name="entity">Entity to evaluate</param>
    /// <returns>result of the evaluation</returns>
    bool Test(TEntity entity);
    /// <summary>
    /// The unique indentifier for a rule.
    /// </summary>
     int RuleId { get; set; }
    /// <summary>
    /// Common name of the rule, not unique
    /// </summary>
     string RuleName { get; set; }
    /// <summary>
    /// Indicates the message used to notify the user if the rule fails
    /// </summary>
     string ValidationMessage { get; set; }   
     /// <summary>
     /// indicator of whether the rule is enabled or not
     /// </summary>
     bool IsEnabled { get; set; }
    /// <summary>
    /// Represents the order in which a rule should be executed relative to other rules
    /// </summary>
     int SortOrder { get; set; }
}

The "Expression" is compiled as the body of the "Test" method when the application first executes. As you can see the other columns in the table are also surfaced as first-class properties on the rule so that a developer has flexibility to create an experience for how the user gets notified of failure or success.

Generating an in-memory assembly is a 1-time occurrence during your application and you get a performance gain by not having to use reflection when evaluating your rules. Your expressions are checked at runtime as the assembly will not generate correctly if a property name is misspelled, etc.

The mechanics of creating an in-memory assembly are as follows:

• Load your rules from the DB
• iterate over the rules and for-each, using a StringBuilder and some string concatenation write the Text representing a class that inherits from IDataRule
• compile using CodeDOM -- more info

This is actually quite simple because for the majority this code is property implementations and value initialization in the constructor. Besides that, the only other code is the Expression.
NOTE: there is a limitation that your expression must be .NET 2.0 (no lambdas or other C# 3.0 features) due to a limitation in CodeDOM.

Here is some sample code for that.

sb.AppendLine(string.Format("\tpublic class {0} : SomeCompany.ComponentModel.IDataRule<{1}>", className, typeName));
            sb.AppendLine("\t{");
            sb.AppendLine("\t\tprivate int _ruleId = -1;");
            sb.AppendLine("\t\tprivate string _ruleName = \"\";");
            sb.AppendLine("\t\tprivate string _ruleType = \"\";");
            sb.AppendLine("\t\tprivate string _validationMessage = \"\";");
            /// ... 
            sb.AppendLine("\t\tprivate bool _isenabled= false;");
            // constructor
            sb.AppendLine(string.Format("\t\tpublic {0}()", className));
            sb.AppendLine("\t\t{");
            sb.AppendLine(string.Format("\t\t\tRuleId = {0};", ruleId));
            sb.AppendLine(string.Format("\t\t\tRuleName = \"{0}\";", ruleName.TrimEnd()));
            sb.AppendLine(string.Format("\t\t\tRuleType = \"{0}\";", ruleType.TrimEnd()));                
            sb.AppendLine(string.Format("\t\t\tValidationMessage = \"{0}\";", validationMessage.TrimEnd()));
            // ...
            sb.AppendLine(string.Format("\t\t\tSortOrder = {0};", sortOrder));                

            sb.AppendLine("\t\t}");
            // properties
            sb.AppendLine("\t\tpublic int RuleId { get { return _ruleId; } set { _ruleId = value; } }");
            sb.AppendLine("\t\tpublic string RuleName { get { return _ruleName; } set { _ruleName = value; } }");
            sb.AppendLine("\t\tpublic string RuleType { get { return _ruleType; } set { _ruleType = value; } }");

            /// ... more properties -- omitted

            sb.AppendLine(string.Format("\t\tpublic bool Test({0} entity) ", typeName));
            sb.AppendLine("\t\t{");
            // #############################################################
            // NOTE: This is where the expression from the DB Column becomes
            // the body of the Test Method, such as: return "entity.Prop1 < 5"
            // #############################################################
            sb.AppendLine(string.Format("\t\t\treturn {0};", expressionText.TrimEnd()));
            sb.AppendLine("\t\t}");  // close method
            sb.AppendLine("\t}"); // close Class

Beyond this I did make a class I called "DataRuleCollection", which implemented ICollection>. This enabled me to create a "TestAll" capability and an indexer for executing a specific rule by name. Here are the implementations for those two methods.

    /// <summary>
    /// Indexer which enables accessing rules in the collection by name
    /// </summary>
    /// <param name="ruleName">a rule name</param>
    /// <returns>an instance of a data rule or null if the rule was not found.</returns>
    public IDataRule<TEntity, bool> this[string ruleName]
    {
        get { return Contains(ruleName) ? list[ruleName] : null; }
    }
    // in this case the implementation of the Rules Collection is: 
    // DataRulesCollection<IDataRule<User>> and that generic flows through to the rule.
    // there are also some supporting concepts here not otherwise outlined, such as a "FailedRules" IList
    public bool TestAllRules(User target) 
    {
        rules.FailedRules.Clear();
        var result = true;

        foreach (var rule in rules.Where(x => x.IsEnabled)) 
        {

            result = rule.Test(target);
            if (!result)
            {

                rules.FailedRules.Add(rule);
            }
        }

        return (rules.FailedRules.Count == 0);
    }

MORE CODE: There was a request for the code related to the Code Generation. I encapsulated the functionality in a class called 'RulesAssemblyGenerator' which I have included below.

namespace Xxx.Services.Utils
    {
        public static class RulesAssemblyGenerator
        {
            static List<string> EntityTypesLoaded = new List<string>();

            public static void Execute(string typeName, string scriptCode)
            {
                if (EntityTypesLoaded.Contains(typeName)) { return; } 
                // only allow the assembly to load once per entityType per execution session
                Compile(new CSharpCodeProvider(), scriptCode);
                EntityTypesLoaded.Add(typeName);
            }
            private static void Compile(CodeDom.CodeDomProvider provider, string source)
            {
                var param = new CodeDom.CompilerParameters()
                {
                    GenerateExecutable = false,
                    IncludeDebugInformation = false,
                    GenerateInMemory = true
                };
                var path = System.Reflection.Assembly.GetExecutingAssembly().Location;
                var root_Dir = System.IO.Path.Combine(System.AppDomain.CurrentDomain.BaseDirectory, "Bin");
                param.ReferencedAssemblies.Add(path);
                // Note: This dependencies list are included as assembly reference and they should list out all dependencies
                // That you may reference in your Rules or that your entity depends on.
                // some assembly names were changed... clearly.
                var dependencies = new string[] { "yyyyyy.dll", "xxxxxx.dll", "NHibernate.dll", "ABC.Helper.Rules.dll" };
                foreach (var dependency in dependencies)
                {
                    var assemblypath = System.IO.Path.Combine(root_Dir, dependency);
                    param.ReferencedAssemblies.Add(assemblypath);
                }
                // reference .NET basics for C# 2.0 and C#3.0
                param.ReferencedAssemblies.Add(@"C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\System.dll");
                param.ReferencedAssemblies.Add(@"C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.5\System.Core.dll");
                var compileResults = provider.CompileAssemblyFromSource(param, source);
                var output = compileResults.Output;
                if (compileResults.Errors.Count != 0)
                {
                    CodeDom.CompilerErrorCollection es = compileResults.Errors;
                    var edList = new List<DataRuleLoadExceptionDetails>();
                    foreach (CodeDom.CompilerError s in es)
                        edList.Add(new DataRuleLoadExceptionDetails() { Message = s.ErrorText, LineNumber = s.Line });
                    var rde = new RuleDefinitionException(source, edList.ToArray());
                    throw rde;
                }
            }
        }
    }

If there are any other questions or comments or requests for further code samples, let me know.

Solution 4

Reflection is your most versatile answer. You have three columns of data, and they need to be treated in different ways:

1. Your field name. Reflection is the way to get the value from a coded field name.

2. Your comparison operator. There should be a limited number of these, so a case statement should handle them most easily. Especially as some of them ( has one or more of ) is slightly more complex.

3. Your comparison value. If these are all straight values then this is easy, although you will have divide the multiple entries up. However, you could also use reflection if they are field names too.

I would take an approach more like:

    var value = user.GetType().GetProperty("age").GetValue(user, null);
    //Thank you Rick! Saves me remembering it;
    switch(rule.ComparisonOperator)
        case "equals":
             return EqualComparison(value, rule.CompareTo)
        case "is_one_or_more_of"
             return IsInComparison(value, rule.CompareTo)

etc. etc.

It gives you flexibility for adding more options for comparison. It also means that you can code within the Comparison methods any type validation that you might want, and make them as complex as you want. There is also the option here for the CompareTo to be evaluated as a recursive call back to another line, or as a field value, which could be done like:

             return IsInComparison(value, EvaluateComparison(rule.CompareTo))

It all depends on the possibilities for the future....

Solution 5

If you only have a handful of properties and operators, the path of least of resistance is to just code up all the checks as special cases like this:

public bool ApplyRules(List<Rule> rules, User user)
{
    foreach (var rule in rules)
    {
        IComparable value = null;
        object limit = null;
        if (rule.objectProperty == "age")
        {
            value = user.age;
            limit = Convert.ToInt32(rule.TargetValue);
        }
        else if (rule.objectProperty == "username")
        {
            value = user.username;
            limit = rule.TargetValue;
        }
        else
            throw new InvalidOperationException("invalid property");

        int result = value.CompareTo(limit);

        if (rule.ComparisonOperator == "equal")
        {
            if (!(result == 0)) return false;
        }
        else if (rule.ComparisonOperator == "greater_than")
        {
            if (!(result > 0)) return false;
        }
        else
            throw new InvalidOperationException("invalid operator");
    }
    return true;
}

If you have a lot of properties, you may find a table-driven approach more palatable. In that case you would create a static Dictionary that maps property names to delegates matching, say, Func<User, object>.

If you don't know the names of the properties at compile time, or you want to avoid special-cases for each property and don't want to use the table approach, you can use reflection to get properties. For example:

var value = user.GetType().GetProperty("age").GetValue(user, null);

But since TargetValue is probably a string, you'll need to take care to do type conversion from the rules table if necessary.

View more solutions

Comments

[Blankman]

what does value.CompareTo(limit) return? -1 0 or 1? Haven't seen that b4!

[Rick Sladkey]

@Blankman: Close: less than zero, zero or greater than zero. IComparable is used for comparing things. Here are the docs: IComparable.CompareTo Method.

[Mrchief]

And you can cache your reflected assemblies/objects which will make your code even more performant.

[Blankman]

Where can I read more about your answer to learn the classes/objects/etc. you have in your in your code? It is mostly expression trees?

[Martin Konicek]

All the classes come from the namespace System.Linq.Expressions, and all are created using factory methods of the Expression class - type "Expression." in your IDE to access all of them. Read more about Expression trees here msdn.microsoft.com/en-us/library/bb397951.aspx

[Martin Konicek]

You are right that the engine can be made more generic and the CodeDOM API is definitely also an option. Maybe instead of the "sb.AppendLine" code which is not very clear, you could show how exactly you invoke the CodeDOM?

[Nahydrin]

@Martin where can I find a list of qualified .NET operator names?

[Martin Konicek]

@Dark Slipstream You can find them here msdn.microsoft.com/en-us/library/bb361179.aspx. Not all of them are boolean expressions - use only the boolean ones (such as GreaterThan, NotEqual, etc.).

[William Daugherty]

Where is your "Rule" class? Can you list that?

[Appetere]

I don't understand why this answer has been up-voted. It violates many design principles: "Tell don't ask" => the rules should each be asked to return a result. "Open for extension / closed for modification" => any new rule means the ApplyRules method needs modification. Plus the code is difficult to understand at a glance.

[Martin Konicek]

@BillDaugherty Rule a simple value class with three properties: MemberName, Operator, TargetValue. For example, new Rule ("Age", "GreaterThan", "20").

[Rick Sladkey]

Indeed, the path of least resistance is rarely the best path. Please see and upvote the excellent expression tree answer.

[user575219]

Can an admin dynamically create these rules? List<Rule> rules = new List<Rule> is hard coded to have just 3 rules. What if we need another rule?

[Taiseer Joudeh]

@MartinKonicek if I have a complex User model (i.e User model which has a list of Addresses) do you think I will be able to build a rule which will work against properties from the Addresses model (i.e var rule = new Rule("Addresses[0].City", "Equals", "Boston"); ) Let me know your thoughts about this and any tips to define rules on complex models. Thanks in advance

[Sumit Joshi]

how can we identify which rule is violated if compiledRules.All(rule => rule(user)) returs false.

[Max.Futerman]

had a problem of case sensitive in Martin Konicek answer, so if you want the rule.MemberName to be not case sensitive just add var tProp = typeof(User).GetProperty(r.MemberName, BindingFlags.IgnoreCase | BindingFlags.Public | BindingFlags.Instance).PropertyType;

[LP13]

This is excellent approach to evaluate conditions dynamically. How would you dynamically do then part? eg. if(CompileRule(Rule r) == true) then set properties

[xgp]

@MartinKonicek, Thank you! This has taken me much further in my work on this. Can you tell me how the MethodCallExpression would be implemented and invoked by the program?

[Martin Konicek]

Hi, I wrote this answer over 10 years ago and don't remember the details. But I think the linked blog post probably had enough info to be able to compile and run the code? coding-time.blogspot.com/2011/07/…