Label Cloud

Tuesday, March 17, 2009

Linq provider for Oracle Coherence (Linq to Coherence)

I am very impressed with Coherence from Oracle. Coherence provides a distributed in-memory cache and processing fabric. However it is a lot more then just a cache. It can be used for everything from messaging to cross platform communication medium. There is too much to talk say about it, so read more information at Oracle: http://www.oracle.com/technology/products/coherence/index.html

Coherence works very nicely with .Net however, in the days of Linq, I wanted to write a Linq provider for it. My code is based largely on the Linq provider documentation on MSDN (http://msdn.microsoft.com/en-us/library/bb546158.aspx) and excellent series on creating a linq provider by Matt Warren (http://blogs.msdn.com/mattwar/pages/linq-links.aspx)

I am using Google Code to host the project under Artistic License. Please check out the full source code at http://code.google.com/p/linqtocoherence/.

Below is a rundown on two main classes. The main part of the code that deals with Coherence is in two classes CoherenceQueryProvider and CoherenceQueryTranslator.

CoherenceQueryProvider accepts a connection to the INamedCache – a reference to coherence cache that will be queried.

public class CoherenceQueryProvider  : IQueryProvider
{
   public INamedCache Cache { get; set; }
   public CoherenceQueryProvider ()
    {
    }

   public CoherenceQueryProvider(INamedCache cache)
   {
       Cache = cache;
   }

In the Execute method, CoherenceQueryProvider translates the Where clause to a Coherence Filter and executes the filter against the Cache objects to return array of values.

public object Execute(Expression expression)
{
  if (Cache == null)
      throw new InvalidOperationException("Cache is not properly set");

  // Find the call to Where() and get the lambda expression predicate.
  InnermostWhereFinder whereFinder = new InnermostWhereFinder();
  MethodCallExpression whereExpression = whereFinder.GetInnermostWhere(expression);
  LambdaExpression lambdaExpression = (LambdaExpression)((UnaryExpression)(whereExpression.Arguments[1])).Operand;

  // Send the lambda expression through the partial evaluator.
  lambdaExpression = (LambdaExpression)Evaluator.PartialEval(lambdaExpression);

  IFilter filter = new CoherenceQueryTranslator().Translate(lambdaExpression);

  object[] data = Cache.GetValues(filter);
  Type elementType = TypeSystem.GetElementType(expression.Type);
  return data;
}


CoherenceQueryTranslater uses the visitor pattern to convert the Linq Expression from the where clause to Coherence Filter. Coherence filters are nested to converting one to the other is relatively simple

protected override Expression VisitBinary(BinaryExpression b)
{
  this.Visit(b.Left);
  object lastGlobal1 = globalFilter;
  this.Visit(b.Right);
  object lastGlobal2 = globalFilter;
  switch (b.NodeType)
  {
      case ExpressionType.AndAlso:
          globalFilter = new AndFilter((IFilter) lastGlobal1, (IFilter)lastGlobal2);
          break;
      case ExpressionType.OrElse:
          globalFilter = new OrFilter((IFilter) lastGlobal1, (IFilter)lastGlobal2);
          break;

There is a lot more code in the classes to handle other filters, but a lot of it is pretty repetitive. The work on the linq provider is not done and I still have to implement some of the coherence functionality. Full code and usage sample is available on google code http://code.google.com/p/linqtocoherence/

Check it out and post your comments / suggestions.


Share/Save/Bookmark

5 comments:

philchungny said...

Hi Timur,
Glad things are going well with you. Looks promising and will keep this project in mind.

-Phil

dunny said...

Hi Timur,

Interesting but in it's current state it doesnt help much on an enterprise level. There is no support for "object" or arrays. I have a project where such behaviour would be a huge help, we often store arrays of objects to mimic the behaviour of a data row. Lists and objects are a common occurance in most of our data objects.

Good work so far

Thanks
Adrian

Timur said...

dunny: can you post or email be an object example, so I can test.

Thanks

dunny said...

Hi Timur,
My sincere apologies, I just noticed POFWriteAsTypeEnum where I can specify my type as object or Array. Opened my mouth without looking properly. Great work with this. Is serialisation of a dictionary supported? If you are interested in how I am using your binary...

I created a generic caching API for my applications, I wrote a coherence implementation using your generic serialiser, basically the caller has the option to register the types they wish to cache for each type they register I dynamically create a new subclass from their type which I mark-up with your attributes so the caller of the API doesn’t have to mark-up their objects (as it may not be the coherence implementation they are using). The only thing I need to do now is ensure I include this POFWriteAsTypeEnum when marking up the generated objects. The only disadvantage is that I have to be careful that the types i use in my data objects are supported by your implementation.

I haven’t yet tested filtering an array property by an index, is this working? The only other issue is that one of my objects mimics a data row thus I have to index in by column name for the filtering, I planned to put a dictionary in the class allowing me to internally map the column name to an ordinal but I’m not sure if your implementation supports serialisation of dictionaries...

Thanks again, this is great work (sorry for the long post)

Regards,
Adrian

dunny said...

Timur,

Have you been able to successfully use EqualFilter with dates using your generic serializer? It doesnt seem to work for me. I havent looked into the code yet just wandering if it is something not supported or could be related to some other issue. A colleague of mine went to a coherence 3.5 training session last week where he reported that this new version actaually ships with a pof extractor meaning it now supports generic serialisation out of the box. Will be looking forward to that.

Thanks!
Adrian