Extending the Data Grid - LocalStorage=False and Extend connectivity

Oracle coherence is an excellent technology. It is extremely powerful in distributing data and processing across multiple physical and virtual servers. This functionality is extremely simple to achieve, you simply start another machine on coherence JVM on a connected network, and the two nodes will find each other and start talking.

Well… with great power comes great responsibility. Here are a few important things you must be aware of and keep in mind when designing and deploying your distributed coherence applications.

1. All Coherence Nodes act as active members of the grid. This statement is true even if LocalStorage is set to false. Setting LocalStorage to false only removes the node from the list of places where distributed data can be located. However, for replicated data, invocation services, node membership, etc the node is still a fully capable node. See: http://forums.oracle.com/forums/message.jspa?messageID=3664294
2. Extend is the only way to make Coherence Client-Server. This is very closely related to the previous point. It is also very easy to overlook. More explanation below.
3. All coherence nodes must have access to all classes included in POF Configuration. Default POF Configuration Context initializes all types included in the POF configuration even if they will not be used. That means that you must provide access to the classes that define them even though you will not be using them. If you don’t, the cache will fail to initialize
4. You DO NOT need to have identical configuration between all nodes in the grid. There is a benefit to keeping a different set of Cache and POF Configurations (see previous point) However, there is also a complexity of managing multiple configuration files.
5. Extend clients require an extra network hop for working with distributed data located in the grid. However, the overhead is small. Yes – there is an overhead of an extra network hop of working with over extend client. This is something that has to be profiled and waited against the benefits of terminating the extend of the grid to at the Extend Proxy. Consider it over making a client a member of the grid, and therefore making it take role in the cluster management.
6. Extend clients require proxy to pre/post process invocation requests. Complexity for processing invocation service requests is bigger for extend clients. By default, invocation request will be processed by the proxy node, instead of being distributed to the multiple nodes. http://coherence.oracle.com/display/COH35UG/Configuring+and+Using+Coherence+Extend

Extend is the only way to make Coherence Client-Server

Here’s an example where this becomes critically important:

Imagine a 2 node coherence grid that holds some transaction data, and a web site that provides views against that data. Fairly simple environment.

As you see, the connections are between every machine. Every machine is responsible for other members in the grid. Now, we create another website with different functionality, but utilizing the data in the same coherence cache.

Every member is again talking to every other member. Pay attention to the connectivity between Web1 and Web2. Creating second website, not only requires connectivity between the new webserver and data nodes, but also between web2 and web1. If running from different DMZ sites, this might mean a new set of rules in the firewalls. If Web2 is a disaster recovery site that is running in warm state, that will mean that connection DR will actively participate in actions within the Production grid. A different layout of the same problem is when you have separate data entry service, and website for data display

Connectivity between Data Entry Service and Website might not bet apparent upfront, however, without it, the cluster will not start.

Even without physical connectivity concerns, the extend of the grid is something to think extra hard about. Coherence includes the concept of Senior Member. From http://wiki.tangosol.com/display/COH34UG/Security+Framework: “Each clustered service in Coherence maintains a concept of a "senior" service member (cluster node), which serves as a controlling agent for a particular service. While the senior member does not have to consult anyone when accessing a clustered resource, any junior node willing to join that service has to request and receive a confirmation from the senior member, which in turn notifies all other cluster nodes about the joining node.” After running the grid for a while, the coherence node running within Web site, might be the oldest running member of the grid, and therefore will become the Senior Member. It will no longer be a “client” of the grid as you might believe, but it will become critical actor during grid membership resolution.

Technorati Tags: Architecture,Oracle Coherence

Extending the Data Grid - LocalStorage=False and Extend connectivity

I am here to help... really

“Why are these guys messing with my design. What makes them think they know better, especially since they don’t actually produce anything?!” As a developer, these are pretty often the first thoughts I used to say when I had to deal with an “Architect”. Now I hear them back. Am I really a Pig…. I certainly hope not.

I am making a reference to an excellent post “Chickens, Chicks, Pigs and Piglets”. My latest interaction with the business development team started out very much like that. It pointed to the problem with many non-business aligned technology teams (such as architecture). We need to prove to others that we actually help the their teams.

It doesn’t matter how good you are, how good your designs are, or how new your technology is. The only thing that counts is “does it help the other team deliver their product and to run it after delivery.” At the end, we are there to support the other business technology teams. It is an extremely important point to remember and to follow through on.

So back to my latest project. I think its back on track after extra effort on both sides of the table. The most important thing to remember: “Commitment to the project”. It is not the technology, or the ideology that should be the center of attention. It should be the project. As an architect, Only after you can show to others that you are there to help their projects, not to derail them by forcing them to go after “The Vision” are you able to successfully influence and deliver that vision.

Technorati Tags: Development,Architecture,Management

I am here to help... really

Source Control Management 201 - Repository Design for efficient code management using any source control

Source control management is an essential part of the development. It should be just as critical part of the developer’s toolbox as a good text editor. There is no project that is small enough not to deserve one. There is no such thing as bad source control management system. While there are probably hundreds SCM systems out there, some are more user developer friendly then others. Some of the more common ones you hear about today are Subversion, CVS, Git, Visual SourceSafe, Rational ClearCase.

A goal of a source control system is to answer the following questions:

• What code is currently running in production?
• What is the latest code a developer should be looking at?
• What code was used to compile version X.XX.XXX?
• How can a developer make changes without affecting other developers?
• What changed between version X.XX.XXX and X.XX.XXY?
• How can a developer incorporate changes between X.XX.XXX and X.XX.XXY to make X.XX.XXZ?
• How can a developer rollback changes from X.XX.XXY to get back to X.XX.XXX?

All source control versions that I’ve worked with can answer all of the above questions. It does take some organizational skills on the developer to allow it to do so. SCM system stores code in a repository. It is up to the developer and SCM team to organize the layout of the repository. Some of the more common strategies are Trunk Focused

All development is done on the trunk. When the code is stable enough to be ready for QA/Production Testing it is brunched into a Beta/Release Candidate branch. The trunk version is incremented. The RC branch should have only minor bug fix related changes applied to it. All major changes are done in the trunk. As bugs are fixed in the RC branch, the changes are migrated into the trunk. When a version is released, the RC branch is moved into a Release branch. CHANGES ARE NEVER MADE IN THE RC BRANCH. IT IS READ ONLY! If a bug fix must be made to the already released version, a branch is created, and a change is done on the branch. The fix is then merged into the trunk.

This repository allows to easily answering developer’s question

• What is the latest code I should develop from: Trunk
• What code is currently running in production: Latest Read Only branch
• How to make changes to version X.XX.XXX: Make a new branch from the version X.XX.XXX. After finishing your changes, merge them into the trunk.

Another strategy is to organize the repository around production version.

Main trunk has the currently released production version. The trunk IS READ ONLY! For development, a branch is created, and development is done on the branch. After the version is released to production, the trunk is replaced with the copy of the released branch, and is again made read only. To develop the next version, another branch is created. If a fix has to be made to the production release, a branch for the fix is created. Once fix is released, a new trunk is created from the fix branch. The changes are merged into the branch under current development.

This layout answers the same questions slightly differently:

• What code is currently running in production: Trunk
• What is the latest code I should develop from: Latest branch
• How to make changes to version X.XX.XXX: Make a new branch from the version X.XX.XXX. After bug fix is released into production, the branch is copied and becomes the new trunk. Changes are merged into the latest development branch.

There are other repository layouts available, and / or your team might make design changes around above structures.

Technorati Tags: Development,Source Control,SCM,Tools,Tips

Source Control Management 201 - Repository Design for efficient code management using any source control

Multithreading will only take you so far

Working with Oracle Coherence, I do a lot of thinking about distributed architecture, parallel processing, and multithreading. Making use of all this technology is a great way to solve many problems. It can often seem, that as long as you can split your problem into small enough pieces, you’ll be able to process data instantaneously.

When thinking about distributed system, we often forget that making a system distributed, we still have a limited number resources to distribute the workload. The application is deployed on a specific number of machines, each with a specific number of CPU Cores. Each CPU core can process one instruction at a time (not completely true, but for simplicity’s sake). For example: In a distributed system where each workload takes 1 second to process, and we have a total of 10 workloads that need to be processed, will take at least 3 seconds to process all of them

Adding more threads will not work, only adding more CPU cores will. That is critically important when you consider that for many complex operations, a result of individual workload is not enough to provide a meaningful result to the end user. Results from all requested workloads have to be aggregated to create a final result.  This makes it relatively simple to figure out how much a calculation will take:

Total Time = Number of Work Loads / Number of Cores * Time Take by Work Load

Another very import point to remember, is that during performance testing, distributed systems behave differently. Requests from multiple clients will interfere a lot more with each other then they do in a straight processing system. In a prior example, a request that takes 3 seconds when ran from 1 client, can take 6 seconds with 2 clients, if the last work item for the first client, is started after all work items of the second client.

That will probably not happen, yet you have take the possibility into account. That is just the nature of the beast. 

Technorati Tags: Development,Performance,Tips,Multithreading,Coherence

Multithreading will only take you so far

Looking for work in a tough market

For over a month, I’ve been pretty quiet. No new posts, comments, tweets, etc… The reason behind it was that I was too busy looking for a job. Yep… A month and a half ago, I was “Right Sized” by my previous employer and was places into the ranks of millions of unemployed Americans.

Being laid off is definitely a learning experience, one that I would not wish on anyone. I got a chance to rethink my ideas on job security, professional relationships and job searching. After 6 weeks, I've started a new job that is quite a bit different then my previous one. I will be doing a lot more software architecture than management (Comes with the Sr. Architect title :). I will also be doing a lot more Java and no .Net development. I will also be working out of New Jersey instead of New York City. That's a lot of changes, a lot of things I didn't quite look for originally, yet all are good. My commute is expectantly shorter, I am doing what I like, and I am expanding my technology footprint.

My experience of looking for a new job was eye opening as well. There are tons of guides to get you through the time when you are looking for work. I don’t have a guide, these are some comments from my own recent experience.

• Ask everyone / Network – Talk to everyone you’ve had good relationship with. Friends, Recruiters (Good Recruiters), Vendors, Technology Partners, etc…
• Use Networking Sites – LinkedIn.com works wonders. Facebook, twitter, plaxo, etc… Do not ignore the sites because you think that no one will notice your post. People do notice, and they do help!
• Be Flexible – Don’t ignore an opportunity because its not a perfect fit. This relates to technology, organization and salary requirements. Don’t be afraid of a different technology. As a manager, specific development language matters a lot less then you think. The more flexible you are, the easier it will be for you to talk to people. You might be pleasantly surprised about the opportunity you might have turned down otherwise.
• Be Patient – This might be the toughest one. There are tons of thoughts that go through your head. It would be great to get a response to your resume the same day – but you will probably not get it that fast.
• Don’t burn your bridges – This is, VERY IMPORTANT. The world is much smaller then it seems. I guarantee you, that you will meet people that know someone who knows you. And every relationship you kept, will be remembered.

To those who are now looking for work, good luck. Please contact me via linked in on http://www.linkedin.com/in/tfanshteyn. If there is anything that I’ll be able to do to help, I’ll gladly do so.

For those managers, who decide on the hiring and firing. Don’t make your decisions lightly. Don’t ignore resumes – people are waiting. And don’t ever let someone go easily.

Technorati Tags: Tips,Employment,Business

Looking for work in a tough market

Oracle Coherence SIG - Presentation and sample code

Oracle Coherence SIG came around. Excellent event. It was twitted live, so go ahead and search : http://search.twitter.com/search?q=%23nycsig

I spend around 30 minutes talking about using Coherence with .Net. First part of the presentation was on general Coherence configuration and setup. Second part was on more advanced use of serialization wrappers and LINQ to Coherence. People asked some great questions. I guess the one important point I wanted to reiterate is that you should not be afraid of using Coherence with .Net. It is a really great product. You will definitely get a lot of benefit out of the box with it. The wrappers I’ve provided will create some overhead, but that should be acceptable to many. Wrappers and LINQ are not a solution that will solve every .Net developers problem, but should definitely get you started.

I’ve uploaded the presentation and the samples (in 4MB large zip file).

Presentation http://tfanshteyn.110mb.com/CoherencePresentation.pdf
Sample Files http://tfanshteyn.110mb.com/CoherencePresentation.zip

Technorati Tags: Coherence,Development,C#,Sample Code

Oracle Coherence SIG - Presentation and sample code

I’ll be presenting at NY Coherence SIG

I will be making a presentation at NY Coherence SIG on June 24th. Check it out.

http://coherence.oracle.com/display/CSIG/24+June+2009+-+New+York%2C+NY
http://craigblitz.typepad.com/rawpower/2009/06/announcing-summer-new-york-coherence-sig.html

Craig Blitz will be twittering live from the SIG. http://craigblitz.typepad.com/rawpower/2009/06/live-twittering-the-next-nycsig.html  Join us by searching for #nycsig on twitter.

Using Oracle Coherence in .NET Environment

Timur Fanshteyn (speaker bio) Technology Manager
Oracle Coherence can be successfully used in a Microsoft Window and .NET Environment. It can be a excellent media for connecting .Net and Java applications. It can provide .NET applications with a caching solution in front of a database server. It can even be used to provide real-time data to the desktop clients. All with minimum or possible no java coding.
The presentation will go through requirements of setting up Coherence .NET Client. Go through use cases and point out some limitations. We will focus on general data caching, object serialization (through POF), event notifications and LINQ to coherence.

Outline

• Oracle Coherence for .Net Environment
  • Basic Setup and Configuration
• POF Serialization
  • Serialization
  • Cross platform data transfer
• Invocations, Eventing and Notifications
  • Using Coherence as messaging bus
  • Moving data to the desktop
• Advanced .NET Integration
  • Attributes base serialization
  • Serializing Metadata
  • Linq to Coherence

Technorati Tags: Oracle,Coherence

I’ll be presenting at NY Coherence SIG

Linq to Coherence + Attributes + MetaData = Cool

First, we got the Linq Provider for Oracle Coherence

Second, we got attribute based serialization

Third, Metadata in the serialization stream

Once we put all that together, what we get is a set of very clean way of storing and querying data in Oracle Coherence.

A Coherence Linq provider now supports passing a CoherenceQueryTranslator as a parameter. I am providing a MetadataCoherenceQueryTranslator that uses getProperty method to access property originally serialized by the Generic Serializer. Here’s almost all relative .Net Code:

Person Class

[POFSerializableObject(StoreMetadata=true)]
    public class Person// : IPortableObject
    {
        [POFSerializableMember(Order=0,WriteAsType=POFWriteAsTypeEnum.Int16)]
        public int ID { get; set; }
        [POFSerializableMember(Order=1)]
        public string FirstName { get; set; }
        [POFSerializableMember(Order = 2)]
        public string LastName { get; set; }
        [POFSerializableMember(Order = 3)]
        public string Address { get; set; }
        [POFSerializableMember(Order = 4)]
        public string Title { get; set; }
        public Person()
        {
        }
    }

Add object function

INamedCache cache = CacheFactory.GetCache("dist-Person");
for (int i = 0; i < 1000; i++)
{
cache.Add(i, new Person()
{
   ID = i,
   FirstName = string.Format("First Name {0}", i),
   LastName = string.Format("LastName {0}", i),
   Address = string.Format("Address {0}" , Guid.NewGuid()) ,
   Title = i % 2  == 1 ? "Mr" : "Mrs"
   });
}

Query using Linq Query:

CoherenceQuery<Person> coherenceData =
 new CoherenceQuery<Person>(
	 new CoherenceQueryProvider(CacheFactory.GetCache("dist-Person"), 
		 new MetadataCoherenceQueryTranslator()));
string likeClause = "%8";
var people = from person in coherenceData
			 where
				(person.FirstName.Like("Test")
				 || person.LastName.Like(likeClause))
				 && person.Title == "Mrs"
			 select new { person.Title, person.ID, person.LastName };
IFilter filter = ((ICoherenceQueryable)people).Filter;
dataGridView1.DataSource = people.ToArray();

Internally, MetadataCoherenceQueryTranslator, will convert the linq query into a filter and execute the query against the Java POFGenericObject

Technorati Tags: Coherence,C#,Development,Oracle,Open Source,.Net

Linq to Coherence + Attributes + MetaData = Cool

99.9% pure .NET Coherence (100 % No Java Code Required)

Its not 100% .NET Coherence, since a developer is required to modify configuration files on the java service to specify the generic POF Serializer.

So to start: The .NET Class:

 

    [POFSerializableObject(StoreMetadata=true)]
    public class Person
    {
        [POFSerializableMember(Order=0,WriteAsType=POFWriteAsTypeEnum.Int16)]
        public int ID { get; set; }
        [POFSerializableMember(Order=1)]
        public string FirstName { get; set; }
        [POFSerializableMember(Order = 2)]
        public string LastName { get; set; }
        [POFSerializableMember(Order = 3)]
        public string Address { get; set; }
        [POFSerializableMember(Order = 4)]
        public string Title { get; set; }
        public Person()
        {
        }
        
    }

Note the StoreMetadata=true argument. When StoreMetadata is specified, Generic serializer will first first write a string array of property names. On server side, we must specify a Generic Java serializer. This is needed to be able to store objects for filtering. Here’s an interesting note. Unless filters are invoked, the object WILL NOT be deserialized on the java side. That means, Put, Get, GetAll calls without a filter, do not require Metadata to be written into the cache. Now. To specify Java Generic Serializer. Distributed Cache Configuration:

      <distributed-scheme>
      <scheme-name>dist-default</scheme-name>
      <serializer>
		<class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
		<init-params>
		  <init-param>
			<param-type>string</param-type>
			<param-value>custom-types-pof-config.xml</param-value>
		  </init-param>
		</init-params>
	  </serializer>
      <backing-map-scheme>
        <local-scheme/>
      </backing-map-scheme>
      <autostart>true</autostart>
    </distributed-scheme>

custom-types-pof-config.xml

<pof-config>
  <user-type-list>
    <!-- include all "standard" Coherence POF user types -->
    <include>coherence-pof-config.xml</include>
    <!-- include all application POF user types -->
    <user-type>
      <type-id>1001</type-id>
      <class-name>com.Coherence.Contrib.POF.POFGenericObject</class-name>
      <serializer>
        <class-name>com.Coherence.Contrib.POF.POFGenericSerializer</class-name>
        <init-params>
           <init-param>
             <param-type>int</param-type>
             <param-value>{type-id}</param-value>
           </init-param>
           <init-param>
            <param-type>boolean</param-type>
             <param-name>LoadMetadata</param-name>
             <param-value>true</param-value>
           </init-param>
         </init-params>        
	  </serializer>
    </user-type>
  </user-type-list>
</pof-config>

For now, you must specify a user-type for each .Net object. On the java side, the server will be using the POFGeneicSerializer, and all values in the object array indexed by the property names. A generic getProperty method is implemented to allow filtering on any property that was used in the serialization. Property evaluation is happening on the server, so only filtered data is returned.

Here’s a simple loop to add an object into the cache

INamedCache cache = CacheFactory.GetCache("dist-Person");
for (int i = 0; i < 1000; i++)
{
    cache.Add(i, new Person()
    {
       ID = i,
       FirstName = string.Format("First Name {0}", i),
       LastName = string.Format("LastName {0}", i),

       Address = string.Format("Address {0}" , Guid.NewGuid()) ,
       Title = i % 2  == 1 ? "Mr" : "Mrs"
    });
}

A cool side effect, data can be accessed from .Net and from Java code in the same fashion.

Technorati Tags: Oracle,Coherence,Open Source,.Net,Development

99.9% pure .NET Coherence (100 % No Java Code Required)

Attributes based Coherence POF Serializer

As much as I love Oracle Coherence, it is a very much a Java product. .Net has lots of cool tricks that can be used during programming, however, they are not implemented in Coherence .Net

Many .Net developers are used to using attributes to define object serialization. A Coherence Generic serializer allows a developer to specify POF Serialization using object attributes as well. Here’s a simple example

    [POFSerializableObject()]
    public class Person// : IPortableObject
    {
        [POFSerializableMember(Order=0,WriteAsType=POFWriteAsTypeEnum.Int16)]
        public int ID { get; set; }
        [POFSerializableMember(Order=1)]
        public string FirstName { get; set; }
        [POFSerializableMember(Order = 2)]
        public string LastName { get; set; }
        [POFSerializableMember(Order = 3)]
        public string Address { get; set; }
        [POFSerializableMember(Order = 4)]
        public string Title { get; set; }
        public Person()
        {
        }
        //#region IPortableObject Members
        //public void ReadExternal(IPofReader reader)
        //{
        //    ID = reader.ReadInt32(0);
        //    FirstName = reader.ReadString(1);
        //    LastName = reader.ReadString(2);
        //    Address = reader.ReadString(3);
        //    Title = reader.ReadString(4);
        //}
        //public void WriteExternal(IPofWriter writer)
        //{
        //    writer.WriteInt32(0, ID);
        //    writer.WriteString(1, FirstName);
        //    writer.WriteString(2, LastName);
        //    writer.WriteString(3, Address);
        //    writer.WriteString(4, Title);
        //}
        //#endregion
    }

To force Coherence to use the serialization attribute, use the the Coherence Generic Serializer in the POF Configuration File

    <user-type>
      <type-id>1001</type-id>
      <class-name>CoherenceSample.Person, CoherenceSample</class-name>
      <serializer>
        <class-name>Coherence.Contrib.POFGenericSerializer, Coherence.Contrib</class-name>
      </serializer>
    </user-type>

Some important points to mention:

• Object must include POFSerializableObject attribute
• Generic Serializer will serialize Properties and Members
• Objects MUST NOT implement IPortableObject. Doing so will force Coherence to use ReadExternal and WriteExternal functions instead of a custom serializer
• Order parameter of the attribute is optional. The order of serialization is Order, Alphabetical Ascending. Meaning. Multiple attributes can have the same order argument, and will be serialized in alphabetical order
• WriteAsType parameter is optional and is usually derived based on the source type.
• Serializer uses Converter.Convert() to convert between object types.
• Most (but not all) main types are implemented. Check out the source code for specifics
• Hardcoding a serializer still provides better performance due to extra boxing and object conversion performed by the serializer.

Please check out the latest code on Google Code: http://code.google.com/p/linqtocoherence

Technorati Tags: Coherence,C#,Oracle,DotNet

Attributes based Coherence POF Serializer

Glad to have my IPhone back

My IPhone didn’t work this morning, so I wound up using a loaner Blackberry. Now… I’ve used a blackberry for over 5 years before my IPhone, and the interface didn’t change much. However, switching back was definitely an eye opener. Let’s just say I am glad my IPhone is back in business.

What I’ve learned:

Pro IPhone

• I can type just as fast on the IPhone as I can on the blackberry. Since the IPhone “keys” are actually bigger, I even like IPhone typing more then blackberry typing.
• I miss all the additional apps that I’ve installed on my IPhone. And they are not available on the blackberry

Pro Blackberry (I really wish IPhone had these)

• Battery Life – It keeps going, and going, and going. Last time I charged it was 2 days ago. And it still has 2 out of 5 battery bars left
• Blinking red light tells me I have new emails. I can see that from across the room.
• EMail Shortcuts (T – Top / B – Bottom / U – Unread email)
• Some additional email functionality – Mark all as read. Delete All Older EMail (from blackberry)
• Quiet mode – The IPhone has only Sound and / Vibrate. Blackberry has multiple settings.
• Brick Breaker – Spent 45 minutes playing it on the bus. I miss it.

At the end, I am glad that I have my IPhone back. Blackberry is a great business device, but a mediocre consumer accessory. I wish IPhone gets a better batter life and a few EMail / Calendar improvements.

Technorati Tags: Hardware,IPhone,Blackberry

Glad to have my IPhone back

Windows Server 2008 R2 is 64bit only

Learned something new from Mark’s blog: Windows Server 2008 R2 is 64bit ONLY( The news is not new, but I only learned it tonight.) It will still support 32 bit applications, but only via OPTIONAL WoW64 functionality that is not added by default. I’ve heard about Microsoft stopping 32bit OS development a while back, but seeing this actually done…. This is BIG.

Technorati Tags: Microsoft,Windows,Server

Windows Server 2008 R2 is 64bit only

Subversion 1.6 is released

Subversion 1.6 (And so is TortoiseSVN 1.6 and Subclipse 1.6) is released.

New Functionality includes

• Improved handling of authentication data
• Repository root relative URLs
• Improvements to svn:externals
• Detection of tree conflicts
• Filesystem storage improvements
• Ctypes Python Bindings
• Improved interactive conflict resolution
• Sparse directory exclusion
• Logging support for svnserve
• New public HTTP URI syntax for examining history
• Command-line client improvements
• API changes, improvements, and much language bindings work
• More than 65 new bug fixes, enhancements
  

TortoiseSVN includes a completely rewritten Revision Graph

Get it while its hot www.tigris.org

Technorati Tags: Subversion,TortoiseSVN,Tools,Source Control

Subversion 1.6 is released

Internet Explorer 8 is released – and I am using it

IE8 is now released and I’ve just upgraded my home laptop to it (I try not to mess around with betas on my main machine) So far so good. Speed of the browser is definitely something to brag about. I’ve also liked accelerators that come with IE. When you select a block of text with a mouse, a menu pops up that allows you to blog, email, search and even translate. Very useful functionality that can be extended by third-parties.

Download the new Internet Explorer at http://www.microsoft.com/windows/internet-explorer/default.aspx

Technorati Tags: Internet Explorer,Microsoft

Internet Explorer 8 is released – and I am using it

Linq Provider for Oracle Coherence – Pt 2

In previous post I published about a Linq for Oracle Coherence. Linq makes it very convenient to add functionality to a provider, however creating a functionally reach and complete provider is a very complicated process.

Latest functionality I’ve added was ability to use coherence extractors in the query. Linq does not allow you to change its keywords and syntax, however, all functions are converted to linq expressions and are passed to the ExpressionVisitor for evaluation. First thing is to create an extended function to provide the functionality

public static T Extractor<T>(this object obj, string Name)
{
    throw new NotSupportedException();
}
public static T ChainedExtractor<T>(this object obj, string Name)
{
    throw new NotSupportedException();
}

The functions themselves do not have any functionality and are used solely to be converted to a linq expression. I’ve created them as Generic functions to be able to strongly type against extracted data

In the QueryTranslator class VisitMethodCall() function, we add functionality to evaluate the expression

if (m.Method.DeclaringType == typeof(LinqFunctions))
{
    if (m.Method.Name == "Extractor")
    {
        this.Visit(m.Arguments[1]);
        string right = (string)globalFilter;
        globalFilter = new Tangosol.Util.Extractor.ReflectionExtractor(right);
        return m;
    }
    else if (m.Method.Name == "ChainedExtractor")
    {
        this.Visit(m.Arguments[1]);
        string right = (string)globalFilter;
        globalFilter = new Tangosol.Util.Extractor.ChainedExtractor(right);
        return m;
    }
}

m.Arguments[0] has the reference to the parent object. m.Arguments[0] contains the argument. This allows us to create a Coherence Extractor that is used in the filters higher in the expression tree.

---

A Note: I hope that the project gets some visibility and comments. Oracle Coherence is an excellent product with great .NET potential. Check out the project on google code: http://code.google.com/p/linqtocoherence

Technorati Tags: Oracle,Coherence,Microsoft,Open Source,C#

Linq Provider for Oracle Coherence – Pt 2

Linq provider for Oracle Coherence (Linq to Coherence)

I am very impressed with Coherence from Oracle. Coherence provides a distributed in-memory cache and processing fabric. However it is a lot more then just a cache. It can be used for everything from messaging to cross platform communication medium. There is too much to talk say about it, so read more information at Oracle: http://www.oracle.com/technology/products/coherence/index.html

Coherence works very nicely with .Net however, in the days of Linq, I wanted to write a Linq provider for it. My code is based largely on the Linq provider documentation on MSDN (http://msdn.microsoft.com/en-us/library/bb546158.aspx) and excellent series on creating a linq provider by Matt Warren (http://blogs.msdn.com/mattwar/pages/linq-links.aspx)

I am using Google Code to host the project under Artistic License. Please check out the full source code at http://code.google.com/p/linqtocoherence/.

Below is a rundown on two main classes. The main part of the code that deals with Coherence is in two classes CoherenceQueryProvider and CoherenceQueryTranslator.

CoherenceQueryProvider accepts a connection to the INamedCache – a reference to coherence cache that will be queried.

public class CoherenceQueryProvider  : IQueryProvider
{
   public INamedCache Cache { get; set; }
   public CoherenceQueryProvider ()
    {
    }

   public CoherenceQueryProvider(INamedCache cache)
   {
       Cache = cache;
   }

In the Execute method, CoherenceQueryProvider translates the Where clause to a Coherence Filter and executes the filter against the Cache objects to return array of values.

public object Execute(Expression expression)
{
  if (Cache == null)
      throw new InvalidOperationException("Cache is not properly set");

  // Find the call to Where() and get the lambda expression predicate.
  InnermostWhereFinder whereFinder = new InnermostWhereFinder();
  MethodCallExpression whereExpression = whereFinder.GetInnermostWhere(expression);
  LambdaExpression lambdaExpression = (LambdaExpression)((UnaryExpression)(whereExpression.Arguments[1])).Operand;

  // Send the lambda expression through the partial evaluator.
  lambdaExpression = (LambdaExpression)Evaluator.PartialEval(lambdaExpression);

  IFilter filter = new CoherenceQueryTranslator().Translate(lambdaExpression);

  object[] data = Cache.GetValues(filter);
  Type elementType = TypeSystem.GetElementType(expression.Type);
  return data;
}

---

CoherenceQueryTranslater uses the visitor pattern to convert the Linq Expression from the where clause to Coherence Filter. Coherence filters are nested to converting one to the other is relatively simple

protected override Expression VisitBinary(BinaryExpression b)
{
  this.Visit(b.Left);
  object lastGlobal1 = globalFilter;
  this.Visit(b.Right);
  object lastGlobal2 = globalFilter;
  switch (b.NodeType)
  {
      case ExpressionType.AndAlso:
          globalFilter = new AndFilter((IFilter) lastGlobal1, (IFilter)lastGlobal2);
          break;
      case ExpressionType.OrElse:
          globalFilter = new OrFilter((IFilter) lastGlobal1, (IFilter)lastGlobal2);
          break;

There is a lot more code in the classes to handle other filters, but a lot of it is pretty repetitive. The work on the linq provider is not done and I still have to implement some of the coherence functionality. Full code and usage sample is available on google code http://code.google.com/p/linqtocoherence/

Check it out and post your comments / suggestions.

Technorati Tags: Microsoft,Linq,Development,Oracle,Coherence,Open Source,C#

Linq provider for Oracle Coherence (Linq to Coherence)