Monday, March 3, 2008

Disconnected LINQ to SQL Tips Part 1

Intro!
In my research on LINQ to SQL and trying to work around the limititation of no "out of the box" disconnected (n-tier) mode, I've come accross a lot of things that others may find useful.

I was somewhat disappointed in this drawback as LINQ to SQL - seeing that it was only intended to be used in "connected" scenario. I saw it as a challenge to figure out ways in which a "disconnected" scenario could be done.

Hence, here I am a first time virgin blogger - who felt compelled to reduce the sweat and tears of others while dealing with this double edged sword.

I'll be blogging how "Disconnected LINQ" can be achieved in a later post, but first up here's some tips for some of you out there that might be struggling with "Disconnected LINQ".

The 'How to do Disconnected' Rules
First up, here are the rules for allowing disconnected LINQ to SQL. To successfully disconnect a LINQ to SQL entity from a Data Context and allow it to re-connect to a different Data Context, you must do the following:

1. Enable Concurrency Tracking
You can enable concurrency tracking by adding a timestamp (rowversion) field to your database and include this in your LINQ to SQL model.

Alternatively, if you don't care for concurrency tracking you can set all columns so that the Update Check is set to false.

2. Disable Deffered Loading, Load Everything or Serialize the objects
Disabling Lazy loading is fairly straight forward, it just a simple matter of setting the DeferredLoadingEnabled = False like so:



using (EntitiesDataContext db = new EntitiesDataContext())
{
    db.DeferredLoadingEnabled = false;
 
    var customers = from c in db.Customers
                    where c.CustomerId == CustomerId
                    select c;
}

This tells link not to query that database when a link association (another object or collection) is referenced, instead it will simply return a null.

Alternatively, you can also load all related objects so there is nothing to lazy load. This can be done by using the datacontext load options object like so:


using (EntitiesDataContext db = new EntitiesDataContext())
{
    DataLoadOptions lo = new DataLoadOptions();
    lo.LoadWith<Customer>(c => c.Dependants);
    db.LoadOptions = lo;
 
    var customers = from c in db.Customers
                    where c.CustomerId == CustomerId
                    select c;
}

Basically, this is telling the data context that whenever it loads a Customer, it should also load the Dependants for that customer. If all the related objects or collections are covered in this way, LINQ to SQL won't even consider deferred loading because it believes it has all the possible connected objects.

As for serialization, this automatically disables deferred loading as above ... and how to serialize is covered next....

Serializing and copying the objects
Linq to SQL entities cannot be serialized using standard serialization techniques, this means you can't just pop it in the view state or ASP.NET state server without running into some trouble.

In order to serialize an LINQ to SQL object graph (a root object and it's child entities) you'll need to use the WCF data contract serializer instead.

But wait! Even before you do that, you'll need to set your Data Contexts serialization mode to "Unidirectional" (which is available in the model properties). This means that only references in the object graph from parent to child will be serialized and therefore therefore child to parent references will be ignored. This is good because it means that circular references won't cause problems. The standard serializer can't be used for exactly this reason, because it would end up with failing because it would try to serialize circular references.

Here's a couple of functions which can be used to serialize/deserialize a LINQ to SQL object graph and also a handy copy function which will make a completely seperate copy of an object graph.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using System.Text;
using System.IO;
using System.Xml;
 
namespace SampleFramework
{
    public static class LINQHelper
    {       
 
        /// <summary>
        /// Makes a copy of an existing LINQ to SQL entity and it's children.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="entitySource">The LINQ to SQL entity to copy</param>
        /// <returns></returns>
        public static T CopyEntityDeep<T>(T entitySource)
        {
            if (entitySource == null)
                return default(T);
 
            return (T)DeserializeEntity(SerializeEntity(entitySource), entitySource.GetType());
        }
 
        /// <summary>
        /// Makes a copy of a list of existing LINQ to SQL entities and their children.
        /// </summary>
        /// <typeparam name="T"></typeparam>
        /// <param name="source">The LIST of SQL entities to copy
        /// </param>
        /// <returns></returns>
        public static List<T> CopyEntityListDeep<T>(List<T> entitySourceList)
        {
            List<T> result = new List<T>();
 
            if (entitySourceList == null)
                return null;
 
 
            foreach (T entitySource in entitySourceList)
            {
                T entityTarget = CopyEntityDeep(entitySource);
 
                result.Add(entityTarget);
            }
 
            return result;
 
        }
     
        public static string SerializeEntity<T>(T entitySource)
        {
            DataContractSerializer dcs = new DataContractSerializer(entitySource.GetType());
 
            if (entitySource == null)
                return null;
 
            StringBuilder sb = new StringBuilder();
            XmlWriter xmlw = XmlWriter.Create(sb);
            dcs.WriteObject(xmlw, entitySource);
            xmlw.Close();
 
            return sb.ToString();
        }
 
        public static object DeserializeEntity(string entitySource, Type entityType)
        {
            object entityTarget;
 
            if (entityType == null)
                return null;
 
            DataContractSerializer dcs = new DataContractSerializer(entityType);
 
            StringReader sr = new StringReader(entitySource);
            XmlTextReader xmltr = new XmlTextReader(sr);
            entityTarget = (object)dcs.ReadObject(xmltr);
            xmltr.Close();
 
            return entityTarget;
        }
    }
}

Wrapping up
Next post, I hope to move further into the disconnected model that I've come up with that uses some of the above techniques to track changes.

Cheers

Matt.

13 comments:

Benjamin Eidelman said...

Matt,

Incentivated by your comment in our blog, I posted a new entry giving details about our disconnected change tracking implementation.

http://pampanotes.tercerplaneta.com/2008/03/implementing-n-tier-change-tracking.html

please check it out, I'll apreciate your comments as we are solving the same problem, don't we?

Matthew Hunter said...

Hi Benjamin,

I really like your idea's!! I've read the blog and will comment soon!

I'm just a bit busy because I put a Beta of my base class on codeplex (http://www.codeplex.com/LINQ2SQLEB) - it's a different method to what you are doing, but there are a lot of similar ideas and our goals certainly are the same!

Anonymous said...

Hi

Good blog and considering trying out your EntityBase component once you've got a version to work with WCF (which we are starting to use here).

Regarding the above blog, are you saying that to disconnect entities from their "originating" datacontext you can EITHER set DeferredLoadingEnabled=false OR load up-front using the LoadOptions OR serialize the entity(s)?
My understanding was that serialization was the only way (unfortunately I haven't got far enough down the LINQ road to try it yet...)

Cheers

Matthew Hunter said...

Thanks, I'll be working on the WCF stuf soon, just quite busy at work at the moment and haven't had time to get back to it.

In regards to your question:

Yes, if you set DeferredLoadingEnabled = False, this basically tells LINQ not to go to the database if you use a relationship that hasn't had objects loaded yet - which is required to disconnect the object.

Even though Microsoft says the following:

"Only call the Attach methods on new or deserialized entities. The only way for an entity to be detached from its original data context is for it to be serialized. If you try to attach an undetached entity to a new data context, and that entity still has deferred loaders from its previous data context, LINQ to SQL will thrown an exception. An entity with deferred loaders from two different data contexts could cause unwanted results when you perform insert, update, and delete operations on that entity. For more information about deferred loaders, see Deferred versus Immediate Loading (LINQ to SQL)."

http://msdn2.microsoft.com/en-us/library/bb546187.aspx

You'll notice that use serialization to make sure that any deferred loaders are disabled, so that attaching the objects later on is safe. However you don't need to serialize to do this, you can explicitly set the "DeferredLoadingEnabled" option yourself instead - so you can pass it through the layers of your application (in the same process) without a Data Context.

Of course, if you are passing it out of process you will need to serialize the objects first to make this possible.

Perhaps they wrote the note mentioned above to keep things simple? But it certainly does work when you explicitly disable deferred loading.

Anonymous said...

Matt,

Have you considered incorporating into your serialize and de-serialize routines a compression routine. I do it to datatables all the time. I serialize them to binary format and then compress them using the compression routines in .net. It may minimize the amount of data being sent across the wire.

Chris.

Matthew Hunter said...

Hi Chris,

that's not a bad idea! I might think about doing it for a later release.

Anonymous said...

In reponse to your previous comment about MS documentation, try re-reading the first sentence as it set's the context

"When you serialize entity objects such as Customers or Orders to a client over a NETWORK" (not in process)

Even though Microsoft says the following:

"Only call the Attach methods on new or deserialized entities. The only way for an entity to be detached from its original data context is for it to be serialized. If you try to attach an undetached entity to a new data context, and that entity still has deferred loaders from its previous data context, LINQ to SQL will thrown an exception. An entity with deferred loaders from two different data contexts could cause unwanted results when you perform insert, update, and delete operations on that entity. For more information about deferred loaders, see Deferred versus Immediate Loading (LINQ to SQL)."

http://msdn2.microsoft.com/en-us/library/bb546187.aspx

Matthew Hunter said...

Hi there,

I re-read the information, context is always important - but what I said is still correct. Could you let me know what you meant specifically?

:)

mkamoski said...

Do you know of a way to get the serialization to write "utf-8" instead of "utf-16"?

mkamoski said...

Mike -- I refactored the code very slightly, in an unelegant yet practical way. I have posted the refactored version here... http://mkamoski1.wordpress.com/2009/12/02/linqhelper-refactor-idea/ ...with full credits to you, of course. Thank you. -- Mark Kamoski

Matthew Hunter said...

Hey Mark, if you check in the LINQ 2 SQL EB the serializer in there deals with Known Types - you'll probably need this too as sometimes with inheritence and other some other conditions, the serializer can't work out all the types.

See http://blogs.msdn.com/sowmy/archive/2006/06/06/618877.aspx for more info.

And finally, you might want to check out [DataContract(IsReference=true)] on your entities, it uses references during serialization instead of repeating entities every time.
http://www.zamd.net/2008/05/20/DataContractSerializerAndIsReferenceProperty.aspx

Anonymous said...

Maintain state of a dynamic variable:
@Matt:
Thanks for such a great article!
I have some problem I hope you can help me with that?

I am using linq and thus I have a few "var" types object created dynamically, as in

var Order = CustomerDataContext.Order.Where(e => e.OrderPending)

and many others like this, so the problem is how can I maintain this "var" across method calls, I am using

ThreadPool.QueueUserForWorkItem(ProcessOrder, var)

but this method (ProcessOrder) accepts object (as per specification of the delegate as specified in QueueUserForWorkItem) and I cannot cast the back the parameter back into Order type. Also, since its dynamic it cannot even be declared as static, so that I may maintain state, so what should I do? How should I maintain this var, or if I cannot, then what is the other way to do what I am doing?

The point is, after I filter, and pass this filtered record to some other method, and not only (QueueUserForWorkItem) how do I get its same recordset or say same filtered list back, also if the method expects an argument of type object, as you cannot convert it into Order type because that var was just local variable to the caller function.

I can still create a new Order, but that would destroy the whole purpose of filtering some records and then passing those records to another method to be processed? If we create a new Order in other method say, (ProcessOrder) in this case, it will be a new Order type and not the one that was filtered on some condition and passed as parameter to this method. I hope I made my point clear.

Any help would be really appreciated.

Matthew Hunter said...

I'm not sure if understand you, but I'll try and answer some of your questions and hopefully you'll get an idea.

1) The Where statement returns a IQuerable (thing of it like a query type) which must be iterated first to get actual values (infact the query is not executed until you iterate). So Add a .ToList() or .First() or .FirstOrDefault() or something similar to force the iteration and assign a list or object of type order instead of IQuerable.

2) You can Extend an IQuerable by adding query methods to it. You can pass IQuerable to another method an have the other method extend/execute the query. So for example:

// Initially create the IQuerable.
var myQuery = orders.Where(x=>x.Id = 2);

// Extend the query (it's still IQuerable!)
myQuery.Where(x=>x.Name = "Smith");

// Actually execute the query and assign the results to a new list
var myList = myQuery.ToList();

3) "var" variables takes on the type assigned to it so they are not a type themselves. You can't have a method parameter of type var (as var is not a type). So for parameters, either use a specific type (IQueryable or Order) or use the object type and cast it to the type you want within the method/delgate.

Hope that gives assistance.

I think the main problem is that you are expecting the where class to give you back the order type, whereas it gives you back an IQuerable.

Cheers

Matt