(Yes, I know I’m behind the times – “LINQ to SQL? Who needs it when there’s the newest preview/alpha/beta of the Entity Framework?" Well, I did start this application in EF v1 and ran away when I saw “unsupported” stickers plastered all over it. So, no thanks, I’m waiting for the proverbial “Microsoft v3.11” (or 3.51, whatever they call it)).

Looking superficially, one would say that all ORMs are alike. Moreover, as one of the newest to come into the world, LINQ to SQL would be expected to have it’s philosophy and design done according to previously accumulated knowledge. Erm, yes, it’s a polite way of saying that I expected it to be a rip-off of NHibernate…

This similarity may exist in general, but there are some areas in which the two are completely separate worlds. The example that I encountered is performance optimization. Coming from the NHibernate background I was surprised to discover that there are not much optimization topics in common with the two. In some aspects, NHibernate has already solved (at least for me) issues that LINQ to SQL has not yet stumbled upon, but in others, LINQ to SQL focuses on performance issues that don’t even exist as topics in NHibernate.

The search started with the (for an NHibernate guy like me common) “N+1” record problem. Well, either I don’t know how to search for “N+1” in Google or this subject is non-existent in the LINQ world. The “N+1” problem happens if you need to read additional data (say, related records) for each record fetched and do this by executing one SQL command for each record. You fetch 100 records in one query and then execute additional 100 queries for 100 records. So, in total you have 101 SQL commands executed for 100 records read.

Of course, this happens only if you want to load joined data – that is, read multiple joined records and the ORM doesn’t know how to do this in one go. NHibernate sometimes join-loads data by itself but LINQ to SQL doesn’t, so you have to explicitly instruct it to do this. For this, it you can use Eager loading – you instruct the ORM (using DataLoadOptions.LoadWith() method) to fetch the related record along with the original one in a single query. The trouble with this is that if you specify too complicated a structure for eager fetching (for ex. multiple eager-fetch collections on a single record), the ORM won’t be able to read everything in a single query and will execute N+1 SQL commands by itself. Both NHibernate and LINQ to SQL actually do this in a similar fashion. But in NHibernate, you can solve this by pre-fetching additional records in a separate query: you separate the data structure in portions and join-fetch each portion in a single query (this way, each of the multiple mentioned collections gets a separate query and a separate join-fetch). All records get cached in the NHibernate session and each query adds a portion of the missing structure. At the end, you have, say, 3 queries executed to get the full structure.

In LINQ to SQL this cannot be done because you must use DataLoadingOptions to say what gets loaded with what, and once you set the DataLoadingOptions, they cannot be changed. Because of this, you cannot load one set of records at a time: either set DataLoadingOptions to load all or you’re left with partial data. Also, you cannot explicitly load joined objects like in NHibernate (e.g. the statement “load Client along with its associated Orders” would in NHibernate sound something like “from Client c join fetch c.Orders”), here joins are used solely for filtering data.

Obviously, if we want to load two different child collections on a single object we cannot do this in one go. But we also cannot do two consecutive queries with different fetching strategies because we are obliged to use DataLoadingOptions which we cannot change. There may be some secret handshake that can allow me to pre-fetch data separately, but I don’t know about it. Probably the EF will come up with a wider and more mature feature set. As I mentioned, last time I checked EF it was pretty much infantile. But testing it paid off because I learned one important thing: using LINQ for all queries allowed me to rapidly move to LINQ to SQL, and this is where I am now with my N+1 problem. How do you think I’m going to solve it? Yes! LINQ to NHibernate, here we come.

To conclude on the note with which we started, the difference between the two engines… To me it seems that nobody even thought of the N+1 problem in LINQ to SQL: here the biggest performance optimization seems to be on the client side as everyone mentioning query optimization talks about compiling queries… I find it strange that optimizing parsing of a LINQ query could give a performance gain bigger than reducing the number of SQL commands executed, but I never checked so I’ll leave it at that. In NHibernate, on the other hand, there are such things as cached and named queries, but if their impact on performance is proportional to the amount of space given to them on NHibernate blogs and in docs, they are not on top of the list. Could it be that the two technologies are so much different that they have to be optimized in different ways? I don’t think so. It is evident that NHibernate has significantly more mileage than LINQ, but NHibernate tech guys are notorious for not documenting basic stuff, so if something is not mentioned in the docs it doesn’t mean it’s unimportant (ah, the irony of it…) On the other hand, Microsoft technologies receive the most attention in the pre-release stages, when most of the information available is extremely superficial (and I must add that the bloggers behave like a flock of parrots, mindlessly replicating the content from Microsoft CTP announcements ad infinitum and flooding search engines). Because of this, most of the high-tech LINQ information has yet to come. Of course, after LINQ to SQL was discontinued and they started to build yet another ORM – the Entity Framework – from scratch, the wait is naturally going to get even longer. I don’t expect that EF will have a radically different philosophy in the end, but it must at one point implement a way to solve the N+1 problem. Maybe it will do this by itself? It could, and so could NHibernate…

Or: how to solve the setup project message “ERROR: An error occurred while validating.  HRESULT = ‘80004005′”

It seems that for a large part of features in Visual Studio .Net, the development stops at the point where they are mostly usable and effectively demo-able. The Microsoft people are very eager to show you how easy it is to solve a trivial problem with a couple of clicks, but they are very reserved once something really serious has to be done.

A case in point: the Setup/Deployment projects in Visual Studio. (Yeah, the ones, Zero Click Deployment – they make it sound like it reads your thoughts – and the like). If you have a missing file in your solution, and if the file is of a non-critical type (e.g. a resource) so that the solution compiles, you have a big problem because the setup won’t. It will fail with a moronic message “ERROR: An error occurred while validating.  HRESULT = ‘80004005′”. Which file is missing? Well, if you really really care – go through all files in your projects and check (don’t forget to expand the controls, some RESX child file may be the culprit!)

Another thing that can happen is that a reference in one of the projects is bad. For example, you had the project reference another project and you moved the other project out of the solution… The first project doesn’t use anything from it so that the build passes but the setup is not as forgiving. In this case, you need to check all references.

If, like me, you have a solution that consists of thirty project and thousands of files, this would mean a major headache. One way to solve it would be to create a new, temporary setup project and add to it one by one all outputs from the original setup. Build after each step and when the error appears you’ll know which project is at fault.

Now, if the first is the case (a missing source file), one possible solution is to automate the manual search by using a macro. Below you’ll find one, modified from the example on the excellent MZ Tools site. As for the missing reference, it is somewhat easier to find since there are considerably less references in a solution than project files (you can detect these using the one-by-one method described above). I’m leaving it to you as a TODO: improve this script to detect missing references, post it on your blog and let me know so I can add a link to it.

Option Strict Off
Option Explicit Off
Imports System
Imports EnvDTE
Imports EnvDTE80
Imports EnvDTE90
Imports System.Diagnostics
Imports System.Windows.Forms

Public Module Module2

    Sub FindMissingFiles()

        Dim objProject As EnvDTE.Project

        Try
            If Not DTE.Solution.IsOpen Then
                MessageBox.Show("Please load or create a solution")
            Else
                For Each objProject In DTE.Solution.Projects
                    NavigateProject(objProject)
                Next
            End If
        Catch objException As System.Exception
            MessageBox.Show(objException.ToString)
        End Try

    End Sub

    Private Sub NavigateProject(ByVal objProject As Project)

        Dim objParentProjectItem As ProjectItem

        Try
            objParentProjectItem = objProject.ParentProjectItem
        Catch
        End Try

        NavigateProjectItems(objProject.ProjectItems)

    End Sub

    Private Sub NavigateProjectItems(ByVal colProjectItems As ProjectItems)

        Dim objProjectItem As EnvDTE.ProjectItem

        If Not (colProjectItems Is Nothing) Then
            For Each objProjectItem In colProjectItems
                For i As Integer = 1 To objProjectItem.FileCount
                    Dim fileName As String = objProjectItem.FileNames(i)

                    If fileName <> "" And Not System.IO.File.Exists(fileName) _
                            And Not System.IO.Directory.Exists(fileName) Then
                        MessageBox.Show("File missing: " + fileName)
                    End If
                Next

                If Not (objProjectItem.SubProject Is Nothing) Then
                    ' We navigate recursively because it can be:
                    ' - An Enterprise project in Visual Studio .NET 2002/2003
                    ' - A solution folder in VS 2005
                    NavigateProject(objProjectItem.SubProject)
                Else
                    ' We navigate recursively because it can be:
                    ' - An folder inside a project
                    ' - A project item with nested project items (code-behind files, etc.)
                    NavigateProjectItems(objProjectItem.ProjectItems)
                End If
            Next
        End If

    End Sub

End Module

I remember that when the first version of Java was released one of the frequent questions was: why doesn’t it have multiple inheritance? This also happened later with C#. The answer in both cases was: you can use interfaces instead. It is not the same, you don’t inherit the logic, but you can at least simulate it by implementing multiple interfaces.

Well, this was obviously forgotten by the people designing C# classes: even worse, they seem not to have been aware of it because most of the classes in v1.0 onward don’t have equivalent interfaces that would allow this.

There’s a real-life example (or should I say everyday example? Because I’ve encountered it a million times, you probably did too) that perfectly illustrates the point: why isn’t there an IControl interface that represents a (windows, web, whichever) control type? The logical answer (and it was probably the real reason the framework designers came up with) would be: well, it would be too difficult to implement. Anything that is supposed to work as a control should really have to be derived from Control. True: but the purpose of interfaces is not only to implement them.

Let’s say I want to create my control type that implements some interface, IMyInterface. How do I reference this component, using a variable of Control type or IMyInterface? Whichever I use it isn’t complete, I have to cast it into the other one and I lose compile-time type safety. I could create a ControlWithMyInterface base class that implements the interface and use this instead of both, but then I’d have to inherit everything from it, which in most cases would not be possible. No, the only solution would be to support the “interfaces instead of multiple inheritance” principle and derive IMyInterface from IControl, but IControl doesn’t exist. Although it wouldn’t be too difficult to implement, one could just put all of Control’s public members into the interface.

There could really be a programming rule enforcing the existence of an equivalent interface for every class, even something that generates them on the fly from its public members. I wonder if there is a pre-processor that can help with this?