I don't have time for this

Monads As I Have Known Them

2010-07-02T18:46:00.001-04:00

Are you really writing yet another Monad tutorial!?

No. This is not a Monad tutorial. If this were a Monad tutorial, I would need to be accurate and complete in my description of Monads. I, a repentant, often back-slidden, imperative programmer, have a tenuous, ephemeral grasp on what Monads are and what they’re good for. If you started from where I came from, just consider this blog post a breadcrumb dropped on the path to Monadic enlightenment…or the path to insanity. I’ll let you know when I get there.

What is a Monad, anyway?

For a complete, correct and nearly incomprehensible (to me, anyway) definition of a Monad, please consult Wikipedia. As far as I’m concerned, a Monad is principally a way to combine ‘monadic values’ with other ‘monadic values’, resulting in…other ‘monadic values’.

Ok, what’s a “monadic value”?

It’s the sort of thing that can be used with Monads. Ha ha, just playing! Well, I mean, that’s not wrong, but I can tell you some other things I’ve noticed about ‘monadic values’. Every monadic value I’ve come across has the following characteristics:

You can get some number of other values from it. Note that the number of values could well be zero.
You almost certainly have more than one monadic value with the same structure (i.e. the same way of getting values out of the monadic value).
There is a sensible way to combine or sequence monadic values together in a single expression such that result of that expression will be another monadic value (of the same type or structure).
There is a way to take an ordinary, non-monadic value and promote it or contain it as a monadic value.

What the heck are you talking about?

Good question. Let’s try to find an example of a ‘monadic value’. Consider point #1: I don’t know about you, but when I think of the sort of thing (value, object, whatever…) that I can get other things from, the first thing I think of is: Lists! And considerning point #2, I don’t know about you, but my life as a programmer is filled with lots and lots of lists! And I get values out of lists the same way you do, one leg…I mean one element at a time. And point #3? Well, you’ve seen LINQ, right? And you’ve seen how you can combine two lists in a single LINQ query (with either a ‘join’ or multiple ‘from’ clauses)? Oh, you haven’t? Well, I know you’ve coded (or at least seen) nested ‘foreach’ loops and that, my friend, is combining two lists. Sure, it’s combining them in a dirty, shameful, imperative kind of way, but I’d expect no less of you. And finally, what about point #4? Well, if you can’t figure out how to take a value into a single-element list, I think you should reconsider your chosen profession.

So Monads are Lists?

Not so good with the logic, I see. Let us review: If all slicks are snicks and all snicks are pricks then…No! Monads are not (necessarily) lists! On the other hand there is a List Monad. But it’s not the only one! For example, in Haskell there’s also the Maybe Monad for combining values of the Maybe data type. If you’re a .NET programmer like me, you should know that the Maybe data type is quite similar to .NET “reference” types or Nullable<T>. In other words, it’s a container that can either contain null/nothing or a single value. The Maybe Monad combines Maybe values in such a way that if any of the combined values are null/nothing, then the overall expression is also null/nothing. What’s the value of the overall expression when all the values are not null/nothing? Well, that’s up to you, but whatever you come up with has to be a Maybe value.

There is also the State Monad for sequencing computations that have access to and can possibly mutate some shared state. In effect, this allows you to return to your sinful imperative ways when the need arises…without break the rules of functional purity! Miraculous! There is the Reader Monad, sequencing computations that have access to a (non-mutable) shared environment. And perhaps most famously, there is the IO Monad, which solves the ‘problem’ of representing IO in Haskell in a rather clever and elegant fashion.

This is hardly an exhaustive catalog of all the Monads there are. In fact, I’d guess there are many more that have yet to be invented!

Well, that’s nice, but I’ve gotten this far without Monads, so…

I know, and it’s very sad. Think of all the key-strokes wasted! Think of all the unnecessary heart-ache caused by errors in trivial “glue code”! You see, you can get along without Monads, but no one should have to live their life like that, writing the same code pattern over and over and over and…Monads are about writing that code once! And reusing it over and over and…Yes, I know, code re-use is what inheritance is for. But let me tell you something, in a programming language without functional-programming features, this level of code re-use is somewhere between very difficult and impossible.

Kind of makes you wonder if ‘objects’ are the right thing to build programs from, doesn’t it?

Is Microsoft Taking the Right Approach with Internet Explorer 9?

2010-04-14T11:59:00.001-04:00

Microsoft has been making some noise lately about the upcoming version of Internet Explorer, IE 9. While it’s still way to early to pass final judgment, some of the things that folks at Microsoft have been saying about version 9 have me wondering if Microsoft is missing an opportunity.

If you’ve heard anything Microsoft have said about IE 9, then you’ve probably also heard them say that the various web standards and performance benchmarks that everyone seems to care about do not necessarily reflect how browsers are used in the real world. Based on hearing this mantra stated repeatedly by various people and in various ways, I’ll be very surprised if IE 9 turns out to be very competitive in the aforementioned benchmarks.

I can see two alternate conclusions that can be drawn from this:

Microsoft is focused on delivering a browser that works well in the real world, and artificial benchmarks are at most a minor concern.
Microsoft realizes that Internet Explorer will not be competitive with the likes of Chrome anytime soon and all this “real world usage” talk is just spin.

Suppose that we give them the benefit of the doubt and assume that the first scenario is close enough to the truth. Can one really argue against being pragmatic? Would anyone be so foolish as to argue that Microsoft should focus on artificial benchmarks, possibly at the expense of “real world” performance? Hi, my name is Daniel…

The thing is, I think Microsoft has a bit of an image problem. I think many folks think “Microsoft” is synonymous with “moribund”. To be sure there is plenty of evidence that Microsoft is still trying hard and is still willing to take risks: Windows 7 is an unqualified success. The Microsoft Office UI reboot was a risk that seems to have paid off. Windows Phone 7 looks fantastic (but we have yet to see if fantastic is good enough -- it wasn’t for Zune). Then there is Natal, which will either be awesome or become the next Segway (a product practically killed by its own hype-machine).

On the other hand, there are also plenty of reminders of that “other” Microsoft, the one that seems lazy and out-of-touch. And currently Internet Explorer is definitely one of those reminders. Considering that Microsoft is the largest software company in the world, and makes the very platform on which Internet Explorer runs and considering that Internet Explorer is a flagship product, the situation is a bit embarrassing, I think.

The next version of IE is therefore an opportunity to surprise and impress, to convince the people still sitting on the fence that Microsoft, despite its occasional gaffes, is still a relevant, vibrant company…and sitting on the fence is darned uncomfortable anyway. One way Microsoft can do that is by making a browser that doesn’t have to apologize or make excuses for its performance or support for web standards.

What do you think?

The Value of Commenting Code

2009-02-15T21:52:00.001-05:00

Never Changing Constants

Our legacy product started life about 10 years ago, about a year ago before I started here. Not including “designer” generated stuff, the entire code base is about 500,000 lines of VB6 code, spread across 35 projects. Considering that those of us who worked on the project from the beginning did not have much combined experience making a Windows application of this size and scope, you can imagine that a lot of mistakes were made. I don’t think there is anything in the code base today that is worthy of The Daily WTF, but that wasn’t always the case.

I’ve seen many things in the code that made me laugh, cringe, or cry, but my favorite is probably this comment:

	'Never changing constants.
	Public Const AUTO_ID_NAME As String = "RecordID"

I guess the commenter wanted to distinguish the above from those constants that do change...

What Is Your Opinion about Code Comments?

I remember being asked about code comments in my interview to work here.

Interviewer: What is your opinion about code comments?

Me: Uh…they’re good?

I guess that’s what they wanted to hear. Since then my thinking around code commenting has become a lot more nuanced. I’ll summarize it this way:

Good comments are good.
Pointless comments are bad.
Bad comments are worse.
Most comments (in my experience) are not “good”.

It makes me wonder: What would have been the interviewer’s opinion of my nuanced answer?

At Least It Has Comments

I recently came across this post on The Daily WTF. Here’s the reader’s digest version: The code has been thoroughly commented, but the comments are pretty much the code translated to English. Helpful, I suppose, if you’re not familiar with the programming language being used. Oh well, we’ve all seen worse. Then I starting reading the comments (the comments responding to the post, not the code comments). There was this one:

“H*ll, at least it has comments...”

And this one:

“More comments the better, even if they are "obvious" ones.”

And a bunch more along the same lines. This was very enlightening for me. Apparently trying to write good comments about the intent of the code has been a waste of my time. I’m going to start littering my code with personal anecdotes in the comments. No…even better, I’m going to create a “Comment-ifier” that will automatically improve code quality through randomly generated comments. Patent pending, so don’t even think about it.

The Problem with the Software Development Industry…

I see a link between this attitude about code comments and the recent kerfuffle about some heretical statements uttered by a one Joel Spolsky on Hanselminutes. From the sounds of things, Joel doesn’t think much of the SOLID principles. Gallingly, Joel’s cohort, Jeff Atwood was right there to lend Joel some moral support.

I’m not ready to get on Jeff and Joel’s bandwagon (I do not wish to be excommunicated), but I think they have a point. ~~Programmers~~ Human beings have a strong tendency to latch on to Ideas to make themselves a better ~~programmer~~ human being. I put this down to intellectual laziness. Being better at anything requires continuous effort. Ideas are much easier to implement.

Learning coding “best practices” will (hopefully) make you a better coder, but that is not the key to being a good coder. For one thing, I think it is vital to understand the “why” of those “best practices”. Thinking back to that WTF post, maybe if the person who wrote that code had ever thought about why commented code is important, he or she would’ve come up with some better comments.

The Secret to Good Code

So there are probably at least as many coding “best practices” or methodologies out there as there are fad diets. Do any of them work? All the time? Everywhere? Obviously not, otherwise why are we still talking about it. For what it’s worth, here’s what’s been working for me so far:

Care
Cheat

Care means care about the quality of your code. Care about the next guy who has to use or maintain what you did. Care about whether your code still works after you’ve retired. Care about your future self, who will probably have to do the same thing again in a different project.

Cheat means copy from the smart kids. Someone somewhere has succeeded at doing something very similar to what you’re working on right now. You just need to find it and copy it as best you can. Of course, to be a good cheat, you’ll need to get your eyes on as much good code or concepts as you can. Even if you don’t need it today, you may at some point in the future. A case in point: One of my all-time favorite books in my library is Writing Compilers and Interpreters by Ronald Mak. I read* it so much, the book is now literally in pieces. Have I ever written a compiler or interpreter? No. Have the techniques outlined in the book helped me with other tasks? Absolutely yes, from simple things like how to properly handle CSV formats with embedded quotes to bigger ideas like how to take a relatively massive problem and break it down into manageable pieces.

Honestly, I don’t think I can say enough about the value of being exposed to as many programming languages, frameworks, architectures, or projects as possible. This is good because:

You have a much better chance of knowing the right tool for the job.
Seeing the same problem solved in multiple different ways will expand your mind.
All those ideas bouncing around in your brain will eventually collide and stick together to make new ideas (not all of them will be good, of course).

* “read” here is intended as past tense. Don’t you just love the English language?

Comparing Functional and Imperative, Part 1: Context determines meaning

2009-01-18T14:25:00.001-05:00

Over the past year, I’ve taken a strong interest in functional programming. I’m not sure where it started, but it may have been Brian Beckman’s Don’t fear the Monads on Channel 9. It went pretty much completely over my head, but my interest was definitely piqued. It has been in my mind to do a series of posts on functional programming once I became an expert on the subject. Well, I am so not an expert at this point, but I’ve decided to consider that an advantage. Once something makes perfect sense to you, it’s often hard to remember why it didn’t make perfect sense before.

Functional programming is an alternate universe of programming where hard things (e.g. concurrency) become easier and easy things (e.g. writing to a file) become at least a bit harder. Besides the fact that functional programming is just plain different from imperative programming, another challenge for an imperative programmer learning functional programming is the fact that the same or similar syntax is used to represent quite different things. Consider:

	a = 3
	b = a + 2

In an imperative language context, this would probably mean “assign the value 3 to variable ‘a’, then evaluate a + 2 and assign the resulting value to variable ‘b’”. In a functional programming language such as Haskell, ‘a’ and ‘b’ can be thought of as being functions. This may not technically be true, particularly in the case of ‘a’, but I think it works to think of them that way. Just to drive the point home, here is an equivalent definition in C#:

	int a() { return 3; }
	int b() { return a() + 2; }

One concept of functional programming that is quite different from imperative programming is the idea that once you define ‘a’, you can’t change it. Of course, if ‘a’ is actually a function, then this is to be expected. Even in imperative programming languages, the definitions of functions generally don’t change in a compiled program. The fact that ‘a’ is a function also explains something else that is at first peculiar about functional programming, which is the order in which statements get evaluated. Consider:

	a = 3
	b = a + 2
	c = b + 1

In an imperative program, the first line would be evaluated first, followed by the second line and so on. But what if ‘a’, ‘b’ and ‘c’ are actually functions? The equivalent C# code might look something like this:

	int a() { return 3; }
	int b() { return a() + 2; }
	int c() { return b() + 1; }

If there were some code that referenced the ‘c’ function, program execution would jump to function ‘c’, then to function ‘c’ and so on. Furthermore, if no other code in the program referenced functions ‘b’ or ‘c’, then neither of those functions would ever execute. So it is in functional programming. The point is that if we think of syntax like “foo = ...” as defining a function, it’s easier to grasp as opposed to thinking of it as a statement with special rules.

When I learned that pure functional programming has no mutable variables and no “side-effects”, my first thought was “How can I actually do anything?”. Well, Haskell (a pure functional language if there ever was one) does have side-effects, depending on how you choose to define them. For example, there are functions that perform IO (reading and writing to file streams). On the other hand, these side-effects are definitely not incidental, as they are in imperative languages. In Haskell, you know when a function may have side-effects. Generally, Haskell treats side-effecting code rather like nuclear waste, encapsulating it in a special container called a “Monad” and posting lots of scary warning signs around it.

Can a pure functional programming language be useful without side-effects? I think the answer is yes. I work for a company that creates LOBs (Line of Business apps) first and foremost. When you take all the layered architecture and technology concepts out of the picture, an LOB looks something like this:

That yellow arrow in the middle of the diagram is a transformation. It transforms the shape of the data as it comes from the user interface into the shape it has in the database. You could imagine that it’s a pipeline, of sorts, constructed of all manner of logical loops and branches. There is no reason that yellow arrow needs to have any mutable state within itself. In fact, for reasons of scalability and consistency, it would be better if that yellow arrow didn’t have any mutable state inside it. As such functional programs are uniquely suited to constructing this sort of pipeline.

Continue to Part 2: Economies of Syntax.

Comparing Functional to Imperative, Part 2: Economies of Syntax

2009-01-18T14:30:00.001-05:00

This is part 2 of a series (consisting of at least two posts) comparing imperative programming languages with functional programming languages. Here is Part 1: Context determines meaning. By the way, Haskell is my prototypical functional language because it’s the purest functional language about which I know anything. C# is my prototypical imperative language because I know it pretty well.

It seems that functional algorithms and programs are quite a bit shorter than their imperative counterparts. For example, here is a possible definition of a function to calculate square values in Haskell:

	square x = x * x

For sake of comparison, here an equivalent function defined in C#:

	static double Square (double x) { return (x * x); }

The Haskell version is less than a third as long as the C# version. As far as I’m concerned, this is a win for Haskell. I much prefer a syntax that does not get in the way of what a program actually does. This is mostly why I gravitated to C# even after several years of nothing but VB. VB syntax may be easier to learn for the beginner, but I find it slows me down because I have to read (or worse write) past a bunch of words before I get to the meaningful part. Of course, C# may be concise compared to VB, but it’s hard to imagine a more concise syntax than is afforded by Haskell.

How can a functional language like Haskell get by with such an economy of syntax? One reason is that Haskell was designed to create functions, first and foremost. Do one thing and do it well, as the saying goes. Another reason is that Haskell is very good about inferring the types of things from other things. You can give things an explicit type in Haskell, but often the only reason to do so is to help the compiler help you when you screw up. For example, Haskell knows that ‘square’ is a function that returns a numeric value. It knows this because it knows that the ‘*’ operator returns a numeric value. It also knows that parameter ‘x’ must be a numeric value, likewise from the definition of ‘*’. In other words, Haskell will not let you do something like this:

	y = square "foo"

Of course, it is possible to define a function in Haskell that does not, in its definition, imply a particular return type at all. In such a case, Haskell will define the function in a generic fashion. Think C# generics without all the syntactic fluff. Consider Haskell’s ‘foldl’ (fold left) function:

	foldl f z [] = z
	foldl f z (x:xs) = foldl f (f z x) xs

Basically, ‘foldl’ applies an operation to each item in a list, accumulating the result of the operation along the way. The accumulated value is the return value of ‘foldl’. As of version 3.5, the .NET Frameworks has a function that serves the same purpose: Enumerable.Aggregate. What does it look like? Hang on to your hat…

	public static TAccumulate Aggregate(
		this IEnumerable source,
		TAccumulate seed,
		Func func)
	{
		if (source == null)
		{
			throw Error.ArgumentNull("source");
		}
		if (func == null)
		{
			throw Error.ArgumentNull("func");
		}
		TAccumulate local = seed;
		foreach (TSource local2 in source)
		{
			local = func(local, local2);
		}
		return local;
	}

Why is Haskell’s version so much shorter? One obvious answer is Haskell’s penchant for exceedingly short parameter names (something I don’t care for, actually). It’s much more than that, though. In Haskell’s version, there are no explicit types, no explicit generic parameters, and much less ‘punctuation’. Does this mean that Haskell is weakly-typed? Not at all. Haskell’s version is at least as strongly-typed as the .NET version. In fact, considering that Haskell will not let you pass invalid parameters to ‘foldl’, it can be considered to be more strongly typed than the .NET function, and also eliminates the need for parameter validation in the body of the function.

I’m not sure what part 3 of the series will be (or even if there will be a part 3), but I’m considering a post about why IO is a challenge for pure functional languages and how Monads make it much better.

A Confession: I don’t have a degree

2009-01-01T08:57:00.001-05:00

I ran into a blogo-sphere discussion recently about the merits of having a CS degree. I found this particularly interesting because, well, I don’t have a degree. Shocking, I know, coming from someone who claims to be a “Software Architect” (well, that is my job title). I’ll give you a minute to recover.

To be very cathartically honest, I wish it were not true. I have a full-time+ job and a full-time+ family, but if I had an opportunity to pursue a degree, I’d seriously consider it. So there you have it. There is at least one non-degree-possessing person in this world who thinks having a degree is a good thing.

On the other hand, a lot of what I read about the purported merits of having a degree is…just crap, in my opinion. I believe and accept the fact that a lot of people have learned a lot of things in a college or university setting, things that they would never learn in “real” life. I believe and accept the fact that for most people, there isn’t a better way to get a well-rounded education. I do not believe those things are true for me.

It has been well established that there are at least a few different ways that people learn best. I think it is also reasonable to conclude that what any person has learned or been exposed to in his or her life is going to be unique to that person. For these and other reasons, concept “A” might really “click” with Alice, while Bob just doesn’t get it. For concept “B”, it may be just the opposite. In a classroom setting, the extra time it takes to teach concept “A” to Bob is a waste of time for Alice. The extra time it takes to teach concept “B” to Alice is a waste of time for Bob. This leads me to conclude that classroom learning is not particularly time-efficient.

I think the primary benefit of learning in a college/university classroom setting is the fact that most people do not have the motivation or, in some cases, the ability to educate themselves.

I love to learn. It might be more correct to say that I’m addicted to information…of any sort, really. Truly, I’m not sure I’ve ever encountered any subject matter or concept that didn’t interest me, at least a little bit. People have told me that they are intimidated to have a conversation with a subject matter expert concerning said subject matter. They’re afraid of asking an ignorant question. I can’t relate to that. I’ll happily make a fool of myself, asking that person a thousand stupid questions, just for the chance to learn something.

One idea I often encounter is the idea that a self-educated person is going to have a “narrow” education. Maybe they can become competent in their field, but they’ll be weak in other areas, which might hamper their overall performance. I think this idea “rings true” with a lot of folks. It seems logical and fair-minded. On the other hand, I’ve never encountered “hard numbers” to support this idea.

Of course, I can’t give you any hard numbers, either. In fact I can only offer one data-point: myself. If I were to be given a general knowledge test covering various aspects of Math, Science, History, English, etc., I feel confident that my score would compare very favorably with scores obtained by administering the same test to a random selection of college educated people.

Having said all that, why do I wish I had gotten a degree when I had the chance? Two reasons: First, I did attend college for three semesters. It did not go well. I spent too much time in the computer lab when I should have been in the classroom or completing assignments. Basically, I feel that I failed at college and that bothers me. Second, “Perception Is Reality”. The vast majority of people I work with and, I would guess, the vast majority of people in my field have a degree. I would venture that the majority of these people “perceive” that having a degree is better than not having one (the example set by Bill Gates notwithstanding).

I know of at least one case where I was not even considered for a position on the basis of not having a degree. I doubt that was an isolated case. If you’re dealing with an overwhelming glut of resumes, you may very well decide to trash all the resumes that don’t appear to “measure up” in all respects.

This reminds me of the recent experience I had doing the initial interview of several candidates for a Software Developer position in my division. This was a first for me, and the experience was…fascinating. I am thinking of one candidate in particular. This person came with all the right credentials. Not only did this person know everything, but this person had been working for several years at a “big name” company, and before that another “big name” company (we verified this person’s employment history). This person had an MBA.

On that basis, I was shocked, truly, at what this person did not know about Software Development. The point in the interview that I began to understand the situation was when I asked “It says on your resume that you have experience with technology ‘X’. Can you tell me more about how you used that technology?” This person’s answer was something along the lines of “Well, I didn’t use technology ‘X’ directly. My job was to review and approve other people’s work and send it up the chain.” Imagine several more responses along the same lines. Finally I asked this person to demonstrate how to solve a particular (fairly simple) problem using the language of their choice. Imagine a blank stare. I found myself describing the problem in simpler and simpler terms. The blank stare remained. I eventually realized that this person could not tell me what a “string” was. Yikes!

Oxite: A good example after all?

2008-12-19T13:40:00.001-05:00

I haven’t been doing much web development these days, but I’m anticipating that will change in the near future. Consequently, I’ve been keeping an eye on the progress of the various web development frameworks, including ASP.NET MVC. When I heard about the Oxite project, I was very interested. Most example projects I find are just that: examples. Oxite is running a production website. I immediately downloaded the project sources and…still haven’t even looked at them.

In the meantime, the community response to Oxite has been…interesting. The general consensus seems to be that they got it all wrong. The last thing I’ve read on the topic (so far) is Glenn Block’s On Oxite post. Ouch! Right now I’m just picturing him running into the Oxite guys at the Microsoft Christmas party…

Once upon a time, I was a minor presence in the MS NNTP forums. This situation reminds me of something I noticed back then: The surest way to get a relevant and timely answer to your question would be to post a completely ridiculous solution, don your asbestos underwear, and wait for the responses to roll in.

Which gets me back to the matter at hand. These days, you can expect that any framework you’d consider using is going to come with with plenty of samples, examples, and documentation. General guidance, especially in the area of “what not to do”, is harder to come by. This is understandable. I’m sure most of us have had the experience of a customer (or co-worker) taking something we’ve done and use it in a manner that makes us cringe. It’s just very difficult to anticipate all the wrong directions that someone might go.

I think this makes the Oxite project almost invaluable as an example. I’ve bookmarked this post and you can be sure that if and when I get around to actually using ASP.NET MVC, I’ll read it again. Of course, this is no consolation for the Oxite folks, who probably feel a bit chastised at this point. No one wants to be thought of as the Gigli of…anything. Well guys, all I can say is thanks for sticking your neck out there on behalf of the rest of us.

Why LINQ is better than SQL, Part 3: Query Composition

2008-12-18T12:01:00.005-05:00

A LINQ query is like a SQL view in at least two ways:

No data is retrieved until you really need it.
LINQ queries can be composed with other LINQ queries.

Now I will show you that LINQ queries are better than SQL views because LINQ queries are more modular than views.

The Problem

Suppose that you are creating a dashboard for your business app (which happens to use the same database schema as Microsoft’s AdventureWorks sample database) and one of the data points you need is the sum of total sales by year and month.

A Solution Using SQL Views

If you implemented a SQL view for this purpose, it might look like this:

create view vSalesByYearAndMonth
as
    select
        SUM(so.SubTotal) SubTotal
        , DATEPART(yyyy, so.OrderDate) [Year]
        , DATEPART(m, so.OrderDate) [Month]
    from
        [Sales].[SalesOrderHeader] so
    group by
        DATEPART(yyyy, so.OrderDate), DATEPART(m, so.OrderDate)

Now we can take our view and compose it with other views or queries like this:

create view vSalesByMonth2004
as
    select
        SubTotal
        , [Month]
    from
        vSalesByYearAndMonth
    where
        [Year] = 2004

This is all very good, but now imagine that you’d really like to filter the results by some other aspect of a sales order: the region, the sales person, a product category, etc…You’d need a view for each scenario! Your best bet (assuming you’re using SQL Server) is probably to make a function. That way you can create a bunch of parameters for everything you might conceivably want to filter by and then make the query itself a lot more complicated with a whack of “…where (isnull(@x, x) = x) and (isnull(@y, y) = y) and…”. Whee!

A Solution Using LINQ

Now I’m going to do the same thing using LINQ and Entity Framework (EF). The source of my queries is an ADO.NET Entity Data Model that I generated directly from the AdventureWorks database (if you’re not sure how to do this, here is an example). If you generate a model in this way and never change it, you’re probably missing out on some of the goodness EF has to offer, but for this example I’m sticking with what the wizard gives me.

The first thing I’m going to do is define a class to contain the result of the query (thanks to anonymous types, this step isn’t always necessary):

        class YearMonthTotal
        {
            public int Year { get; set; }
            public int Month { get; set; }
            public decimal Total { get; set; }
        }

Now I’m going to write the LINQ query and encapsulate it in a function (all C# functions must have an explicit return type, which is why the previous step is necessary):

        IEnumerable<YearMonthTotal> GetSalesByYearAndMonth()
        {
            return
            (
                from so in context.SalesOrderHeader
                group so by new { so.OrderDate.Year, so.OrderDate.Month } into sog
                select new YearMonthTotal { Year = sog.Key.Year, Month = sog.Key.Month, Total = sog.Sum(so => so.SubTotal) }
            );
        }

And for the sake of completeness, here’s what the second view looks like in LINQ:

        IEnumerable<YearMonthTotal> GetSalesByMonth2004()
        {
            return (from sbm in GetSalesByYearAndMonth() where sbm.Year == 2004 select sbm);
        }

Fantastic! But, uh, I haven’t done anything in LINQ that I couldn’t easily do with SQL…yet.

Making It Modular

Essentially, I want to be able to pass a “where” clause as a parameter of my function. The new parameter will have the same type as the predicate parameter from Queryable.Where, but I’ll specify SalesOrderHeader as the generic parameter because that is what I’m filtering. Here is the revised function:

        IEnumerable<YearMonthTotal> GetSalesByYearAndMonth(Expression<Func<SalesOrderHeader, bool>> salesOrderFilter)
        {
            var filteredSalesOrders = context.SalesOrderHeader.Where(salesOrderFilter);

            return
            (
                from so in filteredSalesOrders
                group so by new { so.OrderDate.Year, so.OrderDate.Month } into sog
                select new YearMonthTotal { Year = sog.Key.Year, Month = sog.Key.Month, Total = sog.Sum(so => so.SubTotal) }
            );
        }

Wonderfully strange, is it not? How does one use such a beast? Here’s one example (sales for the Northeast):

            var neSalesSummary =
                GetSalesByYearAndMonth(so => so.SalesTerritory.Name == "Northeast");

And something more convoluted (sales for the great state of Maine):

            var maineSalesSummary =
                GetSalesByYearAndMonth(so => so.SalesTerritory.StateProvince.Any(state => state.StateProvinceCode == "ME"));

The Result

Retrieving the results of that last query causes the entire composed LINQ query to be translated into a single SQL query that looks like this:

    SELECT 
    1 AS [C1], 
    [GroupBy1].[K1] AS [C2], 
    [GroupBy1].[K2] AS [C3], 
    [GroupBy1].[A1] AS [C4]
    FROM ( SELECT 
        [Filter2].[K1] AS [K1], 
        [Filter2].[K2] AS [K2], 
        SUM([Filter2].[A1]) AS [A1]
        FROM ( SELECT 
            DATEPART (year, [Extent1].[OrderDate]) AS [K1], 
            DATEPART (month, [Extent1].[OrderDate]) AS [K2], 
            [Extent1].[SubTotal] AS [A1]
            FROM [Sales].[SalesOrderHeader] AS [Extent1]
            WHERE  EXISTS (SELECT 
                cast(1 as bit) AS [C1]
                FROM [Person].[StateProvince] AS [Extent2]
                WHERE ([Extent1].[TerritoryID] = [Extent2].[TerritoryID]) AND (N'ME' = [Extent2].[StateProvinceCode])
            )
        )  AS [Filter2]
        GROUP BY [K1], [K2]
    )  AS [GroupBy1]

It may not be the most readable query, but I can't find any obvious flaws in its logic.

Why LINQ is better than SQL, Part 2: From comes first

2008-12-16T20:09:00.001-05:00

[Note: I have another of this “series” on stand-by, but it’s long and slightly complicated, so it made sense to me to post this one first.]

I’ve recently upgraded my developer workstation to SQL Server 2008 (from 2005). For me the killer feature, hands-down, is IntelliSense. Of course, there’s still room for improvement:

Great. How about telling me what I can never remember without firing up the documentation, namely which “expression” is the thing I’m searching for and which is what I’m searching in? Oh well, they’ll probably address this sooner or later. Then there is this: How can IntelliSense be provided for field names when you haven’t specified the source? It can’t, of course. If you want IntelliSense for field names you can type something like “select from tableA as a [join tableB as b]…” then go back and complete the “select” clause. I know I should be grateful, but I find this mildly annoying.

LINQ doesn’t have this problem because…“from” comes first.

IntelliSense is not the only reason I prefer that “from” come first. The biggest reason is the way that I think about queries. Basically, I think of them as a pipeline, with data flowing from one end to the other, getting transformed and filtered along the way:

Coincidentally, this is pretty much the same order of elements in a LINQ query.

Granted, putting the “select” clause first does put SQL slightly closer to natural English sentence construction. Does that make SQL easier to understand? I think not.

Overriding LINQ

2008-12-09T21:35:00.002-05:00

As I’ve said previously, I love LINQ. I love the fact that so much interesting and useful functionality has been built up around a very simple interface (that being IEnumerable, of course). But, as the saying goes, not everything that can be done should be done.

Take for example Enumerable.Count (in case you hadn’t noticed, the LINQ-to-objects extension methods are hiding in the “System.Linq.Enumerable” class). Given an IEnumerable<T>, it returns the number of items in the sequence. How does it do that when IEnumerator<T> does not have a Count property or method? One way would be to enumerate over the entire sequence, incrementing a counter along the way. Of course, that’s potentially very inefficient and fortunately Enumerable.Count will only take that approach as a last resort. If the source container implements ICollection<T>, then Enumerable.Count will simply return ICollection<T>.Count.

The strategy LINQ uses makes a lot of sense, I’m sure you’ll agree, but I think there is room for improvement. For one thing, the relationships between LINQ and interfaces like ICollection<T> are not documented (probably because they are tenuous). If you were doing something “weird”, LINQ operations might not behave as you expect. Furthermore, the primary reason we have so many data containers to choose from is that different data containers are optimized for different operations. As it is currently implemented, the onus is on LINQ itself to know and choose the best implementation for each LINQ operation. Of course, if the data container in question did not come “in the box”, this is not merely difficult, but impossible.

I think there should be some way to tell LINQ that your data container knows how to do operation X in the most efficient manner possible, and that LINQ should defer its implementation of X to the container. Perhaps the simplest way to accomplish that would be to organize LINQ operations into some number of logical groups and then define interfaces which, in-turn, define the operations in each group. If a container implements one or more of these interfaces, LINQ will defer its implementation of the corresponding operation(s) to the container. Such an interface might look like this:

namespace System.Linq
{
    interface ICountable<TSource> : IEnumerable<TSource>
    {
        int Count<TSource>(Func<TSource, bool> predicate);
        int Count<TSource>();
    }
}

The implementation for Enumerable.Count could then look like this:

        public static int Count<TSource>(this IEnumerable<TSource> source)
        {
            if (source == null) throw Error.ArgumentNull("source");

            var countable = source as ICountable<TSource>;
            if (countable != null) return countable.Count();

            // business as usual...
        }

A system like this would also allow a data container to “opt-out” of a particular operation by immediately throwing an exception. For example, suppose that you wrapped IEnumerable around some kind of stream. You wouldn’t want someone to try to “Count” that, would you?

Ideally, there would be a way to only partly implement an interface and otherwise fall back to the default LINQ implementation, but it would be hard to avoid a “stack overflow” scenario without making it much more complicated (or perhaps I’m missing something obvious).

How about some syntactic sugar for IEnumerable<>?

2008-12-06T18:48:00.001-05:00

Recently, my attitude has changed a bit toward IEnumerable<>. It has always served a very good purpose (foreach is so nice), but it has seemed to me to be an interesting, but mostly hidden implementation detail of the framework. And then there was LINQ.

LINQ is a ton of functionality built around that simple little interface. Which makes me think about using it, exposing it directly in my own API. Why would I choose to expose IEnumerable<>, instead of IList<> or something else? First of all, what other interface is there for exposing a read-only set of “things”? Second, I adhere to the philosophy APIs should expose only the necessary functionality and no more, because:

It makes it easier to understand how to use the API (and how not to use it).
It makes it easier to change or completely re-implement the original API.
The less you can do, the less there is to go wrong.

Of course, IEnumerable<> may be useful, but it certainly isn’t pretty. Instead of IEnumerable<Foo>, I would rather see Foo* or Foo+ or…well, you get the idea.

This is not a new idea (I’m sorry, but you probably won’t find those on this blog). There is, of course, Foo?, which means Nullable<Foo>. And then there is the experimental language Cω (thank you, cut-and-paste), which proves how unoriginal this idea is: In Cω, Foo* is a “stream” of Foo. According to the Cω Overview, “Streams in Cω are closely related to IEnumerable<>…”. Well, who’s going to argue with the brains inside MS Research? Not me!

Why LINQ is better than SQL (Part 1?)

2008-12-05T12:05:00.002-05:00

To be very honest, LINQ warms the cockles of my heart. The mere fact that I can write a compiler-checked query in C# that references my C# entity class and that query, through some magic incantations, gets executed as a SQL statement against my database...it makes me happy.

But maybe you're more pragmatic than I am. Maybe you think all this LINQ hoo-ha is pure sophistry. Take my word for it, I could go on and on about the wonders of LINQ, but then "I don't have time for this" do I?..Well, maybe one tiny example (since you asked so nicely):

    var r = (
        from e in context.Employee
        where e.Manager.Contact.LastName == "Word"
        select new { e.Contact.FirstName, e.Contact.LastName });

What's the big deal? This query traverses four logical tables. I didn't have to specify how those tables get joined together. Maybe this is kind of obvious, but it surprises me that standard SQL does not have a similar capability. Databases have these things called foreign keys that formalize the relationship between tables. That being the case, why do I have to spell out the relationship between every table in every query.

Don't get me wrong, I love SQL joins. Like the other day, when I needed to identify value gaps in a set of rows. Once upon a time, I probably would've resorted to some process involving cursors, but a simple self-join does the trick. The problem is that the chance to do something "clever" with a join doesn't come up very often. Most of the time (for me, at least), a join is identical to the relationship defined by a foreign key.

Obligatory Introduction Post

2008-12-04T09:11:00.001-05:00

Hello Blog World. My name is Daniel. I live in Eastern Maine with my beautiful wife and four affable children. My passion and profession is software development. Presently, I am employed as a Software Architect with Tyler Technologies. I heard about this "blog" thing (invented by some guy by the name of Dave Winer, I think) and I had to give it a go. Wish me luck, my imaginary audience!

Visual Studio Team System 2008 Database Edition GDR - RTM

2008-12-04T18:38:00.001-05:00

Last week, Visual Studio Team System 2008 Database Edition GDR was released to the developer public. I haven't had a lot of time to evaluate it, but my initial impression is positive.

Of course, I had similar feelings about the VSTS 2005 edition of "Data Dude", but my initial excitement quickly turned to disappointment when I realized it just wasn't going to work for us. I couldn't really blame the product...let's just say our db schema is rather sadistic. The SQL parser didn't stand a chance and died in a flurry of spurious error messages.

The parser in the new version is obviously much improved. Once I got the projects/references set up, I was only left with a manageable number of legitimate errors. On top of that, they've managed to remove the dependency on the "Design DB", which makes the product feel a lot less cumbersome. Add to that a lot of other compelling improvements (including basic code analysis and refactoring tools), and I'm a happy customer.

I can't say that all is roses, yet. When I attempt to "deploy" my changes, targeting the original database, the generated script file is blank. The deployment process generates a ton of warnings, but no errors. Curiously, if I instead do a "schema compare" between the project and the aforementioned database, it works.

Data Dude : Visual Studio Team System 2008 Database Edition GDR - RTM

January '09 BAND Gig

2008-12-04T15:03:00.001-05:00

I've agreed to be the presenter for the January '09 BAND (Bangor Area .NET Developers) gig. I'm not exactly sure what day it is (hopefully I'll show up), but I do know the topic: Microsoft's Entity Framework (EF). Here's the blurb I came up with to describe the talk (for the BAND site):

Using entity classes has many advantages over more generic data access methods. On the other hand, entity classes can be time-consuming and surprisingly difficult to implement correctly. In this talk, Daniel will show how Microsoft's new Entity Framework does the heavy lifting to make entity data access simple and powerful.

I'm not sure I'll ever get used to referring to myself in the third person.