The hillbilly turned forty last weekend! I’ve been anticipating this giddily because now, I can be crotchety and people will still find me adorable.

I hate QA. Hate, hate, hate, HATE, HATE, HATE it. I despise it so much, I think I might swear. Here goes….I f—, okay, I can’t do it but I still &*%$ hate it.

Side note: I know all QA people weren’t raised by trolls and/or gnomes but I’m going to generalize here. If you are a QA person and were raised by evil leprechauns, please don’t take offense.

As a hillbilly, I’m an innately optimistic and forgiving person. An ugly codebase to me is a sea of opportunity. Changing requirements? Ha! I can put my OCD up against any client. I can see the positive in almost anything. Except the emotional rollercoaster that is QA.

The end of an iteration is a time of celebration and relief for me. We’ve completed a significant piece of functionality on our application. We’ve followed best practices, left the code in a better state than when we started, and all our unit and UI tests are passing (more or less). Despite my past experience, I always feel great pride when I email our QA department:

Hey there, QA folkery! I come bearing tidings of great joy! We have a new version of the application for your revelry and astonishment! You will surely faint in awe at the sheer glory of it! I await your token sign-off. Acclaim and praise is optional (but recommended).

After that, I sit quietly for a few minutes to bask in my accomplishment of the last couple of weeks then dive into the next iteration, the previous one long since gone from the vestiges of my memory by that point.

The inevitable spartan response:

Please fix the following issues:

• As per the spec, emails should be sent every hour, not daily
• I got a 404 error when I went to the public booking URL
• I know this wasn’t part of the iteration but we should display a message when someone confirms an appointment. Can you include this?
• You spelled it “cancelled” in the settings screen but “canceled” on the calendar.

Let me know when a new version is up for testing.

By now you will not be surprised when I tell you that the QA process was invented by the Marquis de Sade.

QA goblins: surely you see my point, no? The iteration is over. Done. Finished. There’s no “D.S. al fine” at the end. In my head, I’ve moved on to new functionality. I’ve got tests written and prototypes completed. And you want me to forcibly re-enter the past?!? All the hard problems from that iteration have already been solved. I don’t care about your “cosmetics” or your “functionality”. That new feature you’ve requested? It’s KILLER! It’ll be a great asset to the app. When I get around to it. That 404 error? Just a configuration issue. It’ll be fixed when we deploy. And come on, spelling mistakes? Have you not been on Facebook lately?

In short: I. Don’t. Want. To. Do. It. Right. Now.

And I have to. Virtually everything brought up in a good QA process is valid from bugs to cosmetic issues to features that we need to include this iteration even though they weren’t in the list. Without them, we don’t have a usable product. Users will complain or worse, move on to something else. But my headspace is somewhere else. I’ve already compartmentalized the new features and have shifted my own personal Eye of Sauron on them. Now, there’s leakage that I thought I’d never have to deal with again.

Another psychological issue: I’ve poured everything I’ve got to deliver the best that I’ve got and it wasn’t good enough. It’s never good enough. I’ve just presented to you the end result of four years of university training and over a dozen years of experience as an “expert in the field”. And with a few phrases, you send me slinking back to my computer to do it over again.

It’s a little strange in some regard. I welcome criticism of my coding choices and style. I actively seek it out sometimes. Even code reviews can be made fun again. But something about the subtly adversarial nature of QA raises my dander. The fact that it’s so essential to quality and that I can’t think of an effective way around it doesn’t help…

So to sum up:

• QA forces me to change my headspace. And I hate it for that.
• QA points out my flaws as a developer. And I hate it for that.
• QA is necessary and makes software better. And I hate it most of all for that.

To end on a more positive note, most QA people (including those that do it as part of other duties) I’ve worked with are extremely nice. There’s no accusatory tone in the responses, no blame on messing things up, and no sense of “you’ve *really* screwed things up this time.” In their minds, they’ve done what’s expected and like me, have made the product better for it.

But I still hate you.

Kyle the Unadulterated

OpenRasta has been there for many years, and is the most widely used alternative framework for ReST applications.

OpenWrap has been there for a year and a half, and has some usage for advanced users wanting a nicely-architected package manager that doesn’t take dependencies on lots and lots of other technologies and doesn’t require VS.

And we have sub-projects, such as OpenFileSystem, an IO abstraction library that ought to  make using the file system easier, shipping with an in-memory version that should work the same way as your target operating system.

Now all those projects are at varying stages in their life, and the amount of work I used to give to only OpenRasta has been vastly disolved, which in turns means that some users are starting to get quite annoyed with our dreadful homepage and our lack of binary downloads.

Add to this the competitive pressure on OpenRasta introduced by Nancy and the future openrasta copycat WebAPI from the photocopier machine Microsoft, and you’ll understand that the current trend is not sustainable.

This year I’ve decided to simplify my life and delegate much more of my work to a community that has been so far very supportive but very unwilling to contribute anything back. I also want to go back to enjoying working on OSS software to build great solutions and not focus my energy on simply competing with any tool and framework microsoft invents to duplicate existing OSS solutions. I have a vision and I want to deliver on that, OpenRasta and OpenWrap are two sides of that, and I want to start making them work together the way I want without having to be dragged in competitive races with Microsoft’s marketing dollars.

So we had a planning meeting with some of the companies that use OpenRasta in anger in London and want to step up their contributions (7digital and huddle, we love you guys!), and decided that we were going to change how things are done in the OpenEverything projects.

Project organization

First and foremost, I’m going to contribute much less to the 2.1 branch of OpenRasta, and patches / merges / new builds are going to be handled by the contributors that are given write access to the git repository. This gives away a lot of my power.

I’m going to be focusing on the outstanding modules for OpenRasta that add a whole bunch of functionality that will be rolled-back in 3.0: the caching module (soon to arrive in 0.2 version), the new config API (with compact fluent API and convention-based config) and the dynamic / json out-of-the-box support. There’s also a new openrasta-diagnostics module that I’ve started building and want to get out there. I’ll also contribute some work towards the async support that has been requested by some.

OpenFileSystem 2.0 is going to be under Henrik Feldt supervision, and will be a merge of the amazing work he did on long-path and transactional file system support.

All of our versioning is going to be a strict application of http://semver.org 2.0 (which is now the only standard in OpenWrap), so that will solve any breaking changes being introduced without knowing.

Of course, OSS is not a democracy, so I keep a veto right on certain changes that would impact future versions, or architectural changes that do not fit with the vision I have for all those projects. This is give and take, I give some of the control of more than 4 years worth of my development time, but I will keep the main architect role for the time being.

As for releases, anytime someone with commit access wants to create a new release, the process is very simple: they email on the mailing list, and if there’s no majority opposition, we have a new build. Simples.

Orphaned projects

Sadly, we do have some orphaned projects in all this. Mostly, the container support for each of the IoC containers is not up-to-date for StructureMap (tests are broken) and Unity (I have no understanding of the code, so while I *think* I fixed the latest build, I can’t try it out). At this stage, anyone with a vested interest in those projects should start contributing and come help out, or risk that code not being maintained anymore.

OSS is not just take, it’s give too

That leaves a real question for those companies that have used OpenRasta to make money and provide a killer-architecture and have contributed absolutely nothing back since the beginning.

I’ll let you in on a little secret. I give the code away on everything, so it is a company’s legal right to take and not give back. As a side-effect, it is my complete right to completely ignore those that make money on our hard work and want to keep it all to themselves, and my responsibility to not prioritize the needs of those that contributes over those that are selfish..

OSS is also about giving back, and this year it’s what is being enforced on the OpenEverything projects. Fostering contributions is what we want to get to. No more free-lunch though, that’s for sure.

Conclusion

This new organization should let OpenRasta absorb patches and get on the 2.1 release cycle faster, and then on to 2.2, etc. This will allow me to focus on what I want to build for 3.0, and this will also allow more contributors to own projects more directly than before.

If you have comments or feedback on all those new plans, please let us know on the now unified openeverything-dev mailing list.

Please don’t use it for user questions, we’ve moved all support of all those frameworks to stackoverflow where google can do its job and users can find their answers more quickly.

The mythical book, Mythical man month quoted that no matter the programming language chosen, a professional developer will write in average 10 lines of code (LoC) day.

After a bit more than 6 years of full-time development on the tool NDepend I’d like to elaborate a bit here.

Let’s start with the definition of logical Line of Code. Basically, a logical LoC is a PDB sequence point except sequence points corresponding to opening and closing method brace. So here we have a 8 logical lines of code method for example:

See here the post How do you count your number of Lines Of Code (LOC) ? for more details.

I already hear the bashing; LoC has nothing to do with productivity, Bill Gates said “Measuring software productivity by lines of code is like measuring progress on an airplane by how much it weighs.“.

And indeed, measured on a few days or a few weeks range, LoC has nothing to do with productivity. Some days I write 200 LoC in a row, some days I spend 8 hours fixing a pesky bug by not even adding a LoC. Some day I clean dead code and remove some LoC. Some other days I refactor exiting code without, all in all, adding a single LoC. Some days I create a large and complex UI Form and the editor will generate automatically 300 additional LoC. Some days are dedicated solely to performance enhancement or planning upcoming code architecture…

What is interesting is the average number of LoC obtained from the long term. And if I do the simple math I obtained around 80 LoC per day. Let’s precise that I abide by high code quality standard. Each LoC must be covered by unit-tests + all possible code contracts must be declared (of course, not all LoC can, nor should, be checked by contracts and some classes are nit relevant for unit-testing, especially UI related classes).

So this average score of 80 LoC produced per day doesn’t sacrifice to code quality, and seems to be a sustainable rhythm. And what is really cool with LoC, is that, once calibrated, caring about counting LoC becomes an accurate estimation tool. After coding and measuring dozens of features achieved in this particular context of development, the size of any feature can be estimated accurately in terms of LoC. Hence with a simple math, the time it’ll take to deliver a feature to production can be accurately estimated. To illustrate this fact, here is a decorated treemap view of most NDepend components.

Thanks to this treemap view, I can compare the size in terms of LoC of most components. Coupling this information with the fact that the average coding score if 80 LoC per day, and looking back on cost in times for each component, I have an accurate method to tune my way of coding and estimate future schedules.

Of course not all components are equals. Most of them are the result of a long evolutive coding process. For example, the code model had undergone much more refactoring since the beginning than say, the dependency graph for example that had been delivered out-of-the-box after some weeks of development.

There is something else interesting this picture is telling. I can see that all these years polishing the tool to meet high professional standards in terms of ergonomy and performance, consumed actually the bulk of LoC. For example just having a clean Project Properties management is implemented through (model + UI control) =(3.751+2.686) = 6.437 LoC. While a flagship feature such as the interactive Dependency Graph only consumes 2.897 LoC, not even a half of the Project Properties impelmentation. Of course the interactive Dependency Graph capitalize a lot on all the existing infrastructure developed for other features. But as a matter of fact, it took less effort to develop  the interactive Dependency Graph than to develop a clean Project Properties model and UI.

All this confirms an essential lesson for everyone that wishes to start an ISV. It is lightweight and easy to develop a nice and flashy prototype application that’ll bring enthusiast users. What is really costly is to transform it into something usable, stable, clean, fast with all possible ergonomy candy to make the life of the user easier. And these are all these non-functional requirements that will make the difference between a product used by a few dozens of enthusiast users only, and a product used by the mass.

I rarely review books however the past few years there’s a few gems I’ve found in my REST travels that I have felt compelled to review. Mike Amundsen’s new book on hypermedia is one such work.

Note: This review is a slightly modified version of what I posted at Amazon.com.

If you are a hypermedia developer you should read this book. If you are building web apis to reach multiple devices then you should read this book. If you are building web apis that will be around for a long time and evolve over many versions, then read this book. Hypermedia systems have been around since the dawn of the web, however they have mainly been HTML pages offered up to a browser agent. It is extremely rare that you see the usage of hypermedia in web apis / machine to machine scenarios.

Hypermedia is basically a black art that very few folks actually understand. I experienced this first hand when I was the PM for WCF Web API as hypermedia was something that we wanted to be able to support. I immediately discovered that there was very little literature on the concept, there was no definitive reference to turn to. The guidance was mainly spread through a set of articles and through thousands of mail threads. Mike is one of the few who actually gets it having many years of experience building such systems. He is one of the foremost experts on the topic. Just head to his blog or the REST discussion list and you’ll see this is the case.

This book is important because with it Mike lifts the veil of mystery of how to build such systems, taking it from a black art to something broadly available to developers. Mike goes into depth defining what hypermedia is and the different ways it can manifest itself. He describes the different types of hypermedia controls and how best to use them. Mike also pays particular attention to using hypermedia in javascript agents running in the browser, something that is extremely relevant today considering the rise of rich ajax style applications using frameworks like jQuery and extjs. The book is not a book of theory, from start to finish it uses real world examples to illustrate the concepts.

The one thing I would have liked to have seen Mike elaborate on a bit more was why hypermedia itself was important. Throughout the book it does become more self-evident however for one who doesn’t understand what it is I felt that the book could have laid more of a foundation / done a bit more selling on the value. Mike does plenty of that in his blog though.

If you are building RESTful systems particularly in node (though the guidance is general applicable) then I highly recommend this book.

InfoQ has made another of my DevTeach talks available online – TDD/BDD as Architectural Tools. Enjoy!

TDD/BDD as Architectural Tools

As architects, we have all experienced the folly of BDUF (Big Design Up Front) – spending weeks or months perfecting an architecture that fails when it meets the real requirements and real code. Is it possible to design in the small? How can we avoid unintended complexity, which cripples so many code bases? Can we build enough of an architecture to start writing code and then flesh out our architecture as the code evolves? In this session we examine how Test-Driven Development (TDD) and Behaviour-Driven Development (BDD) allow us to solve these conundrums. We will see how we can use TDD/BDD to focus our architectural efforts in the high-value areas of our code base to achieve just-in-time architecture.

Preamble: The intent of this post is to educate on how HTTP was designed to be used , clarify misconceptions and to give folks food for thought on different ways they can design a system. It is not to ignite a war against RPC. There are tradeoffs to consider when your design your system. If you are not designing a system that is to be consumed by many clients over a long time and where the server needs to evolve independently of the client, then using HTTP for RPC may be absolutely fine.

Recently there was a question on the forums asking why we encourage usage of HttpRequestMessage<T> / HttpResponseMessage<T> in the signature of a web api implementation. The point made in the post is that if you have an ICalculator contract which your API implements, then it’s violating SOC / inappropriate to have those messages as params and in the contract. The argument is valid when looking at HTTP from the standpoint of an RPC mechanism, which is actually a quite common view.

Not of the HTTP authors though. If you read Roy Fielding’s dissertation you will see very clearly that:

HTTP is not a transport protocol and HTTP is not RPC.

You might be surprised, but it is true. There are real quantifiable reasons for this that underlie the foundations of how and why the web infrastructure was built. Roy explains it much better than I can, though I still continually attempt to do it  I recommend reading what Roy has to say on the subject here in sections 6.5.2 and 6.5.3.

Back to our HTTP messages. This has deep implications in your API design. Once you accept that the thing that you are exposing is an HTTP resource which clients interact with via the uniform interface and not a class with standard OOP methods, you design them differently. Your APIs become a gateway where HTTP concerns are addressed and then work is delegated off to the business domain rather than being part of the business domain. Once they do, you can safely use HttpRequestMessage<T> and HttpResponseMessage<T> without fear as they are specifically for addressing HTTP concerns.

Mike Amundsen has a really nice post where he emphasizes this last point and why it is just better for us to embrace HTTP in our code and stop fighting it:

“the better way to deal with HTTP is to embrace it. let HTTP lead system designers in the direction of lossy, chunky, state-less designs. accept HTTP as the stable rarely-changing foundation for your implementations. learn to ‘think’ and ‘speak’ HTTP so that all your dist-net designs reflect the power, stability, and reliability of HTTP itself. and don’t fuss with it.”

You can read more about the design implications from a Web API perspective in my reply on the thread.

Interested in your thoughts.

AppEngine has new pricing as of November 7 which has generated much discussion, most of it as exciting as deciding which wine to serve with raccoon (answer: dandelion). So with the segue established, I shall pontificate on what I believe is the single biggest thing keeping it from being a truly great product and a worthy competitor in the cloud hosting space.

First I’ll get out of my system a laundry list of lesser issues:

1. Confusing pricing/configuration

This could be a product of my Microsoft indoctrination but I still have trouble figuring out how we get charged for stuff. It’s based on usage which is a great model on paper. But the usage is broken down into, among other things:

• Front-end instance hours
• Back-end instance hours
• Datastore Write Operations
• Datastore Read Operations
• Datastore Small Operations
• Stanzas Sent
• Channels Created

Furthermore, you can tweak things like the minimum and maximum idle instances and the minimum and maximum pending latency.

All this requires a lot of research and testing. And the only way to test various settings is: a) in production, or b) using load testing with a separate paid environment. Both of which will end up costing you dollars. But it must be said, those dollars probably won’t add up to the cost of setting up your own test environment. So far, we spend about the cost of a smoothie every day for all versions of our app (depending on where you find the ingredients).

2. No built-in backup and restore

Backing up the datastore is your responsibility. Now, it’s easy to set up a batch process running nightly. But you’re almost guaranteed to hit your Datastore Read Operations quota on a nightly basis once you reach a certain size. Also, the new-fangled High Replication datastore makes things more interesting by not actually being supported for this scenario.

3. No built-in reporting mechanism

The only way to interact directly with your data is through code you’ve written yourself or with GQL, a SQL-like language for the datastore. But you quickly hit limitations. First is that you can return only 20 results at a time (which you can up to 200 by adding a limit parameter to the URL…manually). Second is that you have to create indexes for fields you want to filter or order by. Doesn’t lend itself to ad hoc querying.

4. Admin console bugs

This one irks me almost as much as the fact that I have to restart my computer whenever I update the glorified Notepad that is Adobe Reader*. Too often for my tastes, when clicking around the console, I’ll get a general “An error has occurred” page. And the error page is not a pretty one:

Even when it does work, until recently, I had to use Firefox or Internet Explorer to access one of the pages (Datastore Admin) because it didn’t load in Chrome.

Which leads me to the point of this post. The one killer issue in AppEngine that keeps its status below world class hosting environment:

Support as a second-class citizen

If you didn’t notice at first glance, take another look at the error page above. In particular, at the URL at the bottom, which is where the “report” link goes. It’s a Google Group page for AppEngine. To their credit, they’ve addressed many of the issues I’ve had with Google Groups recently. Even so, for a world-class hosting environment and especially for apps we’re paying for, I’d much rather see something like this.

Also, my experience with the group site has been pretty dismal. Much of the time, my questions get not a single response except mine. Even reporting issues on the Google Code site leads to sporadic responses. I get slightly better averages on StackOverflow.

This came to a head for us recently when we converted from Master/Slave to High Replication late one Saturday night. Several of the problems I outline here occurred that night, including the 1990s error screen above. And I couldn’t find a single email address or phone number anywhere that I could contact for help and be assured of a response with even an unreasonable time. I made a post to the forum about an issue we saw the next day. The link is included above in the list of posts that have received no response.

Another support-related area that could use work is the roadmap. Something I’ve noticed in all dealings with the AppEngine team is an almost fanatical abhorrence of delivery dates. Even at Google IO, the best we could get from the team was “we’re working on it” with a lot of nervous laughter when someone asked about SSL access on custom domains, which is one feature we’ve had to make decisions around. I gather it’s a hard problem but even if they said “probably 2012”, that would at least indicate to us “okay, it’s not anytime soon, time to decide which is more important in the short term.”

Had I written this post a couple of months ago, after our High Replication migration, it would have been a lot more acidic in tone. For a few weeks after that, I was actively checking out Amazon Web Services. A Microsoft rep reached out to us serendipitously that same week wanting to talk to us about Azure. If he had not dropped the ball and postponed our meeting at least three times, this here blog thingy might be back in .NET land by this time. (Likely not, but this wouldn’t be a blog if I didn’t make grandiose unsupportable claims that back up my argument.)

These days, I’m not so much frustrated as I am disappointed. For all its faults, there is a lot of good being offered by AppEngine. Like many startups, we have discussions about Google and AppEngine and the general consensus is that we’re happy to see Google focusing on what they’re good at. (Do I lose points if I point out my subtle digs?) But when it comes to support for AppEngine, it feels like it’s run by engineers for engineers. Yes, support is boring and customers can be confrontational and much of the time, the answer is a variation of “you’re doing it wrong”, etc, etc. But it’s not just another IoC container; it’s a cloud hosting platform. I believe it needs a higher level of professionalism than what I’ve seen so far.

Kyle the Untenable

*But not nearly as much as the fact that said update invariably adds an icon to my desktop. C’mon Adobe, who actually opens Reader and then opens a PDF file?

A somewhat unorthodox and non-traditional suggestion I admit, but as Brad Meltzer often asks: “This may seem unlikely, but go with me on this…”

How many times have you had to build that piece of furniture from Ikea, install window blinds, etc? In each case, you are solving a problem. You have to analyze and organize. It’s really not that different from development. It’s a project like anything else. It has a beginning, middle, end. Like development, if you misinterpret the requirements, you have to go back a few steps. That next thing you were going to do has to get pushed to a later time.

This past weekend, I had to install some Levolor Blinds into our spare bedroom. I also happened to be listening to the  This Developer’s Life Podcast. I was listening to Episode 1.0.6: Abstraction. A great episode (as they all are – and like really good things – you have to go through them a few times because you are likely to pick up something new and useful that you missed before.) The part that really struck me was the interview with Ward Cunningham. It was then that I decided to use my blind installation task as an analog of sorts for development. The interview with Ward was the inspiration. The end game is to then reverse the flow – to perhaps use development as an analog for the blind installation task – assuming of course the blind installation task is successful!

It begins with us…..

The one thing that really struck me about Ward was his motivation – that he builds things form himself. The same was true with John Ressig and Dan Bricklin. Then it struck me that this is really a craftsmanship issue. True craftsmen always reserve the best stuff for themselves. Indeed, much of that good stuff accrues to the benefit the things that are released for public consumption. Craftsmen work product also shares another characteristic – what they produce is pleasing to look at and use by the harshest critic of them all – the craftsmen themselves.

Turning back to my window blind example, I wanted to ensure they would function properly, be pleasing to look at – much like what we strive for in our software.  For a satisfactory end result however, there has to be some measure of planning. Since I had two blinds to install, I wanted to make sure the second one had the benefit of the first – as to process and as to a completed product. My benchmarks were simple:

Did I thoroughly understand the requirements?

Did I have a plan?

Was I organized?

As to tooling, did I have what I needed and were those tools readily available?

Was there improvement in the process?

The great thing about tangible projects – particularly those you have to do around the house or wherever you may be, those projects provide immediate feedback. You know whether they look right. You know immediately whether something was hacked together. You know whether the finished product is pleasing to look at.  You also know what is lurking beneath the surface. I’m in the middle of the Steve Jobs bio by Walter Isaacson. Jobs was obsessive about quality (perhaps to an unhealthy extreme!!). His [Jobs] obsession was centered around the fact that quality of thing not seen was at least as important as what is seen.   That consideration absolutely applies to the software world. When we see properly working software, we can take pride that the things people don’t see are well constructed. With home projects – we know that something is not going to break or fall apart because the things people don’t see (like screws into the wall being well anchored) are well done.

Food for thought as you contemplate ways to continuously improve.

In case you missed it, the WebAPI Developer Preview is up to version 6. The CodePlex site provides all of the information you need to get started. To illustrate how to build a simple WebAPI, this example uses ASP.NET MVC as the host. While the example makes reference to the fact that you can create this example in self-hosted mode (not using ASP.NET/ASP.NET MVC) – there is no code that illustrates how to do this.

To get started, create a Console Application and follow the steps in the ASP.NET MVC example to use Nuget to download the WebAPI references and to create the Contact Manager classes. Once you have those items in place, all you need to do is outfit the application to fire up the web host. To do that, use this code:

using System;
using ContactManagerSelfHost.APIs;
using Microsoft.ApplicationServer.Http;

namespace ContactManagerSelfHost
{
    class Program
    {
        static void Main(string[] args)
        {

            using (var host = new HttpServiceHost(typeof(ContactsApi), "http://localhost:9000/api/contacts"))
            {
                host.Open();
                Console.WriteLine("Press any key to stop host...");
                Console.ReadKey();
            }
        }
    }
}

Simply run the app and navigate to the URL. That’s it!

So given that a class that is nothing more than a data bag. After all, each object is just a bag of dict’s which are pointers to some byte array some place. What can we introspect about the byte array in each language?

The following is my attempt to document my metadata experiments.

Let’s start with .Net, its what I know the best and what I am ultimately comparing everything too.

.Net

public class Loan
{
    public decimal Amount { get; set; }
}

var aLoan = new Loan {Amount=5};
aLoan.GetType();
//an instance of Type filled with data about Loan
for(var prop in aLoan.GetType().GetProperties())
{
    Console.WriteLine(prop.Name);
}
aLoan.GetType().GetProperty("Amount");
//returns an instance of PropertyInfo

Links:

Type: http://msdn.microsoft.com/en-us/library/system.type.aspx
PropertyInfo: http://msdn.microsoft.com/en-us/library/system.reflection.propertyinfo.aspx

Ruby

class Loan
    attr_accessor :amount
    def initialize(initialAmount)
        @amount = initialAmount
    end
end

aLoan = Loan.new 5
aLoan.class #Loan
aLoan.method(:amount) #a method object

After playing around, I couldn’t find anyway to get the type returned by the ‘amount’ function. This is exactly what I expected, but wanted to confirm. I could get the number of arguments via the ‘arity’ method. That said, I was able to extract a lot more information than I had expected. Special thanks to @jflanagan for the much need schooling in ruby.

Links:

http://en.wikibooks.org/wiki/Ruby_Programming/Syntax/Classes#Accessor_Methods

http://ruby-doc.org/core-1.9.3/Method.html

Python

class Loan(object):
    def __init__(self, initialAmount):
        self._amount = initialAmount

    @property
    def amount(self):
        return self._amount

aLoan = Loan(5)
aLoan.__class__ #Loan
aLoan.amount #nope, this returns 5
dir(aLoan) #return 'all the things'

Links

http://docs.python.org/library/functions.html#property

JavaScript

function Loan(initialAmount) {
    //as recommended by Ryan Rauh
    this.amount = initialAmount;
    return this;
}

var aLoan = new Loan(5);
typeof(aLoan); //object
Object.getPrototypeOf(aLoan).constructor.name; //Loan
for(prop in aLoan) {console.log(prop)}; //prop is a string
typeof(aLoan['amount']); //number
//this is actual a typeof(5)

I was again surprised at how much I could extract. Special thanks to @rauhryan for the JavaScript schooling.

Beyond my reach (for now)

So I wanted to do some other languages outside my comfort zone like lisp, haskell, and erlang. But my brain was exploded by the very different nature of the languages. What I do have is how I would set the structure up roughly in the various languages. I hope to come back overtime and update this as I learn or people share with me.

Lisp

('Loan, ('Amount, 20))

Haskell

data Loan = Loan { Amount :: Int }

http://www.cs.auckland.ac.nz/references/haskell/haskell-intro-html/moretypes.html

Erlang

-record(Loan, { Amount })

Links:

http://www.erlang.org/doc/reference_manual/records.html

 

Any feedback on how to do these kinds of things in other languages will be most appreciated.

-d

I’ll be touring the north of Britain from the 18th of January with two talks: HTTP caching 101 and Links, forms and Unicorns.

The details are at http://www.gep13.co.uk/blog/?p=609 so if you’re interested in HTTP or/and ReST, do come along!

I began writing tests around 8 years ago and I must say this revolutionized my way of developing code. In this blog post, I’d like to group and share several non-trivial feedbacks gained over all these years of practicing real-world testing.

Test Organization

There must be a one-to-one correspondence between classes tested and test fixture classes. Very simple classes like pure immutable entity (i.e simple constructors, simple getters, and readonly backing fields) can only be tested through test fixture classes corresponding to more complex tested classes.

The one-to-one correspondence stated above makes the following operations easy and straightforward:

• writing a new test suite for a currently created class,
• writing some new tests for an existing class,
• refactoring tests when refactoring some classes,
• identifying which test to run to test a class.

Tests must be written in dedicated tests assemblies because product assemblies must not be weighted with test code. Hence tests and tested code lives in different VS projects. These projects can live in the same VS solutions, or in different VS solutions.

Having tested code VS projects and tests VS projects living in the same VS solution is useful to navigate easily across tested code and tests.

If tested code and tests live in different VS solution, a global VS solution containing both code and tests VS project is still needed. Indeed, such global VS solution is useful to use refactoring tools to refactor at the same time tested code and tests

Test-First and specifications

I don’t abide by the test-first theory. Tests concerning a code portion should be written at the same time than the code portion. The reason I advocate to do so, is because not all edge cases can be faced up-front, even by expert of a domain and professional testers. Edge cases are discovered only when thoroughly implementing an algorithm.

Test-first is still useful when tests can be written up-front by a domain expert or a client, with dedicated tooling such as FitNesse. In this conditions test-first is a great communication mean that can remove many pesky misunderstanding.

However only the code itself is the complete specification. Tests can be seen as a specification scaffold, that is removed at runtime. Tests can also be seen as a secondary less-detailed specification, useful to ensure that the main specification, i.e the code itself, respects a range of requirements, materialized by tests assertions.

Test and Code Contracts

Calls from tests to tested code, shouldn’t violate tested code contracts. When a contract is violated by running tests, it means that the tested code is buggy or (non-exclusive) that the test is buggy.

Assertions in tests are not much different than code contract assertions. The only real difference is that assertions in tests are not checked at production-time. But both kind of assertions are here for correctness, to buzz as soon as something goes wrong when running the code.

Test Coverage

The number of tests is a completely meaningless metric. Dozens of [TestCase] can be written in minutes, while a tricky integration test can takes days to be written. When it comes to test, significant metrics are:

• percentage of code covered by tests and
• number of different assertions executed.

Not all classes need to be 100% covered by tests or even covered at all. Typically UI code is decoupled from logic, and UI code is hard to test properly, despite the progress in this domain. Notice that UI code decoupling, which is a good design practice, is naturally enforced by tests.

When testing a class, covering it 100% is a necessary goal. 90% coverage is not enough because empirically we observe that the 10% non-tested code contains most of unidentified bugs.

If to cover 10% of the code of a class it takes as much effort as covering the other 90%, it means that this 10% of code needs to be redesigned to be easily testable. Hence the 100% coverage practice is costly only when it forces the code to be well designed.

Having zero dead code is a natural consequence of having code 100% covered.

It is a good practice to tag a class 100% covered with an arbitrary [FullCovered] attribute because it documents for future developers in charge of an eventual refactoring, that the class must remain 100% covered.

Code 100% covered by tests + containing all relevant contracts, is very hard to break. This is the magical practice to be very close to bug-free code.

Personally I consider that the job is done only when all classes that can be covered by tests are indeed 100% covered by tests.

Test Execution

The concrete difference between unit tests and integrations tests is that unit tests are in-process only, they don’t require touching a DB, network or the file system. The distinction is useful because unit-tests are usually one or several orders of magnitude faster to run.

Tests must be run from the build machine to validate any code before it is committed.

Test should be fast to run, like a few minutes maximum, because tests must also be ran often on the developer machine, to detect regression as soon as possible.

Having a one-to-one correspondence between tested class and test class + 100% coverage, makes easy to identify which tests to run for a particular piece of code.

Test and Refactoring

Last released version of the product that is working fine in production, should stress out that bugs are more likely to appear on newly developed code and refactored code. Hence tests must be written in priority for newly developed code and refactored code.

When some refactored code is not properly tested, it is a good practice to write tests that should have been written in the first place. However this practice of testing refactored code is not mandatory because it can be pretty time consuming.

Code 100% covered by tests + containing all relevant contracts, can be refactored at whim. In these conditions, there are very few chances to miss a behavior regression.

Test and Design

Code that is hard to be covered by tests is bug-prone and need to be redesigned to be easily coverable by easy tests. This situation often results in creating one or several classes dedicated to handle the logic implemented formerly by the code hard to cover by tests.

The previous point can be restated this way: writing a complete test suite naturally leads to good design, enforcing low coupling and high cohesion.

Singletons should be prohibited because they make the code hard to test. It is harder to re-cycle and re-initialize a singleton object for each tests, than it is to create an object for each test.

Mocking should be used to transform integration tests into unit-test, by abstracting tests from out-of-proc consideration like DB access, network access or file system. In these conditions mocking helps separating concerns in the code and hence, increases the design value.

Mocking is also useful to isolate code completely un-coverable by tests, like calls to MessageBox.Show().

Test Tooling

Personally I am happy with:

• NUnit, I like the fact that only the NUnit.Framwork.dll needs to be referenced from VS tests project and that NUnit is supported by all testing tools.
• NCover, it is the fastest code coverage tools available. Its performance are almost magic since in coverage mode tests are executed almost as fast as without coverage. Contrary to VS coverage, the assembly instrumentation phase is completely transparent for the user, and coverage results files are easier to work with. Also NCover is very easy to use and integrates with most testing tools.
• TestDriven.NET, its flawless VS integration made me addicted to it. Resharper also comes with cool testing facilities but my addiction to the simplicity of TD.NET is too high.
• When it comes to testing NDepend is especially suited to create rules to continuously check for code coverage by tests, to ensure that new and refactored code is properly covered by tests and to navigate between tested code and tests across VS solutions opened in different VS instances.
• It is worth also keeping an eye to ContinuousTests currently in beta to identify which tests to run when a portion of code is touched, and run them in background.

I’ve had a great 7 year run at CodeBetter, but I’ve decided to start my own little blog at http://jeremydmiller.com (RSS is http://jeremydmiller.com/feed/).

Thank you to Brendan and everybody I’ve met through the CodeBetter community over the years for a great experience.

 

Although I left WCF Web API, I am very passionate about continuing on my journey toward understanding REST and how it fits with Web API. For that reason you will likely continue to see posts on REST and Web API here along with my new found love node.js.

If you head over to our codeplex site, one question commonly asked is “Does Web API support hypermedia?” To understand this question you have to first understand what hypermedia is. I am not going to cover that here, but there are many articles about it, a ton of discussions in the REST Discuss Yahoo group and a new book by Mike Amundsen which is well worth reading. (I need to write a review for the book at some point). I gave a quick primer which doesn’t do justice, but will get you through this post here.

The answer to the question depends on your notion of support:

• If by support you mean, “Can I embed links within my responses which will indicate to my client available resources / state transitions?” then the answer is YES.
• If it means, “Will Web API automatically embed links for related resources out of the box?” the answer is no”.

Web API was designed to be highly pluggable, especially as it relates to how a request is handled, or how the content is processed. You can take advantage of this for plugging in your own custom code for generating hypermedia.

In the rest of this post I will discuss various strategies for how as well as point you to several samples.

Why formatters are a good choice

There are several different ways you can technically attach hypermedia including formatters, message handlers, and operation handlers. Formatters are the way we would recommend as they are specifically responsible for taking the result of an operation (could be anything) and transforming it into some representation. That representation can contain hypermedia.

How conneg and formatters work in Web API

Each Web API host has a collection of formatters. Each formatter specifies information about the media types it supports. On any incoming request, if content is sent, Web API matches up the incoming request’s Content-Type against the available formatters to choose one to appropriately handle processing the content in the request. It invokes the matched formatter passing in the content of the request. The formatter then outputs a CLR object which is passed to the operation.

Once the operation has completed, Web API performs server side content-negotiation to determine the response representation. It looks at the accept header supplied by the client in the request and matches up against the available formatters. The algorithm will look for a best match based on the specified media type / media type ranges in the accept along with the q-values. Once a formatter is identified it will be invoked passing in the return value of the operation. The formatter takes the the incoming value and writes out the representation.  

As the formatter is writing out the response, it can also embed hypermedia controls i.e. links and forms. If you read the REST in Practice book they show a media type “application/vnd-restbucks+xml” which contains hypermedia. You can create a custom RestBucksFormatter that handles that media type. If you check the RestBucks sample for Web API on codeplex, you’ll see an example of this.

Url / Link generation

In a hypermedia based system, links are generated based on a state machine.  Depending on the state of the resource, different links apply. As soon as you start writing the formatter you will have to deal with how to handle the generation of links. There’s a several approaches you can use.

Generate links within the formatter.

With this method, your formatter contains the state machine logic for generating links. In the case of an Order management API, the API implementation for GET can return an order. The formatter then comes a long and grabs the order and adds appropriate links based on the state of the order. I used to be a big fan of this approach as it keeps links as separate concern from the API code itself. I’ve since changed that opinion, as I don’t view links as a separate concern

The upside of this approach is everything is nicely encapsulated within the formatter. The downside is that it makes it harder for one to follow the logic of the system if they are reading the API code. They must also read the formatter. It also means that the link code is duplicated if you support different variants like xml and json that require the hypermedia.

Example: Hypermedia formatter for contacts

A while ago, Mike Amundsen visited Microsoft and I was fortunate to have the chance to brainstorm with him around Media Type design. We took the ContactManager example and spiked on how it “could” look if it used hypermedia. The implementation used the approach I just described. Below is a snapshot of how the media type itself looks. The “.” for the uri is a placeholder as the uri doesn’t matter, it is opaque to the client.

<?xml version="1.0" encoding="utf-8" ?><Root>  <Contacts href=".">    <Contact href=".">      <Name>Mike Amundsen</Name>      <Address>123 Any Street</Address>      <City>Erlanger</City>      <State>Kentucky</State>      <Zip>41018</Zip>      <Email>mca@mamund.com</Email>      <Twitter>        <Tweets href="."/>        <DM href="."/>        <Mention href="."/>      </Twitter>    </Contact>  </Contacts>  <Queries>    <Query name="query-twitter-contacts" href="."/>    <Query name="query-search" href=".">      <Name/>      <City/>      <State/>    </Query>  </Queries>  <Response>    </Response></Root>

view raw gistfile1.html This Gist brought to you by GitHub.

One thing to note here if you are familiar with hypermedia, is that it is not using the standard link element with “rel” and “href” attributes that you are probably accustomed to seeing. Instead a domain specific format is used to to represent links and forms. In this example, Tweets has an href indicating where to go to get tweets. DM is a form for sending a direct message. The query named “query-search” includes a form with fields for specifying search criteria. This type of approach of using domain specific hypermedia deserves it’s own post. It has several advantages in that it is more intention revealing, offers richer semantics than just being limited to a rel and href , and also doesn’t abuse the rel tag.

As to the implementation, here you can see the operation within the ContactsApi. Then in a custom formatter all the hypermedia affordances are attached.

You can find all the code for Mike’s example at his github repo here

Generate links close to the operation.

I mentioned above how my perspective changed on this. My current understanding is that links relate completely to the core business logic of the system as opposed to being a cross-cutting concern like logging. They are NOT a separate concern. Think of your Web API as a bridge between HTTP and your business domain, not the domain itself. Thus the model being returned from your API being tailored to HTTP is completely valid. This is not about DTOs.

With this approach links will get generated by the operation. Your operation returns a model that has links on it. The formatter’s responsibility is then to take the model and encode the links correctly within the representation. This allows you to add new formatters easily without replicating the linking logic. You could have an ILinkable<T> type which your formatter understands for attaching links. This type is not sufficient for collections as in that case you have links nested at multiple levels. For that you can have an ILinkableCollection<T> that contains items of ILinkable<T>. Thus you can return an ILinkableCollection<Order> that contains within ILinkable<Order> instances each with their own links attached.

public interface ILinkable<T> {  T Instance {get;}  IList<Link> Links {get;}}
public interface ILinkableCollection<T> : IList<ILinkable<T>> {  IList<Link> Links {get;}  void Add(string name, Uri uri, string rel="");}
public class Link {  public Uri Uri {get;set;}  public string Rel {get;set;}  public string Name {get;set;} }

view raw gistfile1.cs This Gist brought to you by GitHub.

The link generation code itself does not have to sit within the operation itself. Introduce a separate service such as IOrderLinkService which has that specific responsibility and have it injected via an IoC container. The service exposes an AddLinks method which accepts  ILinkable<T> or an ILinkableCollection<T>. Although the link logic does not exist within the service, it is easy to step through the code to debug the logic. 

Example: Domain specific media type for Orders

Here’s a snippet of the Order API.

public class OrderApi {
  private IOrderLinkService _linkService;
  public OrderApi(IOrderLinkService linkService) {    _linkService = linkService;  }
  [WebGet(UriTemplate="{id}")]  public Get ILinkable<Order> Order(int id) {    var order = //get the order from somewhere    return _linkService.AddLinks(order);  }}

view raw OrderApi.cs This Gist brought to you by GitHub.

The link service is injected into the API. The Get method then invokes it passing the order.

public class OrderLinkService : IOrderLinkService, ILinkService<Order> {  public ILinkable<Order> AddLinks(Order order) {    var linkedOrder = new Linkable<Order>();    if (order.State == OrderStates.Created) {      linkedOrder.Links.Add(OrderLinks.Approval, GetApprovalUri(order.ID));    }
    //other state logic here    return linkedOrder;  }
  private Uri GetApprovalUri(orderid) {    ...  }}

view raw OrderLinkService.cs This Gist brought to you by GitHub.

Once the service is called it looks at the state of the order and adds links appropriately. Above you can see that when the order is in the created state, a link is added for an approval resource.

The code can definitely be refactored / made cleaner, but it illustrates the basic idea. As an example of something more sophisticated, the RestBucks port for Web API contains a ResourceLinker service that generates links via accepting a lambda expression.

Lastly there is the formatter itself. It will take the linked order and write out the appropriate links.  Below is a skeleton of such a formatter.

public class OrderMediaTypeFormatterJson : MediaTypeFormatter {  public OrderMediaTypeFormatter() {    SupportedMediaTypes.Add(      new MediaTypeHeaderValue("application/contoso.orders+json")    );  }

  public override object OnWriteToStream(Type type, object value, System.IO.Stream stream, HttpContentHeaders contentHeaders, System.Net.TransportContext context)   {    var linkedOrder = value as ILinkable<Order>();    dynamic json = new JsonObject()    //write the order to json  }
  public void OnReadFromStream(Type type, System.IO.Stream stream, HttpContentHeaders contentHeaders) {    throw new NotImplementedException();  }}

view raw OrderFormatter.cs This Gist brought to you by GitHub.

Example: HAL formatter

HAL is a media type in draft at IANA that is designed specifically for expressing resources and links. Steve Michelotti has created a formatter for HAL. The way it works is your operations return a HalResource model. HalResource can recursively contain other resources thus you can have a very deep resource map. You can use it both on the client and the server. The formatter then takes that model and writes it out appropriately as HAL, it also do the reverse taking a HAL response and turning it back into a HAL model. You can read more about how to use it in Steve’s post. Hopefully we’ll see this on nuget soon!

Example: XHTML using the plain old XML formatter

Mike Amundsen talks quite a bit about XHTML in his new book. What makes XHTML interesting from a hypermedia perspective is that it contains native support for hypermedia affordances, and browsers naturally understand it. You can express links very easily using an <a href=””> construct. You can embed forms as well as it supports them natively.  It also offers annotations like name, id and style which we can use in order to attach domain specifics to the elements. For an rich example of what you can achieve with XHTML check Mike’s ALPS micro-blogging specification.

Web API supports the ability for you to return XML dom objects directly, for example you can return XmlDocument/XDocument instead of a model. The beauty of this is that you can easily represent any XHTML document using the XML dom. Thus in your operation you simply create an  XmlDocument, populate it appropriately and return it. The formatter will very happily then write it out to XHTML.

Here’s a skeletal snippet illustrating what I mean

pubic class OrderApi{  [WebGet(UriTemplate="{id}")]  public XmlDocument Get(int id) {    var order = //get the order    var document = //create an XmlDocument for the order    return document;  }}

view raw gistfile1.cs This Gist brought to you by GitHub.

If you do use this approach one thing you will need to do is add the media type to the default XML formatter. You can do that off of the configuration object’s Formatters collection using code similar to the following:

config.Formatters.XmlFormatter.SupportedMediaTypes.Add(new MediaTypeMapping("application/vnd-contoso.orders.xml"));

view raw gistfile1.cs This Gist brought to you by GitHub.

Example: XHTML formatter example using Razor

Recently I delivered a talk at Tulsa TechFest where I demonstrated RestBugs. A sample app written by Howard Dierking. It  contains a web API for bug tracking which and supports XHTML via a custom formatter. The app also includes a console hypermedia client which I hope to cover in  another post.  You can watch the video from that talk here. If you jump to the last 30 minutes that’s where I get into RestBugs where I cover the API, formatter and the hypermedia client.

One interesting thing about the app is the formatter that is used. Instead of manually writing out the XML, Howard uses a custom Razor formatter. The formatter looks for a custom content header which specifies the template. It then uses ASP.NET’s Razor engine to take the model passed in along with the template to generate the XHTML. To illustrate here is the template for the bug list.

<html>	<head>		<title></title>	</head>	<body>    <table border="1">        <tr>            <td>ID</td>            <td>Name</td>            <td>Status</td>            <td>Priority</td>            <td>Rank</td>            <td>Assigned To</td>            <td>History</td>            <td>Item Link</td>        </tr>
        @foreach (var item in @Model)        {            <tr class="bug-data">                <td class="id">@item.Id</td>                <td class="name">@item.Name</td>                <td class="status">@item.Status</td>                <td class="priority">@item.Priority</td>                <td class="rank">@item.Rank</td>                <td class="assignedTo">@LinkFromEntity(item.AssignedTo, "assignedTo")</td>                <td>@LinkFromEntity("history", item, "history")</td>                <td>@LinkFromEntity(item, "self")</td>            </tr>        }    </table>    </body></html>

view raw gistfile1.html This Gist brought to you by GitHub.

I really like this approach because it gives you absolute freedom to format things exactly the way you want them without jumping through a 1000 hoops in code. I know from discussions with Mike that using templates is really his only way to fly.

You don’t have to wait for that future post I eluded to (which might never happen) to try the code in the talk as I’ve put it on Dropbox here. You’ll find RestBugs including the Razor formatter and the client.

Generate links through middleware.

This approach is more of a hybrid of the other two. Instead of generating links in the formatter or the operation, you use an operation handler. The operation handler takes the model that is returned like ILinkable<Order> and then adds the appropriate links.

Taking the above code, you can simply move the logic to an operation handler to achieve the same.

public class OrderLinkHandler : HttpOperationHandler<ILinkable<Order>,ILinkable<Order>>{  public ILinkable<Order> OnHandle(ILinkable<Order> linkedOrder) {    if (linkedOrder.Instance.State == OrderStates.Created) {      linkedOrder.Links.Add(OrderLinks.Approval,        GetApprovalUri(order.ID));    }
    //other state logic here    return linkedOrder;  }
  private Uri GetApprovalUri(orderId) {    ...  }}

view raw OrderLInkHandler.cs This Gist brought to you by GitHub.

The pros of this is that it still allows the link logic to be used cross formatters, and it makes the code within your operation much tighter. The downside, and why I no longer prefer it is that it makes the code harder to follow. You have to know that a handler was registered which applies the links and you have to find that handler to understand the link logic.

How to generate the uris themselves

Aside from the strategy to embed links there is the question of how best to generate uris. Here’s a couple of recommendations on how you can do this.

Relative uris

If you can get away with relative uris do it.  What I mean by that is this, let’s say you have a link to an Order approval resource for order 5. You can generate a link for “/order/approval/5”. The client will take the uri and append it to the host url in order to navigate. This alleviates the need for you having to get access to the base url for the site when you generate urls.

Absolute uris

If you have multiple domains where your resources live, or if you want clients to always get the full uri so that they can simply copy/paste, use absolute uris.

For absolute uris, create a service which is configured with a map of all the available uris and exposes methods for retrieving them.

public interface IUriService {  string OrdersBaseUri {get;}  string InventoryBaseUri {get;set;}  string UspsBaseUri {get;set;}}

view raw gistfile1.cs This Gist brought to you by GitHub.

OrdersBaseUri returns “http://contoso.orders.com” the address of the order system, while InventoryBaseUri returns “http://constoso.inventory.com” for the inventory system. UspsBaseUri is a the address for a third party site for shipping .

The uri service can then get injected into the link uri service which can use it to generate absolute uris.

Your mileage may vary

As you can see from this post, there are many different ways to approach implementing a hypermedia system with WCF Web API. As building hypermedia systems for web apis is still a pretty young topic, I don’t think there’s a clear winner yet on which hypermedia design approach to use. By extension there is also no clear winner on implementation. Over time and with more people building hypermedia based systems, hopefully this will change and there will be more accepted standards.

What has worked for you? Do you have a particular approach you’d like to see better support for in the box?

For the record, I’ve never even been offered a Microsoft MVP. How’s THAT for street cred! That said, if the MVP lead in my area is reading: even though I don’t speak at user groups these days and hardly blog (and even then, rarely about Microsoft products anymore), I still feel my lack of contribution to OSS projects should count for something…

Now back to our regularly scheduled hoe down.

One of the issues with GWT apps that’s only really discussed in hushed whispers in the back alleys of Google Groups is how to handle new versions. The nature of pure JavaScript applications is a bit of a hindrance in this case.

When converting a Java application to the necessary Javascript, GWT generates (depending on your set up):

• a .nocache.js file
• several .cache.js and .cache.html files
• several .gwt.rpc files

The .nocache.js file has the same name every time you compile. But if you’ve changed any code, the cache files and rpc files will not. Here’s what my folder looks like today for BookedIN:

 

The next time I GWT-compile (provided I’ve changed some code), the folders and files in red will be deleted and replaced with new ones with different names. Among other things, the scheduler.nocache.js file is used to locate these files on demand while the app is running.

We use the Google Plugin for Eclipse to deploy our application to AppEngine. We almost always deploy to a new version in AppEngine as well so that we can play around with it ourselves before unleashing it on an unsuspecting public. The upshot of this process is that the new version will have new .cache and .gwt.rpc files but not the old ones.

So let’s run through a potential scenario:

• Our faithful user logs into BookedIN and uses the default version, which I will call “Dandelion”.
• We deploy a new version, called “Thistle” and make it the new default version
• The user makes a request for a page that is, say, behind a code-split. One of GWT’s nice optimization features that lets you split JavaScript among several files and loads them dynamically as needed.

At this point, the user has the main page and the .nocache.js file loaded in memory. When it tries to satisfy the request, it will look for a .cache.js file from version “Dandelion”. Only by refreshing the entire browser page will it then load the new .nocache.js file, which knows about version “Thistle”. But this being a GWT-type, AJAX-ified application, there is rarely much call for them to refresh the entire page.

Predictably, we get a 404 error:

This leads to some pretty nifty dancing when it comes to deployment time. For example, how do you take down the application for maintenance cleanly? If the user has the page loaded in memory and is just making AJAX calls, you can’t just throw up an appoffline.htm file and redirect all your traffic to it (says the guy who thought differently a few short months ago).

Even if you can take the app down for maintenance, I don’t want to. We’re trying to shorten our deployment cycles which doesn’t lend itself to a page that says “hey, paying user, we’re adding some cool new features so pardon us interrupting you using the old ones” every week even if it just shows for a few minutes. In short, what I’d really like is a hot deployment.

Based on my research and questions, this isn’t 100% possible but we can get close. As was suggested in the previous link, we can trap the appropriate exception in RPC calls and display a message to the user asking them to refresh their browser. Similarly, for code-split .cache.js files, we can trap the 404 error in the onFailure of the RunAsyncCallback (or better yet, use GWTP and have some main presenter implement AsyncCallFailHandler to make this easier) and do the same thing: notify the user that the page needs to be refreshed.

Initially, this kind of left a bad taste in my mouth. But from a marketing perspective, it’s not bad. We have a little popup that we display to users when we’ve implemented something new so this is a nice way to ensure they see it.

Another suggestion that was made (by one of the creators of GWTP, no less) was to use the Channel API to detect when a new version has been released. This has an advantage in that you don’t need to wait until the user does something before informing him of the change.

So far, much of this is theoretical for us because it was easier to write about it than to actually implement it. In any case, few people seem to be discussing it. Besides which, I had to write *something* after that little Microsoft MVP commentary.

Kyle the Filler

For the record, I’ve never even been offered a Microsoft MVP. How’s THAT for street cred! That said, if the MVP lead in my area is reading: even though I don’t speak at user groups these days and hardly blog (and even then, rarely about Microsoft products anymore), I still feel my lack of contribution to OSS projects should count for something…

Now back to our regularly scheduled hoe down.

One of the issues with GWT apps that’s only really discussed in hushed whispers in the back alleys of Google Groups is how to handle new versions. The nature of pure JavaScript applications is a bit of a hindrance in this case.

When converting a Java application to the necessary Javascript, GWT generates (depending on your set up):

• a .nocache.js file
• several .cache.js and .cache.html files
• several .gwt.rpc files

The .nocache.js file has the same name every time you compile. But if you’ve changed any code, the cache files and rpc files will not. Here’s what my folder looks like today for BookedIN:

 

The next time I GWT-compile (provided I’ve changed some code), the folders and files in red will be deleted and replaced with new ones with different names. Among other things, the scheduler.nocache.js file is used to locate these files on demand while the app is running.

We use the Google Plugin for Eclipse to deploy our application to AppEngine. We almost always deploy to a new version in AppEngine as well so that we can play around with it ourselves before unleashing it on an unsuspecting public. The upshot of this process is that the new version will have new .cache and .gwt.rpc files but not the old ones.

So let’s run through a potential scenario:

• Our faithful user logs into BookedIN and uses the default version, which I will call “Dandelion”.
• We deploy a new version, called “Thistle” and make it the new default version
• The user makes a request for a page that is, say, behind a code-split. One of GWT’s nice optimization features that lets you split JavaScript among several files and loads them dynamically as needed.

At this point, the user has the main page and the .nocache.js file loaded in memory. When it tries to satisfy the request, it will look for a .cache.js file from version “Dandelion”. Only by refreshing the entire browser page will it then load the new .nocache.js file, which knows about version “Thistle”. But this being a GWT-type, AJAX-ified application, there is rarely much call for them to refresh the entire page.

Predictably, we get a 404 error:

This leads to some pretty nifty dancing when it comes to deployment time. For example, how do you take down the application for maintenance cleanly? If the user has the page loaded in memory and is just making AJAX calls, you can’t just throw up an appoffline.htm file and redirect all your traffic to it (says the guy who thought differently a few short months ago).

Even if you can take the app down for maintenance, I don’t want to. We’re trying to shorten our deployment cycles which doesn’t lend itself to a page that says “hey, paying user, we’re adding some cool new features so pardon us interrupting you using the old ones” every week even if it just shows for a few minutes. In short, what I’d really like is a hot deployment.

Based on my research and questions, this isn’t 100% possible but we can get close. As was suggested in the previous link, we can trap the appropriate exception in RPC calls and display a message to the user asking them to refresh their browser. Similarly, for code-split .cache.js files, we can trap the 404 error in the onFailure of the RunAsyncCallback (or better yet, use GWTP and have some main presenter implement AsyncCallFailHandler to make this easier) and do the same thing: notify the user that the page needs to be refreshed.

Initially, this kind of left a bad taste in my mouth. But from a marketing perspective, it’s not bad. We have a little popup that we display to users when we’ve implemented something new so this is a nice way to ensure they see it.

Another suggestion that was made (by one of the creators of GWTP, no less) was to use the Channel API to detect when a new version has been released. This has an advantage in that you don’t need to wait until the user does something before informing him of the change.

So far, much of this is theoretical for us because it was easier to write about it than to actually implement it. In any case, few people seem to be discussing it. Besides which, I had to write *something* after that little Microsoft MVP commentary.

Kyle the Filler

TL;DR:  I don’t think there’s any need to give in to paranoia about the MVP program or look at it as a conspiracy or believe that it either damages or enhances your credibility as a developer.

A couple folks that are fairly visible and arguably very valuable OSS leaders or contributors didn’t get renewed as Microsoft MVP’s this year and it’s caused a minor fuss online.  After 5 straight years of receiving the MVP award for C#, I didn’t either.  No big deal, but a couple thoughts anyway:

• I was on 4-5 podcasts and spoke at CodeMash last year, but I didn’t do much writing and speaking otherwise.  I think I made it to one NUG meeting just because a colleague was speaking, but that was it.  The point is, I didn’t really do anything to meet the criteria as I understand it for being an MVP — plus I was distracted and didn’t get around to filling out the “why I deserve to be an MVP” form anyway.  I’m not exactly Mr. “Good at doing paperwork”
• I did spend a very large amount of time devoted to OSS efforts last year, but as you probably know, the MVP program doesn’t really recognize OSS contributions.  That’s an old complaint and one that I’ve raised myself in years past.  Nuget notwithstanding, Microsoft has never really embraced OSS development on the .Net platform.*  I don’t think this will ever really change so we might as well just stop getting upset about it.**
• MS apparently does have a program going where OSS contributors can get MSDN licenses and I’ll be getting myself hooked up at some point.  At some point this year I think we might be completely switched over to developing on Mono and MonoDevelop, but for now I want a working copy of VS.Net and ReSharper and I’ve got both.
• I have no entitlement to any sort of reward or recognition from Microsoft.  It’s their program and they can do whatever they want with it using any criteria they come up with.
• I really don’t need vindication or recognition from Microsoft.  You shouldn’t either.
• My personal experience interacting with Microsoft teams is mixed.  I can point to a few things in Microsoft tools that were influenced by OSS work that I’m involved in, but I don’t think that the direct feedback I’ve given over the years has ever made any difference.
• At no point did I ever feel pressured to self-censure my opinions online or at development events just because of my MVP award (probably should have anyway;)).  Neither did I particularly feel forced to toe the Microsoft line or push Microsoft tooling.  Heck, I got the MVP several times after helping author the silly EF Vote of No Confidence thing (I do think that my original verbiage was a lot less eye-poking than what got in there).  A couple people told me that I would be able to be a lot more honest and open with my opinions after losing the MVP.  I plan to do the opposite and stay out of nerd rage events like, say, claiming that FubuMVC is better than Rails just because someone did something dumb with ActiveRecord one time.

* Yes, someone is going to rightfully say that Nuget has been a huge boon for .Net OSS development, but I’ll automatically respond by reminding you that Nuget effectively trashed a pair of ongoing OSS efforts from the community.  You might also say that Microsoft has embraced jQuery and I’ll tell you that they had absolutely no chance of competing with jQuery’s ecosystem and had to do that.  You’ll also remind me that the Azure team is suddenly really active in supporting Node.js and I’ll tell you that I think the Azure team had to do that for the sake of Azure adoption and remind you that Node.js didn’t originate from the Microsoft development community at all.

** As an aside, I’m no expert on Microsoft internal politics but it definitely seems to me that the Windows team has more or less put DevDiv under its thumb in the pecking order. I think this could help developers on the Windows platform by shifting the focus squarely towards making Windows a good platform to develop software for versus trying to push a pure Microsoft created development ecosystem from DevDiv. If that means supporting tools like Node.js or Nginx or PHP or even Rails on Windows instead of just pushing their own tools, we the developers can only win. Again, I’m no Mary Jo Foley, just my 2 cents.

This post is somewhere near the border of being off-topic but I consider it geeky enough myself to share it here.

Like many others of my generation, living their high-school day in the end of the sixties/ beginning seventies, I’m a guitar kid. Despite my heroes like Hooker and Hendrix I never had more myself than a nylon stringed Spanish instrument. And love that. This holiday I tried to bridge some gaps using today’s technology.

There are wonderful apps for an iPad like Amplitube which emulates classical Fender guitar amps, complete with distortion and the like. Using an iRig you can plug an electric guitar straight into your iPad (or iPhone). A pickup does not work on a nylon strings (no, don’t get me started on Takamine) but I did have an old Aria (a brand known over here mainly for its recorders) contact microphone. The output level of a (contact) microphone is less than that of an electric guitar, but perhaps it could work with an iRig. Googling around brought up a lot of ways to build your own contact microphone (on Youtube) but no answer to my question. So I went to a real life music shop (Westerhaven in Groningen). They shared my doubt but offered to give it a try. In case it wouldn’t work I could just return the iRig. Where do you find that in a web-shop ?

And I’m going to keep it, it does work ! This is my setup:

The signal is weak, just raise the input level in Amplitube’s settings.

Of course the sound does not have the electric punch. The level of noise is acceptable with the noise filter switched off. In most settings switching that on filters out to much of the tones as well.  The tremolo of a Fender Twin is not as deep as I had hoped. But the Super-Sonic really let’s the Christmas blues scream out.

That’s it, a nice combination of classic old stuff and new technology. I’m looking forward what surprises 2012 will bring. Happy new year !

If you’ve my last few posts around the new Windows Azure SDK for Node.js and you have not been following me on twitter, then you might be suspicious. Let me remove the suspicion, I’ve moved on from the Web API team about 5 months ago to help with our Node.js on Azure story.

My ride with Web API

I’ve had a thrillingly fantastic ride being with the Web API team. When I first joined WCF after leaving the MEF team, I had a ton of folks pinging me on twitter asking me what in the world I was doing / why would I go to WCF. I told them there was an opportunity for WCF to do the right thing for customers building HTTP/RESTful APIS. I also said that we would not do it alone and we would work with the community. Sebastian Lambla (@serialseb) can attest to this as he and I chatted before I even agreed to take the job. Seb was supportive and agreed to help us along with a bunch of other folks.

Since then we think we’ve delivered on that promise. At PDC last year we released our first drop of Web API on Codeplex and published our Mercurial repo, since then we’ve been releasing as frequently as we could including on Nuget.

So far, we’ve heard that people like like what we are doing. We’ve been developing a library that really helps you take full advantage of HTTP and REST (if you desire) to support an evolving set of clients. Not only that, but it’s complexity bar is low and the simplicity bar is high. Also it plays much better with good principles of software design like separation-of-concerns, and offers good compatibility with good software practices like test-driven-development.

We’ve had a core set of advisors from the community helping us at every stage to build something useful in the real world, telling us what to do and more importantly what NOT to do. We’ve taken a ton of feedback from the broader community and driven it back to the product. Finally we’ve seen some great efforts from the community both in education on how to use it and in extensions to the core product.

<mushy-stuff-follows>

I am indebted deeply to the people I worked with on the Web API team. We accomplished a lot with the odds stacked against us. I am also indebted to our advisors who invested a lot of personal energy and time to helps guide us. I feel privileged to have worked with such stellar people, there’s too many of you to name, but you know who you are!

As for my part well, I felt like the phony on the team with all the great minds that were around me. I did my part, but the real work was of the folks I mentioned.

Web API IS still moving and it will ship

Since my last few posts and throughout my tweeting over the past few months, I’ve received a lot of questions from folks asking me “What about Web API”. Is it another dead carcass left in the framework graveyard? Is it another sacred text to be stored in the gigantic warehouse of Raiders of the Lost Ark*? The answer is an emphatic no!

* – Yes I am delusional

Yes, I’ve moved on, but Web API is moving on as well to bigger and brighter places. The team is pumped up and the product is moving like a high speed train going at 200 mph! You’ll have to take my word for it, but I believe it is headed in the best possible direction than I could have ever hoped for. I also believe it is the best outcome for you, our developers.  I can’t talk about it now, but you will be hearing about it shortly in a big announcement. I am sure many of you can imagine what that will be.  You can find a big teaser in this discussion thread.

The meta point here is if you are investing in Web API, you’ve made a good decision. And if you haven’t, well you should probably start There’s much more goodness to come!

Keep watching ScottGu’s and Howard Dierking’s blog for details on the extremely bright future of Web API!

Why Node.js?

I have been an avid mostly observer of Node.js for the past two years. I first learned about it at Øredev. It offers a simplified model for building highly scalable web applications using server side JavaScript. I talked a bit about my own perspectives on Why Node in my interview here. Since it’s inception we’ve seen an explosion of interest in Node across the industry. Recently we’ve also seen a lot of interest around making Node.js work well for Windows. We’ve worked with Joyent to make that a reality, and we’ve also look at how to better integrate Node with Windows with efforts like iisnode.

A few months back I was asked if I’d be interested in helping to take that support for Node.s to Windows Azure. Seeing the excitement around Node.js, the opportunity to make Azure a great place for developing Node.js applications, the opportunity to make Azure a truly open platform, and the bonus of doing it in ScottGu’s org, I jumped at the chance.

And so, if you look for me now going forward, you’ll find me on the Azure SDK team! Going forward you’ll see much more about the work we are doing in this space on my blog.

<more-mushy-stuff>

Thank you all for your support! I am truly grateful for the opportunities you (and Microsoft) gives me to help you build better software. See you in the cloud!

It’s been 10 days since we release our first preview of Windows Azure SDK for Node.js. So far the community reaction has been really good. Feedback is starting to trickle in with feature requests / bugs, etc. We are listening and really appreciate the feedback, so keep it coming.

Staying true to our commitment to release early and often, and in keeping with the holiday tradition, we bring you this gift, the December update.

  

Highlights of this preview:

• Windows Azure PowerShell for Node.js 0.5.1 update, including new cmdlets for easily enabling Remote Desktop, as well as SSL support.
• iisnode 0.1.13 update, including significant performance improvements (>2x throughput of the previous version)
• Node.js 0.6.6 update from Joyent, including a more stable npm
• Installer in-place upgrade support for all these components, so you can just click the WebPI link (linked from the dev center) and it will upgrade your existing installation of these components if you already have the previous version, or install them all if you don’t.
• Numerous bug fixes in all three components

Enabling Remote Desktop access the easy way

Remote Desktop allows you to connect directly to your VMs. It’s very useful in particular for debugging and diagnostics.

In our previous preview to enable RDP support to your VMs you needed to follow a bunch of manual steps which included among other things modifying config files, creating a certificate, and uploading the cert to the portal.

Now, all of the steps are taken care of for you via a single cmdlet, “Enable-AzureRemoteDesktop”!

Say these three words with me, “Enable, Publish, Connect”.

A picture tells a thousand words, so just glance at the screenshots below.

1. Enable:

2. Publish:

3. Connect:

Any questions?

The release is also accompanied by an update of the Windows Azure Node.js dev center content which includes new “How-To content” and “Common tasks”.

Enjoy (download) and have a happy holiday.js!