Chris Whitmore · Notes

Loving SourceTree

Chris Whitmore — Wed, 03 Apr 2013 07:15:00 UTC

Being a relative newbie to git I really appreciate the Atlassian's new SourceTree app.

Free
Easy to start
Deep feature set

The last few GUI git clients I've tried attempted to mask the complexity of the git feature-set whereas with SourceTree I feel like I can actually explore that feature set. I've already begun embracing branching way more as a result of the nice visualizations and as a bonus my command line git usage is also improving. Great little tool.

iTerm2 vs Terminal.app (iTerm2 FTW!)

Chris Whitmore — Sat, 16 Feb 2013 02:34:00 UTC

I just switched from the built-in Terminal.app on Mac OS X (which I've always liked) to iTerm2. Here's some of what I like about iTerm2 :

borderless windows
better scrolling and mouse support (or better defaults?) I can use my scrollwheel in vim by default and clicking on a word moves the cursor which is a nice bonus (behaves like unix)
feels faster; I was having issues with Terminal which seemed to be related to using Microsoft's Consolas font coupled with transparency and blur which was causing ever so slight but hugely annoying key lag in vim. (key lag in vim? really?)
remap the modifier keys!!! This was actually the reason I started looking for an alternative because my mac's "option" key is where I expect my control key to be and it was a source of continual frustration in vim and elsewhere. I remapped Left option to be a control key and life got better.

Check out the features page for a full list of goodness offered over the default Terminal application shipping with Mac OS X 10.8 and below... bunch of features I have not even started using yet

http://www.iterm2.com/#/section/features/expose

I am a technology tourist (or first impressions of Dart)

Chris Whitmore — Fri, 15 Feb 2013 15:35:00 UTC

So this week I met with someone who was very excited about working with Google Dart, a web programming language aimed at being a better JavaScript running both on the server and ultimately the client where the VM could run in browsers (only chromium for now). When Google first announced Dart in September '11 I thought "cool, too bad it will never work" and basically dismissed the project as doomed since it would presumably face too much pressure from competitors and open standards. After the failures of Flash, Java (applets) and most recently Silverlight it's hard to imagine Google having an ability to convince the world to abandon JavaScript in favor of Dart. Even GWT seems to be losing Google's attention somewhat. Having just been burned my MSFT on Silverlight I am overly wary of any proprietary tech these days.

But what I overlooked when I initially dismissed dart was the dart2js compiler and the fact that Dart doesn't really need the Dart VM to succeed for the language to get some traction. So skepticism temporarily put on hold I decided to port a small existing project from jQuery to Dart. The project was pretty simplistic, about 175 lines of Javascript served up statically and allowing users to navigate a full page grid with vim-like keybindings, entering free flowing text in "contenteditable" divs within the grid.

Some thoughts on the experience:

Dart editor based on eclipse is handy and the integration with chromium made the whole experience of getting up and running with the SDK really easy
Classes! Optional Types! (or ?) Libraries! Isolates! Lexical Scoping! All very nice and immediately familiar in the resulting code.. although I did find myself momentarily confused by:

lack of object notation (use Maps instead)
lack of dot access to those map properties
lack of explicit private keyword (use _prefix to indicate private)
lack of string concatenation (use string interpolation, which I actually really like)

handling of concurrency is really interesting not just in the Isolates but also in the "Future" type which makes for some really clean chaining of asynchronous calls such that you are not nesting N levels of callback functions. Didn't use them but they looked cool.
There are equivalents in the "dart:html" library for a lot of what you'd commonly reach for in jQuery, but in general I found myself writing more verbose code than what I was doing equivalently in jQuery. That said...
I had WAY fewer error cycles in Dart, and the semi-static typing made the whole conversion process take only a couple hours including all the time spent looking things up..
... and looking things up was pretty easy, you can tell it's largely from Google but they are investing enough in their docs that it was always pretty easy to get what I needed
A nagging feeling... While dom manipulation was straight forward I found myself cut off from Javascript. It looks as though calling between them is totally doable but feels hacky as soon as you go down that path. Dart itself assumes just enough responsibility that leaving the core libraries feels like foreign territory effectively isolating you off from all of the innovation happening out there in Javascript land. I may be making a mountain out of a molehill here but having to choose between Dart native libraries and Javascript community for common tasks is not pleasant.

So all in all it was a really positive experience, the tooling is solid, the language itself is very comfortable for someone with my background and I can see clear advantages especially on a larger project. But I'm now at crossroads on this little project, do I go with the comfortable tools and language with questions around the long term viability and community support for the proprietary Dart (to be fair it's open source, but open source doesn't remove the fact that this is very much a Google initiative) or do I stick with the "open web" and just deal with all the pains?

.....

After a couple days and writing this out I decided I just can't commit to Dart yet. There's too many exciting developments happening in the open web and Dart feels too much like unnecessary fragmentation of the collective efforts to improve things. Will be keeping an eye on this though, just because it's cool. The project's ambition is admirable and it feels like they are just getting started.

How does vim keep sucking me in?

Chris Whitmore — Thu, 14 Feb 2013 20:59:00 UTC

I "grew up" on vim, which is to say my second professional programming job 15 years ago required me to spend vast quantities of time in a terminal to a Solaris machine on which I used vi exclusively to get work done. In those two years I got relatively proficient at navigation, search and replacing, using registers, and tweaking .vimrc with custom settings and macros. I did NOT get into folding, window management, syntax highlighting, or plugins. Things have changed a lot in vim-land.

Watching a friend operate vim last week triggered weirdly intense pangs of jealousy and masochistic visions of how I too could suffer through horrible learning curves and bouts of frustration in order to eke out those extra few moments of productivity per day. How I too could spend hours meddling with plugins and hacks, custom scripts and a terminal to get features which every modern IDE user now takes for granted. I romantically pictured my hands rooted on the homekeys, flying through mountains of text like a boss, and the legions of adoring co-workers who would wonder at my elite skills. Totally will happen.

So for what feels like the Nth time, I re-approached my old friend vim. It was shocking how much of my muscle memory had been retained, but it was also shocking how much I miss features like "find usages", "refactor...[anything]" and quick project based shortcuts for getting around files. One of the plugins my friend was using was NERDTree, which provides a pretty nifty tree project/folder navigator as a window in vim. Almost but not quite a project view.

And when it comes to plugins, all the cool kids these days are using the social git-goodness of github combined with a neat plugin called Pathogen to create "dotVim" repositories which make the whole process of managing, sharing, and installing your perfect vim environment far far simpler. Branch a dotVim repo, symlink it into your user folder and you're off. You'll find a ton of documentation out there on how to make it work so I won't repeat it here. I based my latest attempt at vim heaven on Kody Krieger's dotVim, which is a slimmed down version of Janus which uses Pathogen. I particularly liked that installation of this is bootstrapped so you can run a single command in the console to get started: (which of course I had to inspect before running because I'm paranoid like that)

curl https://raw.github.com/codykrieger/dotvim/master/bootstrap.sh -o - | sh

Anyway, after dealing with some issues as a result of missing plugins (slim, which looks to be fixed since I installed) I was off and running. I ended up deleting the ruby specific plugins and adding a few color schemes but have otherwise only changed the .vimrc (macros, and tabs not spaces damnit! unless it's python, then spaces are ok)

NERDTree is great, but I find myself still relying on good ole :e for most things
Ctrl-P is fantastic but I find it doesn't always work, I'm guessing I need to feed more information about my context to it so that it searches the right paths. When it does work though it's basically :e on steroids.
Fugitive is amazing, and the ability to quickly pull up a diff on what you're working on is lovely. I still hit the command line more often than I probably need to.

There are a bunch of others in here that are probably making life nicer without me even knowing...still much to learn.

I quite like the git repo approach and the fact that I was able to tweak everything on my laptop and then just clone to my desktop at home was pretty fantastic. I will say though that I find the sub-modules a bit of a nuisance. I don't really need to be on the bleeding edge of vim plugins and I'd rather have full control so I removed all of the submodule aspects to the above dotVim and will just pull updates when I hit bugs or need new features. The end result of my efforts are on bitbucket.

To commit myself more to learning the editor I've also started using it for writing simple text documents as well like the government forms I'm in the process of filling out. It's fun to make a terrible job an opportunity to refine your tool skills. Vim out of the box is not great at handling paragraphs of text however and so a few tweaks are required. I have the following in my .vimrc which allows me to quickly enter "prose" mode (fixing line up line down in wrap mode being the biggest element for me)

" setup the current vim instance for writing prose style
command! Prose call SetupForProse()
function SetupForProse()
"break lines visually, and don't group hardbreaks on previous word
set formatoptions=1
set linebreak
set wrap
set nolist

"set breakat=\ |@-+;:,./?^|

"change default cursor keys to work on visual not actual lines
nnoremap j gj
nnoremap k gk
vnoremap j gj
vnoremap k gk

"border on the left hand side
set foldcolumn=7

endfunction

On with the self torture!

From GitHub to BitBucket in 60 seconds

Chris Whitmore — Thu, 27 Oct 2011 10:40:00 UTC

Three weeks ago I finally decided to pay for GitHub so that I could keep some of my new projects private without giving up the beauty that is their cloud based source control. Last night I decided to try BitBucket's new FREE offering and see whether I could save myself $7/month. Literally fifteen minutes later I was deleting my private repo's on GitHub so that I could downgrade my account back to free.

I am of two minds on this, because on the one hand I want to see GitHub succeed. They are the innovators as far as I'm concerned and I've been extremely happy using their service. BitBucket feels like a cheap knock-off in almost every way, which is great because it's familiar and easy, but I almost don't want to use it on principle. On the other hand, I would rather clearly rather pay nothing then $84 a year for my personal projects which I want private. Should any of these projects take off I will need to pay when the number of users goes past 5, which makes sense. Until then these projects don't have any monetary value anyway and so are not worth spending money on just for privacy.

Anyway enough opinion.... here was the process :

Create a user on bitbucket.org. Love that they have OAuth support - but you do still need to pick a username and password for use from the git client.

I was able to select the same username I had on GitHub which simplified moving things a bit

Under the repository menu select "Import Repository", then select github, provide your credentials and click go
Magic! you have your repo moved along with all of your commits and history. Very simple.
Go here and add your ssl key which you should already have from using GitHub (~/.ssl/id_rsa.pub)
Now for each repository you moved, on each machine you use, edit the following file:

/repo_location/.git/config

If you've kept the same username then the only change to make is the domain: (obviously change the username if you need to ;-) )

OLD: url = git@github.com:your_user_name/your_project.git
NEW : url = git@bitbucket.org:your_user_name/your_project.git
Done. Commit, push and pull as before. Your project is happily oblivious.

So far the experience has been good. No mac client for bitbucket but I was having problems with the mac GUI client anyway and always ended up on the command line. The UI is familiar and because I hadn't really gotten into the social features and other aspects of github I have noticed no real difference in service.

I wonder how long before GitHub changes their model slightly to compete with this....

Timeline tools

Chris Whitmore — Fri, 27 May 2011 09:45:00 UTC

My memory is awful, it really is. I maintain a private personal blog which I use for capturing extremely short pieces of content that I want tagged and timestamped. Works well for those random thoughts that are maybe not suitable for sharing but which I want to capture nonetheless. Anyway, I'm looking for sometime similar to aid my memory at work and have been considering something more along the lines of a timeline of sorts where I can jot down major company moments, hires etc in a format that's easy to explore and I came across the following online timeline makers which all seem to fit the mold of what I need....

http://www.dipity.com/

http://www.timetoast.com

http://timerime.com

http://www.preceden.com/

http://www.allofme.com/

http://www.xtimeline.com

Once I actually decide which one to go with I'll report back. Data portability will be important for me as I'll need to invest quite a bit of time into putting it together and would hate to lose it into the cloud. Easy of entry is probably second highest and visuals are third.

Also check out http://moreofit.com, great tool for finding competing tools/solutions once you have found one you like.

lowering impedance of TDD with python mock

Chris Whitmore — Thu, 07 Oct 2010 21:56:00 UTC

So after my post about gaeunit a few weeks ago I’ve since completely thrown out what I was doing there and moved to vanilla python unit tests. I ended up making this move for a few reasons.

1. I was never running my tests.

GAEUnit was nice, but slow. Even when running tests in parallel I still had to go through the process of opening the browser, navigating to the right place and letting the tests run. Compare this to ctrl-f9 and seeing the following:

2. AND I love seeing code coverage numbers and while it might actually be possible with GAEUnit, it wasn’t this easy…

In the end it was relatively simple once I started employing mocking to deal with sessions and realized that I could actually issue real request objects to my handlers with relative ease…

webObReq = Request.blank(url)

webApReq = webapp.Request(webObReq.environ)

handler.initialize(webApReq, webapp.Response())

For the mocking I eventually decided to go with this framework:

http://www.voidspace.org.uk/python/mock/

It has so far been fine for my needs and at least my handlers have almost achieved 100% coverage. The syntax is a little foreign compared to Moq, but hasn’t been a blocker to me actually getting work done so I’d recommend it.

Engineering Management (link)

Chris Whitmore — Tue, 21 Sep 2010 21:48:00 UTC

Great article(s) on some of the management principles in the engineering group at Facebook from Yishan Wong who was at Facebook through some very interesting growth times. I found reading this to be inspirational so posting for posterity…

http://algeri-wong.com/yishan/engineering-management.html

I just quit my job....

Chris Whitmore — Wed, 15 Sep 2010 22:18:00 UTC

Ha! No I didn’t. But starting on the premise that I had and I had already saved a bunch of cash and decided to finally become my own boss, what would I do first?

For me this is hypothetical, but for my good friend who’s about to make the leap out of full time employment to self employment it is very very real. And so I will live vicariously and imagine what I’d do.

The Goal:

build a business that can at a minimum support me before my year of savings has run out so that I can continue to build said business long enough affect real change and or make loads of cash

My Challenges:

I have enough money to sustain myself for the year, or to invest and sustain myself for less
I love to program. I love to program, and specifically solve technical problems, so much that I focus on it to the detriment of other tasks. (like writing business plans, talking to users or bathing)
I have a set of real technical hurdles to clear in order to have anything of value

My Assets:

A good idea
Mad programming skills
Friends in medium places

(caveat that this is an hour’s worth of dumping thoughts based on a few conversations and some latent thought) ;-)

My first steps:

Set some very high level goals (these can change)

Month 1 initial Product Plan is ready (see below)
Month 1 website is up, domain is secured (even if a teaser)
Month 2 delivering live usable software with weekly updates for the remainder of the year
Month 3 gut check milestone - pull the plug or keep going?
Month 4 gut check milestone - go alone or go big? do I need more money?
Month 5 gut check milestone - pull the plug or keep going?
Month 6 target - engaged user community is built and driving the backlog (uservoice)
Month 6 target - marketing (adsense? viral? user communities?)
Month 8 target - earn my first dollar
Months 9 gut check - is this a venture someone would buy? should I start talking to those people?
months 10-11 - rinse and repeat, drive drive drive
Month 12 - profit!

Make a workspace

Seclusion, powerful machine, dual monitors, natural lighting, huge whiteboard, music and snacks
block reddit.com from this network
have a laptop available for games, surfing etc - try however possible to keep these separate

Make a development environment

IDE, tools, etc
source control
continuous integration with all the trimmings

Start building!
For at least two days a week early on, or maybe a couple hours a day in the beginning besides building I would ALSO do the following…. (Product Plan)

Write myself a short Vision and Scope document which would include:

The elevator pitch, preferably in a single sentence which I would then post on a wall in my workspace
The high level guiding principles for the need I’m trying to solve, how I’ll translate that into some revenue (not overly specific), who I’m targeting, who my competitors may be and what the biggest risks are.
I would do this loosely, but I would do it. Just the act of looking and solidifying my direction would be motivating.

Build a backlog of features, ideally written in the style of “As a <<user>> I want <<to do something>> so that <<some value>>"
Make a persona or two or three based on the largest value user segments
Pick one persona who I can deliver something of value to
Prioritize my backlog with this user type in mind, and work backwards to find the minimum viable product that could actually be used by a user. This is my first milestone.
Look for ways to accelerate getting to revenue sooner.
Look around for ways to extend the 12 months without too much distraction (grants etc)

Network network network. A lot of my milestones might be easier with help. Socialize that I’m up to something even if I’m not giving away the secret sauce.
Force myself to join some local developer user groups that align to my technology choices

force myself to attend some tech talks and socialize with other technologists
guarantee that at least some evenings are spent away from my work

Refer to my 12 month road-map on a weekly basis. (Monday morning when I’m slow)

Change it whenever
Where will I deploy? How will I deploy? I only have two months before it needs to be live in some form so figure that out.
Will finding my initial target user be difficult? Start hunting early
What questions will I need to ask in order to answer my gut check milestones? Ask those early.
Are there components I should look to buy or rent rather than build? How do those align to my backlog?
How could I accelerate delivery of features? Can some things be outsourced? Should my backlog be altered to allow my investment decision in month four?
What else might I need to make a go big decision in month four? Do I need a more formal business plan? Do I need to incorporate?

Continue to build like mad!!
Profit!!

welcome criticism, be open to review and invest in process

Chris Whitmore — Tue, 14 Sep 2010 14:20:00 UTC

TLDR:

External audit and review is important
and can have value not easily measured
Be open to investing in processes even
if you think those processes are already
optimal

When someone asks you why you aren't better at what you do, how do you react? Is your impulse to defend yourself? Do you look for comparables? Do you start to question yourself or your team? Can you begin to break down what exists today and imagine it built up in a new form?

I work for people who are largely from a non-technical background. These same people are investing eight digit figures in our teams and products. Naturally they need to know that those dollars are being spent wisely, efficiently... maximized.

For myself, facing this question and having been so recently embedded in those same teams being questioned, I admit my impulse is to defend my compatriots. I have a lot of trust in the people I've been in the trenches with and to a lesser degree trust in the process we've put in place there (which is in a word scrum). This defensive impulse is well intentioned, but in my mind definitely the wrong place to start. Of course I need to defend my peeps, but at the same time I need to be completely open to inspection, evaluation and change. (left brain fighting right brain)

The inspection and change is taking the form of the following:

A few day audit by a high priced and well networked consultant with a ton of experience in software
Scrum Training for our whole team
A week long more technically focused audit and coaching session

I was reticent about any of these at first, concerned that we were wasting money on a problem that could be tracked down to an inability on my part to assure my employers of the quality of our group. Despite my internal struggle with the idea of subjecting my teams to external audit, and the spending, and the potential for fallout, I acted as best as I could to be open and welcome to the idea, knowing full well that anything else can be suicide for the group. Before I understood the value in these exercises I already had a sense for the political dangers in what closing myself off to them would mean.

It's a few weeks later now and we've already largely completed numbers 1 and 2. I have to say I'm pretty happy about the value for the money and the impact on us. For 1, it has been somewhat shocking to see the informal chat like approach the contractor has taken with us but it really has been somewhat shallow. Honestly what are you going to learn in a couple days anyway? My guess is that it's all going to be about structure, alignment of views and ultimately focus for the company. Very high level stuff. Despite the shallow nature of the actual time spent with this guy though, there was a very focused frenzy of activity to prepare for what we thought the audit would look like. In particular the questions ahead of our interviews and requests for particular types of documentation had me seeing our knowledge and goal management in a completely new light. The todo's that I have as a result of preparing to be audited are immensely valuable. Another less tangible benefit was socializing the idea of the audit itself. I don't necessarily want us to feel complacent, and having someone external come look at you is a good way to fight that.

The biggest surprise for me has been the training however. And maybe we were just lucky, but the scrum training we had was fantastic and had so many positive side-effects that I didn't anticipate. Here are a few examples...

1. The value of common terminology. Simply having the same vocabulary does wonders for conversations around process and expectations. Not that we're 100% of the way there, but it really does make a huge difference in day to day conversations across our larger team and product management.
2. Bottom-up process improvement. The teams know best, and the learnings and suggestions coming from the people who are on the team are invaluable. The training really gave the team a collective voice to management that may have been lacking.
3. Hope/Change/Inspiration. Maybe these are separate points, but they are very related. Introspection and seeing the potential for positive change leads to hope which leads to inspiration. People were positively jacked coming out of the training sessions and very eager to put into action the incremental changes they discovered through the process.
4. Socializing. The group had basically two uninterrupted days together to socialize and work together on something other than the daily grind. Everyone got a little closer as a result and had this common vision they could work towards on leaving. So hard to measure, but very important.

What's really amazing about the impact of the this is that the training was on a process we were already using. And in fact at the end of the day there was very little genuinely new learning. It was all small incremental change and having everyone look at the problems with fresh perspective and a critical eye.

I think for me this highlights what would be my biggest challenges in running a business which is an openness to spend money. It's true of my house (in which I've only invested the bare minimum) and it would be true of my company I fear. Dr. Deming had a critical phrase for an accounting centric decision making process that I always loved "managing by the visible numbers only". As much as that resonated with me when I read his book a decade ago, I still catch myself getting caught up in the visible numbers and missing the value that comes from investments like this training and audit.

In any case, next time the external questions swirl around I'll definitely be a lot more open to what I may not know.

blogquotes version b02.1

Chris Whitmore — Tue, 24 Aug 2010 21:50:00 UTC

New release of blogquotes is out, with the biggest change being a switch from exclusive reliance on google accounts for login to leveraging a cool service at http://www.janrain.com/ that gives me a simple way of integrating authentication from a bunch of different sources. So you no longer need to be a google user to use blogquotes!

Note that if you were previously using google to login this should just work albeit with a slightly different UI. And if you find other methods to be more preferable they should also “just work” provided that you use the same email address for those services. Note the logout button at the top of the page which will tell you what email or ID you’ve logged in with. So for example if I login with facebook all my old quotes are there, but if I login with blogger which doesn’t use the same email address I get nothing and am essentially starting fresh.

- enhancement : support for Facebook, Google, Windows Live ID (includes hotmail), Blogger, Wordpress and LiveJournal logins (have yet to validate Wordpress + LiveJournal)
- bugfix : javascript remote inclusion could still barf on some unicode, added clientside utf-8 handling

In the background I’ve also made a number of restructuring changes to the code which should help me release changes faster.

Please leave comments on this post if you are having trouble getting access or for some reason can’t see the quotes you’ve already entered. (no data has been deleted!)

Unicode woes and Python unit testing in GAE

Chris Whitmore — Fri, 20 Aug 2010 17:50:00 UTC

One of the really cool aspects of deploying to Google’s cloud offering (GAE) versus the more machine oriented Microsoft Azure and Amazon EC2 approaches are that you really are only dealing with computing resources. You deploy your app not to any particular server, but to the cloud itself. Despite the very real challenges in distributing work across data centers I am still filled with visions of automagical propagation and distribution and unlimited elastic computing. Beautiful. Anyway I was inspecting the logs and was SHOCKED to discover that there have been close to a thousand quotes added to system by 37 users. THIRTY SEVEN USERS!! Now, these are small numbers I know, but given the only publicity I’ve ever given this has been the two posts on this blog I was absolutely amazed to find that people had not only managed to find the app but were able to use it! (It’s quite ugly)

There is nothing quite as motivating as having users.

So I started looking at log files and discovered that I am actually throwing errors for at least some of those users who are not English speaking and are using Unicode characters. Oops. You’d think I’d know better by now. See : The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

In most cases for these bugs it just meant calls to unicode(var) rather than str(var) . So for example when parsing url variables starting with the example provided here by Google and adding my own processing of an array of arguments to a method like so…

  def update_field(self, fieldName, *args):
   strings = [str(arg) for arg in args]
   ….
   return

and then calling that code with a unicode character such as ‘’–" would produce an error that looked like this…

UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\u2013’ in position 0: ordinal not in range(128)

the same code as above corrected is below. There were a few variants on this but this is a good example of a function where user defined input dictated a change to accommodate.

def update_field(self, fieldName, *args):

strings = [unicode(arg) for arg in args]

….

return

So in the course of digging in and finding/fixing these bugs I also recognized I was far overdue for some unit testing. So starting with the premise that I needed a way to reproduce the issue in the lab and write a failing test for this bug I went in search of the best method to do so. Python actually has [pretty good unit test support] built-in but I was also looking for something that would work in the context of GAE. It doesn’t take much looking before stumbling on GAEUnit, which is a useful extension that provides some scaffolding for tests that will run within GAE. The major negative to this is that your tests are part of your application, but I don’t have a problem with that as it’s easy to secure them to administrator access only but for some people it’s a block.

The next issue I ran into with this was the fact that the python SDK for GAE doesn’t include a method for testing content behind authentication. So whereas in Java you can say this :

private final LocalServiceTestHelper helper =
        new LocalServiceTestHelper(new LocalUserServiceTestConfig())
            .setEnvIsAdmin(true).setEnvIsLoggedIn(true);

 public void testIsAdmin() {
        UserService userService = UserServiceFactory.getUserService();
        assertTrue(userService.isUserAdmin());
    }

But today there is no equivalent for that in python. I wasted a few hours trying to work around this issue before deciding to just login before I run tests. My login lasts for hours so this isn’t much of a constraint. In the long run I plan to drop Google’s authentication any and move to janrain/rpx to allow users to login using whatever they like. That will mean abandoning or at least shimming the infrastructure provided by GAE, but has the additional benefit of reducing my applications direct dependency on Google. Once I have my own definition of Session and session provider I can of course mock it out and control it a lot easier.

Here’s the test, no asserts but if there is a problem we’ll see an exception

  def test_unicode_to_json_error(self):
   badQuote = q.Quote(quote=‘hi there ’ + u"\u2013")
   result = badQuote.jsonSafe()
   simplejson.dumps(result)

So one error-type in the log file later I’ve spent a couple days, done a bunch of refactoring and have produced no new functionality… but I do have this!

Of course then, after all that I STILL ran into an error with the JavaScript, for which I have yet to add unit testing for. Sigh. I’ll have to choose between qUnit which I’ve used before and YUI-test which is probably better aligned to the rest of my application.

The JavaScript portion of this bug was basically because I was calling unescape() on a url-encoded variable. URL encoding though only supports 8-bit characters and doesn’t truly allow unicode, so my unicode characters became three ascii characters which could include control characters or other undesired results. There seems to be a lot of people standardizing on UTF-8 within url-encoded variables but it’s purely a convention at this point. Thankfully I found this pre-written code on webtoolkit.info that knows how to do the look ahead at the next three characters in order to handle UTF-8 input properly.

Now I hope it’s safe to say that blogquotes can handle unicode! (and has some unit tests to boot!)

Blogquotes version b01.2

Chris Whitmore — Mon, 09 Aug 2010 01:15:00 UTC

Didn’t expect to get more released this weekend or I would have combined these posts, but another quick release of blogquotes, the following are added :

- enhancement : quotes table is sortable by either “who” or “quote”
- enhancement : click to edit on any cell in the table for convenient fixes
- bugfix : missing robots.txt

In the background I’ve also added analytics to the header so I can get a sense of how much traffic the site is seeing.

The table features have both been things I’ve felt missing and which were remarkably easy to add since I’m just leveraging the Yahoo User Interface library YUI for major widgets like the table. Lots of low hanging fruit now that the guts are in place.

Blogquotes version b01.1

Chris Whitmore — Sat, 07 Aug 2010 13:03:00 UTC

A new version of blogquotes has been published this morning. Not a lot of new features, but at least the wheels are turning again and some annoying bugs have been addressed.

- bugfix: adding quotes with some types of punctuation caused page errors
- bugfix: unicode characters in quotes caused errors when pulling those quotes out for includes
- bugfix: hitting enter on the “who” input field caused a page error, now hitting enter is equivalent to clicking “Add”
- enhancement: updated to a new faster version of the YUI data table
- enhancement: moved quote entry form above the quote table to avoid scrolling down when adding to an already long list
- enhancement: adding a quote will blank out the form and put focus back to the quote textbox
- enhancement: more obvious navigation in left column for moving between managing and including quotes
- enhancement: minor cosmetic improvements (title bar, fonts, spacing etc)
- enhancement: new welcome page that includes more than the word “Login”
- degredation : banner ad now in right column (sorry… appengine ultimately is not free and I’m worried about potential rising costs)

Please comment on this post if you use blogquotes and want to see specific features.

curiosity killed the addictive gamer in me

Chris Whitmore — Wed, 30 Jun 2010 00:25:00 UTC

I can see it now. Homeless, slightly demented and jabbering about inane details of imaginary characters. This will be my life in six months from now after suffering through a debilitating addiction. An addiction to the ridiculous time suck that is world of warcraft (aka world of walking). I’m almost embarrassed to admit that I have now played through the 10 day free trial (damn you free trial!) and have paid to play for another month. What happened?! How could I go from so jealously guarding my time to pissing it away into the ethereal fantasy world that is the “world” of warcraft?

And yet…. And yet, I am completely fascinated. I’ve read a few articles about professors studying the economics of the game, and about the staggering number of people (11 million?) who play the game but never really understood the allure. In fact, just the idea of playing games “online” is largely unappealing to me. And by online I mean amongst hoards of teenage kids and angry shut-ins throwing slang at each other. I mean internet gaming, the kind where you show up and get completely slaughtered before you even know what’s going on. LAN gaming on the other hand, with a room full of people you knew and could taunt… that’s a different story. How old am I?

I got to witness my young (11) cousin not too long ago playing on his XBox. It was eye opening to see the way his generation has adopted online anonymous gaming in a way that I have yet to really get to. Jump online, form casual alliances, piss some people off, play basically completely ad-hoc and move on to the next one. It’s social too, with banter and chat being casually tossed around. I still didn’t really get it though. (Is this just the introverted programmer in me?)

[time passes]

I started writing this post in December, and after five months of not playing the game I am all of a sudden well into this thing now. I had to stop playing after the holidays were over just to get back into work, and once I stopped I stopped. Course all it took was to login again after a stressful week at work and I was back at it. No different than any other game I suppose except that this game has no ending. I really can see how this game could destroy school, relationships, careers. Thank god I already have a family.

And I have to say after a few weeks the social elements are starting to be far less baffling to me. I still find that given constraints with kids in the house and limited time windows I rarely have the opportunity to commit to a game socially, (very poor form to just turn off the game midstream like I do when I play alone) but having now run a few quests both with planned partnerships and adhoc allegiances I am starting to really get the appeal. It’s amazing too the diversity of people online (teens to doctors, moms to executives), and you have an ability to really choose who of that diverse group you actually interact with for the most part. The game includes two main modes of play, either player vs player or player vs environment. Both are collaborative, but PvE is primarily groups of humans versus AI and bosses. I may yet play PvP but I fear that’s where you really lose control over your experience. I don’t relish the thought of being ambushed by a group of players who’s sole purpose is to run around finding people to annoy/destroy/humiliate. And even when playing against the environment you still have arenas in which you can pit yourself against other players which honestly is enough for me.

Biggest distraction for me so far? The remote auction house on my iPhone. I can be walking to a meeting and purchase or sell gear for my toon, check out upgrades etc. It’s insane. I lament the thought of what these hours amount to in terms of lost productivity to global economies. ;-)

Anyway this is way off topic but I’m cleaning up old posts and thought I’d get it out. For the record I can’t yet say I’d ever recommend playing to my real-life friends, but these MMORPG worlds are insanely rich and engaging, not to mention wildly popular.

Flying at the right altitude - advice to a slightly younger me

Chris Whitmore — Sun, 30 May 2010 23:06:00 UTC

No posts in five months! Almost exactly the same time I’ve been in a new position at work.

Disturbing trends, this completely predates just the last six months….

I’ve moved from being a team lead to being the head of our R&D group. I now have as many teams (7) as I did people to think about, and a whole new world of politics, strategy and planning. With a group of this size HR issues seem to be at least a weekly occurrence and I am now fully and completely on the manager’s schedule.

However now that I’ve somewhat got my sea legs in this position I am increasingly allowing myself to get back into technical issues where I think I can add value. “Where I think I can add value” is the whole challenge here though. I have literally not opened visual studio in five months of running around like a chicken with my head cut off. How often does having the boss swoop in and give their less informed perspective do more harm than good? Impossible to answer without knowing the boss in question of course, but surely there are ground rules some sage masters can teach me.

I have to say I really enjoy looking at things from this altitude, in that you really can see patterns in development across teams, opportunities for reuse or better design across teams, efficiencies to be gained etc. It all can be very fun, but at the same time I worry about losing relevance, and I worry about sidetracking teams. Even worse is the feeling that by interfering I actually hurt the autonomy and self-sufficiency of these teams by interfering (which almost outweighs the fear of divergent or teams).

Here are some things I think I would tell the me of six months ago: (advice to a new executive)

Don’t stop the old routine completely. I actually think a lot of my comfort these days is precisely because I’m getting back to things like my reader and blogger. Just imagine how nice it would be if I could do the occasional code review or bug fix! (unlikely) My life has gotten decidedly more stressful (father died, promotion, second child all within the last six months) and my routine has out of nowhere incorporated quite a bit of gaming all of sudden. (COD2 MW, WoW) Good stress releases but I need to get myself back into creating rather than just consuming things.
Embrace the manager’s schedule, but respect the maker’s schedule. It’s a very tricky thing not to let your new found schedule dictate awkward length and times for meetings with the people who are there to actually do the development. Resist the urge and find ways to be more effective with meetings. It is SO easy to forget the real costs around this.
Don’t seek consensus on every decision. It can work on a team of seven or eight to drive for consensus and try to win minds but on much larger teams you’re going to have to get used to making some calls without the comfort of knowing you have everyone on board. Of course reasonable efforts still need to be made but it becomes about key people and finding the influence in the group.
Be open and give status frequently. Still one I struggle to do properly but it really is key. I am trying right now to really focus on problems at the request of my boss, but don’t forget to acknowledge the wins as well. The more open you are about the warts the easier it is to excise them. In my opinion, maybe counter intuitively, more shit should flow up and more sunshine down. (though that’s an easy one to over generalize)
Trust and trusting your gut. Not really anything new to the position, but the frequency with which I’ve run into this has gone through the roof. You often just know the right course of action. Waiting to find out the hard way (or expensive way) that you were right can be painful. Trusting yourself is only second to trusting your people. Trust your gut on your people, make sure you have the right people and put that trust in them.
There will be a lot of water sprayed at your back, be the duck. Or, in other words, find a way to regain your slack.

google chrome software updates make everything else feel broken

Chris Whitmore — Mon, 14 Dec 2009 22:59:00 UTC

I am growing more and more annoyed at the Apple's and the Adobe's of the world who are constantly interrupting my work to tell me that there are updates waiting for me to install. Why do I have to manage this? Yes I know that I can go in there and tweak the settings so that I don't get annoyed... but why should I even have to do that? I would need to do that across every user account on every machine I use on a regular basis! (5) This is noise, and it isn't at all necessary for me to have to think about it.

I believe this is part of the convenience of applications delivered in the cloud. (sorry to throw that term out there) It is part of the convenience that has me accepting fewer features in order to get that functionality.

There are companies out there who understand this, and are working hard to create a better user experience. Google for example went to great lengths in order to be able to update their entire browser in under 100 KB of download just to make the experience more efficient, user friendly and safer. Not only that but chrome updates happen automatically, again to help with security. It's very close to the experience I get by logging into gmail, which is always up to date. Now I can already hear rattles of complaint about "control" over your own machine, but I have to ignore this or I'll get completely derailed. I'll say in short that I think that control is an illusion, and the alternatives are worse.

If you have not already read this post by the chrome developers I highly recommend it.
http://blog.chromium.org/2009/07/smaller-is-faster-and-safer-too.html

And then you have the Apple, who has gone from a position of pushing hard to avoid needing to reset the operating system unless absolutely needed to requiring reboots for Quicktime, Safari and sometimes even iTunes (especially in windows). You can argue all you want that it's because of kernel extensions or who knows what else - but at the end of the day it doesn't matter why. This is bad design and leads to a diminished user experience.

It seems like more and more often I am prompted with a software update dialog that looks like this one. Three out of four of the updates require a reboot! Are you serious? And chances are good that I will continue to ignore this dialog because closing all of my browser sessions, and ending the long running handbrake encode, and stopping the download I have going, and turning off the TV program I'm watching all amount to a whole lot of trouble for something I don't think I should even have to think about.

Seriously what is so special about Safari that they require me to download 36.2 MB and restart my machine for a point release?? Maybe this didn't seem so offensive before Chrome, but now it feels dated and clunky.

I really should pick on Adobe here for the horror that is the Adobe Updater, but honestly I really do expect more from Apple. Adobe is too easy of a target.

Please fix this Apple, you can do better. Where is the Steve Jobs who wanted to save lives by reducing boot times??

build it (so it's easy) and they will come (make the right decision)

Chris Whitmore — Sat, 05 Dec 2009 22:59:00 UTC

One of the biggest lessons I think I’ve learned over the past few years is that you have to be very careful with what you make easy to do in a software system.

When you are working within a preexisting system, it is very hard to work effectively outside the bounds of that system. Whether you are limited by time constraints, peer pressure, political decisions or just pure technical inertia, those early/legacy decisions in a system will have long reaching impacts on the decisions of those who follow.

I’ll give you an example. On a product I worked on for years, the decision to use an object relational mapper (ORM) in early stages was based on a desire to eliminate boilerplate code, reduce the learning curve for new developers and generally push the development of new entities down to the entire team rather than specializing this role in one person. All in all the reasoning was sound, but the inability to see some of the psychological aspects that would impact the future had some serious impact on the future of the system.

Developers stop thinking about database impacts because they never really SEE the database
The object model and the schema become inexorably tied
Accessibility to the DAL ends up being given to junior developers who may not have otherwise dealt with it yet.**
Things that could reasonably be built OUTSIDE the ORM end up dropped in without consideration because developers are following the template.

This can be avoided by having data contracts that are specific to the DAL
** There is nothing inherently wrong with ORM, and if your DAL is properly abstracted so that the ORM isn’t propagated throughout the stack then this too isn’t necessarily a problem.

I add those two caveats because I really don’t have an issue with ORM, in fact I think used properly it makes way more sense then to waste days of development doing repetitive and simple CRUD work.

However the deeper issues that arose for us still focused on convenience. It was convenient to expose the friendly querying methods of the domain objects that mapped to our tables directly to the business logic assemblies. It was convenient to let junior developers write code that accessed those objects as if retrieval and persistence were magically O(1) operations. Of course in reality we discovered embarrassingly late that we had more than a few graphs of objects that were being loaded upon traversal, leading to a separate mapper triggered SELECT for each object and its children. This is the kind of thing that only becomes apparent when you test with large datasets and get off of your local machine and see some real latency.

And yes, in this case, I think you could argue that QA dropped the ball here. But as a professional software developer you really never want to see issues like this get that far.

I’ve picked one example, but there are many many others in the system I’m referring to. Including but not limited to a proliferation of configuration options and files, heavy conceptual reuse of classes and functionality that are only tangentially related to each other, and an increasing reliance on a “simple” mechanism to do work outside the main code base.

Ultimately, this post has less to do with ORM and proper abstraction and more to do with understanding how your current (and future) developers will react to those decisions. I think a conscious effort has to be paid to how a human will game your system. You need to come up with penalties for doing dumb things if possible, and the path of least resistance for the right ones. There are entire books dedicated to framework and platform development that encompass some of these ideas, but they apply at every level really in my opinion. (except maybe the one man shop?)

way to go, LRO

Chris Whitmore — Wed, 23 Sep 2009 23:36:00 UTC

LRO does it again, water on the moon! That’s so cool! NASA is important people, we’re laying the foundation for future generations here.

http://www.space.com/scienceastronomy/090923-moon-water-discovery.html

And of course finding water is not the same as finding lakes, but imagine the potential for fuel sources and or human sustenance. Water is damn heavy, and not something we can easily take with us.

Pretty cool stuff.

hanselman tools 2009!!

Chris Whitmore — Wed, 02 Sep 2009 20:58:00 UTC

Saw this on reddit tonight, hanselman has updates his legendary tools list for 2009. So what was going to be an evening of actual coding is slowing turning into an evening of trying out cool new tools that have made his list. (I’m writing this blog post in windows Live Writer after seeing it in the list)

But what’s an hour or two of my time compared to the time that must go into compiling this list? Totally worth checking out, especially if you’ve never had a look before.

It’s nice, my mac envy always takes a slight dip in moments like these.

LRO sends us some underwhelming evidence!

Chris Whitmore — Tue, 01 Sep 2009 23:07:00 UTC

I remain a huge fan of projects like LRO, and personally still believe that the disbelievers are crackpots but I also have to admit to being a little underwhelmed by the photos listed here on NASA’s site for the Lunar Reconnaissance Orbiter.

http://www.nasa.gov/mission_pages/LRO/multimedia/lroimages/apollosites.html

Still, I’m excited “we’re” (go NASA) going back, and if anything this just really highlights for me how damn big our “little” moon is. Easy to forget that the satellite that’s taking those photos is still 50 kilometers! away from the surface, so somewhat understandable that we’re not getting the close ups I’d love to see.

There is a pretty cute video of the LRO launch party here (check side bar) that is worth checking out. The highlight for me was seeing the communications in action, the laser ranging support for the data coming from LRO looks like a large robot with big green blinking eyes.

Congratulations and good luck LRO team, if you can now just convince Sarah then your mission can surely be called a great success.

lessons learned from online gambling - predicting scalability

Chris Whitmore — Tue, 01 Sep 2009 22:42:00 UTC

I work with someone who has spent a few years working for an online poker company who shall remain nameless. This company was responsible for a poker platform that supported both their own branded poker offering as well as being an engine for other companies who would layer on their branding. My colleague played an important role in taking their fairly well built existing system from thousands of users to tens of thousands of users, and in the process exposing a large handful of very deep bugs, some of which were core design issues.

Looking at this site http://www.pokerlistings.com/texas-holdem and seeing just this small sample of some of the top poker sites is a bit insane. We’re talking close to 100,000 concurrent players at peak hours JUST from the 16 top “texas hold-em” sites listed here. Who are these people? (I’m totally gonna waste some money online one of these days by the way) I’m sure this is just the tip of the iceberg too.

It’s a great domain for learning critical systems in my opinion. Real time, high concurrency, real money, third party integrations for credits and tournaments and the vast reporting that goes on for all that data being generated.

My colleague’s experience in dealing with real time load and breaking the barriers of scalability are truly fascinating and a good source of learning for me. While I realize there are bigger puzzles out there, but it’s not every day you have direct access to that experience where you work. In any case, one such learning that I am in the process of trying to apply is a more forward looking approach to load modeling. That is, rather than to simply design and test for scalability; to actually drill down into the theoretical limit of what you are building in an attempt to predict failure.

This prediction can mean a lot of different things of course, being on a spectrum with something like a vague statement about being IO bound to much more complicated models of actual transactions and usage to enable extrapolating much richer information about those weak points in the system. In at least one case, my boss has taken this to the point where the model of load was expressed as differential equations prior to any code being written at all. Despite my agile leanings I have to say I’m extremely impressed by that. Definitely something I’d throw on my resume. So I’m simultaneously excited and intimidated at the prospect of delving into our relatively new platform that we’re building in the hopes of producing something similar. I definitely see the value in at least the first few iterations of highlighting weak points and patterns and usage. How far I can go from there will be a big question mark.

For now I’ll be starting at http://www.tpc.org/information/benchmarks.asp and then moving into as exhaustive list as I can of the riskiest elements of our system. From there I’ll need to prioritize and find the dimensions that will impact our ability to scale. I expect with each there will be natural next steps to removing the barrier (caching, distributing load, eliminating work, etc) and I hope to be able to put a cost next to each of those.

Simple!

sometimes it's helpful to think about what NOT to do

Chris Whitmore — Tue, 01 Sep 2009 21:52:00 UTC

Came across this list of “anti-patterns” on wikipedia tonight. I’m tempted just to copy and paste the contents here but that would make me feel dirty.

Definitely a good list though and something worth reminding ourselves of every once in a while when thinking about the systems we build.

http://en.wikipedia.org/wiki/Anti-pattern

the rise and fall of myspace (and twitter)

Chris Whitmore — Fri, 07 Aug 2009 21:11:00 UTC

This is a great post on how myspace rose and fall and how the same thing applies to Twitter (and I’d imagine Facebook as well) Some really good thoughts. Getting popular before you have your mission can forever trap you into that identity vacuum where popularity is everything.

http://codybrown.name/2009/08/06/myspace-is-to-facebook-as-twitter-is-to-______/

A good read, and the level of blogging I’d like to work towards (more pictures!).

manager schedule vs maker schedule

Chris Whitmore — Thu, 23 Jul 2009 01:09:00 UTC

Popular comp-sci essayist and lisp hacker extraordinaire Paul Graham recently posted this article on the difference between a manager’s schedule and a maker’s schedule. This is really inline with my own views on this issue and really sums up a big problem we have where I work with meetings being scheduled with the makers and the impact that has. We’ve had tons of discussions around the cost of mental context switching, but even that’s an understatement of the problem…

Great work, it’s always helpful hearing echoes of these types of thoughts beyond my own everyday sphere. Who knows maybe I can use to add some weight to my arguments.

http://paulgraham.com/makersschedule.html

performance tuning to an insane level

Chris Whitmore — Wed, 22 Jul 2009 15:22:00 UTC

Ok, so I have to admit that I’ve been one to disregard figures around performance when arguing with co-workers over the merit of managed code vs C/C++. I’ve even used the argument that statically typed languages like Java and C# offer more hints to the compiler that allow for optimizations not possible in unmanaged code. I still have a fairly pragmatic view of the spectrum of cost to deliver (skill set/maintainability) vs performance gains… but regardless of all that….. wow this article completely humbled and inspired me.

I don’t know shit.

http://stellar.mit.edu/S/course/6/fa08/6.197/courseMaterial/topics/topic2/lectureNotes/Intro_and_MxM/Intro_and_MxM.pdf

taking a step back/up/sideways (thebrain)

Chris Whitmore — Mon, 11 May 2009 22:39:00 UTC

Around 2003, 2004 I had a bit of a mild obsession with organizing my life into digital form, creating as many mappings as I could from my everyday existence into some kind of digital form. This of course included ideas and thoughts, writings and paintings, music and movies, friends and bits of information about those friends etc. This amass of data have gone from one disjointed medium to another (.txt files in folders, emails, blog posts, napkins) without ever really achieving any chohesiveness or real improvement in my ability to actually synthesize or act on all that information.

At that time the closest tool I found that could come close to mapping ideas and thoughts in a way that made sense to me was “TheBrain” a piece of software intended for visualizing information, and as importantly, the links between that information. As a user, you add “thoughts” to your brain which become nodes in a large graph of ideas, thoughts, attributes, urls etc. You can then very quickly build child, parent and sibling links between those nodes to add more context and information. Those links can then be categorized to add even more context to the relationship between ideas which is often very important information. Unlike a lot of “mind mapping” tools I’ve used though, the brain does not force you into a tree structure. Your thoughts can have multiple parents and siblings or “jumps” to thoughts anywhere else in your brain, whether they are directly related or not. Hyperlinks, imagine that! I think for the actual exercise of brainstorming and free flow thought this is critical. It also mimicks how I visualize my own thoughts working. I’ve since tried personal wiki’s which give you a lot of the same flexibility but lack the very fast keyboard entry for connections and nodes and force you to think at the level “inside” the node, by editing the page. Whereas the brain allows you to think and work at the level of the topology, which is really effective.

I stopped using the software basically because of the overhead of attempting to keep the futile mapping exercise up to date. Imagine if every idea and thought you had needed to be compulsively cataloged manually into a program in order to keep the overall picture intact. It just doesn’t work, at least not for me, and not for long and detracts from my overall goals. I found that every time I opened my brain there was just too much catchup work to do in order to get things synced up and in order.

The other thing that changed for me since 2004 has been search. Between Google and Spotlight on my mac I basically stopped categorizing information in the same ways I used to have to. It becomes less and less necessary to build nested categories of programs or emails or documents etc. Search has exposed the entire hierarchy in a glance, in many cases keeping the context intact that would have derived the categories. Still there is value in the information of that structure, but in a far less visible way.

Well I’ve resurrected the brain and am using it in a new way that seems to be actually working for me. For starters I don’t keep any notes or real content in the brain, this is one of the clumsiest facets of the tool and almost seems like an afterthought. It really is all about the topology, which for me is fine since the bulk of what I need is often in dedicated stores (subversion, sharepoint, team foundation server) I’ve found any attempt to use the clumsy palettes and data entry forms to just be too cumbersome to bother with.

Focusing on the nodes and connections, and learning all the keyboard shortcuts have enabled me to use the brain in a way that is a lot like I sometimes use notepad when I need to brainstorm. Except rather than dashes, asterisks and plus-signs I’m using jumps, parents and children of small snippets of text. It’s really a cool feeling being able to navigate this very large graph of interrelated concerns and ideas with very little effort. Wiki’s and other hypertextual forms have served me well too, but never with so little impediment.

For high level thinking and free form brainstorming this tool is the best I’ve used. And provided I keep it to just the free flowing jumping and navigating it’s incredibly useful.

Simple Extensibility in .NET

Chris Whitmore — Fri, 08 May 2009 22:03:00 UTC

I’ve used this approach a few times when I essentially need a really simple plugin / provider model within my applications so I thought I’d jot down the relevant details here for posterity using an old project for adding post commit hooks to subversion.

Consider this a somewhat simplistic approach, not suitable for production code without a bit more plumbing. If you are going all out and need true add-in’s for your .NET based product I recommend checking out the managed add-in framework , very robust stuff and not that hard to implement. In a lot of cases though the isolation, discoverability, communication pipelines etc are a bit overkill. The example I’ll show is a subversion hook that allows for very simple addition of new .NET “actions” to execute on PostCommit. In this case the “add-ins” are only written in house, and editing a config file to hook them up is completely acceptable etc etc.

The solution

Subversion.Contracts : This project is the bridge between our dispatcher and the plugins that will do the work.
Subversion.Plugins : Any of the actions we wish to take post commit are added here, but could just as easily be distributed across as many assemblies and projects as necessary as long as they reference the contracts.
Subversion.Dispatcher : This is the console application that actually receives the arguments from subversion and translates them into our contracts, then executes the appropriate actions (note no references to the plugins project)

The Contract

The contracts are relatively simple, but whatever you put in them this is the interface for the “plugin” that will need to implement. In our case this is IPostCommitHandler :

using System;

namespace Subversion.Contracts
{
 public interface IPostCommitHandler
 {
 void ExecuteCommand(PostCommitArgs a);
 }
}

Pretty simple, essentially just a “do whatever you want” method that passes the arguments from subversion wrapped up in a simple class. See the attached zip if you want the guts of the subversion specific stuff.

The Plugin

using System;
using Subversion.Contracts;

namespace Subversion.Plugins
{
 public class ExecuteForAllCommits : IPostCommitHandler
 {
 #region IPostCommitHandler Members

 public void ExecuteCommand(PostCommitArgs a)
 {
 SendEmailNotification.SendEmail(a.Argument, a.Revision);
 }

 #endregion
 }
}

Again, very simple and in this case we’re passing off the execution to a static class that again is not shown, but what gets executed isn’t all that important in this case.. simply fill in what you need.

The Dispatcher (Plugin Host)

using System;
using System.Collections;
using System.Text.RegularExpressions;
using System.Configuration;
using Subversion.Contracts;

namespace Subversion.Dispatcher
{
 ///
 /// 
 /// Summary description for PostCommit.
 /// 

 class PostCommit
 {

 private static string subversionPath = ConfigurationSettings.AppSettings[“SubversionPath”];

 static void Main(string[] args)
 {
 SubversionRevision rev = ParseRevision(args);
 ArrayList commands = DispatchGlobalCommands(rev);
 DispatchNamedCommands(rev, commands);
 }

 private static SubversionRevision ParseRevision(string[] args)
 {
 SubversionRevision rev;
 if (args.Length == 2)
 {
 rev = new SubversionRevision(subversionPath, args[0], args[1]);
 }
 else
 {
 rev = new SubversionRevision(subversionPath, string.Empty, string.Empty);
 }
 return rev;
 }

 private static void DispatchNamedCommands(SubversionRevision rev, ArrayList commands)
 {
 string[] commitLines = rev.CommitLog.Split(Environment.NewLine[0]);
 // Handle Named Commands
 string registeredCommands = String.Join("|", (string[])commands.ToArray(typeof(string)));
 Regex CommandSearch = new Regex(@"(" + registeredCommands + @")\s*:\s*(.+)?", RegexOptions.IgnoreCase);
 foreach (string line in commitLines)
 {
 string lowerline = line.ToLower();
 for (Match Matches = CommandSearch.Match(lowerline); Matches.Success; Matches = Matches.NextMatch())
 {
 string handlerString = ConfigurationSettings.AppSettings[“command:” + Matches.Groups[1].ToString()];
 DispatchCommand(handlerString, Matches.Groups[2].ToString(), rev);
 }
 }
 }

 private static ArrayList DispatchGlobalCommands(SubversionRevision rev)
 {
 // Handle global commands 
 ArrayList commands = new ArrayList();
 for (int i = 0; i < ConfigurationSettings.AppSettings.Count; i++)
 {
 string key = ConfigurationSettings.AppSettings.GetKey(i);
 string val = ConfigurationSettings.AppSettings.Get(i);
 string[] cmdParts = key.Split(’:’);
 if (cmdParts.Length == 2 && cmdParts[0] == “command”)
 {
 if (cmdParts[1].StartsWith(""))
 {
 DispatchCommand(val, cmdParts[1].Substring(cmdParts[1].IndexOf(",") + 1), rev);
 }
 else
 {
 commands.Add(cmdParts[1]);
 }
 }
 }
 return commands;
 }


 /// 
 /// Call the appropriate method for the command name given with the argument given
 /// no processing of the argument happens here. 
 /// 

private static void DispatchCommand(string handlerString, string argument, SubversionRevision rev)
 {
 // We don’t want properly configured commands to stop working because of errors so trap
 // everything here…
 try
 {
 if (handlerString != null && handlerString.Length > 0)
 {
 string[] typeAndAssembly = handlerString.Split(’,’);
 if (typeAndAssembly.Length == 2)
 {
 System.Reflection.Assembly a = System.Reflection.Assembly.Load(typeAndAssembly[1]);
 System.Type t = a.GetType(typeAndAssembly[0], true);
 object handler = System.Activator.CreateInstance(t);
 if (handler is IPostCommitHandler)
 {
 ((IPostCommitHandler)handler).ExecuteCommand(new PostCommitArgs(argument,rev));
 }
 }
 }
 }
 catch (Exception) 
 { //TODO: log errors 
 }
 } 
 }
}

There is some plumbing in this class that isn’t directly related to this post, but I’ve left it all anyway. Subversion will run this command every time a checkin is made, and the process ends and starts over again each time. This allows for some pretty simple handling of loaded assemblies and whatnot, if you have a longer running process or are dealing with some scale be cautious. ;-)

The Main function has two jobs, parse and create the revision, then read the application configuration file and start issueing commands for the received revision. Commands are in two parts, those defined in config to be executed always (global commands) and those that are interpreted from the subversion commit log itself, parsed out and executed with arguments from the revision log.

Here are some example commands defined in the config

<!– Commands –>
<add key=“command:,chris” value=“Subversion.Plugins.ExecuteForAllCommits,Subversion.Plugins” />
<add key=“command:,check-ins” value=“Subversion.Plugins.ExecuteForAllCommits,Subversion.Plugins” />
<add key=“command:bug” value=“Subversion.Plugins.UpdateBugTracker,Subversion.Plugins” />
<add key=“command:cc” value=“Subversion.Plugins.SendEmailNotification,Subversion.Plugins” />

in the key we have “command:[name]” signifies a command arriving in a revision where somewhere in the revision log we’ll see the command name followed by a colon, anything following the colon is then passed to the plugin as an argument. If the name is an asterisk then we simply execute for all, with an optional argument being passed to the plugin. (so the first example emails chris for all revisions, and the second emails an account named check-ins
the value portion here is what directs the program where to look for the appropriate plugin and class to execute. I copied the format I found in a web.config file which is to put the class name followed by the assembly name separated by a comma.

In retrospect if I were doing something similar again I’d probably create a better structured format rather than relying on all this string parsing… but old code is what it is in this case.

Finally we call DispatchCommand for each parsed out command which is the last piece of this old code that I’m attempting to document here for reuse. DispatchCommand will read the class name and assembly name, load the assembly name and attempt to instantiate the class/type named in order to call it using our IPostCommitHandler interface.

There are a few ways to do this, and for this project I’m simply calling “System.Reflection.Assembly.Load” which relies on the fact that my plugins are located in my bin directory. I’ve also done this using a “plugin store” which is a fancy way to say I had a dynamic path configured that I could read my assemblies from. In this case you can use LoadFile or LoadFrom, LoadFrom will load dependencies automatically while LoadFile loads just the assembly and will potentially load duplicate copies. (see the documentation) In order to get the dll’s in place for this project we just simply add a post build event like so…

copy $(TargetDir).* $(SolutionDir)\Subversion.Dispatcher$(OutDir)

If after instantiating the named type from the loaded assembly we actually have an IPostCommitHandler then make the call! Done.

System.Reflection.Assembly a = System.Reflection.Assembly.Load(typeAndAssembly[1]);
System.Type t = a.GetType(typeAndAssembly[0], true);
object handler = System.Activator.CreateInstance(t);
if (handler is IPostCommitHandler)
{
 ((IPostCommitHandler)handler).ExecuteCommand(new PostCommitArgs(argument,rev));
}

So that’s that. You can download the code here - it should basically work as is if you are looking for a shortcut to extending subversion with .NET. I was relatively lazy with getting this posted - so if you got this far, can use the code, and have problems with it leave a comment and I’ll try to help if I can.

Transcendent Man!

Chris Whitmore — Wed, 06 May 2009 20:49:00 UTC

I read the singularity is near last year and really enjoyed it, despite a few misgivings for Kurzweil’s ego and some dubious use of statistics. One of the things I found myself really intrigued by was Kurweil himself and this movie looks like a fun look at the man and his ideas.

Do I believe him? Part of me wants to, definitely. The ultimate end-game of the singularity is fascinating and wondrous, but I actually found some of the more intermediate steps in his projections to be more fascinating. Maybe that’s just a factor of what I can relate to. One example of this was the idea that nano-technology will lead us to self-assembling products from base materials and an instruction set transmitted as information. So much of my life is already so information focused that the idea of being able to go 100% information based and the implications on how society is structured etc… it’s mind numbingly cool. Try to imagine how much energy, time and effort we put into moving goods around this planet and how incredible it would be for all of that to end.

Anyway, looking forward to renting this one when it becomes available.

From test spy to Verify() with Moq

Chris Whitmore — Thu, 30 Apr 2009 13:12:00 UTC

Moq is now my favorite unit testing framework for .NET, and a great poster child for the power of the lambda expression support added to C#. If you are not doing unit tests or Test Driven Development you should, and if you already are and have not checked out Moq, you should.

My tests previous to Moq were using NMock, a very handy tool that looks like a lot of other mock frameworks. In order to setup a mock call you would write something similar to this :

[Simple NMock example]

Mockery mocks = new Mockery();
 IWidgetAdapter mockAdapter = mocks.NewMock();

 IList mockWidgets = new List();
 Widget mockWidget = new Widget();
 mockWidget.Name = “Mock Widget”;
 mockWidgets.Add(mockWidget);

 Stub.On(mockAdapter).Method(“LoadWidgets”).WithNoArguments().Will(Return.Value(mockWidgets));
 WidgetManager widgetManager = new WidgetManager(mockAdapter);

The ugliest thing in the expression above for me was the literal string that describes the method name that will be called. All of a sudden my fancy refactoring tools don’t quite reach all of my code and things become brittle. Sure you say, but I run these tests all the time! So it is caught right away anyway right? Yeah, but who wants to be searching and replacing these values after every refactor? Just does not feel right.

Here’s the Moq equivelent:

[simple Moq Example]

IList mockWidgets = new List();
 Widget mockWidget = new Widget();
 mockWidget.Name = “Mock Widget”;
 mockWidgets.Add(mockWidget);

 Mock mockAdapter = new Mock();
 mockAdapter.Setup(cmd => cmd.LoadWidgets(It.IsAny())).Returns(mockWidgets);
 WidgetManager widgetManager = new WidgetManager(mockAdapter.Object);

See that the “LoadWidgets” string disappears, and refactoring code now properly refactors tests right along with it, very very handy. Some find the need to add .Object when referencing the underlying mocked type annoying (on the call to WidgetManager) but personally I find this a small price to pay.

When I first started using Moq a few weeks ago I didn’t go much beyond that example. Which speaks to Moq in that it is VERY easy to get started without much effort and more advanced features really don’t get in the way of the simple features.

For a while I was able to do a lot of the testing I had in place by Asserting on values I either had access to or were being returned to me. In those cases where the values I needed were being returned to someone else (say a Service for example) I was in the habit of building stub classes (Test Spy in this case) to handle the outgoing data.

So using the generic service as an example, and wanting to observe and Assert that I am sending the correct requests to that service my previous code would have looked something like this:

[Test spy example]

public class AuthenticationSpy : IAuthenticationService
{
 #region Test Helpers
 public IList ReceivedReqeustContexts = new List();
 public AuthenticationResponse ExpectedResponse { get; set; }
 #endregion

 public AuthenticationResponse AuthenticateUser(AuthenticationRequest request)
 {
 return ExpectedResponse; 
 }

 public AuthenticationResponse RenewAuthenticationTicket(RequestContext context)
 {
 this.ReceivedReqeustContexts.Add(context);
 return ExpectedResponse;
 }

}

[TestMethod]
public void RenewExpiredTicketTest()
{
 AuthenticationSpy _authenticationMock = new AuthenticationSpy();
 Mock _respondingMock = new Mock();

 Mock mockServices = new Mock();
 mockServices.Setup(cmd => cmd.GetAuthenticationService(It.IsAny())).Returns(_authenticationMock);
 mockServices.Setup(cmd => cmd.GetRespondingService(It.IsAny())).Returns(_respondingMock.Object);

 ServiceWrapper.Current.ServiceProvider = mockServices.Object;

 _authenticationMock.ExpectedResponse = GetGoodAuthenticationResponse(DateTime.UtcNow.Add(ServiceWrapper.Current.TimerSleepTimeSpan.Subtract(TimeSpan.FromMinutes(1))));

 // initialize will call authenticate() in the service wrapper
 ServiceWrapper.Current.Initialize(“testing”, “Password1”, “http://auth”, “http://resp”);

 // now setup and call any method to trigger a renew of our now expired authentication ticket
 SetupCreateResponse(Guid.NewGuid());
 SurveyController.StartSurvey(new StartSurveyArgs());

 // confirm renew was actually called
 Assert.IsTrue(_authenticationMock.ReceivedReqeustContexts.Count == 1);
}

This works, and in some cases the control given to you with your test spy can be really helpful, but if I can avoid it I will every time. More classes and more code means more maintenance, even if it is in the test code. So I finally read the docs on the Verify() method on Moq objects and it is awesome. ;-) Here’s the same code handled with Moq properly and without the need for a whole new class imitating the authentication service.

[using Verify example]

[TestMethod]
public void RenewExpiredTicketTest()
{
 Mock _authenticationMock = new Mock();
 Mock _respondingMock = new Mock();

 Mock mockServices = new Mock();
 mockServices.Setup(cmd => cmd.GetAuthenticationService(It.IsAny())).Returns(_authenticationMock.Object);
 mockServices.Setup(cmd => cmd.GetRespondingService(It.IsAny())).Returns(_respondingMock.Object);

 ServiceWrapper.Current.ServiceProvider = mockServices.Object;

 _authenticationMock.Setup(cmd => cmd.AuthenticateUser(It.IsAny()))
 .Returns(GetGoodAuthenticationResponse(DateTime.UtcNow.Add(ServiceWrapper.Current.TimerSleepTimeSpan.Subtract(TimeSpan.FromMinutes(1)))));

 // initialize will call authenticate() in the service wrapper
 ServiceWrapper.Current.Initialize(“testing”, “Password1”, “http://auth”, “http://resp”);
 
 // now setup and call any method to trigger a renew of our now expired authentication ticket
 SetupCreateResponse(Guid.NewGuid());
 SurveyController.StartSurvey(new StartSurveyArgs());

 // confirm renew was actually called
 _authenticationMock.Verify(cmd => cmd.RenewAuthenticationTicket(It.IsAny()), Times.AtLeastOnce());
}

Check out Part 2 in the 4 part series “Beginning Mocking with Moq 3 ” gives a short description of how Validate works.

Not bad eh? Again the power of the lambda expression here jumps out at you. Full intellisense and compiler support for describing exactly what you expect that method to receive. The “It” class allows for no description “It.IsAny<t>()” or very precise description as above. The “Times” check also allows you to narrow Significant savings in code and maintenance and actually using the testing framework as intended (imagine that) ! My only slight annoyance so far is in having to keep count of the number of times a method has been called in order to check that the last piece of code actually resulted in a call and not some code way earlier.

silverlight 3 - after the high

Chris Whitmore — Wed, 25 Mar 2009 09:40:00 UTC

I failed to convince my manager at work that sending me and a few members of my team to MIX was a worthwhile expense in this economy. So instead I spent a couple days this sprint with http://live.visitmix.com/ on one screen and visual studio in the other. I have to say, Microsoft did an amazing job with MIX in terms of getting me excited and having me “tuned in”. If you are at all interested in web development on the Microsoft stack and haven’t checked out the keynote I’d recommend it. I really enjoyed Buxton’s presentation and Guthrie was amusing.

So now that it’s been a week, and “the Gu” and all those dancing flashy lights are no longer influencing my opinion… I’m STILL excited about Silverlight 3. Sadly the development tools can’t be run in parallel with Silverlight 2 and we’re near the end of our sprint so can’t afford the risk. Which is really too bad because one of the things our current application is leveraging is the wcf duplex polling module. A lovely little COMET like implementation for server push. The version of the duplex polling that made it into the Silverlight 2 toolkit was a little more bare than your typical Microsoft module. And while it works pretty well, it leaves a lot of plumbing code in the hands of the programmer, specifically a lot of asynchronous channel handling code that is a bit of pain to deal with. (though a bit educational too) Anyways, this is one of those areas that Microsoft is improving on in Silverlight 3, and one of those things I’m excited about. Right next to the simpler duplex polling usage for me is the introduction of binary serialization for web services (including duplex!). When comparing to Flex and the myriad of tools and options for using AMF Silverlight was really behind the ball on this one. When we eventually decided to build our tool in Silverlight as opposed to Flex we basically committed ourselves to rolling our own binary serialization. I’m very happy we’re not going to have to follow through on that. Read more from the web services team :

http://blogs.msdn.com/silverlightws/archive/2009/03/20/what-s-new-with-web-services-in-silverlight-3-beta.aspx

Another great addition in the realm of things-that-were-annoying-but-possible-and-already-in-flex is the new navigation uri support within Silverlight 3. Check out Tim Heuer’s typically great post on all the silverlight changes here. (link specifically to the nav)

Lastly to round out my list of really exciting enhancements to SL3 are the network monitoring API, which gives developers events to subscribe to detect when the network is and isn’t present - as well as assembly caching which is huge, allowing Silverlight to cache assemblies like the toolkit so that once a user has been exposed to it they don’t necessarily have to download it again until a new version is required. This in turns makes XAP’s smaller which is always a good thing.

So to summarize, I think the top five features from the slew of enhancements that I’m looking forward to are :

Binary Serialization
Duplex polling enhancements
Network detection API
Assembly Caching
Navigation and Deep Linking suport

My perspective on Silverlight is very biased to the needs of our application of course. And our application will live and die on the network, with performance being a top concern in everything we do. Controls are nice but we can buy those from vendors like Telerik, animation and media are cool for demos but likely won’t do much for us in the short term. The out of browser story is huge, but again with a SaaS app that relies on the network we don’t envision a whole lot of offline work happening in the early versions of our app.

Honorable mentions for features go to GPU acceleration (performance) and the SaveFileDialog (control) and Expression Blend 3. I don’t use Blend much myself, but the current version is a huge pain for our team. Maybe more on that in a separate post.

decision making : flip a coin then check your gut

Chris Whitmore — Wed, 18 Feb 2009 15:53:00 UTC

I once heard an interesting anecdote about how to make a difficult decision between two paths. When you find yourself spinning, alternating between one choice and then the other, it can be helpful to simply assign each choice “heads” or “tails” and flip a coin. When you reveal what side the coin landed on pay attention to your emotional reaction… are you relieved or are you disappointed? Try it sometime, it really can work.

I recently spent about three weeks or so doing an in-depth analysis of Adobe Flex vs Microsoft Silverlight for an enterprise application and I really feel like I ultimately decided via the coin flip method (without actually flipping the coin). Our company is about to embark on a new product aimed at the enterprise that will require levels of functionality and control that Ajax alone can not provide. We are essentially looking to take a workflow that has been heavily dominated by Word and Outlook and drag it into the future with real-time collaborative tools in the spirit of Google Docs.

I ended up choosing Silverlight, despite the potential risk adoption may pose. At the end of the day we believe our target market will be willing to accept the Silverlight install process, and that the underlying engine (.net) provides far more robustness for building the kind of application we’re looking to build. Honestly this is a whole other post, but the nail in the coffin for Flex ended up being the lack of threading support for developers. On nearly every other level the two were neck and neck, with very subjective “wins” for either and Flash being the clear winner when it comes to adoption etc.

What’s interesting though is that my first choice was Flex. After weeks of agonizing I decided we needed to build this thing in Flex, working around the lack of threading where necessary and going with the safe route of next to zero adoption barriers. It only took a weekend after making that decision to flip-flop. I was supposed to be making the call as if this were my company on the line, and with a clear vision of the unknowable future… at the end of the day though taking the safe and compromised route just didn’t feel right. I could see the complexity of our application snowballing in the future, I could see the legacy of the flash runtime catching up with us, I could see a competitor choosing to build their offering in Silverlight and spanking us in the next year. Making the decision from a technical standpoint the only winner was Silverlight, if the business deemed the adoption risk too great then fine we could do Flex, I was prepared for either.

My proverbial flip of the coin had basically taken those three weeks of opinions and research and testimonials and flame wars had all gelled together once I had made an actual commitment to choosing Flex. It was only then that my gut told me what I needed to know and I have not looked back from Silverlight since.

Flex data services limitations (FlexBuilder generated wsdl code sucks)

Chris Whitmore — Thu, 12 Feb 2009 08:54:00 UTC

The post saved me a ton of time. It’s a bit embarrassing for Adobe in my mind to ship something this buggy. I was seriously running into these issues within an hour of trying to connect Flex to our .NET Soap based services.

“MyMethod can’t return an object of with the type name MyMethodResult."

You’re fracking kidding me right? Wow. (and there are more along these lines)

http://lukesh.wordpress.com/2008/11/24/very-important-limitations-of-flex-data-services/

After fighting with the above and other bugs I was rewriting a lot of the generated code from FlexBuilder and it was just pointless. And sure, generated code isn’t the greatest to rely on anyway, but give me a break. In the end I used the WebORB presentation server to handle the communication to our .NET code, as well as for generation of the initial proxy classes for the client and I have to say it was an excellent experience compared to the crap built into FlexBuilder.

QWERTY Myth and the entrenchment of Flash

Chris Whitmore — Sat, 31 Jan 2009 19:54:00 UTC

This is a great article about the myth of how the best technology doesn’t necessarily win. Granted, sometimes the best technology does not win, but there is a persistent and pervasive sense that the populous often chooses the “VHS” over the far superior alternative. The article addresses the VHS vs Beta debate directly as well as the victory over Dvorak by QWERTY. To encourage you to read the original I won’t reveal the clever arguments made.

http://www.reason.com/news/show/29944.html (Read Me!)

I’m posting this because there seems to be a real sense of fait accompli when it comes to the Flash vs Silverlight debate. Critical mass has already been acheived, why would content producers or development shops choose to target any other platform than the Flash runtime when users have clearly already made their choice? How could Beta possibly make a resurgence against an already entrenched VHS? It would take an entire round of evolution before DVD would come along and supplant the status quo. There are a couple reasons why this article has relevance for Silverlight, and why the VHS / Beta argument doesn’t hold water.

Flash vs Silverlight is about a producer investment in technology NOT a consumer investment. Machines are powerful enough, and installations simple enough that the relative cost of owning both technologies is nothing like owning two peices of hardware.

If there is a competitive advantage for a producer to be gained via a specific technology they will use it. Any differentiators in a competitive field like software has a high potential of making a return. This is a very different decision process than it is for consumers.
Consumers don’t really care or even know which technology is driving their rich content. They care that it “just works” (like flash based video in comparison to WMP or Quicktime) and that the functionality they desire is there. Without a right-click most users won’t even realize which is which behind the curtain once they have both installed.
“Owning” everyone (high adoption) is really not that big a deal when your competition can also have 100% adoption at the same time. This is not like choosing a computer or an operating system. Only Microsoft can prevent themselves from achieving their penetration goals.

Better technology does win. I’m not saying that Silverlight is necessarily the better technology right now, Flash maintains an edge on some specific rendering speeds it appears, and their designer tools are clearly better… but Silverlight has the benefit of coming at this with second mover advantage. They didn’t start from scratch, they built out a proven technology (.NET) into new ground by largely copying and improving on the entrenched technology. (sure looks copied from my perspective but that’s a different post) The .NET runtime, threading, compiled/managed code and the lack of legacy in Silverlight will all combine to produce demonstrations of browser based technology that will be extremely difficult and expensive to reproduce on the Flash runtime.
Silverlight does not have to “kill” Flash to win, it only needs to join Flash in the 90% adoption numbers to be a great success.

I like both technologies by the way, I’m just entertained by some of the almost religious like statements on those on the Flash side that sound a lot like any attempt to improve or even add to the status quo is a total waste of time. (or somehow an affront to their own efforts)

Silverlight controls

Chris Whitmore — Wed, 28 Jan 2009 16:59:00 UTC

Silverlight 2 may not have the control set that Flex developers are used to seeing out of the box but there are a significant number of control vendors who are stepping up to the plate to fill the void. It seems as though Microsoft’s strategy has been to get the Silverlight 2 runtime out as quickly as possible (and as lean as possible) always knowing that this type of extension to the framework would exist.

I do still hope to see Microsoft push a little further in controls that are downloaded once and only once with the framework itself thereby making our applications leaner - but it’s a pretty serious tradeoff until the runtime has the kind of penetration that Flash enjoys.

Anyway, here’s a nice post from Tim Heuer that does a good round up of where to find those missing controls.

http://timheuer.com/blog/archive/2009/01/28/comprehensive-list-of-silverlight-controls.aspx

1 TB drive won't format using Disk Utility

Chris Whitmore — Wed, 31 Dec 2008 16:43:00 UTC

I just bought a 1TB external HD, the “Maxtor OneTouch 4 Plus” this weekend on sale at London Drugs. A bit of an impulse purchase but I’ve been digiizing all of our dvd’s lately into iTunes and had completely run out of space…

The drive has a bunch of automated backup features I’ll never use, so I skipped all the software and went to use the drive directly from Mac OS X. First step here is to convert the format of the drive from it’s default of NTFS to something Mac can natively use. My impulse was to simply “format” and continue but unfortunately every time I tried the format disk utility would abort with nothing useful showing in the console logs. I went through this a few times with different file system settings and nothing worked. (See this ArsTechnica link for how to choose your filesystem)

Odd.

I then tried to partition the drive - and two partitions or more would all of a sudden work. Again odd, but I didn’t want more than one partition in this case, but reverting back to one partition would cause the same problem all over again. I resorted to Google at this point and came across this very useful although somewhat poorly formatted post on Seagate forums. (seems to be a generic problem with disk utility)

http://forums.seagate.com/stx/board/message?board.id=freeagent&thread.id=1062

The gist of the problem being that the default partition table format needs to be changed to GUID. You can apparently only acheive this by partitioning (in this case to two partitions) so you can change the setting for partition table format and then partitioning again back to one partition with the new partition table format intact. Annoying but easily worked around once you stumble on the right answer.

Note this issue is actually new to 10.5.* and you can also solve the problem by formatting from an older version of Mac OS X (from a boot CD for example).

FlexBuilder 3 First Impressions

Chris Whitmore — Sun, 21 Dec 2008 21:27:00 UTC

Where we’re coming from

So at the beginning of the year I was tasked with evaluating a number of technologies for RIA development for the next evolution of my company’s product. Up to this point we had been relying extensively on ASP.NET forms with a traditional post-back model that was responsible for a lot of wasted time and bandwidth. We’ve leveraged a lot of Ajax in the past few years, starting with simple fixes like trees and list based controls that use load on demand and going all the way up to full fledged single page applications that consumed purely services.

This has worked, but the cost is overwhelming for a development team of our size and makeup. We hire smart generalists for the most part, favoring developers with C++/Java/C# backgrounds. Some of our developers have acquired some deeper skills on the client side, but where possible we attempt to leverage control vendors like Telerik and ComponentArt as much as possible. They do an excellent job of hiding some of the complexity involved in cross browser web interfaces, but you will inevitably have to “hit the metal” and get your hands dirty. Relying on third parties also removes a lot of the control needed to do things the way you need them done. Regardless despite being a huge fan of the http://docs.google.com suite of tools, I have witnessed far too much ugliness in our organization with supporting multiple browsers (including having to support IE 6) and pushing the limits of complicated UI in the browser. As the size of the DOM increases and the size of our data sets increase we see wild variance in client performance with respect to things like drag and drop. I know it can be done, I know we are not at the limit yet, but seriously this is not pragmatic for our software and our market and our developers. I am a big fan of the view that JavaScript is becomming the assembly of the web, those who do this shit well, do it well by lifting themselves out of the muck with good abstractions like GWT.

One thing I think should add here in defense of Ajax though; UI design plays a really important role in the effectiveness of the DHTML approach and honestly I believe part of our problem has been designing a far richer interface than we could afford in the technology we were leveraging at the time. Take a close look at Google’s lack of decoration, images etc. These things certainly matter.

Next steps… evaluation

Anyway, I’m getting off topic as usual. In the beginning of 2008 my feature matrix analysis really narrowed our options from about a dozen technologies (including XUL, ActiveX, Applets, JavaFX, Silverlight, ClickOnce, Ajax, Flex) down to three. Silverlight, Flex or Ajax. At the time of my evaluation Flex was at version 2, JavaFX was vapourware and Silverlight2 was in Beta. Given that we are a .NET shop and already have the C# programmers, the Silverlight option was looking like it was going to cleanly win out over Flex. Ajax was honestly only at the table still because we needed to justify our position and show we clearly evaluated all our options. Flex was seen as less desirable due to being based on ECMAScript and having to retool and retrain.

For the most part we’ve seen this as two relatively equivalent technologies with different stories for the developers. While there are important differences between how code is delivered and executed in Flex vs Silverlight, but at a high level we believe technically we can deliver our application in either technology very effectively. We prefer to keep working in C#, but the limited penetration of Silverlight is a serious risk for an application delivered in a SaaS model. That single fact has transformed the whole exercise into largely a business decision. I don’t doubt Microsoft will be able to push their offering significantly, but I would not bet money on where they will be in 1 year. (Windows Media Player STILL doesn’t equal flash in penetration)

Tool support however remains something that is extremely important to developers, and is one of those things Microsoft often trots out in arguing the superiority of their platform. We swallowed that line pretty easily at first, knowing that under the hood all the code written for Flex is just a variation of ECMAScript (JavaScript) is enough to scare us off. How can you acheive the refactorability and tool support provided by current and future versions of Visual Studio with a loosely typed language like ActionScript?

Trying it out

This week I downloaded FlexBuilder3 after one of our senior executives setup a call for us with Adobe evangelists to get more details on why to go with Flex. Again the motivation for this coming back to penetration and wanting to ensure we are making the right decision for what will become a million plus dollar iniitiave to re-engineer. I wanted to get some hands on time with the latest version of FlexBuilder (3) that had come out since our initial research.

I was immediately surprised by the leaps Flex had taken since I last really dived in. I’ll admit there was some bias here though as I am also a huge fan of Eclipse, so the fact that FlexBuilder is built on Eclipse is in my mind a huge win. (not new btw)

The effort in actually building an application that connected to our existing .NET web services was embarrassingly trivial. FlexBuilder has a simple tool for generating and managing proxy classes to represent your web services. So after literally pasting a url into a wizard I had code for talking to our .NET SOAP based web services. (seemed to only support SOAP 1_0 not 1_2) I then got started with the Form Designer and had a simple application talking to our backend in under an hour even counting the little things that tripped me up like where to add my event handlers which wasn’t immediately apparent. (too reliant on double clicking controls apparently ;-) hint : <mx:Script> tags and dom style event callouts)

The concept of states in Flex and the ease with which I was able to create a number of them in the designer and bind those to a dropdown for switching between them was pretty eye opening. A state in Flex is defined by the differences between your main UI (or just another state) and the state you wish to have/be in. The IDE allows you to visually manage these states and then visually modify each one to represent application states. I don’t have an early sense of whether this actually scales for complex applications, but at first glance it’s very cool. (Think hierarchical state machine) Couple this with the data binding model and you have some very effective UI management tools at your disposal. Maybe this only looks cool coming from our antiquated asp.net approaches, but this stuff is exciting. (Silverlight/WPF have the same capability, maybe even a little more advanced but with more overhead in my opinion) Having your model drive all changes is so much more manageable, scalable… and just correct than having explicit assignments in page PreRender methods that set visibility based on the state of that model. Barf.

The control toolkit out of the box with Flex is also extremely impressive. Check out this post for a list of all the FlexBuilder 3 Controls included out of the box . For now at least this control set will mean being highly more productive in the early stages of development than if we were either having to roll our own or rely on third party vendors. And of course you can roll your own in both Silverlight and Flex and each can be just about anything imaginable.

So I’m sold, at least sold on the fact that Flex deserves considerably more attention than what we had previously given it. I’ve bought the “Flex 3 Cookbook”, and “Adobe Flex 3 Training From the Source” and I’m intending on spending at least some of this Christmas holiday catching up on just what’s possible with that silly little flash technology.

FlexBuilder 3 Controls

Chris Whitmore — Wed, 17 Dec 2008 15:17:00 UTC

Controls included with FlexBuilder 3 out of the box below… check out some third party components here .

Notes will be updated as I actually get a chance to put some of these to use.

FlexBuilder 3 Controls
Control Name	Notes
AdvancedDataGrid	Professional version only + multi column sorting + grouping + tree view + printing support – Still no paging support out of the box
AlertControl	not sure how this gets grouped with these other controls as it is not an explicit control but a static method “show()” which can be used from where ever.
Button	check out the FlexLib CanvasButton for a more flexible option
CheckBox
ColorPicker
ComboBox
DataGrid	See this article for some hints on implementing paging
DateChooser
DateField
HSlider
HorizontalList
Image
Label
List	very fast and flexible, lots of issues with scrolling but workable
LinkButton
NumericStepper
OLAPDataGrid	Professional version only
PopUpButton
PopUpMenuButton
ProgressBar
RadioButton
RadioButtonGroup
Repeater	SLOOOW when binding to many objects, look at List based controls instead
RichTextEditor	Limited html formatting, seems to work ok
SWFLoader
Text
TextArea
TextInput
TileList	see list, very fast control but takes a bit more work for smooth work
Tree
VSlider
VideoDisplay	FLV based video player with simple cue and playback control
FlexBuilder 3 Chart Controls
AreaChart
BarChart
BubbleChart
CandleStickChart
ColumnChart
HLOCChart
Legend
LineChart
PieChart
PlotChart
FlexBuilder 3 Navigation Controls
Accordion	Great control, but needed to go custom almost immediately ( see FlexLib for custom header)
ButtonBar
LinkBar
Menu
MenuBar
TabBar
TabNavigator
ToggleButtonBar
ViewStack
FlexBuilder 3 Layout Controls
ApplicationControlBar
Canvas
ControlBar
Form
FormHeading
Grid
HBox
HDividedBox
HRule
ModuleLoader	not really a layout control? ModuleLoader allows you to load components of the application on demand, lowering initial download size and improving encapsulation
Panel
Scrollbar
Spacer
Tile
TitleWindow
VBox
VDividedBox
VRule

visual c++ lesson 0.0.0.0.1 precompiled headers

Chris Whitmore — Mon, 15 Dec 2008 16:03:00 UTC

I come from a background of managed memory and interpreted languages. I’m a big proponent of pragmatic approaches to problems and as little re-inventing of the wheel as humanly possible. I don’t think the world needs another text editor, and I personally don’t feel the need to write my own version of the stack I rely on for application development. (.NET Framework and IIS)

This however gives me less credibility with all those “real” programmers out there. The ones who read assembly for fun and don’t believe in memory management or virtual runtimes/machines. I consistently find myself in battles with control freaks who argue that building an application on top of an application server like Tomcat or IIS is dangerous and excessive when it’s so much simpler to just write your own daemon and connection handling.

Regardless. It is difficult to argue without having credible experience with the alternatives. Not only that, but a number of my dream jobs require extensive C/C++ knowledge (Google) and many important FOSS projects require the same. So I am finally diving in and (re-)learning some C++ with an initial task of writing an XML to ASN.1 converter. (don’t ask why)

I’m doing this in visual studio 2008 with Visual C++, which as I’m learning has it’s own learning associated with it. First is the question of ATL , MFC , WIN32 or just Blank. Visual studio doesn’t give you a whole lot of background on why you might choose one or the other, but some simple wikipedia reading spelled out that for my project I wanted the simplest option. So I went ahead with Win32 Console as it seemed to have the least amount of overhead with easiest start.

(Why would anyone use the C++ CLR option?)

From here I moved to some very simple HelloWorld love with some file IO. I’ve taken a course that used C for half of the assignments so I am not completely new to this, but I definitely needed a reminder. Here again I was presented with another option, stdio or iostream ? More wikipedia love, apparently stdio is the old C way of doing things and iostream is the new object oriented way of doing things. There seems to be a lot of contention still about when to use which when, but for my purposes the stream approach seemed more appropriate.

And include how?

#include “iostream”
//OR
#include <iostream>
//OR
#include <iostream.h>

Well visual studio doesn’t allow the last one so that’s easy. Using either of the first two work though works because using the double quotes option will check for an implementation defined location for the file before falling back to the same behavior provided by using the angle brackets. Ok, one more down.

Don’t forget that in Visual C++ adding the above import does not explicitly import the associated namespace. So in order to actually use cin or cout you need to either prefix every instance with std::cin or use the statement “using namespace std;” in order to simply use those identifiers normally.

The next couple hours were learning how to create classes, use namespaces and some simple iterative build up of some basic classes to represent the document I needed to import. Still barely outside the realm of a hello world really.

Now, for this little project I am actually more interested in the ASN part of the problem than I am in the XML parsing so I looked for a parser. I came across the autumn project which references a parser written by Dr. Ir. Frank Vanden Berghen which was appealing as it was relatively small, portable and self contained. You can find the files here.

Now for the fun part. When attempting to compile my newly added class was throwing a dozen or so errors that didn’t exactly make sense to me. I was prepared for some pretty ugly work in trying to port this thing from GCC to the Microsoft compiler, so I didn’t really question these errors. To begin with it was mostly types that were missing. So I would search for the definition of that type and find it in a header file that I was sure was being included. At this point I naively began to move code around in an effort to understand the error. Moving one struct from the header to the cpp file seemed to resolve one error and cause a few more. This seemed to validate my guess that the header was somehow not being included. I had no expectation that this should just work out of the box of course though, so perhaps some of this code was just wrong. I started to chase down the parts of the code that were dependent on flags such as #ifdef WIN32 (I really like how visual studio grays out the code that will not be included based on those conditionals, very nice).

This went on for maybe an hour before I was convinced that this had to be easier. Looking more closely at the build output rather than the error log (which should always be done much sooner than this) revealed this warning :

1>c:\documents and settings\c\my documents\visual studio 2008\projects\xmltoasn\xmltoasn\xmlparser.cpp(82) : warning C4627: ‘#include <Windows.h>’: skipped when looking for precompiled header use
1> Add directive to ‘stdafx.h’ or rebuild precompiled header

This is one of those steps where I know I should have asked more questions when I first started my project. The project by default included my “main” file, but it also included these two stdafx files (header and cpp) which I briefly looked at but didn’t dig into. The comment at the top of stdafx.h shows this :

// stdafx.h : include file for standard system include files,
// or project specific include files that are used frequently, but
// are changed infrequently

Which if you don’t know what pre-compiled headers are may not make a ton of sense. And it sounds to me like this is optional in any case. Well, it isn’t, at least not if you have the build options on your project set to use precompiled headers which by default I did. Simply adding “#include <windows.h>” to the stdafx.h file resolved all the problems. So in fact the xmlParser module WAS portable, and I just didn’t have a clue.

The other way to solve this problem is to actually change the precompiled headers setting for your project to not use precompiled headers at all.

So this was a bit frustrating, but all in all a good first foray into this shit, and I’m at least a few steps closer to having a program that actually does something.

ajax for mac lovers

Chris Whitmore — Fri, 12 Dec 2008 15:05:00 UTC

Another Ajax Framework : (or rather, an Application Framework)
http://cappuccino.org/
http://cappuccino.org/learn/

Demo app built using it:
http://280slides.com/

And a teaser for all those interface builder lovers out there :

http://ajaxian.com/archives/nib2cib-use-interface-builder-to-design-your-ajax-apps

I came across this in reader this morning and was totally blown away by how well the 280 slides application worked. It's really impressive... on non IE browsers. I sent the link to a co-worker to check out and he basically dismissed it as too slow and unresponsive, "typical web app". It took a minute or two to realize the browser was the problem at which point I launched the app in IE 8 Beta (IE 8!) and it performed terribly.It looked terrible, it was jerky and generally just a big let down after the speed of Chrome.

Whatever the technical reasons are, it sucks. Let's hope the new generation of javascript engines (FF3.1 and Chrome) are able to push Microsoft into stepping up. I'm excited about Flex and Silverlight and JavaFX but really I want to believe we can keep pushing the browser without the plugin.

C++ linking

Chris Whitmore — Fri, 28 Nov 2008 12:55:00 UTC

This is a post for myself, to basically bookmark the excellent work of someone else. My post is contributing practically nothing (maybe adding some context/weight for his article) but here it is anyway. ;-)

http://blog.copton.net/articles/linker/index.html

Despite not being an active user of C++ I really enjoyed this post. I actually feel a little smarter and better informed for having read it. Despite the mess in the C++ tool chain being described, this kind of reading actually makes me feel more inclined to dig into this stuff not less. Anyway, filing this one away as something to potentially come back to.

Lunar Reconnaissance Orbiter

Chris Whitmore — Tue, 11 Nov 2008 14:31:00 UTC

A good friend of mine Sarah believes that the moon landings were a hoax. Despite being a huge science geek , a fan of NASA and a member of the planetary society she subscribes to the idea that man has not in fact walked on the moon, and the entire thing was a lie perpetrated in an effort to win the political war with the USSR. Or something along those lines.

Somehow this deeply disturbs me. Anything coming from Sarah carries considerable weight, so I can’t just discard her opinion. How could it be that the same person avidly following the mars rover mission also believes that we couldn’t land someone on the moon (or rather, didn’t)? I can say that I have not watched/read/been brain washed by the same materials that she has, but I’m expecting to be convinced to do so after this post.

Skepticism is vitally important to science, no doubt. I think scientific thinkers must challenge anything that doesn’t make sense to them. I myself am open to hearing the counter arguments, and I even spent a half hour or so reading the respective wikipedia articles on the matter…

http://en.wikipedia.org/wiki/Apollo_Moon_Landing_hoax_accusations
http://en.wikipedia.org/wiki/Independent_evidence_for_Apollo_Moon_landings

I also think while conspiracy theories are fun (x-files we miss you) they are also wrong 99.99% of the time. In a time where science is considered elitist and unnecessary in one of the most important political, economic and scientific centers on earth, and when NASA continues to face diminishing budgets and smaller mandates it seems terribly unproductive to undermine those efforts being made by real engineers and scientists by giving credence to crack pot theories about hoaxes.

Consider how difficult it is to keep a secret. Imagine the incentive to those cynical people out there who wish to undermine the real achievements of science. Remember the movie contact and Ocham’s Razor? Do we make hundreds of assumptions about the ability for hundreds of people in the government, at NASA and elsewhere all keeping a secret about the landings? That the telemetry mirrors left behind were done by unmanned missions (and that we had the robotics capable at that time to do so?) That the long documentation trail left behind from all the steps leading up to Apollo 11 were somehow part of it? That the FIVE additional moon landings (after the public had already lost interest) were also faked just to add weight to the first faked landing? Or can we assume that no such plot exists and that NASA’s account is roughly accurate? I realize you can flip this and play the other side, but you can read the articles for the details rather than me iterating through the arguments that have already been made on both sides.

I have to admit, I want to believe. I want to believe that we really did achieve the mantel of a moon landing. That we as a culture were able to step outside of our regular bullshit to come together and accomplish something truly spectacular for mankind.

LRO - finally time to shut up the crackpots

In any case, the only real reason I started this post, besides to provoke you Sarah, was so that I could mention the upcoming Lunar Reconnaissance Orbiter , a mission I will be really looking forward to. It looks like we’re going to get a lot more familiar with our friend the moon. This can only be a good thing as privatized space exploration steps up and produces more tourism and public interest in things beyond our humble planet. The moon may seem a bit provincial at this point, but if you were to seriously consider visiting it (when you make your millions on the internet) do you not get totally stoked? It seems like the next logical jumping point for our more grandiose visions. LRO is launching in early 2009 from Cape Canaveral this mission will include (from wikipedia)

Selenodetic global topography

Characterization of deep space radiation in Lunar orbit

The lunar polar regions, including possible water ice deposits and the lighting environment

High-resolution mapping (max 0.5 m) to assist in the selection and characterization of future landing sites

And onboard instrumentation will most importantly include :

LROC — The Lunar Reconnaissance Orbiter Camera (LROC) has been designed to address the measurement requirements of landing site certification and polar illumination.^[11] LROC comprises a pair of narrow-angle cameras (NAC) and a single wide-angle camera (WAC). LROC will fly several times over the historic Apollo lunar landing sites, with the camera’s high resolution, the lunar rovers and Lunar Module descent stages and their respective shadows will be clearly visible. It is expected that this photography will boost public acknowledgement of the validity of the landings, and discredit the Apollo conspiracy theories.^[12]

It will be nice to put this to rest. Long live Elvis.

What you are reading is consuming energy

Chris Whitmore — Sun, 09 Nov 2008 22:56:00 UTC

Consumption is one of those things that is on my mind a lot. Both economically as I aim to live debt free and with as little "stuff" as really needed as well as in other forms of energy. Buy local, buy less packaging, drive less, eat less! It goes on and on.

One of the really interesting things with looking at google appengine is their metering and logging of your application. The GAE has limits on how much disk, cpu and bandwidth your application can consume before you have to pay for those resources.

(some stats on a very infrequently used blogquotes app)

This model of utility computing has been something that has been kicked around and toyed with for literally four or five decades. In the beginning it was envisioned that computing would be just like the electrical grid, and you would pay for computing resources in much the same way as you do now. This was back when no one ever believed that a household could use or would need a computer that took up an entire room.

That of course all changed, and we drifted to the current state where everyone has their own (or three). Are we are drifting back to a utility model again with the usefulness of having your data live in the cloud? I know I certainly care less and less about the machine I happen to be using when accessing my data. If the data is in the cloud and therefore accessible anywhere you go, and more importantly from any device you choose (iPhone!) then it naturally just makes sense to perform operations on that data within the cloud as well. Why bring it all down to the client to compute values? Why own multiple computers and have idle processors and half empty disks? (wish I had that problem actually) I think it's a bit early to signal the death knell for the personal computer, far from it, but it certainly gets you thinking.

In terms of energy consumption these are all having significant impact on the overall picture. There still exists an incredible amount of power on the client machine that is largely going untapped with increasingly thin clients (but of course that is reversing now too) and that power is going under utilized when we have servers that are performing the same poorly crafted functions doing the work millions of times over for every page view etc.

There was a blog post in Jan 2007 that talked about how much energy would be saved if google switched from white to black. This post evolved into a full on article on the topic, and a website (Blackle) with a counter for energy saved.

This is all very interesting to me, but to get to my point... Using GAE and looking at very precise measurements of the resources my code and application are using was an incredible moment of perspective for me. Here I am, looking at a direct correlation to the algorithm I choose and a measurable amount of resources being consumed by that decision, amazing really. This is just profiling on the aggregate, but it feels profound. Somehow being in the utility computing frame of mind and looking at my "bill" I am compelled to rethink every aspect of my design to find ways to use less resources. This can only be a good thing.

software fundamentals are exciting?

Chris Whitmore — Sat, 08 Nov 2008 13:47:00 UTC

I came across a nice list of fundamental axioms of development on reddit this morning that made me a little pumped. Pumped because I’m in the middle of a big transition at work that in a lot of respects has me starting over with a new team and a new mandate.

I’ll be focusing on solutions, custom work and a view towards short term revenue vs long term research and development for products. Given the economic climate, it’s a shift I can understand and on a personal level one I’m looking forward to. I am saddened of course to be leaving the product I’ve spent the last four years working on, but at the end of the day software is software and this is going to be a big challenge for me.

Here’s the list (http://www2.computer.org/portal/web/buildyourcareer/fa035) highlights for me :

EF1. Efficiency is more often a matter of good design than of good coding. So, if a project requires efficiency, efficiency must be considered early in the life cycle.

Q4. Trying to improve one quality attribute often degrades another. For example, attempts to improve efficiency often degrade modifiability.

T1. Most software tool and technique improvements account for about a 5- to 30-percent increase in productivity and quality. But at one time or another, most of these improvements have been claimed by someone to have “order of magnitude” (factor of 10) benefits. Hype is the plague on the house of software.

T1 I believe after having fallen for the tools pitch more than a few times. At the same time though I think one of the differences in the “great programmers are 30 times more efficient than mediocre programmers” comes down to mastery of the tool set. Watch a proficient developer fly through their code and it’s easy to see. On the other hand I’ve seen excellent “users” who fly through a terrible design and become constrained by EF1.

Anyway, for me this is reminscent of the pragmatic programmers list, which as obvious as a lot of it is really made me focus on the core of my craft. See Jeff Atwood’s site for a quick reference if you have not seen this list before :

http://www.codinghorror.com/blog/files/Pragmatic%20Quick%20Reference.htm

While I’m at this, I’ve had some accumulated Martin Fowler wisdom around estimates and scoping that I’ve been meaning to post about. Working in custom solutions will mean writing a lot of proposals and giving fixed cost estimates which is going to be a new game for me…

Martin Fowler: Estimates
http://www.martinfowler.com/bliki/ThrownEstimate.html <– Technical debt, casting quick estimates
http://martinfowler.com/bliki/XpVelocity.html <- Nebulous Units of Time

and on dealing with fixed scope….
http://martinfowler.com/bliki/ScopeLimbering.html <– dragging clients towards a more agile process
http://martinfowler.com/bliki/FixedPrice.html
http://martinfowler.com/bliki/FixedScopeMirage.html

frequent beachballs in mac os x caused by bad fonts

Chris Whitmore — Sat, 06 Sep 2008 21:46:00 UTC

I’ve been testing the latest nightly builds of firefox 3.1 over the past few days and while generally impressed with the performance improvements in javascript was quite disapointed that it was causing my iMac to go into frequent hangs where I would see the spinning beachball of death for many seconds before I could continue working again. It became bad enough that I finally had to ditch my firefox testing efforts.

Much to my chagrin the problem continued well after I had stopped using good ole Minefield. I began to explore running processes via activity explorer and just generally clean up my machine. So after uninstalling a bunch of apps and services I wasn’t running anymore and still experiencing the same problem.

At this point I was worried because the cpu was not the issue, iTunes would continue to play with no problem (and even respond to the hotkeys on my keyboard for switching tracks) it was just the UI that was freezing, and generally as I was opening files. Disk issue? Memory? Maybe even something where the network was introducing some latency? After some googling and consternation over my potentially failing disk I finally did what I should have in the first place and started to dig into the system logs via Console.

Sure enough one group of messages stood out right away…

06/09/08 9:21:29 PM com.apple.ATSServer[14462] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
06/09/08 9:21:29 PM com.apple.ATSServer[14462] 2008.09.06 21:21:29.63
06/09/08 9:21:29 PM com.apple.ATSServer[14462] ATSServer got a fatal error (status: -4) while processing a message (id: 20) from pid=14309.
06/09/08 9:21:29 PM com.apple.launchd[386] (com.apple.ATSServer) Throttling respawn: Will start in 10 seconds
06/09/08 9:21:39 PM com.apple.ATSServer[14465] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
06/09/08 9:21:39 PM com.apple.ATSServer[14465] 2008.09.06 21:21:39.90
06/09/08 9:21:39 PM com.apple.ATSServer[14465] ATSServer got a fatal error (status: -4) while processing a message (id: 20) from pid=14309.
06/09/08 9:21:39 PM com.apple.launchd[386] (com.apple.ATSServer) Throttling respawn: Will start in 10 seconds
06/09/08 9:21:41 PM quicklookd[14463] [QL ERROR] ‘Creating thumbnail’ timed out for ‘<QLThumbnailRequest /Library/Fonts/LiberationSans-Regular.ttf>’
06/09/08 9:21:50 PM Console[14309] Failure with ATSFontGetUnicodeCharacterCoverage(). Disabling font fallback optimization for characters not renderable.
06/09/08 9:21:51 PM com.apple.ATSServer[14468] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

So the ATSServer (Apple Type Server) was choking and it looks like the beachballs I was seeing were related to the throttling of the respawn of the server. So I’d be staring at a beachball for up to 10 seconds while the rest of the OS hummed along fine. A quick google search revealed the most common reason for this kind of failure is a corrupt font. At this point I shot around in my chair and asked my wife pleadingly and only somewhat accusatorily whether she had by any remote chance installed any fonts lately….. Yes. So I gave her control of my screen so she could clean up what had been added and like magic the beachballs ended. No reboot or anything required, ATSServer has not crashed since and I am writing this in Minefield 3.1b1pre with no problems! (sorry to blame you firefox)

Sadly I do not have the patience to go through the exercise of finding which fonts specifically caused the problems. I really don’t have much use for the extra fonts so I’m just as happy to have them all gone. Still hopefully this helps someone.

Now if I could just clean up all these damn mds errors that keep cropping up …

mds[34]: (Error) Import: importer:0x84b600 Importer start failed for 501 (kr:268435459 (ipc/send) invalid destination port)

From Chrome with Love

Chris Whitmore — Wed, 03 Sep 2008 08:38:00 UTC

A million bloggers all posting on the same topic, why shouldn’t I join in. The Google chrome team has got to be enjoying themselves right now. I read the comic yesterday and really enjoyed it. Seeing a company I can’t help but admire sit down and rethink the browser in so thorough of a manner is inspiring. Even just the QA involved is pretty damn impressive.

I’ve been using the chrome browser for a day now and have to same I’m pretty happy with it. I can already feel a need for some of the firefox extensions that I rely on so heavily, but at the same time I feel more productive and less distracted in this browser than I do in firefox. It’s FAST, really fast on my machine at work. I can’t wait until this is available for my mac.

I really enjoyed this article from John Siracusa, it sums up nicely what I find so inspiring about this and points out the real motivation for Google to create yet another browser.

http://arstechnica.com/staff/fatbits.ars/2008/09/02/straight-out-of-compton

(I’m almost regretting my recent commitment to developing an RIA in silverlight! Where oh where are my first class developer tools for browser based development…. )

Regression Ratios

Chris Whitmore — Sat, 09 Aug 2008 00:12:00 UTC

Regression is a nasty issue. Ongoing regression from bug fixes can be a pretty clear indicator that there are some serious problems with your code base, your process, your team or all of the above. As an example case consider the effect of moving from 1 in 5 of all bug fixes causing an unrelated issue to crop up to 1 in 10. Given an imaginary scenario where 1000 bugs are found (a medium size project) and a team is closing 20 of those bugs a day then in an extremely simple model we have just added two weeks to our timeline simply from regression issues. (handy chart from google spreadsheets below)

Of course its much more insidious then that though in the real world. For one, regression bugs only pop out at you the minute they are introduced if you’re lucky (they are obvious). More likely is that about half of them will show themselves and the other half will start cropping up near the release date during regression passes. If you alter the model above to show that you’ll see little bulges near the end of the line that start to make time lines feel very elusive and untrustworthy. And given that no QA process is perfect you have to believe that whatever is causing you to introduce those bugs that you are seeing is also causing bugs that are yet to be found.

The importance of quality can never be over-stated, and the effect of poor quality can have a really dramatic impact on a team as well as your customers. Here you are, slaving over your own code, crafting a well thought out and tested solution to a problem, only to have someone else come along and break your code with their fix. Here you are, proudly about to successfully bring a feature out of the QA phase and into the hands of customers and all these elements that you had already seen work are failing. How quickly your trust in your fellow developer begins to wane.

Here though is a problem that I have come to see the light on as being truly systemic. Yes there are horrid developers, and yes there are some difficult technologies, and yes some level of regression is inevitable…. but levels of regression like those that I’ve seen can usually be traced back to process, to your design and architecture, to your requirements and most importantly to your culture.

In terms of process there are some stock answers to bringing this number down like TDD and constant refactoring. I believe in both of these, but in my experience these can be difficult to enfuze into a culture. Without full buy-in and a cultural shift in developers these are just more TLA’s to throw on the steaming pile of buzzwords that will help you magically improve.

Having just come to the end of a release of our product in the past few weeks I am looking forward to whittling away at our own regression ratios.

Changes we’re making :

No more sharing of the bug load. Every developer owns their module/feature/code from design through to maintenance. This include assigning ownership to old modules whose owners may no longer be around. If you write bad code, you will fix more bugs. If you fix more bugs, you will have less time to spend on new bug-ridden features. This will mean our actual bug trends won’t be the pretty descent we’ve worked so hard on achieving. But the overall effect of a self correcting system should help.
More atomic checkins. Checkins that span branch points, or handling merges of bugs can be extremely error prone with subversion and our current branching strategy. We hope to address this somewhat by eliminating bug fixes that span multiple revisions… Ideally this will involve :

Patch files attached to the bugtracker for code reviews to avoid multiple checkins when reviews spot errors
When changes still need to be made to a bug when things are caught in QA then we also revert the original checkin, and checkin again with all changes at once in the new revision.
Bugs will be closed diligently and new bugs opened rather than morphing the bug over time as new side effects or slightly related bugs are spotted.

More automated testing. This is last on the list not because it’s least important, but because we don’t expect as quick of an impact as with the other two steps. A major number of the issues we run into remain UI issues. And while WatiN does do wonders for us, it’s painstakingly slow to get the number of tests to where they should be (and keep them up to date).

Lastly just for interest here’s the graph of how we’ve actually done. Black shade are high severity bugs. The first mountain was the release previous where we amalgamated all previous releases into this (migration of existing clients) we had some pretty serious regression in that release after Christmas. The second range is the most recent release which is looking amazing in comparison, though at this resolution you are missing some definite climbs amongst that fall. Definite progress anyway….

Documenting architecture

Chris Whitmore — Mon, 21 Jul 2008 17:51:00 UTC

One of my side projects at work right now is documenting the architecture of a product that has already been built but will be going through a re-architecting with a focus on a more robust schema and applying some of the learning we've gone through in discovering exactly how our product is being used and ways in which our users want to extend the platform. SaaS and SOA are two good buzz words we'll be throwing around a lot, although to be honest we've been in the SaaS model for years now, just not following all of the best practises. (examples, check out litwareHR)

So despite documentation being at the heart of the architect's role I find it extremely difficult to find good documentation on how to approach a task like this. I have Craig Larman's book Applying UML Patterns which I've enjoyed, but I still find myself grappling for where to even begin sometimes.

These articles on IBM have been good reads for this and I'd reccommend giving them a read if you are facing similar challenges.

Part1
http://www.ibm.com/developerworks/library/ar-archdoc1/index.html?S_TACT=105AGX20&S_CMP=EDU

Part2
http://www.ibm.com/developerworks/library/ar-archdoc2/index.html?S_TACT=105AGX20&S_CMP=EDU

Part 3
http://www.ibm.com/developerworks/library/ar-archdoc3/index.html?S_TACT=105AGX20&S_CMP=EDU

I'm assuming there will be more of these which I'm looking forward to.

Networks, Assignment 1

Chris Whitmore — Sun, 20 Jul 2008 16:39:00 UTC

I just finished my first assignment in a beginning networking course I’m taking and I am so far pretty impressed with how interesting this stuff is. I have a working knowledge of networking that includes decent understanding of the application layer, high level knowledge of the transport layer and basically just awareness of the link layer. It’s pretty rare that in my position as a developer that I need to answer questions about the link layer. (thank you my friends in IT)

Some of the questions are actually kind of fun in that they had me visualizing data flowing through networks in ways I had not before. For example given a link between two hosts X km apart, with a transmission rate of R and a propagation delay of N….

2.4.d What is the width (in meters) of a bit in the link? Is it longer than a football field?

Kind of useless, but super fascinating at the same time, imagining the physical manifestation of all this work I do day in and day out. Pulling these bits from all over the world is so effortless, so fast and so transparent that it’s easy to forget the actual resources behind it.

The football field question actually relates to a pretty interesting concept called bandwidth-delay, which refers to the amount of data that exists “on the wire” or “on the air” at any given moment. Data that has been sent but not yet acknowledged. It’s helpful in determining minimum buffer sizes for receivers and transmitters over a given link.

http://en.wikipedia.org/wiki/Bandwidth-delay_product

Another element to this first assignment was to setup apache as a proxy server on your local machine which was a bit surprising. I assumed at first that the assignment meant squid, but no, apparently apache itself can be configured to be a proxy server for a number of protocols including both ftp and http traffic.There are numerous articles out there on using it as a personal ad blocker, or caching server.

For reference this is what I had to do to Httpd.conf to make it work :

LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so

<IfModule mod_proxy.c>
ProxyRequests On
<Proxy *>
Order deny,allow
Deny from all
Allow from 10.0.1.2/255.255.255.0
</Proxy>
</IfModule>

<IfModule mod_disk_cache.c>
CacheRoot “c:\apachecache"
CacheDirLevels 5
CacheDirLength 3
</IfModule>

Interestingly if you get that configuration wrong, you actually get a big “It Works!” page shown in your browser for any page you try to visit. Go figure. My mistake at that point was just not having uncommented the right modules, so apache was just serving the It Works page rather than attempting to proxy my request.

blogquotes prototype is working

Chris Whitmore — Sat, 19 Jul 2008 21:50:00 UTC

I’m supposed to be studying for a challenge exam I’m writing this week in “Advanced” Operating Systems. Instead I spend a good chunk of the day today working on blogquotes in between watching/playing with my daughter.

I can justify this time spent because I did all of this work exclusively from a bash shell using vi to refresh myself on some of the content for the course. In this session I was able to use wget,grep,awk,vi,a shell script and some file permissions. I’m not so sure that will get me through the exam but it was fun and I was finally able to put some time towards my random quote include for the blog.

You can see the quotes being pulled in now in the top right hand corner of this page. So far only my wife and I are using it, as this is very much a proof of concept. It works basically from end to end, but without a lot of features that are going to be necessary as this grows. Paging, searching, caching, tags, some UI polish, and some testing. It’s very gratifying though to reach this first phase and actually get something working. Adding features will probably happen a lot quicker now that I actually have a user. ;-)

I also took the opportunity to try out some of Yahoo’s client side api’s in the YUI. Currently I’m using XHR, Layout and DataTable. I was amazed at how quick it was to basically “assemble” my application. Google’s app engine makes the CRUD a total cake-walk, and yahoo’s user interface library has no dependencies on server side code but works seamlessly with a JSON backed RPC scheme in Python. It’s a whole new world! Now as long as my application doesn’t get popular and I have to start paying for resources. ;-)

Some interesting snags while working on this latest revision:

Randomly selecting an entity in GQL
Django Utils simplejson can’t serialize google’s db.Model classes, so I had to actually proxy my model class to a simpler structure that I serialize to the client via JSON and XHR
No unique id’s for google user accounts, all you have is email which isn’t exactly something people are going to appreciate me passing on url’s for the random inclusion widget. The solution was simple for what I needed, a user preference entity keyed on google’s User db type and storing a UUID as the publishingKey, that id now becomes my unique id which won’t change even if you change your google account name
Google has a very cool AJAX Library SDK for sharing hosting/serving up of all the most popular frameworks like dojo and jquery

I think I’ll use google code to start some documentation around bugs and features for this tool rather than simply the blog so look for those details elsewhere.

code samples in blogger are a pain

Chris Whitmore — Thu, 17 Jul 2008 22:58:00 UTC

I can’t say I enjoy writing in the blogger post interface, in fact it’s pretty frustrating. For a while there I was using google docs to write posts(which I loved), then I would just publish to my blog. That actually worked great until the actual publishing process which doesn’t allow you to control the title very effectively and totally messed up my rss feed even if I did fix the title. Then I tried scribefire which again was really promising but it’s a cramped UI and again the publishing process was really clunky for my workflow. (things remain drafts for me for weeks at a time)

Anyway, I’m looking at my last post and those code samples are embarrassingly poorly formatted. Not only that but if you check the source the blogger editor is introducing tons of html space entities which drives me nuts considering I’m using whitespace:pre on my blockquotes anyway.

I’m really inclined to just use the tools I have when it comes to this site, primarily so that I focus on writing and not tinkering. Since moving my website from a hosted environment to blogger I have actually started to focus again on my writing and my projects rather than tinkering with a wheel that’s been built a thousand times (photo gallery scripts, php and perl cgi trickery for mundane templating etc). So while I will probably end up spending time on this at some point I really just want to find something that “just works” for showing code in blog posts. More to come I’m sure.

Microsoft's add-in framework and the need for diligence

Chris Whitmore — Thu, 17 Jul 2008 21:21:00 UTC

We've recently put Microsoft's managed add-in framework (part of .NET 3.5) into very effective use building a plug-in system for a large asp.net application at work. Essentially the framework in place allows other developers (and our own team for out of stream releases) to develop new functionality for our platform that runs the entire life-cycle for a given widget. In our case for this particular widget we're talking about plugins being responsible for up to 4 asp.net controls in different contexts (for example data collection and reporting as two separate controls) as well as a script injection point where plug-ins are able to extend the scriptability of our platform.

For us going with the framework gave us a few things we didn't have with our original design for the add-ins.

Tools to help enforce the pattern
An extra layer of versioning over the somewhat naive approach we started with
Built in discovery, provisioning, and a communication pipeline for serializing types and calls across the contracts that make up the interface between host and plugin
And last but not least support from Microsoft. This is somewhat more minor than the points above, but it helps legitimize our design when we are following the best practices laid out by Microsoft and used by others in similar situations. The documentation and training available also make getting other developers up to speed on the framework that much easier.

There have been numerous challenges in using the framework, but perhaps the most surprising of all for me was the human element and how simple it became over the life of the project to break the pattern by coupling components across or outside the pipeline.

Examples :

Referencing an assembly from both the addin and the host that shared code that should have been passed across the pipeline.
Bypassing the pipeline completely by calling web services from the addin code (client side or server side calling code)
Conditional code in the host making decisions based on the type of the addin
Loose coupling based on common knowledge (that shouldn't be common)

These all basically come down to a breach of contract or an absence of contract for various operations that we needed addins to handle. On some level all of these things can be excused and safely done without compromising the framework if they are done right. It's a slippery slope though and requires a commitment to not be lazy to avoid the temptation to sidestep the pipeline.

In the case of #1 above the shared assembly started off very benign. Essentially some shared utility code for handling urls and some common resource tasks. Why rewrite when that code already existing the main project? Break it off from the project so that it has no dependencies then drop it in. Except that slowly the terrible pain of building contracts, views and adapters for every little interface or interface change drives you towards shortcuts. "Oh I'll just put this code here to test and then fix it later" Even worse are those cases where you've chosen the path of least resistence in dealing with a bug resulting from unexpected behavior with serialization across the pipeline. It only took a few weeks of not being completely on top of this before I discovered our project was littered with types that were being shared directly between host and addin. Any change meant a recompilation of both projects, completely defeating the purpose.

#2 is a legitimate need in our scenario, and we've found ourselves needing to creating proxy services that wrap our own services just to protect against the inevitable change that will follow. Given that third party developers may be writing code for the platform we have to make an effort to protect from change in all of our interfaces, web service or otherwise. In retrospect I think it would have made more sense to strictly enforce a team division so that no one writing addin code was also writing host code.This probably would have gone a long way to preventing these types of problems.

#3 and #4 are a little more insidious and harder to spot without strict code review. #3 for us isn't technically breaking anything in terms of the interface or future versioning, but adds cruft and generally points to a missing method or property on the interface. The last thing you need as the host is to have case statements littered throughout your code looking for addins. #4 took many forms, and in some cases it's fine. An ok example might be sharing enums, which provided they are defined in the contracts or slightly worse something like a utility class is ok. A not ok example for me was code like this : extension.GetSetting("Menu_Text"); which in this case has two errors. One "GetSetting" shouldn't really exist because how an addin chooses to configure itself should be transparent to the host. Second this code depends on the addin having a value defined in it's config file for the key "Menu_Text". This is next to impossible to enforce and can of course easily break.

Replacing this with extension.MenuText; should be trivial, and a no-brainer. When we started using the framework back in December we were rolling the supporting code by hand. To give you a sense of what this entails, this is how you would define an extension who's only job is to return MenuText as in the code above :

IExtensionContract.cs

using System.AddIn.Pipeline;
using System.AddIn.Contract;

namespace SimpleExtensionContracts
{
 [AddInContract]
 public interface ExtensionContract : IContract
 {
 string MenuText { get; set; }
 }
}

IExtension.cs

namespace SimpleExtensionContracts.AddInViews
{
 
 [System.AddIn.Pipeline.AddInBaseAttribute()]
 public interface IExtension
 {
 string MenuText
 {
 get;
 set;
 }
 }
}

IExtension.cs

namespace SimpleExtensionContracts.HostViews
{
 
 public interface IExtension
 {
 string MenuText
 {
 get;
 set;
 }
 }
}

IExtensionContractToViewHostAdapter.cs

namespace SimpleExtensionContracts.HostSideAdapters
{
 
 [System.AddIn.Pipeline.HostAdapterAttribute()]
 public class IExtensionContractToViewHostAdapter : SimpleExtensionContracts.HostViews.IExtension
 {
 private SimpleExtensionContracts.ExtensionContract _contract;
 private System.AddIn.Pipeline.ContractHandle _handle;
 static IExtensionContractToViewHostAdapter()
 {
 }
 public IExtensionContractToViewHostAdapter(SimpleExtensionContracts.ExtensionContract contract)
 {
 _contract = contract;
 _handle = new System.AddIn.Pipeline.ContractHandle(contract);
 }
 public string MenuText
 {
 get
 {
 return _contract.MenuText;
 }
 set
 {
 _contract.MenuText = value;
 }
 }
 internal SimpleExtensionContracts.ExtensionContract GetSourceContract()
 {
 return _contract;
 }
 }
}

IExtensionHostAdapter.cs

namespace SimpleExtensionContracts.HostSideAdapters
{
 
 public class IExtensionHostAdapter
 {
 internal static SimpleExtensionContracts.HostViews.IExtension ContractToViewAdapter(SimpleExtensionContracts.ExtensionContract contract)
 {
 if (((System.Runtime.Remoting.RemotingServices.IsObjectOutOfAppDomain(contract) != true) 
 && contract.GetType().Equals(typeof(IExtensionViewToContractHostAdapter))))
 {
 return ((IExtensionViewToContractHostAdapter)(contract)).GetSourceView();
 }
 else
 {
 return new IExtensionContractToViewHostAdapter(contract);
 }
 }
 internal static SimpleExtensionContracts.ExtensionContract ViewToContractAdapter(SimpleExtensionContracts.HostViews.IExtension view)
 {
 if (view.GetType().Equals(typeof(IExtensionContractToViewHostAdapter)))
 {
 return ((IExtensionContractToViewHostAdapter)(view)).GetSourceContract();
 }
 else
 {
 return new IExtensionViewToContractHostAdapter(view);
 }
 }
 }
}

IExtensionViewToContractHostAdapter.cs

namespace SimpleExtensionContracts.HostSideAdapters
{
 
 public class IExtensionViewToContractHostAdapter : System.AddIn.Pipeline.ContractBase, SimpleExtensionContracts.ExtensionContract
 {
 private SimpleExtensionContracts.HostViews.IExtension _view;
 public IExtensionViewToContractHostAdapter(SimpleExtensionContracts.HostViews.IExtension view)
 {
 _view = view;
 }
 public string MenuText
 {
 get
 {
 return _view.MenuText;
 }
 set
 {
 _view.MenuText = value;
 }
 }
 internal SimpleExtensionContracts.HostViews.IExtension GetSourceView()
 {
 return _view;
 }
 }
}

IExtensionAddInAdapter.cs

namespace SimpleExtensionContracts.AddInSideAdapters
{
 
 public class IExtensionAddInAdapter
 {
 internal static SimpleExtensionContracts.AddInViews.IExtension ContractToViewAdapter(SimpleExtensionContracts.ExtensionContract contract)
 {
 if (((System.Runtime.Remoting.RemotingServices.IsObjectOutOfAppDomain(contract) != true) 
 && contract.GetType().Equals(typeof(IExtensionViewToContractAddInAdapter))))
 {
 return ((IExtensionViewToContractAddInAdapter)(contract)).GetSourceView();
 }
 else
 {
 return new IExtensionContractToViewAddInAdapter(contract);
 }
 }
 internal static SimpleExtensionContracts.ExtensionContract ViewToContractAdapter(SimpleExtensionContracts.AddInViews.IExtension view)
 {
 if (view.GetType().Equals(typeof(IExtensionContractToViewAddInAdapter)))
 {
 return ((IExtensionContractToViewAddInAdapter)(view)).GetSourceContract();
 }
 else
 {
 return new IExtensionViewToContractAddInAdapter(view);
 }
 }
 }
}

IExtensionContractToViewAddInAdapter.cs

namespace SimpleExtensionContracts.AddInSideAdapters
{
 
 public class IExtensionContractToViewAddInAdapter : SimpleExtensionContracts.AddInViews.IExtension
 {
 private SimpleExtensionContracts.ExtensionContract _contract;
 private System.AddIn.Pipeline.ContractHandle _handle;
 static IExtensionContractToViewAddInAdapter()
 {
 }
 public IExtensionContractToViewAddInAdapter(SimpleExtensionContracts.ExtensionContract contract)
 {
 _contract = contract;
 _handle = new System.AddIn.Pipeline.ContractHandle(contract);
 }
 public string MenuText
 {
 get
 {
 return _contract.MenuText;
 }
 set
 {
 _contract.MenuText = value;
 }
 }
 internal SimpleExtensionContracts.ExtensionContract GetSourceContract()
 {
 return _contract;
 }
 }
}

IExtensionViewToContractAddInAdapter.cs

namespace SimpleExtensionContracts.AddInSideAdapters
{
 
 [System.AddIn.Pipeline.AddInAdapterAttribute()]
 public class IExtensionViewToContractAddInAdapter : System.AddIn.Pipeline.ContractBase, SimpleExtensionContracts.ExtensionContract
 {
 private SimpleExtensionContracts.AddInViews.IExtension _view;
 public IExtensionViewToContractAddInAdapter(SimpleExtensionContracts.AddInViews.IExtension view)
 {
 _view = view;
 }
 public string MenuText
 {
 get
 {
 return _view.MenuText;
 }
 set
 {
 _view.MenuText = value;
 }
 }
 internal SimpleExtensionContracts.AddInViews.IExtension GetSourceView()
 {
 return _view;
 }
 }
}

Yeah, seriously. One interface and one string accessor requires nine class/interfaces and over 200 lines of code (which obviously could be made less with formatting etc). It's also possible to share the views between addin and host but then you lose part of the more compelling robustness of the framework. If you are interested in where these classes come into play and how the add-in framework actually works check out this link for a good description.

Anyway, I can sympathize with the developers in wanting to speed up the process a bit, but the answer is not to bypass the pipeline. The answer is code generation! Thankfully by the time we realized our mistake Microsoft had released a CTP of their pipeline generator which is a nifty little visual studio addin which picks up the output of the Contracts project and uses reflection to find all of the contracts and generate the necessary projects and files for the pipeline. It literally saved us tons of hours and made the addin framework actually usable. Of couse the code generation is only going to work until we version one side or the other, but at that point we should have solidified those interfaces considerably so it will matter a lot less.

Anyway, long story short, the add-in framework is great, but it's really important for the entire team to understand the goal and be diligent in ensuring that all that extra framework code isn't just being wasted by introducing dependencies.

Bill C61 notes

Chris Whitmore — Thu, 26 Jun 2008 23:59:00 UTC

So I have a great personal distrust and disgust in the way copyright law has continually degraded and been abused by large corporations over the past 30 or 40 years . (Thanks Mickey!) I cringe at the idea of the RIAA sueing people to protect their broken business model and laugh my ass off when bands like Metallica (2) and Kiss make total asses of themselves while those artists that are still relevant embrace new ways of engaging their fans. Anyway, that’s the context for this post. I am legitimately interested in seeing how Canada will follow other countries in protecting artists, content producers and consumers.

As much as I abhor the record industry I do think the law should reflect the reality of the new digital landscape. Content creators need to be protected, and consumers should get their money’s worth when buying or consuming copyrighted material.

Anyway I watched these videos about the new Bill C61, and as painful as it is to listen to Jim Prentice repeat the same meaningless quip over and over in response to these questions, I did find it interesting. I’ve been really concerned about Canada following in the steps of the DMCA, and I imagine that under the covers this is largely similar, particularly where “digital locks” are concerned but I’ve so far not heard anything really terrible.

http://www.digital-copyright.ca/node/4761 (videos)

Some highlights from the videos

Need for international cooperation, if it lets me watch more movies online for cheaper then going to the video store then yeah let’s go. How long has that been possible in the US?
“and of course new technologies such as mp3 players and memory sticks” I just included that because it struck me as funny.
Time shifting and format shifting is preserved
BUT “digital locks” which are chosen by businesses (ie stopping format shifting) will be legally enforceable and will allow those time shifting and format shifting rights to be circumvented
Having personally already rented videos on iTunes I can see some benefit to these locks and some of the new models they enable (death to blockbuster)
Also though owning a bunch of iTunes tracks I can’t use on other devices makes me hate the locks. But ultimately this is my decision on what to buy so I can’t complain too much. It’s important to let the market decide on some of these issues I think. Although how much of a market is it really when there are only a handful of really big labels and studios producing all the content?
no liabilities for ISP’s is great I think
New limits on the liability for “personal use” of copyrighted material to $500 PER INFRINGED WORK (they kept playing up $500 limit as a good thing for consumers but it sounds like downloading 4 movies is still a $2000 hit)We all know how quickly this would completely sink most households. In Prentice’s example this goes from five videos at $20,000 each = $100,000 to $500 total… he didn’t seem to really have that part down and I’m still not sure if it’s $500 per work or per incident or whatever?
Not that that matters as this is totally unenforceable, the law will enable companies to more confidently invest in delivery mechanisms that rely on locks and have a clearer understanding of rights, but none of it will help anyone actually enforce it. (Unless the companies do it themselves ala the RIAA sueings)
This whole bill was rammed through right at the end of summer with little consultation, seems ugly
When timshifting you can’t store those as a library of recordings, again very vague and would seemingly limit PVR software quite a bit.
You can’t import devices into Canada that enable bypassing locks (vague, could encompass a lot of devices )
Teenagers seem to be the “they” in all these videos. Makes the questioners and the lawmakers seem out of touch. I understand teenagers are big offenders but they are by far not the only ones.
What are the whitewood treaties Mr Prentice brings up? I’m assuming I didn’t hear him because I didn’t see anything on my initial Googling of it.

The business network video is the best video of the bunch, if you want a summary just scroll down and watch that one.

One of the things I was most surprised to hear was from Mr. Sookman saying that time shifting and format shifting is currently NOT legal? Really? I had always been under the impression that this was actually legal in Canada.

Check out this website for more information on Bill C61
http://www.digital-copyright.ca/

Enough rope to hang yourself (C# Extension Methods)

Chris Whitmore — Thu, 26 Jun 2008 11:16:00 UTC

So I ran into an interesting “gotchya” with C# extension methods tonight. And of course it happens at the 11th hour on a project that is being demoed at 9:00am tomorrow morning. Of course.

Extension Methods

Extension methods are a really cool new feature of C# that were introduced in version 3.0 of the language. Essentially they are static methods that act like instance methods, allowing you to extend objects you don’t own. Ok, seeing those words on screen makes me think this is a terrible idea, but it really does have it’s place. (LINQ relies heavily on it)

When I first saw these I got quite excited because we had a number of scenarios where they would make our code much much cleaner and easier to maintain. I can name a couple examples in our own project where this has the potential to make the API cleaner.

Example 1: Helper/Utility/Static Methods

Try as they might, the .NET framework guys will not anticipate every piece of code that ends up repeated hundreds of times across your project to get around a common case. Before the framework added String.IsNullOrEmpty() in 2.0 we had StringHelper.IsNullOrEmpty(string arg)in our code base along with about half a dozen other methods. Now string may not be the best example, because in my mind I think it’s a bad idea to write extensions to framework types, but it does illustrate the problem well.

In our project we have maybe a dozen of these Helper classes full of static methods (utility pattern). They are useful, but the biggest problem is the team’s ability to consistently discover those methods. Emails and other forms of communication helps, code review helps but ultimately you end up with repeated code fragments where helpers could have been used or even worse you end up with competing helpers in different namespaces that need to be consolidated once discovered.

Example 2: Enhancing Functionality Based on Context

Another useful place for extension methods in our project is to have our domain model be extended differently based on the area of the application and what rights the users executing that code have. Essentially what we have is two sets of functionality that are implemented very differently based on the context. For example in one context our object model is mapped using an ORM to the database directly, where as in another context that same object model is used with pre-populated data sets that are cached and completely scriptable by our end users. We still have code that needs to understand these objects across this context however, leading us to create interfaces for every single object in that domain that needs to work across these two contexts. I’m a fan of interfaces, but I think in this scenario we’ve clearly lost something in terms of the DRY principle and code readability.

I’ve yet to really map out what this will look like for our project, but I see what’s been done with Linq and am excited about how simple it could be for example to include the “.Scripting” namespace to attach methods to our domain model that are exposed to end users. Similarly an “.API” namespace for our internal privileged code base with everyone sharing the same core objects.

The Gotchya that kept me at work an extra hour

Symptoms

Your extension method is no longer ever being executed

Intellisense shows the correct method signature, reports no problems

Right click “go to definition” takes you to the method you think will be hit

Stepping through the code shows you never reach your extension method

I’ve created some fake code to illustrate the problem. Imagine we have a simple factory class providing some useful functions like so :

namespace ExtensionMethodsTest
{
public class FactoryA
{
public ObjectA GetInstance()
{
return new ObjectA(“empty object”, -1);
}

public ObjectA GetInstance(string name)
{
return new ObjectA(name, 0);
}
}
}

But then we decide that we actually need to grab instances of ObjectA using an int id, so we add that using an extension method like so :

using ExtensionMethodsTest;

namespace MyExtendingNamespace
{
// a container for my extensions
public static class ExtendFactoryA
{
// Extend our factory to look objects up by id
public static ObjectA GetInstance(this FactoryA factory, int id)
{
return new ObjectA(“got by id”, id);
}
}
}

Blamo! Now anytime we’re using the “MyExtendingNamespace” our FactoryA includes the third way to grab an instance of ObjectA.

Here is how I am calling both of these :

private void ByIdButton_Click(object sender, EventArgs e)
{
FactoryA fa = new FactoryA();
ObjectA a = fa.GetInstance(42); // call extension method with int
this.label1.Text = a.ToString();
}

private void ByNameButton_Click(object sender, EventArgs e)
{
FactoryA fa = new FactoryA();
ObjectA a = fa.GetInstance(“gotten by name”); // call instance method with string
this.label1.Text = a.ToString();
}

This seems ok, our calls to GetInstance are consistent and we have everything working. Now transplant yourself a couple months into the future when the original owner of the FactoryA class has a need to alter the behavior to allow for more scenarios for retrieving instances of ObjectA.

public ObjectA GetInstance(object proxyObj)
{
return new ObjectA(proxyObj.GetType().Name, 0);
}

Now from the calling code everything still appears to work fine, intellisense continues to show the “int” signature that you think you are calling, and in fact if you right click the GetInstance call and say “Go to Definition” it takes you to the MyExtendingNamespace version of GetInstance.

At runtime however the CLR looks first for something that matches your call to fa.GetInstance(42); within the main namespace/class before looking at any extensions. Because our int gets automatically boxed we actually match “object proxyObject” and will never reach the extension method. Depending on the specific implementation in your code this can be a particularly insidious error or it may fail outright. Either way on a large project it can be a real annoyance to track down.

To me there are a few mistakes here. 1) Why use an extension method here at all? 2) In general I think extension methods as overrides make little sense given how the CLR matches these and 3) GetInstance(object proxyObject) should probably be something more specific like ProxyBase or something else that is not an object. (Akin to catching System.Exception, bad… be specific)

Fixing this issue is as simple as renaming the extension method to GetInstanceById, or moving the method onto the Factory itself, or fixing the factory method to not use object. Personally I say drop the extension method.
My example is over simplified, and in our scenario at work there was a little more justification for not changing the class that defined the original functionality. To me though this seems like the tip of an iceberg of problems.

Why negotiate with a module maintainer when I can just add functionality right here right now from my own code?
Why stop to understand that class and how it’s been constructed when I can just impose my view of how it should work overtop?
Intellisense makes it easy to find the method, so I don’t have to worry about how I organize this code.

Terrifying.

here here - use var sparingly

Chris Whitmore — Sat, 21 Jun 2008 21:21:00 UTC

When I first read Jeff Atwood’s latest post on his love of C#’s new “var” keyword I was deeply bothered that my co-workers would find the article and latch on to the argument as a justification for laziness. While I do understand his point of view I was bothered by the idea of var statements littered throughout the code base making things more difficult to read for the next developer.

Saving key strokes is never justification for obscuring the code base. If you want to save keystrokes improve your environment, don’t sacrifice your code.

I came across this post on reddit tonight that very nicely counter’s the post on coding horror. Thanks Richard for a voice of reason.

http://richarddingwall.name/2008/06/21/csharps-var- keyword-jeff-atwood-gets-it-all-wrong

Dynamic vs Static... no wait make that DBC!

Chris Whitmore — Sat, 14 Jun 2008 21:14:00 UTC

I need to blog this basically to toss it in my archive. There have been some interesting posts on the religious debate of static vs dynamic languages. I don’t know why I always get drawn into these lines of thought, but I do. (in fact I just added a “versus” label)

I say drawn in because my underlying philosophy in all of these things is to choose the right tool for the job and leave it at that. I know, hardly original thinking, but despite the mantra and the collective nod that this is true we still get very heated on issues that are not actually at odds with each other.

I’m NOT arguing that with you. I’m not arguing that with YOU. I’m not ARGUING that with you. I’m not ARGUING that with you Harry! Harry… Harry… Yeah Harry… but can he DO the job. I know he can GET the job but can he do the job?
Mr. Waturi, Joe vs the Volcano

Still there is fun to be had in the whole exercise. For the record I tend towards the static languages. Despite my recent fun with Python I have spent the last four years neck deep in C# and really am loving it. Our application uses over 75,000 lines of JavaScript, and every opportunity I have to decrease that number I will take. (Part of my excitement about Silverlight is just not having to write as much JavaScript anymore) I see the power in dynamic, I’ve done some really cool things with JavaScript and I’ve really enjoyed working in Python again…. but I believe there is less of a ceiling for static languages then there is for dynamic in terms of tool set, performance and the ability to handle large projects with large teams.

Anyway, Mat Podwsocki presents a great summary of the debate with links to a few bloggers here :

http://codebetter.com/blogs/matthew.podwysocki/archive/2008/05/28/static-versus-dynamic-languages-attack-of-the-clones.aspx

The original Steve Yegge presentation is here :

http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html

And the response that I enjoyed the most is here :

http://codebetter.com/blogs/gregyoung/archive/2008/05/18/revenge-of-the-statically-typed-languages.aspx

I enjoyed Greg’s response because it just really got me thinking. I honestly know nothing about the design by contract research that’s going on, but it just feels like something that makes sense. I got excited just imagining seeing errors like the ones described showing up in our project at compile time. I believe that if we were using such a system we would see a real improvement in quality. I can see how some would see this as an unnecessary burden on the developer, but these are probably the same crowd not writing their unit tests.

Design by contract is really a great example of what I mean when I’m thinking about the potential for statically typed languages in tool sets.

Gadgets in Blogger

Chris Whitmore — Fri, 13 Jun 2008 22:30:00 UTC

So I’ve actually started my blogquotes project which was intended as a widget style provider for random quotes (from a personal library) to appear somewhere on my site. I’m using the new Google app engine for this project and so far I have to say it’s pretty damn easy. From barely knowing python to having a django templated, gql driven tiny quote engine. Granted it’s the “hello world” of web apps, but still considering I have an application running on Google, with built-in authentication and datastore …. I’m pretty impressed.

I’ve played with ruby on rails before, but so far I’m far more comfortable with what I’m building here than I ever was with Ruby. (I don’t think I’ll start detailing that here and now as this post is more about my gadget plans and I don’t care)

So I have my application, and I have the beginnings of the backend for my quote providers. I just went to draft.blogger.com where google tests out new features and clicked “add gadget” from the layout view of the blog. I actually thought that what I was looking at was a mistake. 43,000 page elements and gadgets? How is that useful to anyone? How am I not just making the problem worse by adding another one that no one will find and no one but me will use? This is just insane.

I did a search for quote on that page and it returned 13, thankfully none of them are what I have in mind. They are all basically syndication style “[fill in topic] quote of the day” style widgets. There actually seem to be a surprising number of gadgets like that. Things that just don’t really allow the user to add anything meaningful or contextual to the web. I suppose a lot of people use their blogs as their home page, but for me having the weather and a stock ticker on my blog just doesn’t make sense. My Google home page sure, but on millions of public blogs not so much.

There also seems to be too many thinly disguised marketing devices and cheap looking ad-style nothing widgets. Personally I would like to see more of the types of gadgets that Google themselves are creating, where user generated content is the focus (my youtube videos, my picasa albums, my google docs etc) as opposed to content that is one size fits all for millions of blogs. At least it gives me some hope that what I’m doing is not a complete right off.

The data won’t stick, but if you have a google account you can log in and add a quote to the pre-alpha-hello-world-version of blog quotes at http://blogquotes.appspot.com

Google's changing the rules again (appspot)

Chris Whitmore — Tue, 10 Jun 2008 22:38:00 UTC

I am super excited about Google’s recent foray into cloud computing, the app engine. They are not first, I realize Amazon has been doing some pretty cool things with the S3 services and their computing services but the ability to simply and easily have an application that can leverage BigTable and GQL, google account authentication, image manipulation, memcache and having that all live within the seemingly infinite scalability of the google platform is extremely exciting for me. In fact it’s the whole reason for my recent resurgent interest in python. App engine out of the gate is a python runtime environment which is robust enough even to run frameworks like Django for your web apps.

ArsTechnica had some good comments on the service and one of their cheif concerns was being locked into the platform. I think this is not an invalid concern, but that the tools will be written to make migration from one platform to another a relatively trivial matter. From what I understand of GQL it is a subset of normal SQL with limitations like no joins which should make data/code migration easy, but scalability and hosting maybe not so much. There are open source implementations such as HyperTable which is modeled after Google’s data platform, but ultimately the article is right. Personally I don’t see myself writing large scale applications on this platform though, I see this much more as a utility model for small useful services.

I registered for the preview of app engine on the same day it was announced but not early enough to be one of the lucky first 10,000. When I did get my invite a couple weeks ago I immediately started to create applications. It’s clear this is exactly what everyone else did and I’m glad Google has capped usage at three applications per developer. Already it’s almost as bad as trying to register a domain. For each of my three applications I had to try a series of names before getting a clear one. In other words none of my ideas are at all unique. :-) On a positive note when I checked the names I was looking for there were also no uploaded applications for any of them. I predict Google may have to have an expiry date on these - I can see myself taking weeks or months to finally get anything uploaded.

http://silitrader.appspot.com (silverlight port of the old game OilBaron)
http://wherediditallgo.appspot.com (personal spending tracker)
http://blogquotes.appspot.com

I actually created that last application as I was trying to find a simple means of having a rotating list of hand picked quotes show up in the header of my blog. Back when somatose.com was hosted with LunarPages this would have been trivial, throw the quotes into a textfile, write a script to randomly choose and insert one as I was building the page. I moved from a hosted environment to using blogger so that I would stop tinkering and focus on actual content. I spent most of tonight tinkering though, building up to actually building blogquotes. (By the way I am sure there are other blog quote like tools out there, but this is a rare case where I don’t mind reinventing the wheel somewhat)

Now the nature of the tinkering has just changed. Rather than write a small shell script that spits out random lines I will actually build something of a small application/service. The quotes will be stored using the datastore api (GQL), and will be publicly visible. Other users will be able to authenticate and store their quotes in a similar manner. A widget will be built and hosted from that application that enables anyone to paste in a snippet of html to add random quotes from their personal db. I as a user can subscribe to my friend’s quotes. It’s remarkable really, the differences and yet familiarity of this new ecosystem. And while I do miss the control of simply writing a little shell script I have to say I love the idea of the internet being populated by these little widgets of functionality. The emergence of ajax and increasing power on the client means that millions of non-technical users who blog on platforms like blogger and wordpress can very easily use my quote engine. How cool is that?

Google’s API’s (and Amazon’s) are proving to be the next Win32. Massively widespread and base functionality that anyone can build on. The plumbing is there for us to be able to chain together these small tools/widgets in much the same way a clever unix user could within their shell. And once the honeymoon wears off and I actually give some though to hosting data in the US and dealing with popularity should one of my apps get hammered and I end up having to pay Google usage fees… well until all those little issues I can just be excited again about the changing landscape.

Moving to the interior

Chris Whitmore — Tue, 10 Jun 2008 08:13:00 UTC

The promise of the electronic office has been so damn close for so damn long now. I know there are thousands and thousands of people out there who now work primarily from home, but I remain on the outside looking in. (or inside looking out?) Visions of a city like Nelson or maybe Prince Rupert dance in my head, with me in my large solitary office overlooking the lake, hardwood everywhere, the distant sound of my daughter playing outside. <deep virtual breath> My neural implants connect me to the office at “ludicrous speed” and my Neo-like abilities make distance immaterial.

Reading reddit too much can make you paranoid (and slightly dysfunctional, and unproductive, and slightly dumber) and these days I’m thinking too much about the impending collapse of our society and my desire to get the hell out of dodge before it happens. Preferably a nice small mountain town on a lake here in BC. Somewhere I can grow and fish/hunt food if the shit hits the fan. Somewhere where I’m not spending 40% of my income on housing. Of course, the idea of me hunting is almost enough to make me cry with laughter… I am so not a hunter.

Anyway, this post is about robots not the apocalypse. There are a few robots now intended to make the awkward ritual of sitting around in a board room and talking to a TV a little more awkward by having the TV come see you! Wonderful wonderful idea, and I’m sure extremely fun for the remote worker. A little less fun for the nervous employee constantly watching their backs for this thing to roll up behind them.

I can’t imagine this thing not having kick-me signs and other paraphernalia hanging from it within minutes in mockery of the whole process. I also can’t imagine being the remote worker and resisting the urge to roll around the office all day looking for interesting things to see. I think an episode of the office where Michael gets one of these would be brilliant and the comedic geniuses on that show could have some fun.

This isn’t what I normally consider a robot because it’s not autonomous, but it’s still cool, and if it helps me unplug from the city while staying connected to work I love it.

Giraffe video conferencing robot to weird employees out - Engadget

HeadThere - the maker of the Giraffe

Do I really need to embrace C++?

Chris Whitmore — Sat, 07 Jun 2008 13:21:00 UTC

My internal debate rages on… I have some C++ courses in the pipeline so I will definitely learn more than the basic understanding/grasp I have now…

Still though. Currently I associate C++ proponents with old-school attitudes on the need for absolute control and a fear of newer levels of abstraction that are allowing increasing complex and large projects. Garbage collection, the .NET framework etc, are all grouped together into a category of tools that only “weak” programmers need. I recoil pretty hard from this line of thinking and associate it with a dangerous cowboy like attitude towards engineering.

There is a lure there, to join the elite, to be one of those cowboys who don’t need frameworks or garbage collection and who’s code can scream past yours with sheer force of will by the programmer. So I’m constantly torn between mockery and envy. I will say that I think it’s important to learn and think in C++ because the deeper our understanding of the systems we develop for the better off we are. I’d say the same thing about assembly though and would be just inclined to write large systems in C++ as I would in assembly.

My own line of thinking is that these abstraction layers we’ve invented are more akin to compilers and file systems and OS API’s. Why should we all re-invent the wheel (and poorly) when these problems can be solved in a generic way?

Anyway C/C++ are probably not going anywhere and have their niche, but I thought I would post this article to preserve it in my memory bank.

http://www.mistybeach.com/articles/WhyIDontLikeCPlusPlusForLargeProjects.html

Power Consumption

Chris Whitmore — Fri, 06 Jun 2008 20:47:00 UTC

I’m a graphing junkie, I have to admit it. So I was pretty pleased to discover that my power company (BC Hydro ) provides some interesting consumption graphs for month to month kilowatts per hour usage. I’m pretty sure my meter is only read every two months so some of the resolution is lost in this, but still it’s helpful.

One of the more striking trends I’ve seen in the data has been the birth of my daughter. Late nights, tons more laundry, cooking and heating bottles and leaving fans on so she can sleep. Power consciousness kinda goes out the window while we’re dealing with an unhappy Maggie.

Summer should see the trend reverse and then I think we’ll have to get creative in terms of keeping it down.

Vampires in Space

Chris Whitmore — Sat, 31 May 2008 22:27:00 UTC

I read a very fun book on my holiday in Manitoba this last few weeks. It’s written by a biologist who has turned to fiction with an eye to producing some good hard core science based fiction. I would compare his work to William Gibson and Arthur C. Clarke.

The book is called “BlindSight” by Peter Watts, and I’ve added it to my LibraryThing, there is a link to my full review below.

http://www.librarything.com/work/1333265/reviews/31302401

As an added bonus (I discovered this after buying the book) the author has published it under a creative commons license and you can read the entire book online for free. How cool is that? (Read it here)

There is a lot to chew on in this book, but my post subject says it all. Seriously though, don’t judge based on that, it’s a good book. No really…

Yes

Javascript session hacking

Chris Whitmore — Tue, 13 May 2008 01:08:00 UTC

I had to blog this, and yeah I'm going to be the billionth person to do so, but that's ok because no one reads my blog and it's basically just my personal public archive. (ppa?)

This guy http://www.thomasfrank.se/sessionvars.html has managed to find a very clever place to store data within the browser without resorting to cookies, flash or anything beyond basic cross browser javascript. Apparently "window.name" which lost it's usefulness when the spammers started making popups obsolete can contain arbitrarily long strings that persist as long as that window is open (equivalent to a session cookie)

This is damn cool, and while a part of me twinges at the hack-ish nature of overloading a window property with a bunch of json strings, well this is javascript we're talking about here so what the hell. You're already standing out in the rain trying to build a fire with soggy sticks and no pants so why not use what you've been given.

This is just one of those things I know I'll want to come back to one day for one of my pet projects so I thought I'd blog it.

Saturday night science link (DNA Visualization)

Chris Whitmore — Sun, 11 May 2008 02:45:00 UTC

Fantastic molecular visualization of how dna, rna and ribosomes work… where was this kind of stuff when I was going to school?

ASP.NET Ajax Pendulum

Chris Whitmore — Sun, 11 May 2008 00:53:00 UTC

I’m currently working on moving a web application from using some ad-hoc javascript and iframes to a full fledged ASP.NET Ajax implementation based around Telerik’s RadControls and a suite of our own controls. This is the third application we’ve given the ajax treatment to and each time we’ve taken a slightly different approach.

Approach 1 … “Javascript + Web Services”

Full view manager/page controller implementation on the client side to handle events thrown by all the components on the page
Homegrown “ServiceManager” to handle brokering calls to the server
- Centralized error handling
- Shared message contract base class to allow “sub-messaging” for the server to send information to the view controller on the client
- Blocking calls when things needed to be “modal” or serialized
- … we had intentions of also building in grouping of calls
- … we had intentions of canceling redundant calls when the user was asking for something that made a previous call unnecessary
Chatty - many calls, small amounts of data (largely json)
Javascript controls that render almost strictly client side based on data delivered over web services
Zero postbacks / Very fast

Expensive/Complex development and maintenance
Prone to memory leaks
More difficult to leverage third party controls

Approach 2 … “Update Panels”

Traditional ASP.NET development with a sprinkling of update panels where it made sense
Chunky communication… partial postbacks, but still heavy
“Simple” development - UpdatePanels were spotty at the time, there were quite a few workarounds hacked in to this interface
Medium reliance on third party controls
Less performance than “Javascript + Webservices”
Less transparency and control to developers

Approach 3 … “Javascript + Web Services + Update Panels”

A view controller/ page manager approach similar to Approach 1 except that in this iteration we are relying on the ScriptManager and it’s events to handle a lot of what we were previously rolling on our own.
A heavy usage of Telerik’s Prometheus suite of controls, which in turn is built against the Microsoft Ajax Framework
Webservice layer for a class of managed actions where we control the control flow via our view controller
Partial updates for more complex control interactions/workflows
- These are sometimes triggered by the controls themselves in which case we are dealing with “chunky” transactions
- Othertimes triggered by the page manager in which case we have small requests and potentially heavy responses
Good use of third party controls
Chunky or Chatty based on need
Medium development complexity
Less transparency/control for developers
High development complexity
Better performance than Approach 2) but less than 1)
Still prone to memory leaks

So you can see across these three different applications we’re dealing with a swinging pendulum of ajax development methodologies and philosophies. My hope with our most recent approach was that we would have the strength of increased reliance on Microsoft’s framework + Telerik’s suite of controls combined with hooks and web services to eek out that extra level of performance whenever needed.

Of course in this latest iteration we also have the added complexity of having to port legacy controls on a very difficult timeline. So we have a collection of editors, some of which are doing chunky partial post-backs whereas some of our newer type editors are using a more… let’s say finely tuned approach where only the necessary data is sent and only limited page updates are performed.

The third option still has some proving out to do. Rather then being the perfect blend of complexity vs performance vs reuse vs transparency is instead turning out to be not especially good at any of these. I’m frankly a bit frustrated by the whole exercise right now an am increasingly eyeing the glorious promises of Silverlight.

Really I don’t see this stuff getting any easier until we’re either seeing browsers that have equivalent capabilities in supporting Html 5 and all the exciting new javascript features of the future or a widespread installed base of Silverlight 2. haha… right.

poetic words = exciters of nearby symbols

Chris Whitmore — Wed, 07 May 2008 00:07:00 UTC

In GEB Hofstadter mentions the complexity in building an isomorphism between two poems written in two languages.

“In ordinary language, the task of translation is more straightforward, since to each word or phrase in the original language, there can usually be found a corresponding word or phrase in the new language. By contrast, in a poem of this type, [Jabberwocky, Lewis Carroll] many “words” do not carry ordinary meaning, but act purely as exciters of nearby symbols. However what is nearby in one language may be remote in another."

Reading that acted as an exciter for my own interest in poetry. I immediately think about some of the poetry I have read which had the most impact for me and it did exactly this. The ability of a good poet to draw scenes by exciting patterns of shared experience and perception is really amazing. Choosing those “words that excite nearby symbols” is a bit of a game and for me what makes trying to write poetry enjoyable. It becomes almost technical or like solving a problem.

I am often cynical that poetry can be really widely appealing if only because the most poignant and effective poems I’ve read require a familiarity with the writer and a shared experience that allows compact transfer of imagery. Then again, some of the power in the compression and lossy nature of these poetic images is that there is room for interpretation in the eye of the beholder.

In a field at dusk, trees made short by distance, a line of black on blacker at a horizon pulled ever closer by night…

I've forgotten how much I enjoy Unix

Chris Whitmore — Mon, 05 May 2008 23:35:00 UTC

I just finished reading “The Dream Machine” and getting a very cool look at some early history of computing including the birth of Multics and it’s spin off and almost more interesting the birth of tools like email and ftp as a way to actually do something with the ARPANET that was being built. A great read for anyone interested in how we collectively have arrived at where we are today in computing. And as it happens, I’m also currently challenging a course on Unix which as part of the challenge requires that I complete a project before I can write the final exam.

The project itself is great fun actually. I’ve been in the world of Microsoft for so long at work that I’ve completely forgotten about the old days of living in vi on Solaris as part of my 9-5 work. (CATI programming back then) More than anything about Unix I love the power and flexibility of combining these small well written tools like awk, sed, grep etc. These are incredible tools and it’s easy to forget how powerful a fully text driven system really is.

I remember being completely stoked about Microsoft’s powershell project a few years ago. On the surface it seemed to take everything exciting about a Unix environment and add a layer of object orientation across it so that not only could you fully embracing piping and redirection with the help of some impressive shell programming capabilities but you also had the full .net framework at your fingertips and the ability to use reflection to “discover” a system at run time. This discover-ability aspect is key, and the importance of “man” to rusty or newbie Unix users can let you appreciate the value in being able to query objects to ask what they can do. The simplest example of where powershell is so powerful is to just imagine good ole ls, but where each element output from ls was an actual file or directory object. If you do nothing then those are simply iterated by ls and essentially .FileName.ToString() gets called for each… but if you pipe that output to your own iterator you can all of a sudden do some really amazing things with very little effort.

Having said that though I never truly fell into powershell like I have unix in the past. In my opinion it’s about the eco-system. The set of utilities in unix work because everything in unix plain old text, meaning the same tools can be strung together to do any countless numbers of tasks without additional support. Compare this to powershell which really loses it’s value where .NET doesn’t exist. True, there are some amazing providers that bridge gaps into SQL Server, WMI, etc etc. But just the fact that you are now talking about .NET development to create those new bridges means we’re half a step behind the accessibility and simplicity of a text file. I do think that powershell will only get better with age, but for now unix as a whole is still king for me even if powershell is sexier.

Anyway, this project I’m working on is just a series of clever questions that force you to construct chains of tools to elegantly (or inelegantly) solve a problem. It’s extremely fun and in the course of a couple weeks I’ve reinstalled cygwin at work, gotten ssh access to my mac at home and have had terminal open for pretty much a week straight on my iMac. It’s hard to keep this up without an actual task - and considering I’m still doing 95% .NET development i don’t expect this will continue for too long but it’s a good reminder.

Patches on a centralized VCS in a small team

Chris Whitmore — Wed, 30 Apr 2008 21:52:00 UTC

My team is always looking for ways to improve on quality and one of the proposed ideas this year was to follow a model similar to large open source projects where basically every checkin is first proposed as a diff/patch and then vetted into the code base AFTER review.

Our current setup is a single large subversion repository where our team (~14 developers) all checkout the entire mainline branch we're working on and make commits to mainline as desired. Bugfixes are typically checked into mainline, then merged as requested by QA into the appropriate build. Once checked in the bug is updated in our bugtracker and handed over to another developer for code review.

But of course, the code is now already checked in, and we're in a state of flux until the second developer gets around to reviewing the code. If problems are found (more often than you might expect) then subsequent checkins are made and not only is mainline in a state of flux (what happens if QA branches here?) but we also have lost the nice clean atomic nature of a single checkin. Merges get harder and tracking the code associated with a bug is just a tiny bit more overhead.

I also believe that knowing a checkin has already been committed does something mentally to the reviewer. It's no longer as easy to request changes that might seem superficial or strictly style based. In the back of your mind as the reviewer you know that your comments will introduce a messy string of commits that you unconsciously weigh against the importance of your review.

Our QA lead has often suggested a solution to this problem where code reviews must be done in person and before the checkin. As much as I can see the improvement this would have on quality I personally have issues with the frequent interruptions and delays this would add to the process. Not to mention the challenge of working from home and/or late. Not a good idea.

So patches it is. I managed to convince my team leads that this was worthy of an experiment and so we chose three people out of a hat and for two weeks it was patches and only patches for all bug fixes. For those two weeks we would use TortoiseSVN to create patches of our changes and attach those patches to bugs in BugTracker. A reviewer would examine the diff rather than a subversion commit log and the process would repeat until the developer got it right.

I liked the process, but overall it just didn't really fly with the test group. Here are a sampling of some of the reasons :

Complexity in managing multiple ongoing bugs at once
Reviewing via diff lacked enough context, whereas reviewing a subversion log using TortoiseSVN actually let you see the entire files with highlighted differences
Adding binary files seemingly couldn't be done in a patch (rare but annoying)
Extra steps required are often slow on our repository because of size

What's interesting is that in retrospect I am having a hard time seeing what the real issue here was. I believe that with diff being as standard as it is and the tool support available out there that almost all of these issues can be solved with some infrastructure and automation around the process. As it is the resistance to the idea and my unwillingness to force unnecessary process has killed this effort for now.

Given the size of our team and the fact we are all working with fast direct access to subversion it's hard to justify needing to work with patch files. Particularly since "revert changes from this revision" is so easy to do.

I do think this is one of those things worth pushing again, I may just have to do the legwork to smooth the process and prove the utility before getting everyone on board.

Terrible ad campaign (seal hunt)

Chris Whitmore — Sat, 12 Apr 2008 09:27:00 UTC

So when I started this blog I wanted to make a point of not posting one angry rant after another. Essentially trying to follow the sage advice of bloggers like Scott Hansleman who says amongst other advice to: “stick to a topic”, “avoid politics”, “don’t blog bile” …

Well I’ll break all those rules now by pointing out my extreme distaste for a series of ads being run by the international fund for animal welfare (IFAW). You can see the ads at stopthesealhunt.ca which basically attempt to use guilt and self perceived shallowness to make people act.

Here are some examples of the ads being run at bus stops in my city :

Let’s put aside the actual question of whether or not hunting seals is immoral or wrong. The point I want to make is that these ads are based on the idea that somehow local/personal concerns should be trumped by global concerns. That any problems that we have in our day to day lives do not justify anger or action because there are bigger problems in the world we should be solving.

If we as humans actually worked like this we’d be frozen by indecision in simply trying to prioritize our actions in the global context. I believe we just do not work like that. We must continue to act locally, as our communities and our daily interactions with each other are the foundation for everything else we can accomplish. Trivializing one issue because a larger one exists does not nullify the original issue. The bus is late, ok fine that’s a problem. Seals are dying, that is a problem. People are starving, that is a problem. Apparently by IFAW’s logic we would be selfish fools to worry about dying seals knowing that humans are dying.

Check out this TED talk on prioritizing the world’s problems :
http://www.ted.com/index.php/talks/view/id/62

Here’s a laundry list of some issues I’d rather put effort towards before worrying about seals. (these are from the video above)

800 000 million are starving
one billion lack clean drinking water
two billion lack sanitation
two million dying from aids

<img style=“cursor:pointer; cursor:hand;” src="/assets/seal_myad.jpg” border=“0” alt=““id=“BLOGGER_PHOTO_ID_5212148345682661666” />

Anyway, the point is that these issues are not mutually exclusive. I think cruelty to animals is important, but I’m offended when this type of marketing uses negative appeals to our emotions with flawed logic.

On the need for re-architecting

Chris Whitmore — Sun, 30 Mar 2008 00:53:00 UTC

I have been grappling with the concept of a rewrite of a relatively successful software product that I’ve been involved with since start-up mode. In truth I was googling for compelling stories about why NOT to disappear for a year to do a complete re-engineering of your product (ala Netscape). Mostly because I’m concerned about the size of the effort and the resources we have to pull it off. Instead came across this article, espousing the need to fire all your developers!

http://mikemason.ca/blog/?p=19

Well I’m the guy who’s been promoted to a more senior developer role (not architect, our group shares that responsibility) but still by Mike’s logic the guy who should be fired (?!). I’m the guy who now is learning the ropes in management and who happens to really know the product and the business but now swimming in very exciting waters when it comes to our rearchitecting plans.

What I do have going for me is an incredibly strong team to work with. I’m undecided yet as to whether we actually need an ivory tower architect or if our guys can work this with the guidance of our CTO. We’ll see, as it’s still not too late to change course.

In any case I think I’ll keep an eye on more perspectives like Mike’s…

University Bureaucracy

Chris Whitmore — Sun, 30 Mar 2008 00:53:00 UTC

Depending on how I choose to count I am now dealing with my fifth or sixth post-secondary institution in my odyssey to finish my computer science degree. One thing that all of the schools I’ve dealt with have had in common is a slow and inflexible bureaucracy.

Of course as a younger man I blamed a lot of my own failings on what I perceived then to be a completely fucked up system (University of Manitoba) that was screwing me over. I paid them for a service and yet the treat me as if I should feel lucky to be there! (Can you imagine dealing with thousands of similar minded young punks who couldn’t be bothered to actually read the calendar??)

In retrospect though I can see that most of what was going wrong for me was based on my own mistakes and a failure to read the fine print. I did terribly in my first year of university (terrible = D average) and was put on probation at the UofM. In my frustration I simply packed up and took my business down the road to the University of Winnipeg. (Which also happened to be much closer to where I was working and living)

Ahh… I remember being so excited about this fresh start, getting away from the inept and massive machinations at the UofM and getting a much more familiar and friendly experience with the UofW (which was about 1/6 the size of UofM at that time). It only took registration to completely rid me of my naivety. I had gone from a large university with an annoying but efficient phone registration system to one where I was actually standing in lines and dealing with a slow human process full of uncertainty and questions. Worse, my crappy year of courses of course comes along with me; I’m still on probation and not every course I passed at UofM is even transferable to my new program.

It wasn’t long before I was grumbling and complaining about the inept staff at UofW. I was a little more on top of things this time around though and was able to get through two solid years of course work having properly navigated the probationary process and meeting requirements from the transfer. It was just as I was settling in that I accepted a transfer with work and moved to Vancouver.

By this time I was much better at being a student, having learned my lessons the hard way and I had a certain expectation that future navigation of the university bureaucracy would be much easier. Had I been starting over that might have been the case, but instead I was now dealing with inter-provincial transfer rules and a struggle to get credit for the years I had already put in. I messed around for a few months trying to get into UBC and finally had someone recommend that I try Langara college as it would give me a simpler route into the BC system and transferring from there to UBC or SFU would become much easier.

Langara was great, and because I wasn’t seeking credit with Langara for past work it was basically just a matter of signing up and attending. It all might have worked but ultimately I could no longer afford the time involved in getting to and from school during the work day. And for reasons that are totally beyond me, computer science is still one of those subjects that just does not seem to offer consistent night time access, at least not the universities I’ve been attending at the times I was there.

Which finally has led me to the wonderful world of distance education. For the last two years or so I’ve been attending Athabasca University which is based in Alberta and at the time of my starting was the only fully accredited post secondary institution in Canada to offer a computer science degree.

I should say now that I have been pretty happy with my experience with this school for the last couple years. My transfer process was smooth, the course selection/planning process was easy thanks to some good help from an adviser via email and about my only concern right now is how much respect this degree will actually get once I’m done all this work.

Of course anytime you want to do something just a little different then that’s when you run into the beuracracy and this school was no different. Here’s a timeline of what was involved in challenging a course for credit :

Feb 24 - Email to exam unit to get the professor’s contact info to gain permission to challenge
Feb 25 - Reply with professor name
Feb 26 - Email to professor with short summary and request
Mar 12 - Email the exam unit again asking for a different contact (still no reply)
Mar 13 - Reply requesting I try again and get back to the exam unit if no reply in 3 or 4 days
Mar 17 - Email to professor again
Mar 16 - Professor reply! (need more background for permission)
Mar 20 - Email professor with a customized resume
Mar 20 - Approval! Please contact the exam unit again to arrange
Mar 20 - Email to exam unit asking for next step
Mar 25 - Reply all from exam unit “can someone help Chris?"
Mar 31 - Still no reply from anyone so I submit what I assume is the right form and notify the exam unit that I didn’t want to continue waiting, let me know if I need to do anything else
Apr 5 - the request is approved! course materials are in the mail.

Wow, 42 days from start to finish just to process my request to challenge an exam. That doesn’t include shipping time, or the time it will take to schedule and take the actual exam etc.

I think the big difference between this experience and previous more frustrating challenges with getting things done was that on two separate occasions in this one transaction I had to reach out again in order to get the process moving again. At eighteen I probably would have just written this off as no longer on my plate and waited. If nothing happened it wasn’t my fault. I no longer have murderous thoughts at other’s ineptitude when deadlines pass and I’m stuck in the lurch, I think I just accept that when humans are involved things get slow and error prone. Treat your education like your job and always assume responsibility.

Still, no one enjoys dealing with this kind of crap do they?

Maggie!

Chris Whitmore — Sat, 29 Mar 2008 17:12:00 UTC

This is one of my favorite pictures of my daughter Maggie. She’s not usually this sweet ;-) but at least a few times a day we get a few moments when she’s active, smiley and interactive. I took this picture last weekend, so she’s about nine weeks old here.

A lot of Maggie pictures tend to really distort her, and while often cute they don’t all equally capture that part of her that really defines her personality (in my eyes). The grin here is very recognizable and actually looks a lot like her cousin Josh. I rarely get any sense of what she will eventually look like, but I can see this smile lasting a few years at least.

clever youtube covers

Chris Whitmore — Fri, 28 Mar 2008 17:14:00 UTC

In the spirit of using the blog as a personal online searchable storage medium here is a playlist of some covers on youtube that I totally get a kick out of… most of these are songs I came across in the past year and am now just logging so I don’t forget them.

Youtube is absolutely full of crap song covers by amateur artists in their basements. I think that’s cool and all, I can imagine myself posting something like that when I was first learning guitar but it’s not something I want to watch (does anyone?). These are a little different and with the exception of the first cover, which just rocks because it’s a wicked song, all do something a little different with the original song.

Fairytale of New York (The Pogues) - The Mahones With Damhnait Doyle (a lot like the original, very good)
Hey Ya (Outkast) - Matt Weddle of Obadiah Parker (very different, I had no idea the lyrics to this song were good. Matt has an amazing voice, wait for the end when he really belts it out)
I might be wrong (Radiohead) - Amnesiac Quartet (jazzy instrumental)
Idioteque (Radiohead) - Matt Weddle of Obadiah Parker (not as good as hey ya, but I happened it across it while looking at the others so here you go)
Who is it (Bjork) - Matt Weddle of Obadiah Parker (it’s a Matt Weddle playlist, this song is much more mellow than Bjork’s version, doesn’t really get good until about half way through)

robots are your friends

Chris Whitmore — Fri, 28 Mar 2008 13:25:00 UTC

OMG, I so love this freaky dog robot thing… we need more like this! Truly exciting the kinds of things that are going on, I can’t wait to have a robot servant.

Watch these all the way through, in the following order. The second one is f’n hilarious after you watch the first.

The original

Quadruped Robot Dog Thing

<o:p> </o:p>
the spoof

Spoof of Quadruped

<o:p> </o:p>

and just for fun

Tennis Ball Cannon

I have some reading to do!

Chris Whitmore — Thu, 27 Mar 2008 21:55:00 UTC

http://listverse.com/literature/top-15-science-fiction-book-series

I came across this list of the best 15 science fiction series ever written. The reason I love seeing this is 1) it contains a bunch of science fiction series I’ve already read and thought very highly of and 2) contains some series I’ve never even heard of, which of course is exciting since it means I have some reading to do.

For all I know the guy who wrote the list is totally unqualified and preying on people’s ego by throwing familiar items on a list with a bunch of obscure ones, but I’m gonna go with it.

The series I’ve read so far :

Space Odyssey

I’ve only read 2001 so far, and only just recently. I had a total Aha! moment reading that book and finally getting bits of the movie that were totally nonsensical to me at the time. The book was great, the movie is overrated.

Rama series

My introduction to Arthur C. Clarke (RIP) and definitely my favorite work of his so far. I loved the matter of fact nature with which Clarke wrote, it helped make everything so believable. That and his respect for actual real science which is a total joy to read.

Hitch hikers guide to the galaxy

Douglas Adams is my hero. He is insanely brilliant in the same way George Bush aren’t.

Ender’s Game

Honestly I’ve lost track of all the Ender books, so I may not have read all of them, but I’ve read three or four and loved them all. For straight pure entertainment value I don’t know that I’ve ever enjoyed a sci-fi book as much as the original Ender’s game. I recently got Caitlin to read it and she loved it (she’s not exactly a hard core sci-fi fan)

Foundation Series

I got through all of these and I think all of the robot books as well. Very quick fun reads that had really nice tie in across books. I’m not sure these deserve the top ranking unless we’re just talking pure volume. They were enjoyable but not life changing enjoyable like Douglas Adams or Arthur C Clarke though.

First books on the list I’ll go for will probably be ringworld. This is probably the third or fourth time in the last year where I thought to myself I should read it.

eighth first post

Chris Whitmore — Thu, 27 Mar 2008 21:50:00 UTC

Testing blogger hooked up to my own domain…. we’ll see….