TheArchitect.co.uk - Jorgen Thelin's weblog

Are WS-* Interop Workshops an Example of Test-driven Development?

It looks like Steve Loughran from HP has been having a bad week, and he's releasing some of his frustration by venting on my previous posting about the WS-* Workshops Process!

Hm, Steve's idea for a "PowerPointUnit" testing tool does sound interesting, but I'll wait to see the working prototype implementation before fully appreciating some of his comments.
However, here are some initial thoughts (in no particular order) to set the record straight on some misconceptions surrounding that posting:

• WS-* Plug-fests are different from the WS-* Workshop Process
  The WS-* Workshop Process is about confirming and/or refining the _specifications_ before they are submitted to standards orgs. WS-*Plug-fests are about making sure _products_ will talk to each other correctly once the specs have been implemented in production code.

  Once a spec is submitted to a standards committee, the change control for that spec is handled by that group. If anyone has feedback on a spec that is in a standards org, they should join the appropriate org / committee and submit the feedback there.

• The test scenarios for the plug-fest ARE available online - and have been since Nov-2005
  I'm not sure why Steve couldn't "resolve" the reference to the tests - that may be just a bug in his manual "prototype RDF-truth-resolution tool".

• WS-* specs DO have associated test cases
  Each WS-* spec that went through the WS-* Workshop Process does have test cases - namely the scenarios use for the Interop Workshop on that spec. Those are basically the same test cases posted on the plug-fest interop endpoint website (modulo some minor changes where the base specs have been updated since the original interop workshop).

  And those interop scenarios are also submitted to each of the relevant standards committee to jump-start the interop test suite there too.

• N-way testing is permitted too
  Yes, Microsoft's primary goal in hosting the plug-fest is to ensure vendor's products will work with ours, but we also don't exclude participants from testing among themselves either. This is spelled out in the FAQ on the plug-fest interop endpoint website (see quote below).

  Q4: We all know how N:N plug-fests are ran, is this 1:N? Can we test with others?
  A: The goal of the plug-fest is to provide opportunity for product-level interop testing with WCF on a wide range of areas and leverage efficiency of the face-to-face. We will provide everything in our capacity to help attendants get to interoperability with WCF. Attendees are welcome to use this opportunity for testing with other companies.

  By making our top product engineers available for a plug-fest event (which is a serious resource commitment from Microsoft!) we'd naturally like participants to use the available face-time efficiently and effectively to iron out any implementation issues between our products, but ultimately your time is your own.

So, let me re-summarize my previous posting:

• WS-* specs have test cases
• WS-* specs have proven implementations
• The interop scenarios from the WS-* workshop form the basis for plug-fest and standards committee test suites
• Interop Workshop testing provides a quality gate for spec release - WS-* specs that are not submitted to standards orgs until they are "technically ready"

In other words, the process is actually "Innovate, Test, Implement, Test and then Standardize"

Granted this is not the pure "Test-Driven Development" approach that Steve may have been hoping for (although I should point out that I never actually claimed this was "TDD", BTW!), but it certainly does sound to me like "apply[ing] software testing disciplines to the way Web services specifications are developed".