mult.ifario.us

Paul Fremantle posted a brief analogy between classic UNIX pipelines and scripting for web services, and I posted a comment about some work going on with Ode that deserves a bit broader audience. Matthieu, Tammo, et al are working on a Javascripty language called SimPEL that maps to BPEL. They're making good progress, and the straw man examples are looking good, both for terseness and for legibility. It's still a way off, but it's on its way.

XML languages are known to be awful (hard on the hands, hard on the eyes, underspecified by document- or data-oriented schemata), and an orchestration DSL that maps onto BPEL has been on various wish lists for a while. (See, e.g., Brian's thought.) WSDL and XSLT should also be on the chopping block, and there are at least a few general approaches to abbreviating an XML syntax from which to draw inspiratiom. (See., e.g., RELAX NG compact syntax and Tom Moertel's PXSL.)

The classic parable of the blind men and the elephant applies to some recent proclamations about BPEL and BPMN (and about BPM and SOA, for that matter) being at odds, and there is long line of blind men waiting to take their crack at sizing-up the BPM elephant. I started out believe that the elephant is defined by its back end, the execution aspects, and that's where FiveSight developed a specialty. Nonetheless, customers (who wanted to ride on the elephant) educated me about its other properties, and combined with the expectation that standards would help define the parts of the BPM animal, the breadth of customer expectations is one of the things that drove our specialization. In 2003, we set out to build the “best back-end in the business”, gave it the code name “JLo”, and took an OEM stance. (Note to any entrepreneurs in the audience — Either you are an elephant vendor or your customers are the elephant vendors. The middle ground is not populated by successful businesses.)

I still recall a customer encounter (probably from 2002) for FiveSight's BPM-ish product that predated our BPEL effort. A customer was interested in an email widget to mass-mail customers. (Never mind that mass emailing requires more than a bit of art in dealing uniformly with bounces, proper MIME enveloping for correct viewing by different clients, delivery throttling by large email account providers, etc.) We built one more or less blind that wrapped up some of the usual JavaMail functionality and took a string as input for the body. We showed it to the customer, including a sidebar about how easy it was to implement the widget complete with a code sample (“Even your IT staff could do it, Ms. Customer!” <wink wink>). The business analyst spoke up and asked “Can it do RTF editing? We'd like to have consistent branding with a logo and specific fonts.” Ummm, yeah...

Aside: Once upon a time, on a trip to Japan, I met someone whose title was actually “Business Analyst”. I was tempted to frame his card.

The point of the story is that there are modeling aspects and execution aspects to a process, and they are closely related but not equivalent. BPMN is a modeling notation for describing processes in business terms. BPEL is a service orchestration language for combining services that was designed specifically for a WSDL-based SOA. It is possible, by design but not without effort, to generate orchestration language artifacts from modeling notation artifacts. One could observe that if nothing below the model mattered, then the orchestration language is irrelevant, and that would be true if the processes lived in a closed, black box. But that's so BPM 0.9 — vendors should know better than to imagine that they can shoehorn their customers into an architectural stovepipe when there are other alternatives available.

Agility requires degrees of freedom, and the combination of BPMN and BPEL to bind the higher-level modeling aspects of BPM onto SOA provides two important degrees of freedom. First, the modeling tool is decoupled from the execution environment. (Note that a good backmapping from the BPEL to BPMN is important, but that's a property of the mapping used.) Second, the processes are first-class citizens of a service-oriented architecture, fueling use and reuse outside of the context of the modeling environment.

There is plenty of talk about service-orientation as a way to decompose software, but I've come to look at it differently. Service-orientation is equally important as a way to decompose work.

As an impermeable barrier between the inside (i.e., implementation) of a service and the outside, service-orientation helps to reduce complexity of a larger system composed of multiple services. (Pat Helland's paper on “Data on the Outside vs. Data on the Inside” makes the case in detail.) This is a good thing, but it's not a great thing. Don't get me wrong — architecture requires a certain sensibility and flair, and there are plenty of deep thoughts yet to be thought about software. Nonetheless, the majority of the complexity in software engineering lies in putting a system into production (and keeping it there), not in coming up with an elegant design.

Service orientation also decouples teams. For example, the team working on the customer information system and the team working on the inventory management system only need to adhere to each other's service interfaces in order to collaborate. There is a huge difference between an interface and an implementation as the vehicle for collaboration. Once one system is built around the knowledge of something inside (i.e., the implementation of) another system, those two systems are rigidly linked, and this constrains the actions of both teams. For two teams trying to evolve two systems, this isn't a significant burden, but for dozens of teams trying to evolve dozens of systems toward different goals, it is debilitating.

Some e-commerce leaders talk the talk and walk the walk, but it doesn't take “interplanetary scale” to realize the benefits. Companies that ship complex, customized software to multiple customers have to manage the collaboration between product development and multiple professional services teams, and if they've done it for any length of time, they've probably encountered the challenge of migrating customizations between versions or even just between patches. (I've even seen systems — from a safe distance — where the customizations cross multiple tiers.) Trading communities are another example where service-orientation (albeit seldom SOAP/WSDL-style or even XML-based) or the lack thereof drives complexity and cost.

At any rate, yet another discussion thread on “business/IT alignment” in my inbox is what got me on the train of thought that spawned this entry, but I think that most such discussions miss the point. I do see service-orientation as a way to help align business and IT by pairing the semantics of the business with the semantics of the services, but it's the business of IT where service orientation pays the strongest benefits.

As I'm getting caught up on some blogs and industry news, I see that the new version of SOA is out, courtesy of Gartner analyst Yefim Natis:

[...] With SOA 2.0, an event-driven architecture is deployed in which software modules are related to business components, and alerts and event notifications are featured. The initial SOA concept has not been event-driven but instead has featured direct calls from one piece of software to another in a client-server process, Natis said. SOA implementations have focused on Web services and subordinates to clients, he said. [...]

As a dyed-in-the-wool pragmatist and skeptic, I've been kind of grumpy about buzzwords and analyst predictions for a while. (I wish that I'd titled that post "Overhype killed the WS-*", but I'm only that clever retroactively.) SOA is a style of decomposing a software system, not a vendor or an analyst property. (Note the lack of a conspicuous "™"...)

A quick search shows that Yefim has had similar views since the Paleolithic (stone tools, yuk yuk...) era of SOA as a buzzword in 2003:

A large part of the integration problem requires implementation of long-running offline business processes. Here, event-driven architecture (EDA), not SOA, is the industry best practice. Unlike SOA, EDA is the design vision for long-running asynchronous processes (SOA is best applied to real-time request/reply exchanges). In EDA, a process node posts an event (in SOA, a process node makes a targeted processing request). In EDA, posting an event reflects results of some past processing (in SOA, making a processing request directs future processing). In EDA, the poster of the event is disconnected from the processors of the event, if any (in SOA, the requestor of service knows the service and depends on its existence and availability).

If nothing else, this sounds like it confuses SOA with RPC and EDA with publish-subscribe. Yefim's on the right track that orchestration (effectively, BPM, once you add the trimmings) is a programming model for building applications in an SOA, but I disagree that SOA or even web services as a specialization of SOA is narrowly scoped to client-server interactions. My minimalist definition of SOA would be:

A service is an autonomous, opaque software unit that consumes and/or produces well-defined messages.

A service-oriented architecture for a software system is a decomposition into services.

Interaction patterns, delivery mechanisms, programming idioms, and data protocols are often aspects of an application of SOA but are neither controlling (in the sense of terminology in a contract) nor defining. That isn't to say that it isn't worthwhile to think about those aspects and approaches. For a detailed, broadly-scoped, pattern-driven perspective on SOA, compare the definition (and dozens of accompanying patterns) from Dragos Manolescu and Boris Lublinsky. (Among other things, they reference the repository of definitions of software architecture from the Software Engineering Institute at Carnegie Mellon.)

As I've commented before, setting unreasonable expectations is one of the biggest risks for service-oriented architecture (or BPM or...), and some people are already looking at “service-oriented” as a dirty word.

From a recent LooselyCoupled blog entry:

The first signs that the initial peak of hype is approaching comes when analysts begin predicting massive market growth. So ZapThink's latest prediction, released as news just before Christmas, that the service oriented architecture market will reach “$43 billion by 2010”, is a sure sign that the current passion for SOA is about to turn sour.

Phil Wainewright is dead-on about semantic integration, repository/discovery, service assembly, and skills/practices as the missing pieces, and these are challenges that are neither new nor unique to service-oriented architecture.

Is the $43B number reasonable? I haven't read the ZapThink report, but if you believe a $150B total size for the 2004 enterprise software marketplace (databases, application servers, ERP/MRP, ...) and assume an aggressive 7% annual rate of growth over the next seven years, that puts 2010's enterprise software market at around $225B. The $43B would be 19%.

That 7% number may be overly aggressive. Versus 2002, the 2003 revenues of the Software Magazine Software500 (registration required) actually slipped 4% from $301.8B to $289.7B. The Software500 numbers include services as well, so while the number is not a pure measurement of the enterprise software licensing marketplace, it is a very accurate measurement of the overall marketplace for software and services. It might not be unreasonable to assume that the Software500 represents a roughly stable market at around $300B, in which case the $43B would represent 14.3% of the total market.

I am willing to believe that SOA will find its way into 14-19% of the software market, but I don't think it will represent a distinguishable segment of the future market any more than any technique or approach (e.g., XML or object-oriented programming) represents a distinguishable segment of the current market.