If your IT is supposed to make your business more agile, then you need to turn to SOA, right? I do not agree. Why? Because everyone is once again mixing up cause and effect! An agile organization will fully utilize IT with or without SOA. SOA will not make a business or its people more agile.

Agility will only come from business and IT being willing to innovate and empower their users by doing away with rigid business processes and the straightjacket like impediment of BPM, CRM and ECM. Encoding your processes into rigid Java for SOA is a business killer.

Why the incredible SOA hype?

The SOA bandwagon is overloaded and subsequently bogged because everyone has jumped on to sell products and services. No surprise that Network Computing Magazine has named SOA the most despised buzzword in November 2006. The ‘If You Can’t Sell It, Rename It Game’ played by the likes of IBM (with Tivoli and WebSphere), Oracle (with Fusion), BEA (with Aqualogic) and others. Something has to change, and it’s not just the product names.

Do we even know what is wrong?

I seriously doubt it. The influential quantum physicist David Bohm wrote in 1980: “Fragmentation is so widespread in our society that it interferes with our perception and stops us from solving the simplest problems.” Sounds familiar, right? We put everything in little boxes, buy best-of-breed software solutions for each and every business problem and now we are surprised that they won’t work together.

On the process level we dissect work into little process pieces because they seem easier to manage this way. Process fragmentation was described by Thomas Davenport in 1993 as such: “A process is thus a specific ordering of work activities across time and space, with a beginning and an end, and clearly defined inputs and outputs: a structure for action. Taking a process approach implies adopting the customer’s point of view. Processes are the structure by which an organization does what is necessary to produce value for its customers.”

I can go along with that but what IT implemented was more aptly described by Foote and Yoder in 1999, who wrote: “A ‘Big Ball of Mud’ system is a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle. These systems show unmistakable signs of unregulated growth, and repeated, expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated.” This perfectly describes most SOA-Java application servers. Why? Because the clean object-oriented encapsulation is destroyed by an irrational requirement to use XML messages. XML conflicts with SOA, because Metadata has to be managed in a repository and not inside the data file.

I propose that David Bohm is right and that fragmentation is needed for our ability to think, organize and plan. However, the world around us is not built in process fragments and thus the current SOA approach will not solve fragmentation but rather, create a software engineering nightmare at a higher level. Business processes and process change management needs to be first looked at before IT systems can be defragmented.

Business Process 1911’s Style

There are those who say that SOA is an extension of process management. BPM proponents are however stuck in 1911’s-Taylorism and SOA makes it worse because there is still fragmented change management. Frederick Taylor believed in fragmentation and specialization and proposed rigidly structured corporations. Hence, each and every IT application represents such a business process fragment, because if not, according to Davenport, we don’t need it. But what is a business process? Rummler and Brache (1990) proposed that ”a business process is a series of steps designed to produce a product or service for a customer.” And that is quite simply, incorrect.

Yes, there are rigid processes and some Ad-Hoc approaches giving more room to the user to choose the course of the process. Strangely enough, hardly anyone seems to realize that the need for an agile enterprise is created by the dynamics inherent in process related business communication. It is obvious that the state of the communication content controls the process and not its meaningless steps. Business communication is not just a document or an email, but can be anything: a selection menu, a web page, a sticker on the document, a data record, images, or even a voice recording or video. No matter how much time and money is spent on business process analysis, there will always be one more communication item needed for a business process once it gets going. This is why collaboration tools and email are now so pervasive. They don’t require analysis to communicate.

In ‘Reengineering the Corporation’ Hammer and Champy made a very important suggestion: “Not the individual task or process is important but only the outcome.” BPM systems and BPR projects miss the ability to adjust to the dynamics of goal-orientation.

Damelio writes in ‘Basics of Process Mapping’: “Maps and flowcharts make work visible … they represent a snapshot in time …” And that is all that business process is supposed to be, as Bohm said. We need fragmentation to understand, but life does not work this way.

How relevant is IT to process?

In ‘Does IT matter?’ Nicolas G.Carr writes that ‘… in the 1990’s many strategic investments went to waste … and business executives have grown skeptical about IT, …’ and claims that IT is now a commodity and no longer producing a competitive advantage. I tend to agree, but the problem is not IT but what the business users request from IT!

Users want to organize their work in their own indescribable way, while management wants to rigidly control the user process. IT has absolutely no clue how to go about both, because the underlying technology (mostly because of Windows, Java and XML) has become nearly unmanageable. SOA simply adds to the already incredible complexity.

Business users are human and point at something to say: This I like and this I don’t. Once users see a cute GUI front-end they think the inside must be cute as well. Users buy a GUI, not architecture, flexibility or long-term manageability.

They demand however 99.99% availability (which is fine for the underlying mainframe-based, bank-transaction system); but when the average mental and physical availability of an employee is at most 50% then the availability expectation linked with the current complexity of technology creates the long rollout cycles. Users expect IT to be inhumane and that’s exactly what they get. And because they cant get what they want business is fighting for control over IT with Outsourcing and Governance. Bad idea!

Adherence to ineffective governance rules is usually more important than what constituents need, in case you have not yet noticed. On his ZD-Net blog Joe McKendrick asks to “Turn your SOA into a HOA (human oriented architecture)” and says, “Without governance, there is no SOA. But maybe we don’t want too much governance.” I want to add: Maybe we don’t want SOA at all if it requires even more rigidity to control and manage our business processes?

Business Process Decision Making:

Let’s face it: The AGILE-SOA-jBPEL-XML-enterprise can’t see beyond Tayloristic 2-D process graphs! The latest buzzword of ‘IT-Industrialization’ describes our plan to create Henry Ford conveyor-belt-IT-processes for lobotomized business users.

It gets worse. In 2005 Davenport and Harris further claimed in a MIT-Sloan article: “Rather than require people to identify the problems or to initiate the analysis, companies typically embed decision-making capabilities in the normal flow of work. Those systems then sense online data, apply codified knowledge or logic, and make decisions all with minimal amounts of human intervention.”

In “Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart” Ian Ayres too claims that huge data sets allow previously impossible predictions and that statistical methods are more accurate than the more intuitive conclusions drawn by experts.

From my perspective, Business Intelligence produces an unintelligible mess of data that has little to no meaning for business decisions, automated or not. Imagine that we not only make the processes more rigid but also the codified decision-making knowledge? That will ruin each and every business because the `statistical data facts’ are illusions and at best average values from the past that do not apply to the current point of decision. Face-to-face interviews with the relevant managers, employees and customers will give you an emotional intuitive feel for the right decision that statistics won’t.

You might ask: Emotions?

I have been asked to replace ‘emotions’ with ‘instinct’ because emotions is seen so negatively. I refuse! Instinct is more in the area of reflexes which do not require any thinking at all. Maybe feelings would be better, but feelings also require emotions deeper down in our human nature. I can go with intuition because intuition is an emotional function.

Today’s assumption is unfortunately that emotion or intuition is the opposite of reason. Maybe that is not so. Why can in the UK and the US the anti-depression drug Prozac be detected in the rivers? Because people are continuously forced to be reasonable and not emotional. Look inside yourself and you will feel that it is against human nature and it makes us sick.

Human decision-making is most likely an emotionally weighted pattern-matching ability of the brain. Rationality is only used for post-decisions justification or verification. Are not strong managers and entrepreneurs mostly very emotional, even unreasonable people? Absolutely! Pioneers such as Broca (1878), Papez (1937), and MacLean (1952) suggested that emotion is related to a group of structures in the center of the brain called the limbic system, which includes the hypothalamus, cingulate cortex, and hippocampi among others. Antonio Damasio – today Professor of Neuroscience at the University of Southern California – has long researched neural systems for memory, language, emotion, and decision-making. In his 1994 book, “Descartes’ Error: Emotion, Reason and the Human Brain,” he documents his discovery that “humans with dysfunctional emotional centers face grave difficulties in decision-making.” Thus human decisions – intuitive or not – are emotionally weighted and how do you want to encode that, Mr. Process-Analysis-Consultant?

To improve decision making with the help of computing we need to invent technology to model intuitive human decision-making far beyond logical rules. While Ian Ayres mentions neural networks and pattern matching he fails to understand that both do not need statistical data but decision-points linked to real data. It is not important how many people took a certain decision, but what data pattern was used by each individual to come to the decision. Therefore we do not need to crunch meaningless and inaccurate data with unproven mathematical assumptions, but today’s computing power has to be used to interactively learn decisions from humans.

And what about ‘Agility’?

I am serious: Agility is a human property and not system functionality. What stops current organizations from being agile? The Java/XML ‘Big Ball of Mud’? Tayloristic BPM? Windows DLL hell? I say it is the lack of agility in people and their small-minded resistance to true innovation.

An agile IT department run by a truly agile CIO supported by an agile board creates an application platform that puts great power into the hands of the skilled business user who is willing to be agile himself.

In their SOA White Papers the likes of IBM, BEA, TIBCO and Oracle name an endless array of technical complexities to create SOA when there is not enough IT manpower available on this planet to even do 2 % of such SOA projects (Source: Max Pucher, 2007, informed guess of 35 years IT). Remember, once the hole is so deep that you can’t climb out, digging deeper will not improve your situation.

KISS – Keep it simple, stupid!

Simplification is needed, and it was I who first humbly proposed in 2000/2001 the consolidation of Inbound and Outbound documents into Closed-Loop CRM processes with Papyrus WebRepository – to the blank eyes of every big-name analyst I spoke to! Today, you find Inbound/Outbound in every ECM vendor’s White Paper. Thank you all, because copying is the sincerest form of flattery. Today, I already propose the consolidation of ECM, BPM, and CRM … So you better stop digging!

Do we need Governance?

I do agree that we need some oversight to make the user happy. We can call it governance if you prefer, but it is simply the requirements of management between users and IT. If you already have a deeply fragmented IT landscape with too many outsourcers and external consultants maybe you do need governance to stay afloat.

SOA or Change Management?

The core issue of IT is change management. Change Management is about metadata: structured data about data that “describes, explains, locates, or otherwise makes it easier to retrieve, use, or manage an information resource.”

Yes, but we do all that with XML, I am told. Listen, WSDL metadata for SOAP and UDDI metadata for WSDL is still not enough because neither links to the user interface and the process. You need DTDs, XSLs, XSLTs, XPATH, BPML and BPEL and a lot of Java code to verify that the data is valid and use it in hard-coded decision blocks.

In ‘SOA For Dummies’ the BEA-friendly authors therefore recommend a repository for the Java programs AND a registry for the dynamically linked SOA services. Their argument for two management products is understandable in the light of BEA’s old Tuxedo, the programmed Java Weblogic and the acquired Aqualogic workflow world.

However, a metadata change in a service interface is NOT automatically propagated to all user interfaces, all Java modules, all process definitions, all XML transforms and all databases. There is NO WAY that a user himself can make any change simply and quickly. As there is no common versioning and no common deployment, we are right where we were before – in a ‘Big Ball of Mud’ – just more complex.

Standards – where art thou?

Others believe that standards are the solution. There are many SOA definitions from various organizations such as the Open Group, Oasis, OMG and others. Most definitions still miss several core issues, such as global transactions, security and event handling. As a consequence there is still no standard way to deliver SOA.

Several vendors offer SOA Change Management with tools such as HP-Mercury, IBM Tivoli and others. These are huge investments to deal with SOA infrastructure and networking but not with the front-end business service needed by the user.

As you can see, confusion reigns. I see RFPs where SOA compliance is requested at the same time as ‘fully-featured APIs’. RFPs request XML and Java standards, when there is no such thing. That is an easy ‘YES’ response in the RFP. Many IT people seem to believe that SOA in some way solves the non-standard situation of Java and XML, when at best it hides some of the problems.

Where are we going with SOA?

I think that in a few years we will have forgotten it. The big new buzzword is already ‘Event-Driven Architecture’.

Why am I not surprised? When I designed WebRepository in 1996 it was built around state/event-driven application models. BPM/SOA vendors claim that you can ‘simply connect’ events to the 2-D workflow graphs of a BPEL process by using Java code (i.e. with Oracle jDeveloper).

From my perspective, to use the word ‘agility’ for SOA that requires Java programmers to create a simple event-driven process is misrepresentation.

Consultants and analysts:

Why do I seem to continuously needle against consultants, analysts and outsourcers? I actually do not. But like everyone else, not all of the ideas that consultants have are good ones.

Service-oriented architecture (SOA) was first described by Gartner in 1996 in an SSA Research paper. Gartner, like most research companies have a cupboard full of skeletons (failed predictions) and I am not holding that against them. SOA is not a wrong idea and neither is EDA, but what the industry made of it is what I am speaking out against. I am asking you to not let the prevailing vision cloud your ability to judge for yourself. I am asking you to not be afraid, to stand up and say what you – as someone who is right there at the grass roots of IT – have to say about it. I am asking users and business department to not interfere with what IT wants to deliver as long as IT has a process focus on the user. Actually, I am asking users and IT to be more agile!

The Vision of the Anointed:

What is it then that stops most large organizations from forward-thinking strategic change and innovation – as requested by Barbara Gomolski above – equally in government or business? Why are otherwise excellent professionals so afraid to do something new?

I have found one explanation: In “The Vision of the Anointed” Thomas Sowell wrote in 1996 about political visions in a way that fully applies to other areas including business and IT visions: “Differing visions, of course, are based on differing assumptions … For a prevailing vision, however, meaning that its assumptions are taken for granted by the so-called thinking-people, those assumptions are never challenged with demands for empirical evidence … A prevailing vision offers more than anything a state-of-grace for those who believe in it.”

The subtitle of Sowell’s book is fittingly: “Self-Congratulation as a Basis for Social Policy.” Only politicians and stock market analysts outdo IT in the art of self-congratulation.

Someone who opposes a prevailing vision is fought with all means possible by claiming that ‘the hidden motives have to be disclosed.’ Much like the dicussion on climate change …

References and innovation:

IT people in large organizations more than others shy away from innovation. I am always asked for reference installations (and no one wants to be one) which is impossible for innovative software. Hey, it’s NEW; no one else has done it before! Additionally there are no two customers who actually do the same thing with our software or have the same infrastructure. As always, a reference should be about the integrity of the people involved and not about technology.

You can obviously choose the safe route and decide not to innovate and make your choice from a Gartner Magic Quadrant. Gartner Group is however a market analyst company and thus their information is from the past. You won’t find innovation there.

SOA is not innovation but an evolution of object-oriented messaging that took a wrong turn because vendors needed to sell what they had – they just renamed it. My position in this paper has in difference translated into ISIS Papyrus products over the last twenty years and therefore some of it has been and is once again very innovative.

I propose that going into a huge programming effort yourself or buying a rigid piece of ‘standard’ software (with rigid processes) involves a much bigger risk than giving something new a try. IT is the most powerful competitive tool there is – if implemented and used by agile people.

You can’t lead by walking in someone else’s footsteps. If you don’t innovate, your IT is not leading edge no matter how buzzword compliant you are. Benchmarking IT against others who don’t innovate will only pull everyone down to the same low level. But then … you can show that you are among the best in your benchmark!

Innovation – doing something new – always bears some risk. Be brave!

Bibliography and References:

Allen, Paul (2006). Service Orientation, winning strategies and best practices. ISBN 0521843367.

Ayres, Ian (2007) Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart. ISBN 978-0553805406

Bohm, David, (1980) Wholeness and Implicate Order, ISBN-13: 978-0710003669

Carr, N. G. (2004) Does IT Matter?: IT and the Corrosion of Competitive Advantage ISBN-13: 978-1591394440

Damelio, R. (1996) Basics of Process Mapping, ISBN-13: 978-0527763169

Damasio, Antonio (2005) Descartes’ Error: Emotion, Reason, and the Human Brain, ISBN 0-380-72647-5

Davenport, Thomas (1993), Process Innovation: Reengineering work through information technology

Draheim, D. & Weber, G. (2006) Trends in Enterprise Application Architecture, ISBN-13: 978-3540327349

Erl, Thomas (2005). Service-Oriented Architecture: Concepts, Technology, and Design. ISBN 0-13-185858-0.

Hammer, M. ; Champy, J. (1993), Reengineering the Corporation: A Manifesto … ISBN-13: 978-1863735056

Hurwitz; Bloor; Baroudi; Kaufman (2006). Service Oriented Architecture for Dummies. ISBN 0-470-05435-2.

Johansson, Henry J. et.al. (1993), BPR: BreakPoint Strategies for Market Dominance ISBN-13: 978-0471938835

Newcomer, Eric; Lomow, Greg (2005). Understanding SOA with Web Services. ISBN 0-321-18086-0.

Pulier, Eric; Hugh Taylor (2005). Understanding Enterprise SOA. ISBN 1-932394-59-1

Rummler & Brache (1990), Improving Performance: How to manage the white space … ISBN-13: 978-1555422141

Sowell, Thomas, (1996) The Vision of the Anointed ISBN-13: 978-0465089956

Taylor F. W. (1911) The Principles of Scientific Management, ISBN-13: 978-1434638205

Watson, Thomas Jr. (1997) Father, Son and Company, ISBN-13: 978-0593020937

Internet resources:

SSA Research Note SPA-401-068, 12 April 1996, “‘Service Oriented’ Architectures, Part 1″

SSA Research Note SPA-401-069, 12 April 1996, “‘Service Oriented’ Architectures, Part 2″

SOA Reference Model Technical Committee, OASIS (2006). A Reference Model for Service Oriented Architecture. (PDF).

Rational Unified Process, IBM, Mar. 2005, www-3.ibm.com/software/awdtoolsrup.

IEEE Std. 1003.1, Standard for Information Technology— Portable Operating System Interface (POSIX), IEEE, 2004.

E. Christensen et al., Web Services Description Language (WSDL) 1.1, World Wide Web Consortium (W3C) note, Mar. 2001; www.w3.org/TRwsdl

SOAP Version 1.2, World Wide Web Consortium (W3C) recommendation, June 2003; www.w3.org/TRsoap

SOA Blueprints, Middleware Research, Mar. 2005, www.middlewareresearch.comsoa-blueprints

Anderson, A. “IEEE Policy 2004 Workshop, 8 June 2004, Comparing WSPL and WS-Policy,” Sun Labs, 2004; www.policy-workshop.org/2004/slidesAnderson-WSPL_vs_WS-Policy_v2.pdf

Brian Foote, Joseph Yoder (1999) http://en.wikipedia.org/wiki/Big_ball_of_mud

Joe Kendricks, (2007) http://blogs.zdnet.com/service-oriented/?p=913

Coenen, Alcedo (2006). SOA agility in practice (PDF). Via Nova Architectura.

Jones, Steve (2005). Toward an acceptable definition of service (PDF). IEEE Software.

Mittal, Kunal (2006). Requirements process for SOA projects, Part 1 of 3: Capturing requirements for an SOA application – Initial requirements to build out your SOA (HTML). IBM Developerworks.

Shan, Tony; Hua, Winnie (2006). Solution Architecture for N-Tier Applications (PDF). In Proc. of the 3rd IEEE International Conference on Services Computing (SCC 2006), pp. 349-356.

Wada, Hiroshi; Suzuki, Junichi (2006). A Model-Driven Development Framework for Non-Functional Aspects in Service Oriented Grids (PDF). In Proc. of 2nd IEEE International Conference on Autonomic and Autonomous Systems (ICAS 2006).

Bieber, Guy (2000). Programming Service Oriented Programming (HTML). Motorola.

Gomolski, B. (2006) Computerworld Opinion, Oct. 2006