Part of The Beginning and The End of Information Management.
The 30 Year Drift
Something strange happened when the discipline of general management met information technology. While a lot has been said about the impact of technology on our organisations, hardly anybody even noticed the subtle drift that was occurring in all of our organisations over the subsequent 30 years.
The question of who is responsible for the data in our organisations shouldn’t be difficult to answer. The fact that it often is shows how it feels to be at the end of the boiling frog analogy when you’re both the scientist and the frog – and you think you’re still home safe in bed.
As a side-note I’m not fond of analogies. The explanation is a little long-winded but basically I think analogies make people feel like they now understand each other more without actually improving their understanding. This, to me, is worse. My dislike for analogies means two things. Firstly, I’m not very good at them if I’m honest with myself. Secondly, when I encounter them – when people force them upon me – my immediate reaction is to try and get to the point where the analogy becomes absurd and unhelpful. You have been warned.
On with our story. Over the last 30-40 years information technologies have been radically integrated into business processes and business models. At the same time, the relatively new discipline of software development has evolved and matured in order to keep up with the capacity, quality, agility, and speed-to-market improvements that business’s dependence on I.T. has demanded.
As these two trends collide, the importance of the software development discipline to the average organisation has peaked. Pure software development, no longer a core competency of the average organisation, is now a commodity.
The paradox is that while software development is now a commodity the need to integrate information technology into business models is more important, and more difficult than ever. Add to this the oft-sighted failure rates of large corporate I.T. projects and it could be said that while software development is a commodity, successful software development is scarce resource!
Less cynically, the real lesson that has been learnt is that successful software development is only possible within the context of a clear understanding of the business needs that drive the software development effort and the business changes that are implemented alongside the development effort.
Software development has been commoditised precisely because successful software development, and a successful IT function, must be managed within an overarching continuum of service-based management disciplines, outsourcing contracts, technology-enabled business transformation programs, and business knowledge.
So the history of the last few decades of the I.T. industry has been to learn that successful business I.T. requires an understanding of the business. Unfortunately, this can be as a profound a lesson as it is practically meaningless. It’s like learning that a hand must fit a glove or that a successful puddle is made when the water exactly aligns with a hole in the ground.
My point is that technology people have been rigorously defining an approach to their discipline over decades. But it’s impossible to define a pure technology discipline without understanding business context. So I.T. people have defined things like enterprise architecture, business architecture, and value-seeking frameworks that attempt to go from business motivation all the way down to code.
So I.T. now has processes and frameworks well beyond what it’s their job to manage. This can only lead to frustration on both sides.
At the same time that I.T. people where perfecting their disciplines, information technology was being applied all over our organisations. With all the current talk about robotics and automation it’s easy to forget the automation that has already occurred.
When on payday you see that your leave hasn’t been correctly processed that used to be the job, and therefore the fault, of a real live human. So you could go to that person and talk to them about it. In a real sense you could say somebody in payroll messed up. Today that would most likely be described as “an I.T. issue”. In fact, despite our ability to also find a way to blame just about anything on human error, it most likely really is an I.T. issue.
While they weren’t trying to develop overarching approaches to get from business motivation to code, I.T. also developed mechanisms to ensure that technically, even an error in an I.T. system was actually an error in a “business requirement”. This externalised I.T. problems into problems in “the business”. It didn’t really matter who in “the business” was responsible, as long as it was somebody outside of I.T.
So when we think something is “an I.T. issue” we are both assuming that just because something is automated the responsibility for it shifts to another department, and we are forgetting that I.T. departments have been putting up with this for so long that these sort of failings don’t even appear in their key performance indicators.
Even if I.T. people had developed a complete approach to ensuring these problems never happen (they haven’t, by the way), by definition this approach would need to include the elements of business context that make the technology components work. You can’t understand if an I.T. system is working correctly unless you also model the context of the system.
If you have an approach that claims to manage everything good luck! Because you aren’t responsible for everything so not everybody will listen to you. This is the situation I.T. departments find themselves in. But because they aren’t responsible for everything they’ve never had to truly test if their over-arching approach works. They don’t have the span of control to make this sort of system work.
So with the best intensions I.T. people go about using the parts of the framework they can control and assume the parts they can’t control are operating as they wished they would. But the parts they don’t control aren’t working the way they assume they do.
Everybody does this, so it’s okay. I’m not blaming the I.T. people for doing as much as they can within their span of control. I do, however, have a problem with some of the practices introduced under the banner of “agile” which actually change the relationship with other business units again. The level of accountability that most agile approaches surreptitiously push to other business units isn’t something that should be implemented by stealth. That perhaps is a topic of another book.
Over on the side of the business units outside of I.T., from their perspective there has been a lot of jobs lost, a lot of computerisation, and a lot of shifting of things that used to be done on paper to now being done “in the computer”.
The shift of a process to a computer isn’t without other organisational impacts. Before you might have your own personal system for filing away your sales leads. Now, you’re likely to have to conform to a system imposed by others (and enforced by the computer). So all around us we are co-working with computers.
It’s natural to blame our co-workers for mistakes that are their fault. So we blame the computers. But that feels a little silly or at least premature in a world not yet dominated by robot overlords, so we blame the I.T. department.
But there are plenty of ways an I.T. department might respond to this logic, some of them legitimate but all of them requiring deep debate:
1. If the I.T. department doesn’t receive requested funding, or in fact doesn’t even have it’s own budget, can it be blamed for the errors and issues on the margin that it can’t address?
2. If the I.T. department can fix a problem, as long as you tell them about it and how to fix it, does that mean they aren’t responsible for the impact of the problem before they were told about it?
3. While I.T. departments run computer systems, they will always have a nominated “business owner” for those systems. So isn’t that business owner the one ultimately responsible?
As the software development discipline has evolved the value of the architecture skillset has been continuously reconfigured and applied to more business critical domains.
While ‘system architecture’ originally might have referred to the physical systems present in microchips and motherboard design, the architecture skill set was then quickly applied to software domains. From there it further expanded and segmented into integration architectures, platform architectures, and application architectures. The term system architecture was then co-opted to include multiple software components, expanded again to integrate the software components and the hardware components following the logic that they make up the entire system.
This expansion and segmentation continued. The software and hardware architectures were combined to become the ‘solution architecture’. But the ‘solution’ wasn’t just hardware and software so the term ‘solution architecture’ was refined again to encompass the context of what was now the technical solution as well as the business process change, financial modelling, and partnerships that supported the solution.
This continuous expansion and segmentation is as complex a history of a profession as you can imagine and it’s been played out over a relatively short period of time. The debate on the exact definitions of various types of architectures will likely continue. But right at the top is the elusive “business architect”.
But what, if anything, is a business architect? Unfortunately, this leaves the most important level of analysis and specialisation of the architecture discipline sitting precariously on top of a towering bundle of seemingly never-ending debates about architecture artefact, views, meta-models, and architecture governance.
Does the whole approach used by I.T. fall apart if this concept of a business architecture created by business architects doesn’t exist?
The Battle (with apologies to Christopher Alexander)
I’m increasingly seeing the management and architecture disciplines as being in a race to control organisations. Both groups show behaviours that suggest they are trying to extend their own discipline to encompass more of the organisation. Equally, both groups work hard to exclude areas they are not comfortable with from their responsibilities. Architects want to control the organisation by controlling knowledge of the structure and value streams at all levels, while avoiding execution issues. The management profession wants to control the organisation by controlling resources, while avoiding responsibility for technical issues.
Both the architecture and the management professions reveal their desires to control the organisation by the manner in which they grow the scope of their approach through ever increasing extensions of their disciplines. The management discipline has grown from supervision, to general management, to strategic management, to change management, and to all of the business unit focused sub-disciplines that form the structure of a management degree (finance, HR, etc). Conversely, architecture has grown from a technical discipline to include information architecture, solution architecture, business architecture, and information architecture.
Christopher Alexander popularised – if his ideas can be considered popular – the idea of generative sequences. In essence, a generative sequence is the process of taking a structure and changing it through a series of structure preserving transformations. After each transformation the whole structure is then evaluated to determine if the transformation has – more or less subjectively – improved the structure. This process is repeated. Alexander also defines the so-called structure preserving transformations that are applicable at each step.
This is an interesting analogy to decision making in organisations. Each time a decision is made the structure of the organisation changes. Structure in this case may refer to anything: including the attitude of an individual team-member, the next task to focus on, or even quite literally a change in the organisation’s structure as we usually use that term.
What is interesting is that from both the manager’s point of view and the architect’s point of view the details of that structural transformation are only selectively considered. Because managers are generally outcome focused, each transformation or decision is evaluated based on its perceived contribution to outcomes. These outcomes may be long or short-term, or they might be project-focused, or they might relate to the entire organisation – but it’s the outcome that’s important.
While management is primarily concerned with outcomes, the architect is concerned with structure as a whole. When decisions are made they not only impact the progress towards goals but they may also potentially impact other structural elements of the organisation. Rather than a distinction between long or short-term time horizons, or between technical and business domains, the distinction between architecting and managing is generally about outcomes versus structure.
Currently, it’s difficult for architects to evaluate the impact of a transformation in terms of the progress towards desired outcomes because a comprehensive view of the desired outcomes is rarely shared, documented, or linked to the structural elements as defined by the architect. Similarly, it is difficult for a manager to utilise the models created by the architect to make decisions because the models which describe the structure use technical language and contain much that is irrelevant to decision making.
This battle is not yet won, of course. To be a successful architect you must manage carefully, and to be a successful manager you most certainly need to be an architect of sorts. The Manage Without Them Model is driven from the theory that this battle will and is ultimately changing the practice of management itself. This may be seen as victory going to the architects but is more likely to mean that successful architects will no longer be able to choose what issues they avoid.
While this might be interesting to professionals on both sides of the battle I’m just as interested in how important this is to the organisations that we work in. As I’ve said before, I believe good I.T. is structural – when you implement an HR system that enables you to re-deploy some employees in the HR branch of the org chart, you should really hang that HR capability embedded in the IT system in their place.
As these structural IT changes are increasingly differentiating organisations and brokering their relationships with customers it is ever more important that organisations can effectively operate and enhance these technology-enabled capabilities. Both managers and architects currently struggle to achieve this and in the organisation of the future (now?) it’s really the only game.
What’s this all got to do with information management? Well, information management is at the intersection of business architecture, general management, and I.T. So basically, you’re in for a bumpy ride.