Beyond agile: building on the roots

Software development changes. Over a decade ago the agile manifesto adduced evidence that the document heavy processes needed to change.
Before that progress was measured by producing documents, a big planning and design phase was held at the start of every project to minimize the risk of producing the wrong software. Requirements were carved into stone. Changes later in the processes were shunned.
Agile changed all that. And besides losing some valuable practices (like documentation) recent developments call for a new form and focus of software development.
What changed? Software is now used by all kinds of people. This is not new. But design, user experience (UX), user focus and so on need to be an integral part of software development.

Business people and developers and designers must work
together daily throughout the project.

Nowadays developers need to interact not only with business people but with designers as well. Especially the waterfall like processes from the UX design or product management world struggle with the iterative and highly unplanned nature of agile.

Deliver working software value continuously frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Agile only thinks in iterations or sprints. Even this changed with continuous delivery and lean development. The focus is not so more on a plannable release schedule but on bringing value to the customer.

Our highest priority is to satisfy the customer and the users
through early and continuous delivery of valuable software.

With a user focussed process not only the customer but also the user needs to be satisfied. UX design has this focus.
But UX design needs a significant research (developers call this analysis), design and test phase up front. So in practice this is masked as a big sprint 0.
Unfortunately agile has no concept of planning, design or analysis. The agile philosophy concentrates on execution. This has lead to the notion that planning can be done in every iteration.

Simplicity–the art of maximizing the amount of work not done– Managing and understanding complexity–the art of finding the right places where to invest into details– is essential.

As little planning, design and analysis as possible. The famous pendulum swings the other way. In order to avoid the planning heavy processes from the past, agile ignored and many practitioners even shunned these practices. As many designers can tell us, you need a good foundation to start with. Research is needed. “But these requirements will change” I hear the cries of the past.

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage and the user’s delight.

A dilemma. On the one hand we need an initial investment for a good foundation. On the other hand we know that many things will change during the course of the project. We are stuck.

The best architectures, requirements, and designs solutions emerge from self-organizing teams.

One key principle that the UX design world has to offer for software development is the reasoning. Everything builds upon another thing. There is an incentive to find the cause, to construct a chain of whys. Requirements do not emerge from teams. They are grounded in user needs and business goals. These form the foundation of every user centered project.

At regular intervals All through the project, the team reflects on how to become more effective, then tunes and adjusts its behavior approach accordingly.

The agile philosophy has one core principle which helps to build a bridge between the worlds of development and design: continuous feedback.
In order to build reasoning you need to test your assumptions. UX has many practices to validate assumptions: interviewing, prototyping, analytics.
Feedback is a vital to a project and needs to be included in every action, not only the technical and measurable ones.
This feedback needs to be continuous. Regular intervals won’t cut it anymore.
We need to remove the notion of phases, iterations and sprints from our thinking. We have to define practices that help to build software that meets goals and satisfies needs. These build upon another by reasoning and on demand, not when a time plan demands it. We cannot freeze features for an iteration. Work needs to be organized around streams not iterations. These are not isolated.
Lean thinking tries to foster the notion of bringing constant value to the customers and users. Value streams.
A stream has no defined time frame or regular interval.

Working software is Goals met and satisfied needs are the primary measure of progress.

To bring design into the boat we need move the focus away from requirements and features to goals and needs. Streams of development need to identify, evaluate and build these jobs to be done, the goals. Designers bring the user needs to the table. Developers add the technical constraints. Together all these three factors, business goals, user needs and technical constraints, are balanced by the team.
All this is not new. And in practice many feedback steps or actions are omitted because of time, budget and other constraints. Information and insights are lost because of communication or documentation problems.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Teams should consist of developers and designers. Both need to be present in a kind of pair. Not pair programming. Pair everything. Decisions are made at every step. All three factors need to be considered.
Right now this seems to be pretty abstract and I think we should break it down.
What are the actions during a project?

Listen

A project should start with listening (but it doesn’t need to). And listening should never stop. Listen to the customer, to the users, to the systems, to the code.

Reflect

With all that information you get from listening you constantly need to filter, to reflect, to think. What is important? What not? What is an assumption? What has changed? What needs to be together? What needs to be separated? What forms a whole? What is missing? What is common? What is different? Are we moving towards the goals or away?
Ordering and prioritizing is one of the most important tools we have to bring sense to the mess.

Imagine

How can all this information be translated into software? Into a user interface. Into parts of the software architecture. How will it be deployed? What is the environment? For the user and for the system.
Design systems, guidelines for design and development need to be constructed.

Test

Is my thinking and imaging, the design, right? Does it meet the goal? Is it fast enough? Are the technical constraints met? Does it help the user? All of this needs to be tested. Whether you use the real software, prototypes or other means: you need to remove assumptions and find proof that your translations of information into software works.

Create

This should be obvious. You need to write the software and the documentation.

Ship

Without ever reaching the user, software is worthless. It needs to be shipped. Continuously.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

Every action happens all the time. There’s no right order of steps. This might sound chaotic.
But a project is more like the exploration of unknown terrain. You do not know beforehand what the terrain will be like. You will need to improvise. For this it is better to have practices and principles which help you recognize and react to change.
This change comes from questioning assumptions and furthering your understanding of the goals and needs. Change does not come out of nowhere. Be prepared to react to it.

Reality

One big problem from talking about development and design processes comes from the huge gap between ideal and real world. In the real world there’s not enough time, money, skills, people.

Agile processes promote sustainable development. The sponsors, developers, designers and users should be able to maintain a constant pace indefinitely.

Be pragmatic. You don’t need the newest tools. Go with universal tools: pen and paper, wiki and most importantly your head. In all this you need to think and you need to communicate. Thinking is hard and good communication is even harder. But they stand at the center of a successful project. Do not avoid meetings if they are needed for communication or decision making. Use what you have and build from there.

Continuous attention to technical excellence and good design thinking and communication enhances agility.

It is essential that you know some practices really well. Practices which help you with the actions: listening, reflecting, imagining, testing, creating, shipping.

(The quotes are taken from the principles behind the agile manifesto.)
(The idea for naming the first three actions came from Putting thought into things