The Agile Business Analyst: The next big thing
The reality facing agents for organisational change, especially Business Analysts, is that Agile is here, and it is not going away. To remain competitive and profitable, organisations are rapidly realising that they have to find new ways of delivering value to their customers at the speed of lightning.
First-to-market has become even more critical in getting customer lock-in. Product differentiation, especially in the financial services industry, cannot be based on price alone but on the speed at which product features can be rolled out to the customer.
Organisations that are able to roll out or enhance new products and services rapidly, benefit from market dominance - while their competitors try to play catch up. In some cases, these companies end up becoming the market-makers rather than the market-takers.
Consequently, if companies are not making attempts to transition from the Waterfall methodology of software development to Agile, they will struggle to increase the speed at which they create and generate new revenue streams. Using the Agile approach is proving to be a crucial vehicle in enabling speed-to-market for new products. That said, transitioning from Waterfall to Agile is not a simple task and cannot be accomplished overnight.
It is not new that Agile is the hot topic in the software development space. I have often heard statements from solution delivery teams saying, ‘We have gone Agile’. But when one probes further, the answer isn’t always as succinct. One then starts to question - how many of these members truly understand Agile and live and breathe this methodology?
During an interview with a Business Analyst recently, I asked him to explain the difference between the Waterfall and Agile methodology. He responded confidently by saying that both styles are exactly the same – the only difference is that with Agile you do things faster. I was a bit thrown by his response, because he had indicated to me earlier that he had been an Agile BA for some time. His response was a bit vague, and lacked the insight of someone who purports to understand the methodology they are using.
The expectation is that Business Analysts must also become evangelists of the Agile methodology. However, as a fairly new methodology, most Business Analysts are not clear on how to deal with this style. A large percentage of Business Analysts have been trained in the Waterfall approach, and have operated in environments that have been using the System Development Lifecycle (SDLC) style for a number of years. Transitioning from being a traditional Business Analyst to an Agile Business Analyst requires a complete mindset change. It requires letting go of the territorial boundaries ingrained in the Business Analyst by the SDLC way of doing things.
In the SDLC methodology, the Business Analyst has clearly defined artefacts that he has to produce. A good BA is measured by his ability to elicit, document and validate the requirements. This is a perfect way in which he shows understanding. A good BA needs to show a clear understanding of what the Business Requirements Specification (BRS), Functional Requirements Specification (FRS), User Acceptance Testing (UAT), Business Case documents are. The BA needs to know what goes into each of these documents. He or she is also expected to know which artefacts are produced at which phase of the cycle, together with the resources responsible for them. The test strategy, test plans, functional test cases, regression test packs are artefacts that a Business Analyst is expected to be aware of, be able to review and sign-off. Not forgetting the Deployment Document (DD), Technical or Systems Requirements Specification (TRS/SRS).
An excellent Business Analyst is expected to know who produces these artefacts and what they are used for. These are some of the key performance indicators used to rate the BA’s performance and level of experience, i.e. junior, intermediate or senior.
The level of detail included in the artefacts, i.e. BRS and FRS, is another key performance indicator used in measuring his or her performance level. Detailed and complete requirements are crucial in the SDLC style, and the BA cannot afford to miss the requirements or state them in an ambiguous manner. These requirements are expected to be complete. All possible attributes linked to the requirement should be clearly stated – otherwise the requirement fails the completeness test, and the Business Analyst gets marked down.
In this methodology, junior and intermediate Business Analysts strive to get to this level in order to be considered as senior Business Analysts. Their mindset gets conditioned in this way - and then the Agile methodology comes in - with a change in emphasis from the factors highlighted above. This poses a serious challenge to the traditional Business Analyst when they now have to transition to becoming an Agile Business Analyst.
What does it take to become Agile?
Agile is about collaboration. The focus is not on the completeness of the specification document before development can start. It is about teams that are self-organising. It is about team members who are capable and willing to swop and play different roles.
The project does not come to a halt because the Scrum Master is off sick for two weeks. In the SDLC way of doing things, the traditional BA is trained to see himself as a bridge between IT and Business. The development team cannot directly engage with the Business, and vice versa, without going via the BA.
To succeed in the Agile way of doing things – traditional BAs need to change the way they view themselves. They are no longer the bridge between IT and Business. They play a facilitative role, ensuring that these two teams are constantly talking to each other, and that Business walks the journey of solution design and delivery together with the rest of the project team members.
The traditional BA needs to let go of the ownership of the requirements in the Agile methodology. This is a problematic aspect of Agile, and may result in the BA resisting change to an Agile environment. Letting go of the ownership of requirements can make the traditional BA feel anxious about his role in solution delivery. This may hinder the BA from embracing this methodology, because of the perceived threat that his role may be jettisoned.
A requirement in the Agile methodology is that the traditional BA is expected to be flexible enough to play the role of tester, designer, product owner (in the absence of the product owner) and assist the developers when the solution is being built. The BA needs to provide them with constant feedback from the user’s point of view. The BA does not walk away after logging the specification documents with the development team, only to meet up with the solution in the last cycle of testing (pre-production testing phase). The BA, together with his or her users, walks the solution delivery journey together via the Scrum meetings.
And because of the expectation for the Agile BA to be a domain knowledge expert, i.e. in terms of his Business – the traditional BA needs to change his or her own perception and embrace this challenge. In the Waterfall methodology, the BA represents his Business in meetings and/or during the conceptual solution design sessions. During these meetings, there may be questions they’re not able to answer, and decisions that they feel they’re not empowered enough to make. In such cases the BA will always go back to consult with his Business for guidance, approval or possibly a mandate.
When the traditional BA, who has worked in such an environment for a long time, is required to make decisions on the solution design without consulting with Business first - he/she may be intimidated. This fear might end up incapacitating the traditional BA making it difficult to transition smoothly to an Agile BA.
The reality is that most traditional BAs who have been practicing for a while have already developed domain knowledge, not only of business analysis but of the businesses they represent. The challenge is in changing their mindset.
The Agile methodology provides Agile BAs with an opportunity to gain an overall understanding of the business. They can also play the Product Owner role – which is key in Scrum teams. Traditional BAs need to overcome the barrier of only seeing their changes and not the business in its entirety.
BAs need to understand more than just the system, the business requirements and the business rules. There needs to be an understanding of concepts like competitor analysis, why is the business embarking on project A and not on project B, long term strategy of the business, etc.
Agile methodology is not about detailed and complete requirements, it is about user stories that are developed in a collaborative approach. For the traditional BA, this may seem as requirements that are poorly stated. From habit, the traditional BA might find that they are caught up in the old ways of doing things, and getting stuck on the detail.
In the Agile environment ownership should be with all members of the Scrum team. To manage and prioritise requirements that are on the backlog sheet - and to do that effectively, the traditional BA needs to work closely with the Product Owner. However, if the Product Owner is unavailable for whatever reason, or does not have enough technical knowledge or understanding to guide the developers, the traditional BA is expected to play the Product Owner role.
One of the concerns from traditional BAs who have to transition to an Agile role is that they lose their role’s definition. In Waterfall, the Project Manager would clearly outline the artifacts and their due dates expected from the BA, however in Agile, the traditional BA is expected to play many roles. The traditional BA might struggle to transition to Agile if they are not able to adapt to the various roles. To minimise the impact of this change, BAs should be a part of design, testing and implementation of the solution. BA’s should desist from seeing their involvement to be more during the Analysis and Design phase of the solution, and less during the Build or Development phase, as well as the Testing phase.
So is collaboration the answer?
Collaboration one could argue is the cornerstone of the Agile methodology. A Scrum team cannot succeed or be effective if they are unable to collaborate and work towards one goal – that of delivering a particular feature per sprint cycle. Collaboration is a tool that helps team members achieve a better understanding of the business strategy. When the developer understands the business strategy and where the business is headed, they gain vital insight, helping them to develop better solutions.
There is also a human element derived from a collaborative environment. It gives the Scrum team members an opportunity to understand each other better. With this understanding, the negative impact of personal egos – which could have a devastating effect on solution design and delivery – is minimised extensively. It also offers the perfect opportunity for the Product Owner to inspire the Scrum team through regular feedback during Scrum meetings. Collaboration within an Agile environment creates the possibility of stronger relationships and friendships between team members. Teams that have a culture of working collaboratively end up building stronger relationships. This in turn improves productivity, team cohesion and communication.
Agile promotes collaboration within each team, and this cultivates the ‘group think’ mentality. It is important to stress the need for team members to see that the success of the team depends on the success of the individuals within it. This is where the beauty of Agile comes in – the creation of self-organising teams, whose members are not constrained by the silo mentality and are able to work beyond their job descriptions.
The Agile team understands that when one member fails, the entire team fails. As a result, there is more willingness to assist those members who struggle - and because there is a collective ownership of requirements, the blame game is eliminated entirely. Traditional BAs needs to lower their guard and avoid the battle of territories. The team jointly develops the user stories and is collectively responsible for the successful implementation of these stories. In short, one can argue that collaboration plays a key role in the performance of the Scrum team.
There is another misconception around Agile. People who have not worked in an Agile environment assume that since development starts with no proper long term planning, and features are delivered per sprint – there is no clearly defined strategic high level Business requirement being worked towards. This misconception may influence or affect the traditional BA into thinking that Agile BAs are not working towards a desired goal. This in turn may lead to failure to convert the BA to a true Agile evangelist. Ideally the BA in transition must be made aware of the high level business requirements which everyone in the Scrum team is working towards achieving.
Traditional BAs also spend a huge percentage of their time documenting requirements. This is the output that is expected from them. The conciseness and clarity of their writing is a non-negotiable characteristic used for quality checks on the document. A well written specification document is what earns them respect from peers, developers and their management team. The thick specification document that bears their name provides a sense of identity. It is the tangible output of their hard work, and if they get positive reviews and feedback, this serves as a validation in terms of their capability.
Conversely, the Agile methodology pays less emphasis on documentation and more emphasis on delivery of the solution. This is a huge mindset shift for traditional BAs and may erode their sense of importance. Traditional BAs needs to work very hard to be able detach themselves from their number one love as a BA – documentation - and focus on other aspects of solution delivery.