That this Gold Rush mentality has emerged in an industry whose primary goal is often equated with efficiency, is ironic to say the least. In this New Wild West, business software line managers and project managers sometimes feel like planning is a luxury that they must do without!
How Did We Get Here?
In its infancy, in the 1980's, software development heavily leveraged the processes and tools that had been used for decades in civil engineering projects. These were adapted ever-so-slightly for software development to become the Waterfall Methodology. This methodology (or model) adhered to the basic notions of a budget, a team of resources, a scope and explicitly or implicitly stated quality, and it followed a simple, sequential flow from requirements gathering to maintenance. The Waterfall (or Traditional) Methodology provided executives and management with assurances that planning and accountability were baked into increasingly greater capital intensive (and hence scrutinized) software development projects.
What has become clear over the past couple of decades, however, is that in software development (when compared to traditional engineering disciplines) there are "moving parts" that are significantly harder to measure. Therefore, delivering a project on time and under budget -- the historical yardstick by which project success is measured -- becomes orders of magnitude more difficult when the project involves software development.
The initial response to this realization was for software development teams to increase time spent on analysis and estimation up-front. Although seemingly logical, this adjustment did not address the unique challenges associated with software development projects. In particular, it failed to recognize the following:
- A given software development resource could be orders-of-magnitude more (or less) productive than his/her counterpart. While there is a broad spectrum of ability in any human endeavor, it is particularly the case in software development. Fred Brooks's book The Mythical Man Month does an excellent job of making this (and other) critical points related to software engineering.
- By their very nature -- industry and company specific processes, for example -- business software development projects are unique in the extreme. This means that it is virtually impossible to even define (Requirements box in the Waterfall model diagram above), let alone build (Implementation box), the optimal solution without multiple iterations.
- Software development quality, when compared to many other traditional engineering disciplines, is notoriously difficult to measure.
Along Comes Agile
This lack of software development project management success has, of course, been well documented. As a counterweight to the Waterfall Model a new approach emerged in 2001. That approach, termed Agile -- backed by its very own Manifesto -- essentially proposed breaking down the silos between the disciplines within software development and repeatedly asking and refining this question: Are we building the right thing?This philosophical break from the rigidity of the Waterfall Model, significantly reduces the pressure on the up-front phases (Requirements and Design) which, as we have already seen, are almost certain to "miss the target". It compensates by recognizing that, collectively, the "team" (both business and technical) will iterate towards a solution that will be revealed during regular proof-of-concepts, demos and pilots. These iterations are typically organized as short "sprints", oftentimes just a week in length. (To further distance itself from "tradition", Agile introduces its own vocabulary, including "sprint".)
Agile has become so influential that some of its principles are being applied outside of software development, including so-called "Business Agile".
Looking to Industry Trends
In order to simplify and improve the quality of software development, and by extension the management of such work, I believe that we can look to the following industry trends that are moving us in the right direction:
- Migration to the Cloud. For all of the hyperbole surrounding it, the en-masse migration to the cloud is a net positive for most business software development teams. Once the new tools and cost structures are well understood, "the cloud" helps to eliminate many of the challenges of managing and tuning complex IT infrastructure, allowing better focus on solving the at-hand business problem.
- Service Oriented Architectures and REST API. The movement away from monolithic applications and suites to "a la carte services" can be challenging, since it introduces "seams" in the software stack that must be "sealed" by the development team. However, as the approach matures and coalesces around the REST API, it increasingly seems to embody just the right levels of flexibility and robustness. And, of course, REST is built on, perhaps, the most resilient and well-tested software protocol (HTTP) ever!
- Open-Source. The rise of open-source has democratized and leveled the playing field for entrepreneurial technologists and helped to provide a wave of low-cost, high-quality tools, facilitating improved team collaboration, source code management, deployment, system administration and testing.
- Browser as Lowest Common Denominator User Interface. The trend towards the browser as the de-facto UI has been something of a mixed blessing, although it ultimately simplifies software development. It reduces the number of user "platforms" that must be considered for deployment, testing, etc, and abstracts away many of the details required in designing usability.
- Improved Integrated Development Environments (IDEs). Features such as code-completion and linters are becoming standard in IDEs, helping to reduce many of the syntax and library reference errors that were common in prior generations of these tools.
What's a Technical Manager to Do?
As we have seen, there is undeniably, and understandably, frustration with the way that software development projects have been managed. This frustration might be experienced by any of the stakeholders, on either the business or technical side. Moreover, with the current “developer power mentality" coursing through the veins of tech companies big and small, there is an increasingly vocal line of argument that business process, project tracking and even managers just get in the way
However, it should also be clear that something as complex as software development, warrants and requires significant planning.
Those of us who have been involved in software project management for a while, have probably observed two competing but equally true phenomena:
How do we make sense of this apparent contradiction and tame the software development process?
I have come to embrace many of the important philosophical adjustments that Agile encourages. In particular, frequent and open demos to key stakeholders to level set and calibrate the direction of the development work with the business need. Simple ticketing systems, such as Jira, and the use of User Stories are also valuable tools. Short "sprints" (development cycles) and "retrospectives" (process reviews) help to keep the lines of communication open between the technical and business teams. Perhaps most important of all, Agile helps to break down silos, most notably between the business and technical sub-teams. Almost as critical, Agile helps to reinforce DevOps and QA as first-class responsibilities of the project, thankfully moving us away from a "throw it over the wall" mentality.
At the same time, Waterfall probably better recognizes some of the realities of business software development projects, such as deadlines and multiple stakeholder communities (both upstream and downstream of the "core customer"). In addition, Waterfall incorporates the Gantt Chart -- still one of the most popular project management "visualizations" available -- and other artifacts that can, with a little creativity, be allied to a generally Agile approach.
Collectively, we might consider this approach to be Hybrid or Pragmatic Agile.
As a supplement to this Pragmatic Agile approach, here is my checklist for technical project and line managers:
[X] Enlist
[X] Communicate
[X] Organize
[X] Simplify
[X] Visualize
As Technical Managers, a fundamental first step is building and maintaining a high-performing team. In general, we should err on the side of smaller (rather than larger) teams and, of course, ensure that roles and responsibilities are clearly established.
Although we don't always have full control over project personnel, we should be on the lookout for the following critical characteristics:
These traits apply equally to us as managers, helping to maintain our respect and credibility.
It is largely accepted that at least 75% of a project manager's time is spent communicating. As technical managers, we deliver this communication in a variety of written forms (text messages, emails, documents, presentations, diagrams, etc) and verbally (by phone, in person one-on-one, in meetings and other group situations, etc). We should take the time to understand the preferred communication format and frequency of each team-member and, where appropriate, adapt accordingly.
When communicating with the team, we should emphasize the importance of the project (to the client). For example, we might quantify the expected revenue, number of transactions, unique visitors or other metrics associated with the solution.
It probably does not come as a surprise that many software professionals view meetings and writing documentation with skepticism. Offering options is a way for us to reduce the cynicism, get engagement and facilitate improved quality. As technical project managers, we may offer the option of remote meeting attendance. We may also offer the enticement of a technical deep dive as a segment of every staff or status meeting with a round-robin opportunity for each team-member to both choose a topic and/or present.
Instead of a traditional user guide to support our software development project, we might recognize more enthusiasm within the team for directing and/or starring in a video that "documents" the application's usability. As a bonus, this might also provide a more unique and superior user experience.
Above all, we should be transparent in our communication, both internally to the team and externally to the client and other project stakeholders. We should encourage participation in the direction-setting. As has always been the case, the keys to managing software are communication, planning, and more communication. There is such a thing as over-communicating, but rarely have I experienced it.
As technical managers, we can facilitate much of the communication through logical and consistent organization of project artifacts. Documents, issue tickets, project charters, team rosters and shared credentials should be easily accessible.
We should strive to review, understand and document, as necessary, the related business processes, working to remove any potential roadblocks that might interfere with smooth and unencumbered project progress. This includes translation of the business vocabulary into the required technical parlance (development, quality control, operations, support, etc). At the same time, we need vigilance to ensure that the processes and supporting documentation truly serve only the needs of the project itself, and do not become a means to an end.
Simplification is another key principle that we, as technical managers, need to apply and re-calibrate on a frequent basis. We can work with sponsors to keep the project goal in focus and reduce the likelihood of scope creep.
We can also simplify by ensuring alignment, as much as is reasonably possible, around development, testing and other tools. The proliferation of programming languages, dialects, integrated development environments, test frameworks and more is a blessing only when the final tool selections are identified, clearly communicated and enforced throughout the team. Without that, our teams will be fighting productivity-sapping compatibility and setup issues.
I have recently been working on large eCommerce projects using the Intershop platform. One of the strengths of this platform is its flexibility and its ability to interface with many other enterprise vendors (such as Microsoft, SAP, Oracle, Salesforce, etc). However, that ability to customize must be used judiciously. Fortunately, the Intershop platform also offers a wealth of out-of-the-box functionality that can be implemented with simple configuration changes only. This, of course, means that you are leveraging tested and resilient functionality.
Bottom-line, promote low code (and similar low customization) approaches within your software development teams, reserving custom coding for the truly critical and differentiating features of your software development project.
As technical managers, we also have the opportunity to help stakeholders across the entire spectrum truly understand the project's status. Every project includes countless informative metrics ranging from budget to resource allocation to estimates and actuals. We must ensure the data integrity of these metrics and, with tools such as Jira, Trello, Smartsheet and Microsoft Project, we have the capability to track, aggregate and visualize.
We can assemble these metrics into reports and data visualizations, perhaps starting with simple but critical calendar-based visualizations that communicate project milestones, resource availability (personal vacations, company holiday/workshops/team events), meetings, etc. We can also include clean and simple diagrams, such as this recent one from Amazon, to help convey complex concepts such as application architecture.
Eventually, we might progress to interactive visualizations provided by business intelligence tools such as Tableau, Qlik, Microsoft Power BI or D3, enabling drill-down dashboards across the entire project portfolio.
A picture really might be worth a thousand words, especially if the team does not have to hear them in another project meeting!
However, it should also be clear that something as complex as software development, warrants and requires significant planning.
Those of us who have been involved in software project management for a while, have probably observed two competing but equally true phenomena:
- No project is exactly the same as another
- All projects have a large degree of commonality
I have come to embrace many of the important philosophical adjustments that Agile encourages. In particular, frequent and open demos to key stakeholders to level set and calibrate the direction of the development work with the business need. Simple ticketing systems, such as Jira, and the use of User Stories are also valuable tools. Short "sprints" (development cycles) and "retrospectives" (process reviews) help to keep the lines of communication open between the technical and business teams. Perhaps most important of all, Agile helps to break down silos, most notably between the business and technical sub-teams. Almost as critical, Agile helps to reinforce DevOps and QA as first-class responsibilities of the project, thankfully moving us away from a "throw it over the wall" mentality.
At the same time, Waterfall probably better recognizes some of the realities of business software development projects, such as deadlines and multiple stakeholder communities (both upstream and downstream of the "core customer"). In addition, Waterfall incorporates the Gantt Chart -- still one of the most popular project management "visualizations" available -- and other artifacts that can, with a little creativity, be allied to a generally Agile approach.
Collectively, we might consider this approach to be Hybrid or Pragmatic Agile.
As a supplement to this Pragmatic Agile approach, here is my checklist for technical project and line managers:
[X] Enlist
[X] Communicate
[X] Organize
[X] Simplify
[X] Visualize
Enlist
Although we don't always have full control over project personnel, we should be on the lookout for the following critical characteristics:
- Project motivation and interest, on both the technical and business sides
- Team-orientation and cross-disciplinary interest (planning, development, testing, documenting, etc)
- Willingness to learn ("agility") and to stay current with industry technologies (see previous paragraph)
These traits apply equally to us as managers, helping to maintain our respect and credibility.
A great reference for building high performing teams is a book titled Beautiful Teams.
Communicate
When communicating with the team, we should emphasize the importance of the project (to the client). For example, we might quantify the expected revenue, number of transactions, unique visitors or other metrics associated with the solution.
It probably does not come as a surprise that many software professionals view meetings and writing documentation with skepticism. Offering options is a way for us to reduce the cynicism, get engagement and facilitate improved quality. As technical project managers, we may offer the option of remote meeting attendance. We may also offer the enticement of a technical deep dive as a segment of every staff or status meeting with a round-robin opportunity for each team-member to both choose a topic and/or present.
Instead of a traditional user guide to support our software development project, we might recognize more enthusiasm within the team for directing and/or starring in a video that "documents" the application's usability. As a bonus, this might also provide a more unique and superior user experience.
Above all, we should be transparent in our communication, both internally to the team and externally to the client and other project stakeholders. We should encourage participation in the direction-setting. As has always been the case, the keys to managing software are communication, planning, and more communication. There is such a thing as over-communicating, but rarely have I experienced it.
Organize
We should strive to review, understand and document, as necessary, the related business processes, working to remove any potential roadblocks that might interfere with smooth and unencumbered project progress. This includes translation of the business vocabulary into the required technical parlance (development, quality control, operations, support, etc). At the same time, we need vigilance to ensure that the processes and supporting documentation truly serve only the needs of the project itself, and do not become a means to an end.
By reviewing the project artifacts, we often find duplication of content. Our goal is to have a single source of truth through consolidation. From my experience, these redundancies are a very common and time consuming occurrence on software development projects. They are overlooked or excused because they are targeted specifically for a given project constituency (end user, business analyst, developer, project manager, etc), using that specialization's nomenclature. As an example, we must clearly determine the level of specificity for each business requirement, and whether we should record them directly as user stories (or similar) in a ticketing system or out-reference tickets to a "specification document". The key is not to maintain both!
Simplify
We can also simplify by ensuring alignment, as much as is reasonably possible, around development, testing and other tools. The proliferation of programming languages, dialects, integrated development environments, test frameworks and more is a blessing only when the final tool selections are identified, clearly communicated and enforced throughout the team. Without that, our teams will be fighting productivity-sapping compatibility and setup issues.
I have recently been working on large eCommerce projects using the Intershop platform. One of the strengths of this platform is its flexibility and its ability to interface with many other enterprise vendors (such as Microsoft, SAP, Oracle, Salesforce, etc). However, that ability to customize must be used judiciously. Fortunately, the Intershop platform also offers a wealth of out-of-the-box functionality that can be implemented with simple configuration changes only. This, of course, means that you are leveraging tested and resilient functionality.
Bottom-line, promote low code (and similar low customization) approaches within your software development teams, reserving custom coding for the truly critical and differentiating features of your software development project.
Visualize
We can assemble these metrics into reports and data visualizations, perhaps starting with simple but critical calendar-based visualizations that communicate project milestones, resource availability (personal vacations, company holiday/workshops/team events), meetings, etc. We can also include clean and simple diagrams, such as this recent one from Amazon, to help convey complex concepts such as application architecture.
A picture really might be worth a thousand words, especially if the team does not have to hear them in another project meeting!
