Google Sheets No Code Shootout

My last post was a primer on the burgeoning low code/no code (LC/NC) movement.  Many regard this progression as a technical democratization movement, freeing the citizen developer in each of us.  While that may be true, the movement may also simply represent the next phase of abstraction within software development.  You can read more about LC/NC in this well-balanced piece in Harvard Business Review.

This new post -- the first in a series of deep dives -- focuses on pure no code solutions that sit on top of  Google Sheets.  This approach is often referred to as spreadsheet-as-a-database.  Specifically, I will be looking at AppSheet and Glide.  If, like me, you are already familiar with Google Workspace (formerly G Suite), this is the easiest way to develop and deliver fully functional, data driven apps that run on mobile devices or within the browser.

It is important to stress there is absolutely nothing you need to do to enable the integration between AppSheet or Glide and the Google Sheets document.  There is no exposed API to understand nor any configuration needed.  You simply select (or create) the tabs in the Google Sheets doc as you develop your app and CRUD operations are implicitly available.

The Challenge

My challenge was to use each of the tools -- AppSheet and Glide -- to build a compelling, intuitive app based on a small dataset that could be managed in Google Sheets.  

For this exercise, I was reminded of an important maxim of software tool "demos": to the degree possible, leverage data that you find interesting, that you understand well, and that your audience will find engaging.  With that in mind, for my proof-of-concepts, I gathered a dataset representing results from national soccer tournaments sanctioned by FIFA (the global governing body for the sport).  This dataset includes both the men's and women's World Cup tournaments.

Per me, table stakes for this class of tool are:

• App development requires only a Google account
• App development requires only a web browser and an Internet connection
• App development is free (a reasonable trial period or a limitation on users / data / features is acceptable)
• App development is truly 100% no code

The Results

With each tool, I created a simple read-only app based on the same Google Sheets document that included my FIFA tournament dataset.  

Creating each of these apps took less time than it took to write and edit this blog post.  That is, even accounting for a learning curve on each of the tools themselves, about 1 day apiece.

	AppSheet	Glide
Positives	Extremely easy to use Excellent out-of-the-box Views (1) and Components Integration with Apigee and REST API datasources (2) Access AppSheet functionality directly from within Google Sheets	Extremely easy to use Excellent out-of-the-box Tab Styles (1) and Components Vibrant support community
Negatives	Cannot create additional Views	Cannot create additional Tab Styles No out-of-the-box REST API integration feature
Other Notes	Warnings and Errors are generally helpful, but there were occasions where a message was confusing / misleading or it required me to exit AppSheet to resolve

(1) Views and Tab Styles are pre-built page templates.  These enable you to present the underlying data in a variety of ways and with pre-defined flows.

(2) Although out-of-scope of this blog post, the ability to interact with other data (outside of the Google Sheets dataset) through industry standard REST API is an extremely important feature for many apps.

The Verdict

Both AppSheet and Glide are excellent choices for quickly designing, developing, and deploying simple, intuitive apps for mobile and web.

AppSheet edges this shootout because of its tighter integration with Google Sheets, Google Cloud, and REST APIs (Apigee).  

Of course, features are changing at a blazing speed across the entire spectrum of LC/NC. 

You can find my solutions here (AppSheet) and here (Glide).  I would love to hear from you about this post, these apps, and your experiences with AppSheet and Glide.

Do-It-Yourself Software

"That looks too much like real programming!"

"Well, it's not quite rocket science!"

These are two of my favorite quotes from ex-colleagues.  

The first quote above is from over 25 years ago, when I was working at a software company called Omnis.  It had developed the first cross-platform (Mac and Windows) software development tool.  The tool included a visual builder, a proprietary database, and connectivity to all other contemporary relational databases, most notably Oracle.  My colleague offered this observation when asked why not use Java (which had recently been released) instead of the Omnis tool.

The second quote above is from 4 years ago, when I was working at another software company that provided an eCommerce platform.  My colleague offered his response when I asked him how easy it was to build custom solutions with the platform.  He then proceeded to talk about actual rocket science and his Master's degree in Aeronautics from Purdue!

While my colleagues' responses were clearly tongue-in-cheek, they each highlight the tendency that we, in technology, are inclined to overcomplicate things.  The fact that there was over 20 years between those remarks, suggests that, perhaps, not so much has changed.  Most, if not all, business-facing problems on which I have worked over my long career, did not need real programming or anything like rocket science.

Which brings us to the burgeoning low-code/no-code movement.  These contemporary tools provide another layer of abstraction on top of "real programming" and aim to democratize software development.  They do this by offering visual development interfaces that eliminate or reduce the need to code and understand complex programming syntax and concepts, replacing it with familiar drag-and-drop gestures and simple property assignments.

The fact that technology heavyweights Amazon (Honeycode), Apple (SwiftUI), Google (AppSheet), Microsoft (Power Apps), Oracle (Apex), SAP (AppGyver), and Salesforce (Lightning) all have skin in the low-code/no-code game, suggest that we have reached an inflection point.  

No longer does every application need to be viewed through the lens of the corporate Information Technology department and the all-knowing high-priesthood of software architects and programmers!

As such, perhaps we can think of these low-code/no-code tools as very distant relatives of COBOL (Common Business Oriented Language).  As its expanded name suggests, COBOL was/is more oriented towards "business users" than "engineers".  Don't be fooled, though, it still sports a fairly arcane vocabulary and syntax.

Intuitively, we all understand that smaller teams are more nimble and efficient.  A key promise of low-code/no-code is its ability to support this small team structure, reducing or even eliminating the increasingly over-specialized disciplines -- architect, tech lead, front-end developer, back-end developer, DevOps engineer, data analyst, business analyst, designer, QA engineer, project manager, scrum master, etc. -- within corporate IT.  The challenge of maintaining consistent messaging across these roles and disciplines is akin to the phenomenon of the Telephone Game/Chinese Whispers.

Over my long career in software development, I have always been struck by this industry paradox -- our mission is to encourage productivity and yet our choice of tools and process are often inefficient (for the job at hand).  In addition to implementing solutions in more traditional languages such as Java, JavaScript, PHP, PL/SQL and shell scripting, I have dabbled in the following tools that aimed to smooth away some of those rough edges and inefficiencies, and could, therefore, be considered predecessors to today's low-code/no-code generation of tools:

• Filemaker
• Omnis
• Kony
• jQuery
• Visual Basic
• Intel XDK
• Hypercard/Supercard
• Microsoft Access
• FrontPage
• WordPress
  

"Real programming" (and real programmers) will always be needed, of course.  Someone has to understand concepts such as pointers, memory buffers, static versus dynamic typing, polymorphism, object orientation, functional programming, instruction sets, data types, registers, and memory management.  However, that is not (and should not) be a requirement for most of the business automation that is being developed right now.  

I have always looked for ways to simplify business software development within my teams, either through improved tooling, process, management, or culture.  In my next blog post, I will take a deep dive and test drive the following low-code/no-code tools*:

• Airtable
• AppSheet
• FlutterFlow
• Glide

Stay tuned for my next post to find out which tool will make me the best citizen developer.

*This subset of the vast (and rapidly evolving) low-code/no-code market, focuses on general-purpose app development in the Google ecosystem.  This is the area that I'm currently most familiar with.  I plan to later create deep dive posts into more specialized low-code/no-code tools focused on AI/ML, web design, eCommerce builders, and business process automation (BPA).

Lightsail - My Journey from GoDaddy to AWS

About 9 months ago, I decided to migrate my 3 domains and 2 websites from GoDaddy to AWS.  I had been a loyal customer of GoDaddy for 20+ years, but as more and more of my projects and customers migrated to the cloud, it seemed logical for me to do the same.

I first attended an AWS Summit event about 6
years ago.  I appreciated the seriousness with which each speaker took his/her presentation and the uniform recognition of the scope of Amazon's mission to simplify and improve software delivery.  Having previously been responsible for maintenance of departmental hardware in my role at Sun Microsystems, I immediately recognized the value proposition that cloud computing offered.

Since then, the vast majority of the software projects that I have managed were either predominantly or completely hosted in the cloud, mostly AWS but also Azure and GCP.

For my simple needs, and to control budget, I determined that I would use the following AWS services:

• Lightsail ("Virtual private server for a low, predictable price")
• Route 53 ("A reliable and cost-effective way to route end users to Internet applications")

The Migration

Here is a guide to the migration, with a few comments along the way!

Provision the Lightsail Instance

Login to your AWS account and provision the Lightsail "instance" here:

• https://lightsail.aws.amazon.com/ls/webapp/home/instances

This was relatively simple, especially given my somewhat outdated and incomplete understanding of infrastructure provisioning.  Note that I also had to account for multiple domains.  This video was helpful:

• https://www.youtube.com/watch?v=upZOhKhefAs

After navigating the Lightsail Create Instance wizard, which takes a matter of a couple of minutes at most, you simply input the Public IP (provided for each instance) into the web browser address field and it renders your "base website".  

I particularly appreciated the following:

• The AWS Lightsail web UI provided an SSH console "out-of-the box"
• I did not have to pfutz with encryption keys

Migrate Content

I used the Aptana SFTP client to migrate, en-masse, my htdocs content, as follows:

• Configure SFTP connection from Aptana client to the Public IP for the instance provided through the Lightsail web UI
• You will need the default username and password generated by Lightsail Create Instance wizard (see the previously noted video for details)

I also used this as an opportunity to clean up my legacy web content.  There was (and still is) "tech debt" that needs to be repaid to minimize future maintenance overhead.  Here are some integrations, specific to my content:

• Github's public API: Worked without issue
• Google Maps integration: Worked without issue
• Google Blogger blog feed: Worked without issue

I performed a side-by-side comparison of the functionality of the existing site (GoDaddy) and the migrated site (AWS Lightsail).  This helped identify 2 minor regression issues, that I have noted and still have to troubleshoot in more detail.

Domain Registration

Refer to this guide that is specific to GoDaddy -> Route 53 domain registration migration:

https://blog.shikisoft.com/transferring-domain-registrations-from-godaddy-to-amazon-route-53/

Key Takeaways

Benefits

In my most recent roles, I am less "hands-on-keyboards" than I used to be.  

However, I still find great value and satisfaction in having a basic understanding of the technologies and services at play.  Performing this migration helped me to better understand the AWS ecosystem and, more generally, the operational mental model and nuances of cloud infrastructure when compared to on-premise.  

It will also provide me with a platform should I need to perform first-hand exploration of other emerging technologies and services.

Complications

I achieved "secure http" -- or https -- as part of the migration to AWS Lightsail.  However, this required a little more work than I was expecting.  These additional instructions helped me navigate this:

• https://docs.bitnami.com/aws/how-to/generate-install-lets-encrypt-ssl/

My legacy content was dependent on a MySQL database.  Therefore, I selected the AWS Lightsail Bitnami LAMP stack image.  I encountered some confusion in the database setup, although this helped:

• https://docs.bitnami.com/aws/infrastructure/lamp/get-started/understand-config/
  

I was also migrating a WordPress-based site to AWS Lightsail.  The following instructions helped with this part of the migration (and eliminated the need for me to respond to a series of GoDaddy alerts to perform WordPress updates!):

• https://lightsail.aws.amazon.com/ls/docs/en_us/articles/amazon-lightsail-using-lets-encrypt-certificates-with-wordpress

Cost Management

Lightsail promises a "low, predictable price".  Amazon also provides this pricing estimator.  For my situation, I also ensured cancellation of my GoDaddy license and disabled auto-renewal.

Of course, I have heard stories about runaway costs and budget challenges given the AWS price structure and, more generally, cloud metered pricing.  As a result, over the months since my migration, I have paid close attention to my charges from either AWS or GoDaddy.  

I am very happy to report, there has been no double-billing or unexpected charges and, moreover, my net monthly hosting expense has been reduced!

Frictionless Software Leadership

Frictionless helps us to make sense of the complexity

Delivering quality software is complex.  This has never been truer than it is today.  Over a long career, I have regarded it my mission to help simplify this process, no matter the role.  While there isn't a one-size-fits-all for every team, product, project and initiative, there are a collection of best practices that have consistently served me well.  I call it Frictionless Software Leadership that draws on the best of Agile, DevOps, Lean, Scrum, Six Sigma, Total Quality Management and even Waterfall.  

Frictionless Software Leadership is guided by principles of integrity, transparency and simplicity.

Please note that I use the expression "delivering quality software" and not "developing quality software".  This distinction helps to underscore the importance of DevOps (infrastructure and deployment), SecOps (security and compliance), UX (usability and experience), QA (quality and testing) and Client Services (tech support and professional services).  Treat each of these as first-class disciplines in teams and projects to help dispel the misguided notion that their considerations can be bolted on after development.

What is Frictionless Software Leadership? 

Frictionless Software Leadership allows us to provide efficient and effective guidance to teams working on software projects, products, and other initiatives.  It helps provide answers to the "Why?  What?  How?  Who?  When?" questions, while ensuring that leadership is not unnecessarily in the critical path or becoming a bottleneck.  

Equally important, Frictionless Software Leadership puts the client (or customer) first.  As W. Edwards Deming asserted: "The consumer is the most important point on the production-line."

There are 4 pillars of Frictionless Software Leadership:

1. Appreciate the (Technical) Art-of-the-Possible
2. Understand the Constraints of the Initiative
3. Drive Planning with Data (Not Documents)
4. Enable Stakeholder Self-Service

I regard these pillars as equally critical and complimentary to one another.  I will dive deeper into what I mean by each of these in a moment.  However, I first want to mention that much of Frictionless Software Leadership has become second nature to me.  That is to say, I do these things almost subconsciously.  This blog post has been a helpful exercise, allowing me to unpack what I have tried in the past and objectively reflect on and document what has worked.  I have also used it as an opportunity to corroborate my first-hand experiences with industry methodologies such as Agile and Lean.

Appreciate the (Technical) Art-of-the-Possible

It is imperative that technical leaders regularly dive deep into the state-of-the-art in tools, frameworks and infrastructure, re-calibrating their leadership approach accordingly.  Recent advancements in our industry -- such as APIs and microservices, containers and orchestration, and, of course, cloud computing -- have fundamentally altered the way that software is architected, deployed, managed, and monitored.  In addition, advancements in no-code, low-code, and BPM/BPA significantly affect the composition of project teams, timelines, and budgets.  It is unthinkable that these fundamental shifts would not have an impact on the way we lead and manage technical teams and projects.

This re-alignment, between our leadership approach and contemporary technology, helps us to:

1. Gain the trust of clients and customers
2. Maintain respect within the technical team
3. More deeply appreciate the business problem(s) being addressed
4. Genuinely appreciate the skill of the technical team, while providing the platform to intelligently probe when necessary
5. Assist the delivery team with prioritization, design, estimation, development, testing, and other hands-on tasks
6. Inquire about vendors' functionality and integration claims
7. Anticipate industry trends*

*For example, 6 years ago, as I was digging deep into the pros and cons of SaaS and PaaS, I identified an at-the-time unmet opportunity to "productize data management".  That notion of DaaS (Data-as-a-Service) -- broadly speaking at least -- is now embodied in products such as Snowflake, Splunk, Mulesoft, New Relic and facilitated by the de-facto industry standard REST APIs.

Incidentally, due to the ubiquity of REST APIs, I would recommend all technical leaders and managers develop a simple web or mobile app to familiarize themselves with the details of a call construct and the returned JSON payload.  At an absolute minimum, familiarity with a tool such as Postman to execute a variety of API calls, provides us with much-needed context and insight into this core architectural style.

Understand the Constraints of the Initiative

Every software project, product or initiative comes with constraints.  In addition to easily quantifiable and understood limitations (for example, the project budget for the year is $200,000), it is important to recognize and document more subtle boundaries before planning begins. 







These constraints loosely fall into 3 buckets (with some examples):

• Organizational 
• Top-down, command-and-control reporting structure?  May not be compatible with some of the team composition ideals noted above.
• Small(er) team size?  Usually preferred.
• All stakeholders identified?  
• Motivations understood?
• Who controls the PM iron triangle parameters?
• Budget?
• Schedule?
• Scope?
• Cultural 
• Predefined thresholds for metrics and standard definitions for Red, Yellow, Green flags?
• Definition of "Done"?  In the context of a feature, sprint, or release, it is critical to get consensus on this "metric".
• Project Begin and Project End?  These are not always as obvious as we might think.
• Client (or customer) engagement?  It is important  to clearly identify ownership and responsibilities around engagement and be sure to properly distinguish project (or initiative or product version) tracking from client (or customer) tracking.
• Are "heroics" encouraged?  Ideally not, since this leads to burnout and is also often an indicator of a single point of failure or knowledge/power hoarding.
• Is product/project secrecy critical?
• Encouragement to "eat our own dog food" and use tools developed in-house?  Ideally yes, since this shortens our internal quality feedback loop. 
• Technical 
• Greenfield** or Brownfield?
• On-Prem or Cloud or Hybrid?
• Integration with legacy or home-grown CRM, ERP or other application?
• Corporate/enterprise mandated tools?

**Counter-intuitively, even so-called Greenfield projects or products will have constraints (mostly Cultural or Organizational).

From my experience, project outcomes are positively influenced by transparent conversations about goals, budgets, schedules/deadlines, and roles/responsibilities before any (meaningful) work begins.  Naturally, it's important for us to regularly review these constraints (and assumptions) to ensure alignment and accuracy.

Drive Planning with Data (Not Documents)

As software managers and leaders, we sometimes mistakenly equate raw effort with progress.  In particular, I have seen the count of project artifacts held up as a meaningful measurement of forward movement.  Instead, I recommend a data-first strategy.  Replace the effort involved in authoring and updating project artifacts with investment in a Project Portfolio Management (PPM) system.  There are some excellent PPM products on the market -- some of which are free -- or we can extend our project management tool (Smartsheet, MS Project, etc) or issue tracker (JIRA, Github, Zendesk, etc) or even develop from scratch.  The solution does not have to be exotic or elaborate; the key is to remove as much of the manual overhead and drudgery associated with project plans, charters, stakeholder analysis, requirements and other project artifacts.  Equally important is ensuring that this data is accessible to all interested parties (see Enable Stakeholder Self-Service, the 4th pillar of Frictionless Software Leadership). 

Depending on policy, there are likely certain project artifacts -- for example, SOWs, contracts, legal papers -- that must be standalone documents.  Make those the exceptions.  From my experience, the efficiency gains in making the transformation to (primarily) data-first are considerable, especially as our number of projects, products and initiatives increase.  This transformation to a data-first strategy should include a clear understanding of our current and future system(s)-of-record and an extensive use of data visualizations, wherever possible.  I have had good success with the following digital visualizations:

1. Calendars***
2. Maps***
3. Timelines
4. Kanban boards
5. Gantts
6. Milestones
7. Value streams and flows (work-, data-, process-)
8. Architecture diagrams
9. Checklists****
10. Cheat-sheets****

***The archetypal visualizations!
****Not visualizations but extremely useful nonetheless.

Enable Stakeholder Self-Service

As technical managers and leaders, we can help bring order to the chaos by establishing a one-stop-shop project portal with optional authentication and authorization, accessible through a single URL via any web browser from any channel.  This will consolidate all the project data and artifacts for convenient access by all stakeholders, ideally through a drill-down dashboard-style interface.  Occasionally, there will be exceptions, but universal access to everything should be the starting goal.  Transparency and simplicity are keys to eliminating project ambiguity and confusion. 

Since we have already made the majority of our project information data-driven, the portal will be real-time.  This helps to eliminate confusion around document revisions and versioning.  However, where there are exceptions and static documents are referenced from the portal, we must be vigilant about concurrency.

It is our job, as leaders, to direct stakeholders to the portal and to encourage input on suggestions for content and format change.  Depending upon the portal's complexity and thoroughness, it also offers us a forum to regularly deep dive into a particular aspect of project tracking.  This helps the team-members to better understand and appreciate disciplines and features in which they might only tangentially be involved.  It also provides an opportunity for stakeholders to showcase their work and achievements or even highlight an area of concern.

Benefits of Frictionless Software Leadership

In summary, Frictionless Software Leadership helps teams to:

• Be more efficient, by...

• Favoring smaller, more focused teams
• Favoring a consolidated toolset
• Understanding communication channels
• Clarifying goals and definitions
• Focusing on DevOps' The First Way ("work should only flow in one direction")
• Empowering adaptation to situational changes (for example, remote collaboration and WFH)
• Be more trusting and transparent, by...
• Accessing a single source-of-truth ("project portal")
• Sharing accomplishments and tips (and frustrations!)
• Collaborating regularly
• Be more content, by...
• Encouraging "pride of workmanship" (one of Deming's bedrocks)
• Reducing frustration (when searching for project relevant information)
• Enabling leaders and managers to spend more time addressing team blockers (instead of attending to busy work updating marginally useful documents)

This blog post documents the ideals and, of course, we recognize that concessions will usually be required.  However, using this approach and applying our much-harder-to-quantify-but-equally-important soft skills, we can have success delivering quality software.

Technical Knockout: How COVID-19 Put Society and Big Tech on the Canvas

It is probably safe to say that few of us anticipated the impact of COVID-19.

The virus has affected every facet of our lives.  Aside from the terrible and tragic human toll, societal norms have been turned upside down.  Social distancing and shelter-in-place have become mandated across the globe.  Board games and jigsaw puzzles, seeming relics of a past era, have enjoyed a renaissance and in some cases are selling in greater numbers than during Christmas.  Sports events, ranging from the Summer Olympics to the NBA to Wimbledon, have been postponed or cancelled entirely.  Schools and universities have been shuttered resulting in teachers, parents and students alike undergoing a trial-by-fire as they struggle with distance learning and homeschooling.  More distressingly, the economy has been harpooned as 10 years of US job gains have been wiped out in the space of just 4 weeks.

Yet, five months on from its initial identification, we still have so many basic questions about this invisible foe:

1. What was the virus's origin?  A "wet market" in Wuhan, China or something more sinister?
2. Exactly how contagious and deadly is this virus?
3. Can we develop immunity to the virus?
4. How different is this year's global and regional mortality rate from the average* and from other key milestone years?

And how has Big Tech, perhaps unreasonably heralded as society's knight in shining armor, fared during this once-in-a-generation crisis?

+ Positives

On the plus side, the sprawling, highly distributed network that we call the internet, has enabled significant work to be performed from homes around the world.  Through Facebook, FaceTime, Zoom, RingCentral, Skype and other videoconferencing and collaboration tools, the internet has also enabled us to connect remotely with loved ones.  Despite the huge additional data loads, we have not -- to my knowledge -- suffered an internet meltdown of any meaningful magnitude.  The free transfer of information provided by the internet, has helped expose and highlight the lunacy of entities such as Harvard receiving a bailout.  Also, data collection and dissemination technology have enabled innumerable dashboards and data visualizations to convey, in excruciating near-real-time, the horror of the disease.  (My particular go-to dashboard has been this one.  Apparently, it was developed and is maintained by a high-school student named Avi Schiffman.)

The fact that we also have a robust physical delivery infrastructure -- with Amazon, UPS and FedEx at the forefront -- has helped to offset the imposed limitations around personal movement and the fear of contagion associated with in-person shopping.  There have been other positive contributions, including monetary donations, job creation, masks from 3D printers and more.

In addition, Apple and Google have partnered to develop a contact tracing API on their respective mobile operating systems.

- Negatives

However, on the other side of the ledger, Silicon Valley's technical might has not, to this point, contributed to the much-needed medical solution of a vaccine.

Furthermore, the fragility of those whose livelihood depends on the oft-trumpeted gig economy has been brutally exposed as Uber and Lyft fleets have largely ground to a halt and AirBnB -- only months removed from being considered the hottest IPO on the planet -- is suffering its own apocalypse.

The pandemic has also exposed, once more, the huge digital divide.

Even the previously mentioned positives are often two-edged swords.

The vast amounts of COVID-19 data available on the internet can leave our heads spinning and are sometimes seemingly contradictory.  Even when the data source is unquestioned -- including my go-to dashboard mentioned above that cites the CDC and WHO among its sources -- other troubling questions emerge such as "what are the precise definitions of the terms ACTIVE, RECOVERED and CRITICAL?"  This can lead to increased FUD (fear, uncertainty and doubt).

There have been significant supply chain and delivery issues that have required arcane "hacks" to get grocery delivery scheduled.

In addition, the contact tracing technology noted in the above article has its critics, mostly based on privacy concerns, and a significant number of devices will not be compatible.

Needless to say, society and big tech are facing a defining moment.  I believe and hope that we can all rise to the occasion by making sustained empathy a cornerstone of our recovery.  This trial has surely illustrated the value of each and every person, irrespective of wealth or status.

* NOTE: It took some sleuthing to address the frequent and troubling objection that the 2020 mortality rate is fairly typical and the COVID-19 numbers are simply manifesting as a "different cause of death".  However, by comparing the CDC's Average Daily Number of Deaths of 7,708 from 2017 (latest data I could find) to a recent 3-day rolling average of US COVID-19 deaths of 2,034 , we can reasonably conclude that 2020 is definitely anomalous.  Furthermore, in the same sample year of 2017 -- influenza and pneumonia which would be the counterparts to COVID-19 in a typical year -- account for only 2% of annual mortality.

My Apple Smartphone is Kinda Dumb

The iPhone is still considered, by most, to be the gold-standard in smartphone usability.  It has been around for 12+ years and my family has owned 4 different models, the latest being the iPhone 8.  I must concede that, aesthetically at least, the device continues to "wow".

However, when it comes to content management (file management, data synchronization and backup), I regard its capabilities as quite primitive, leading me to question how smart we should actually consider the device.  I will discuss specific shortcomings later, but I first want to speculate on why this apparent blind-spot from a company usually so visionary.  I hope you recognize the interdependent nature of these observations:

1. An unnecessarily tight coupling with Apple's own iTunes and iCloud software, each of which has been criticized for its own usability issues.  
2. Apple does not appear to have seriously invested in iPhone file management, data synchronization and back-up strategies beyond its own iCloud service.  This means, effectively, that the company is up-selling its customers additional iCloud storage as the "free tier quota" is quickly consumed. 
3. Since the roll-out of the original iPhone, Apple's primary focus appears to have been on consumer-oriented features such as increased camera/photo resolution, video recording, voice control (Siri), touch ID and improved battery life.  Each of these enhancements play well in advertising campaigns as they are relatively easy to quantify.
4. Although data synchronization as a concept is simple for most of us to grasp, it is deceptively nuanced and complex to implement.  There are offline and online trade-offs around caching for performance and real-time access to systems-of-record that have to be considered as the device switches between WiFi and cellular networks.  Furthermore, each application has its own notion of data attributes, think an iPhone Contacts app record versus a Microsoft Outlook Contact record.
5. Apple corporation's notorious tendency to internally silo products and teams, likely make this type of enterprise-wide effort more difficult to anticipate and execute, even assuming the appropriate corporate stakeholder prioritization is in place.
6. The smartphone class of devices has, in this regard, become a victim of its own success and ubiquity, as it increasingly becomes the hub of our digital lives.  Ever more data management responsibilities -- mostly around our social, entertainment and financial well-being -- are being piled onto a device that was originally conceived as a simpler computer for the masses.

The first two observations are manifestations of the (in my opinion) misguided vision of an Apple-only ecosystem.

While I deeply admire Apple and even many aspects of the iPhone itself, I am underwhelmed and disappointed with some of the iPhone's content management (file/synchronization/backup) capabilities.  Here are some specific problems:

• Inconsistent photo and video synchronization behavior.  I have observed, across iPhone models running different iOS versions, the inability of the device to locally delete uploaded images and videos, even when explicitly instructed to do so.  As a result, the device becomes "gummed up" with copies of images that I considered archived, requiring serious detective work to resolve.
• App organization.  While it has improved over time, the device's ability to organize and locate apps remains quite crude.
• No unique device identifier appended to files.  This could help in a variety of ways during synchronization, upload, backup and organization.  At a minimum, it would help eliminate the possibility of file namespace collisions when multiple devices are archived to the same backup storage.
• Limitations of Apple ID model.  Authentication through an Apple ID governs the features and data available to an iPhone user.  For example, it controls access to data stored in iCloud and purchases made through iTunes.  However, higher level data abstractions, such as an Apple customer's family and device inventory do not appear to be properly accounted for in the underlying architecture.  This results in Apple's customers having to employ unnecessary and clumsy workarounds to share content.

I challenge Apple to think different again.  Below are suggestions on how to make an even smarter phone:

1. Offer an all Apple-ecosystem architecture and content management workflow that you (Apple) are able to simplify and optimize.  However, you must also offer a hybrid ecosystem architecture and content management workflow.  Most of us, in either or both our personal and professional lives, must straddle the Apple and non-Apple digital worlds.
2. Continue (and complete) your evolution away from client-installed applications (e.g., iTunes and iCloud for Windows) towards a fully cloud-based architecture.
3. Revisit the design that regards the Apple ID as the highest level data abstraction.  Consider the notion of a household entity that would allow multiple Apple IDs and devices to be associated, smoothing the opportunities to exchange and synchronize content.   
4. Make your pricing model, for both the Apple-ecosystem architecture and the hybrid ecosystem architecture, simple and transparent.
5. Provide the option for the device to "auto-organize" apps by categorizing using App Store attributes or user-defined tags or properties.

Taming the Business Software Development Life-cycle

In this age of seemingly weekly JavaScript framework releases, countless web APIs, and cloud-based everything, it seems that business software development has lost its pragmatism.  The pace of change – propelled by hyperbole, first-to-market pressures, endless updates to underlying software stacks and insatiable consumer and corporate demands for the “latest and greatest” – is dizzying.    

That this Gold Rush mentality has emerged in an industry whose primary goal is often equated with efficiency, is ironic to say the least.  In this New Wild West, business software line managers and project managers sometimes feel like planning is a luxury that they must do without!

How Did We Get Here?

In its infancy, in the 1980's, software development heavily leveraged the processes and tools that had been used for decades in civil engineering projects.  These were adapted ever-so-slightly for software development to become the Waterfall Methodology.  This methodology (or model) adhered to the basic notions of a budget, a team of resources, a scope and explicitly or implicitly stated quality, and it followed a simple, sequential flow from requirements gathering to maintenance.  The Waterfall (or Traditional) Methodology provided executives and management with assurances that planning and accountability were baked into increasingly greater capital intensive (and hence scrutinized) software development projects.  

What has become clear over the past couple of decades, however, is that in software development (when compared to traditional engineering disciplines) there are "moving parts" that are significantly harder to measure.  Therefore, delivering a project on time and under budget -- the historical yardstick by which project success is measured -- becomes orders of magnitude more difficult when the project involves software development.

The initial response to this realization was for software development teams to increase time spent on analysis and estimation up-front.  Although seemingly logical, this adjustment did not address the unique challenges associated with software development projects.  In particular, it failed to recognize the following:

1. A given software development resource could be orders-of-magnitude more (or less) productive than his/her counterpart.  While there is a broad spectrum of ability in any human endeavor, it is particularly the case in software development.  Fred Brooks's book The Mythical Man Month does an excellent job of making this (and other) critical points related to software engineering.
2. By their very nature -- industry and company specific processes, for example -- business software development projects are unique in the extreme.  This means that it is virtually impossible to even define (Requirements box in the Waterfall model diagram above), let alone build (Implementation box), the optimal solution without multiple iterations.
3. Software development quality, when compared to many other traditional engineering disciplines, is notoriously difficult to measure.

Naturally, without the ability to accurately estimate a resource's productivity (point #1 above), define up-front and in detail the scope of the project (#2), and ensure its quality (#3), it becomes almost futile to try to develop a schedule and budget.

Along Comes Agile

This lack of software development project management success has, of course, been well documented.  As a counterweight to the Waterfall Model a new approach emerged in 2001.  That approach, termed Agile -- backed by its very own Manifesto -- essentially proposed breaking down the silos between the disciplines within software development and repeatedly asking and refining this question: Are we building the right thing?

This philosophical break from the rigidity of the Waterfall Model, significantly reduces the pressure on the up-front phases (Requirements and Design) which, as we have already seen, are almost certain to "miss the target".  It compensates by recognizing that, collectively, the "team" (both business and technical) will iterate towards a solution that will be revealed during regular proof-of-concepts, demos and pilots.  These iterations are typically organized as short "sprints", oftentimes just a week in length.  (To further distance itself from "tradition", Agile introduces its own vocabulary, including "sprint".)

Agile has become so influential that some of its principles are being applied outside of software development, including so-called "Business Agile".

More recently, others have gone even further in proposing the no projects movement.  The movement argues that the project paradigm and its associated vocabulary is oftentimes unrepresentative and misleading and results in harmful practices (for example, the creation of temporary/cross-functional project teams).  The movement asks some interesting questions.

Looking to Industry Trends

In order to simplify and improve the quality of software development, and by extension the management of such work, I believe that we can look to the following industry trends that are moving us in the right direction:

1. Migration to the Cloud.  For all of the hyperbole surrounding it, the en-masse migration to the cloud is a net positive for most business software development teams.  Once the new tools and cost structures are well understood, "the cloud" helps to eliminate many of the challenges of managing and tuning complex IT infrastructure, allowing better focus on solving the at-hand business problem.
2. Service Oriented Architectures and REST API.  The movement away from monolithic applications and suites to "a la carte services" can be challenging, since it introduces "seams" in the software stack that must be "sealed" by the development team.  However, as the approach matures and coalesces around the REST API, it increasingly seems to embody just the right levels of flexibility and robustness.  And, of course, REST is built on, perhaps, the most resilient and well-tested software protocol (HTTP) ever!
3. Open-Source.  The rise of open-source has democratized and leveled the playing field for entrepreneurial technologists and helped to provide a wave of low-cost, high-quality tools, facilitating improved team collaboration, source code management, deployment, system administration and testing.  
4. Browser as Lowest Common Denominator User Interface.  The trend towards the browser as the de-facto UI has been something of a mixed blessing, although it ultimately simplifies software development.  It reduces the number of user "platforms" that must be considered for deployment, testing, etc, and abstracts away many of the details required in designing usability.
5. Improved Integrated Development Environments (IDEs).  Features such as code-completion and linters are becoming standard in IDEs,  helping to reduce many of the syntax and library reference errors that were common in prior generations of these tools. 

What's a Technical Manager to Do?

As we have seen, there is undeniably, and understandably, frustration with the way that software development projects have been managed.  This frustration might be experienced by any of the stakeholders, on either the business or technical side.  Moreover, with the current “developer power mentality" coursing through the veins of tech companies big and small, there is an increasingly vocal line of argument that business process, project tracking and even managers just get in the way

However, it should also be clear that something as complex as software development, warrants and requires significant planning.

Those of us who have been involved in software project management for a while, have probably observed two competing but equally true phenomena:

• No project is exactly the same as another
• All projects have a large degree of commonality

How do we make sense of this apparent contradiction and tame the software development process?

I have come to embrace many of the important philosophical adjustments that Agile encourages.  In particular, frequent and open demos to key stakeholders to level set and calibrate the direction of the development work with the business need.  Simple ticketing systems, such as Jira, and the use of User Stories are also valuable tools.  Short "sprints" (development cycles) and "retrospectives" (process reviews) help to keep the lines of communication open between the technical and business teams.  Perhaps most important of all, Agile helps to break down silos, most notably between the business and technical sub-teams.  Almost as critical, Agile helps to reinforce DevOps and QA as first-class responsibilities of the project, thankfully moving us away from a "throw it over the wall" mentality.

At the same time, Waterfall probably better recognizes some of the realities of business software development projects, such as deadlines and multiple stakeholder communities (both upstream and downstream of the "core customer").  In addition, Waterfall incorporates the Gantt Chart -- still one of the most popular project management "visualizations" available -- and other artifacts that can, with a little creativity, be allied to a generally Agile approach.

Collectively, we might consider this approach to be Hybrid or Pragmatic Agile.

As a supplement to this Pragmatic Agile approach, here is my checklist for technical project and line managers:

[X] Enlist
[X] Communicate
[X] Organize
[X] Simplify
[X] Visualize

Enlist

As Technical Managers, a fundamental first step is building and maintaining a high-performing team.  In general, we should err on the side of smaller (rather than larger) teams and, of course, ensure that roles and responsibilities are clearly established.

Although we don't always have full control over project personnel, we should be on the lookout for the following critical characteristics:

1. Project motivation and interest, on both the technical and business sides
2. Team-orientation and cross-disciplinary interest (planning, development, testing, documenting, etc)
3. Willingness to learn ("agility") and to stay current with industry technologies (see previous paragraph)

These traits apply equally to us as managers, helping to maintain our respect and credibility.

A great reference for building high performing teams is a book titled Beautiful Teams.  

Communicate

It is largely accepted that at least 75% of a project manager's time is spent communicating.  As technical managers, we deliver this communication in a variety of written forms (text messages, emails, documents, presentations, diagrams, etc) and verbally (by phone, in person one-on-one, in meetings and other group situations, etc).  We should take the time to understand the preferred communication format and frequency of each team-member and, where appropriate, adapt accordingly.

When communicating with the team, we should emphasize the importance of the project (to the client).  For example, we might quantify the expected revenue, number of transactions, unique visitors or other metrics associated with the solution.

It probably does not come as a surprise that many software professionals view meetings and writing documentation with skepticism.  Offering options is a way for us to reduce the cynicism, get engagement and facilitate improved quality.  As technical project managers, we may offer the option of remote meeting attendance.  We  may also offer the enticement of a technical deep dive as a segment of every staff or status meeting with a round-robin opportunity for each team-member to both choose a topic and/or present.

Instead of a traditional user guide to support our software development project, we might recognize more enthusiasm within the team for directing and/or starring in a video that "documents" the application's usability.  As a bonus, this might also provide a more unique and superior user experience.

Above all, we should be transparent in our communication, both internally to the team and externally to the client and other project stakeholders.  We should encourage participation in the direction-setting.  As has always been the case, the keys to managing software are communication, planning, and more communication.  There is such a thing as over-communicating, but rarely have I experienced it. 

Organize

As technical managers, we can facilitate much of the communication through logical and consistent organization of project artifacts.  Documents, issue tickets, project charters, team rosters and shared credentials should be easily accessible.

We should strive to review, understand and document, as necessary, the related business processes, working to remove any potential roadblocks that might interfere with smooth and unencumbered project progress.  This includes translation of the business vocabulary into the required technical parlance (development, quality control, operations, support, etc).  At the same time, we need vigilance to ensure that the processes and supporting documentation truly serve only the needs of the project itself, and do not become a means to an end.

By reviewing the project artifacts, we often find duplication of content.  Our goal is to have a single source of truth through consolidation.  From my experience, these redundancies are a very common and time consuming occurrence on software development projects.  They are overlooked or excused because they are targeted specifically for a given project constituency (end user, business analyst, developer, project manager, etc), using that specialization's nomenclature.  As an example, we must clearly determine the level of specificity for each business requirement, and whether we should record them directly as user stories (or similar) in a ticketing system or out-reference tickets to a "specification document".  The key is not to maintain both!

Simplify

Simplification is another key principle that we, as technical managers, need to apply and re-calibrate on a frequent basis.  We can work with sponsors to keep the project goal in focus and reduce the likelihood of scope creep.

We can also simplify by ensuring alignment, as much as is reasonably possible, around development, testing and other tools.  The proliferation of programming languages, dialects, integrated development environments, test frameworks and more is a blessing only when the final tool selections are identified, clearly communicated and enforced throughout the team.  Without that, our teams will be fighting productivity-sapping compatibility and setup issues.

I have recently been working on large eCommerce projects using the Intershop platform.  One of the strengths of this platform is its flexibility and its ability to interface with many other enterprise vendors (such as Microsoft, SAP, Oracle, Salesforce, etc).  However, that ability to customize must be used judiciously.  Fortunately, the Intershop platform also offers a wealth of out-of-the-box functionality that can be implemented with simple configuration changes only.  This, of course, means that you are leveraging tested and resilient functionality.

Bottom-line, promote low code (and similar low customization) approaches within your software development teams, reserving custom coding for the truly critical and differentiating features of your software development project.

Visualize

As technical managers, we also have the opportunity to help stakeholders across the entire spectrum truly understand the project's status.  Every project includes countless informative metrics ranging from budget to resource allocation to estimates and actuals.  We must ensure the data integrity of these metrics and, with tools such as Jira, Trello, Smartsheet and Microsoft Project, we have the capability to track, aggregate and visualize.

We can assemble these metrics into reports and data visualizations, perhaps starting with simple but critical calendar-based visualizations that communicate project milestones, resource availability (personal vacations, company holiday/workshops/team events), meetings, etc.  We can also include clean and simple diagrams, such as this recent one from Amazon, to help convey complex concepts such as application architecture. 

Eventually, we might progress to interactive visualizations provided by business intelligence tools such as Tableau, Qlik, Microsoft Power BI or D3, enabling drill-down dashboards across the entire project portfolio.

A picture really might be worth a thousand words, especially if the team does not have to hear them in another project meeting!