As someone who spent most of a career in process automation, I've decided doing it well is mostly about state limitation.
Exceptions or edge cases add additional states.
To fight the explosion of state count (and the intermediate states those generate), you have a couple powerful tools:
1. Identifying and routing out divergent items (aka ensuring items get more similar as they progress through automation)
2. Reunifying divergent paths, instead of building branches
If you want to go back and automate a class of work that's being routed out, write a new automation flow explicitly targeting it. Don't try and kludge into into some giant spaghetti monolith that can handle everything.
PS: This also has the side effect of simplifying and concluding discussions about "What should we do in this circumstance?" with other stakeholders. Which for more complex multi-type cases can be never-ending.
PPS: And for god's sake, never target automation of 100% of incoming workload. Ever. Iteratively approach it, but accept reaching it may be impossible.
> Well-designed automation should look like a funnel, rather than a subway map.
Do you have any examples of this? Like an example project?
> never target automation of 100% of incoming workload. Ever.
A new product owner came in last year and wanted to "automate everything". They wanted a web page where builds, branches, commits, and more were all on one page that showed exactly which commit was deployed where, by whom, etc etc. They wanted this extravaganza for a 2-person application that was in maintenance with no new features.
They're also the kind of person who consults chatgpt or copilot on a subject while you're explaining that subject to them, to check that the LLM agrees with what you are saying. They'll even challenge people to prove some copilot output is incorrect. It seems to me that they consider LLMs more reliable than people.
Oh my goodness, I wouldn't last a day with someone who did that. That sort of casual disrespect while you're talking to someone is wholly unacceptable behavior.
There's only one person you work with that's like that, right? Right?
So far they are the only one.
They're definitely "leadership material"!
> They wanted a web page where builds, branches, commits, and more were all on one page that showed exactly which commit was deployed where, by whom, etc etc.
Sounds like they would have been a fan of ungit [0], which I have seen used for that kind of flow overview, though it has looked more impressive than actually proved helpful in my experience.
> A new product owner came in last year and wanted to "automate everything". They wanted a web page where builds, branches, commits, and more were all on one page that showed exactly which commit was deployed where, by whom, etc etc. They wanted this extravaganza for a 2-person application that was in maintenance with no new features.
you know, this seems very reasonable until that last sentence.
Why a web page and not directly the git log? If style is necessary, reformat the log data with some fancy ASCII art?
With your experience, anything you'd recommend to read in the process automation space?
(I'm a founder of an automation SaaS where we've made "human interface" one of the core features of the product)
I wouldn't consider my career being in process automation, but I feel like you just described my approach to managing new product development on top of existing machinery while ensuring no breakage of what exists.
Not directly relevant to the post, but seems like a good place to share.
My team and I once took on a very tricky automation project. At the time we had a complex software deployment done about once per month that involved a team of about a dozen people showing up at 4am to do it while traffic was low.
The deployment involved many manual steps and coordination of everybody involved. The person leading each deployment followed the documented list of steps and got each person to do their bit at the right time; people to run database migrations, people to install RPMs on particular servers, people to test and verify functionality. Mistakes and missed steps were not uncommon.
The very first thing we did was take the documentation and write a Jenkins job to post each step into a Slack channel specifically for coordinating the deployments. Someone clicked "go" and each step was posted as a message in that channel with a 'done' button to be clicked when that step was done. Clicking the button caused the next step to be posted.
The next release we did used that instead of one person reading the steps out of confluence. Everyone involved in the release could always see what step we were at, and when it was their turn to do their bit. This helped ensure no steps were ever missed too.
Over the following months we chipped away at that job a bit at a time. We'd pick a step in the process and automate just that step, starting with the low-hanging fruit first. The Slack message for that step went from "click to confirm you've done it" to "click to do it", with the result posted once it was done; followed by the next step to perform.
It was a long process, but it allowed the rest of the business (and us!) to gradually gain confidence in the automation, and lowered the risk of the project dramatically. Once several steps had been automated and battle-tested we removed the 'click to do' bits in between and the whole release became a couple of clicks followed by the odd bit of manual QA.
> Then one day the business owner (my father) said “hey this customer is going to come by and pick up the order so don’t print a shipping label.” Yikes had to figure that out. Then one day another customer said “hey can we pay cash on delivery?” C.O.D. used to be a thing. Then I had to add that.
What happens if you just say no? I have a feeling a lot of complexity stems from the fact that these exceptions are easy to handle when you’re doing things manually, so it’s hard to say no. But if you think about the business case I would not be surprised if the correct answer is no more often than not.
This reminds me of Seeing Like a State[0] (I know, it's not new... but I only got around to reading it earlier this year.)
Automating a process makes it more standardized and legible, but takes away a lot of the nuance and of the resilience that human institutions tend to bake in. Do it too little, you get chaos; do it too much, you're destroying the very thing you were trying to nurture.
It's certainly deformation professionnelle, but the parallel with software is eerily relevant.
One thing on my list for CRUD software is an „I know what I’m doing” button that bypasses validation. Usually it’s behind manager or admin approval.
It’s pretty tricky to get right and is very case by case in what it specifically means. But it’s been critical to handling edge cases.
For example, in 99% of cases, you can’t progress to stage X without doing Y. But in this case „I know what I’m doing”.
Though this puts a lot of pressure on downstream processes to handle missing data. Fortunately that hasn’t been an issue too much because validation grows and changes over time and must naturally deal with records that were created under prior validation when their state was originally valid.
Related - I highly recommend reading Lisanne Bainbridge's paper "Ironies of Automation" which points out why we need to keep human factors in mind when designing automation.
http://www.complexcognition.co.uk/2021/06/ironies-of-automat...
It doesn’t have to be so hard, though. One thing you can do is learn to just say “no”. Such as no to cash on delivery.
Being flexible is good, but it comes at a cost. When that cost is too high, don’t do it. Realize that the customer who wants that workflow probably isn’t going to be your make-or-break moment. And in fact, they might have flexibility of their own. For example, you don’t accept Amex but they have a backup credit card or cash. It might be annoying to them if they are fussy, but it’s normal enough that the consequences are minimal. And yes, you may occasionally get a customer who doesn’t have that flexibility, but you shouldn’t be pinning your business on rare events. Figure out what’s most common among your target customers and support a few simple workflows. Say no to everything else until you have the resources to do it properly.
Another thing you can do is have a hybrid low tech/high tech solution. Automate structured inputs with software. Write down the rest as notes in a logbook. Over time you will probably see patterns in the logbook that you can automate.
Lastly, remember what Morpheus says in The Matrix, “Some [rules] can be bent. Others can be broken.” For example, you could simply pay the bill on behalf of the customer who wants cash on delivery. Now you assume some personal risk but the computer system doesn’t have to support their workflow. Is it worth it? Maybe, maybe not, but it’s a choice you can make.
We don't automate things because it's easy. We automate things because we thought it would be easy
Much of the pain can be reduced by considering if you should bother automating the thing in the first place.
Some kinds of automation are absolutely essential. Aligning wafer stages during photolithography, shipping logistics, high frequency trading, etc. The business absolutely wouldn't work without it.
The other kinds of automation are more questionable. Developing E2E CI/CD automation for an internal tool that is redeployed once every quarter might be a more difficult sell to management. For these cases, even if the rate of manual process invocation is somewhat high (i.e. frustrating for some employees), the customer can't see it. They won't pay one additional cent for this.
There is also this entire meta time wasting game where the mere debate about "should we automate" takes up so much resources you could have manually invoked the thing more times than it would ever be feasibly invoked automatically during its projected lifetime. Alternatively, you could have just let your developers build the damn thing to get it out of their systems.
Always reconsider the process itself before you automate it. Human based processes are different than computer based ones. Trying to directly emulate human actions is what can make automation fragile.
One of the best pieces on here recently (and didn’t realize it was by Sinofsky!)
Anyway, one other insight I would add is that the issues tend to come up at the interface between the systems. Your automation has to get some input and send some output and both those are pain points. That’s why we sometimes prefer imperfect monolith software.
Don't forget https://xkcd.com/1319/ !
The thing about software is that it can only ever define its own process. "Software as spec". The goal of automation is to find processes whose architecture and dynamics are already amenable to being represented in software.
Public reaction to automation has been mixed for obvious reasons, but also because when software is applied to structure- and/or determinism-resistant systems it fails to capture some essential components that ends up degrading the services that system can provide.