AI Automation: Automating the Work Around a Human Decision

tl;dr Let the AI work on the vague information, then continue with deterministic behaviors in real workflows.

I want our administration colleagues to stop checking a mailbox for listing deletion requests. They would still approve the deletion, but the reading, translation, lookups, and follow-through should happen without someone driving every step.

The idea is to use AI to do the intent detection, then a deterministic workflow with stepfunction to control the flow. This way we hope to remove repetitive work, and leave the important decision to huamn. We haven't measured the results, and if it's actually improved, yet.

What Are We Actually Removing? #

Today, someone emails a shared mailbox asking for a listing to be deleted. An administrator reads the message, finds the listing, opens its page, and checks whether it's a free listing collected online or a contracted customer page. They decide whether to proceed, do the deletion, and reply using a template in the requester's language.

If they don't speak that language, there's also the copy-paste back and forth with a translation tool.

From experience, Free-listing requests almost always lead to deletion. A contracted page needs closer attention. Why is someone asking us to delete a customer page? It could be legitimate, mistaken, or unauthorized. The status tells us to look more carefully; it doesn't settle whether the requester can authorize deletion.

Most of the routine work happens around the decision. That's where I think we can remove a lot of effort.

Who Keeps the Process Moving? #

One idea being considered in parallel is to share Claude skills among colleagues and let them run the work from their own machines.

I think that could work to an extent. An interactive workspace is useful when a person is checking the result and deciding what comes next. But that person is still carrying the process. They need to start it, notice what's missing, and make sure the request gets finished.

For this mailbox, I want incoming requests to enter the flow on their own. The administrator should receive something ready to review.

There's also coordination to consider. If two people run a skill against the same email, what stops both from acting on it? Sharing instructions helps reuse the approach. We still need shared ownership and a record of what has happened. A central workflow needs those controls too.

The flow we're planning is fairly straightforward:

1Email → interpretation → lookup and checks → human approval
2      → deletion → reply

AI is useful for understanding the different ways people ask for the same thing, across languages, and extracting references to the listing. We then need to resolve those references against the listing system. Whether the page exists or is contracted is something we can check there. There's no reason to ask a model to guess.

Step Functions gives us a place to track the steps, wait for approval, and continue afterward. Approval will trigger the actual deletion, including during the initial observation period.

What Does the Administrator Need to See? #

The planned approval evidence shows which listing we're talking about, whether it's free or contracted, and a link to the live page in case someone wants to double-check.

It should also make the proposed action and any uncertainty clear. If the reviewer has to translate the original email and repeat the lookups, we haven't removed much work. We've added an approval screen.

Some details still need defining: how to verify the requester's authority, which facts to recheck after waiting for approval, and how to prevent duplicate processing. The workflow moves accepted emails into a separate folder, with an agreement that people leave them alone. That's a useful handoff convention, but people can still touch them.

After approval, the system needs to complete the deletion before sending a fulfillment reply. We already have reply templates in different languages, so those are a useful starting point.

How Will We Know It Helps? #

We plan to run it for a while and observe the accuracy. But I also want to know how much work the administrator still has to do.

I'd compare review effort with the current process, track incorrect listing matches and corrections, and check whether approved deletions complete with the right replies. We should also find out whether people can actually stop monitoring the mailbox for these requests.

If classification looks accurate but colleagues still read every email, translate it, and check everything again, we haven't achieved the goal. The useful result would be that they can review the relevant evidence, make the decision, and let the workflow finish the request.