--- title: "AI Agent Idempotency: Test Retries Safely." description: "Test AI agent idempotency across retries, double clicks, restarts, and replay. Use stable intent keys and verify that only one action commits." canonical: "https://scalewithsearch.com/articles/test-ai-agent-idempotency-and-retries" date: "2026-08-19" modified: "2026-09-19" --- ## Site navigation - [Scale With Search](https://scalewithsearch.com/) - Real estate - Real estate - [Real estate](https://scalewithsearch.com/for/real-estate) - Work - Start here - [Send your brief](https://scalewithsearch.com/work#send-your-brief) - [Prepare your six-question brief](https://scalewithsearch.com/work#prepare-your-six-question-brief) - Build - [Site build, content library with SEO, signal desk](https://scalewithsearch.com/work) - For your business - Trades and home services - [Auto body and collision shops](https://scalewithsearch.com/for/auto-body-and-collision-shops) - [Foundation and home repair contractors](https://scalewithsearch.com/for/foundation-and-home-repair) - [Garage door and fencing contractors](https://scalewithsearch.com/for/garage-door-and-fencing-contractors) - [HVAC contractors](https://scalewithsearch.com/for/hvac-contractors) - [Janitorial and commercial cleaning companies](https://scalewithsearch.com/for/janitorial-and-commercial-cleaning) - [Locksmiths](https://scalewithsearch.com/for/locksmiths) - [Moving companies](https://scalewithsearch.com/for/moving-companies) - [Pest control companies](https://scalewithsearch.com/for/pest-control-companies) - [Plumbing and electrical contractors](https://scalewithsearch.com/for/plumbing-and-electrical-contractors) - [Restoration and water or fire damage companies](https://scalewithsearch.com/for/restoration-and-water-fire-damage) - [Roofing companies](https://scalewithsearch.com/for/roofing-companies) - [Towing companies](https://scalewithsearch.com/for/towing-companies) - [Tree services and landscaping companies](https://scalewithsearch.com/for/tree-services-and-landscaping) - [Solar installers](https://scalewithsearch.com/for/solar-installation) - [General contractors](https://scalewithsearch.com/for/general-contractors-and-construction) - [Paving, concrete, and flooring contractors](https://scalewithsearch.com/for/paving) - Practices and professional services - [Bookkeeping and tax practices](https://scalewithsearch.com/for/bookkeeping-and-tax-practices) - [Dental practices](https://scalewithsearch.com/for/dental-practices) - [Family and criminal defense law firms](https://scalewithsearch.com/for/family-and-criminal-defense-law-firms) - [Med spas and aesthetics practices](https://scalewithsearch.com/for/med-spas-and-aesthetics) - [Personal injury law firms](https://scalewithsearch.com/for/personal-injury-law-firms) - [Veterinary clinics](https://scalewithsearch.com/for/veterinary-clinics) - [Gyms and fitness studios](https://scalewithsearch.com/for/fitness) - [Therapy and outpatient health practices](https://scalewithsearch.com/for/therapy-and-outpatient-health) - [Medical billing companies](https://scalewithsearch.com/for/medical-billing) - [Insurance agencies](https://scalewithsearch.com/for/insurance-agencies) - [Financial advisors](https://scalewithsearch.com/for/financial-advisors) - [Property management companies](https://scalewithsearch.com/for/property-management) - [Recruiting and staffing agencies](https://scalewithsearch.com/for/recruiting-and-staffing) - [Architects and interior designers](https://scalewithsearch.com/for/architects-and-interior-designers) - [Logistics and supply chain companies](https://scalewithsearch.com/for/logistics-and-supply-chain) - Agencies, MSPs, and manufacturing - [IT and managed service providers](https://scalewithsearch.com/for/it-and-managed-service-providers) - [Machine shops and precision manufacturers](https://scalewithsearch.com/for/machine-shops-and-precision-manufacturing) - [Marketing agencies and freelancers](https://scalewithsearch.com/for/marketing-agencies-and-freelancers) - [SEO agencies and consultants](https://scalewithsearch.com/for/seo-agencies-and-consultants) - [Small manufacturers and fabricators](https://scalewithsearch.com/for/small-manufacturers-and-fabricators) - Restaurants, shops, studios, and nonprofits - [Restaurants and hospitality businesses](https://scalewithsearch.com/for/restaurants-and-hospitality) - [Retail stores and ecommerce sellers](https://scalewithsearch.com/for/retail-and-ecommerce) - [Photographers, event planners, and travel agents](https://scalewithsearch.com/for/photographers) - [Churches and nonprofits](https://scalewithsearch.com/for/churches-and-nonprofits) - [All industries](https://scalewithsearch.com/for/) - Learn - For your office - [Office job guides](https://scalewithsearch.com/guides/) - [Browser calculators](https://scalewithsearch.com/tools/) - Start here - [How it works](https://scalewithsearch.com/how-it-works) - [Free Starter Kit](https://scalewithsearch.com/kit/business-memory-starter-kit.zip) - [Synthetic specimen](https://scalewithsearch.com/specimen/working-session-specimen.zip) - Guides - [The Complete Guide to Business Memory for AI Agents](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [The Complete Small-Business Guide to AI Agent Governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [The Complete Guide to Leaving Vendor AI Memory](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - Articles by cluster - [Business memory](https://scalewithsearch.com/articles/business-memory-for-ai-agents-guide) - [Agent governance](https://scalewithsearch.com/articles/ai-agent-governance-guide-small-business) - [Migration and ownership](https://scalewithsearch.com/articles/leaving-vendor-ai-memory-guide) - For machines - [llms.txt](https://scalewithsearch.com/llms.txt) - [llms-full.txt](https://scalewithsearch.com/llms-full.txt) - [Machine view](https://scalewithsearch.com/?view=machine) - Company - Evidence - [Proof](https://scalewithsearch.com/proof) - Company - [About](https://scalewithsearch.com/about) # Test an AI Agent for Duplicate Runs, Retries, and Double Clicks. A follow-up agent sends two hundred emails. The provider response times out after the sends commit. The agent retries twice. Six hundred messages reach the same two hundred leads. The retry logic worked. The business action happened three times. Idempotency means one intended action produces one side effect, even when the request repeats. Buyers should test duplicate paths before an agent receives send, payment, booking, deployment, or live-record capability. ## One intent needs one stable key Give each business action a caller-generated idempotency key. The key should represent the intent, not one network attempt. A retry uses the same key. A new approved action uses a new key. AWS defines an idempotent operation as one that can be retried without additional side effects. Its Builders' Library recommends unique caller-provided request identifiers so a service can recognize repeated intent. [AWS Builders' Library: Making retries safe with idempotent APIs](https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/) For an approved email, derive or store the key from stable fields: ```text action_type:: send_email business_id:: BIZ-01 account_id:: ACCT-104 recipient_id:: PERSON-882 artifact_hash:: 7ce4... approval_id:: APR-1902 intent_id:: FOLLOWUP-2026-0819-001 ``` Do not use the current timestamp alone. Each retry would receive a new key and look like new intent. ## Test five duplicate paths Buyers need five scenarios: 1. Double click before the first response. 2. Timeout after the destination commits. 3. Process restart after local intent is recorded. 4. Overlapping scheduler runs. 5. Manual replay of an old run. Each scenario should create exactly one downstream object or effect. Test concurrent arrival, not only sequential replay. Two workers can receive the same intent before either one records completion. The idempotency store needs an atomic claim or equivalent transaction around the key and effect. Record which worker won the claim. The losing worker should return the existing result or a duplicate state. It must not wait, generate a new key, and act again. The [pre-production agent test](/articles/test-an-ai-agent-before-production) should use a recorder, test account, or reversible fixture. Do not learn about duplicate charges from a live customer. ## Record intent, commit, response, and receipt separately One "success" field cannot describe distributed action. Record four checkpoints: **Intent recorded.** The system accepted one exact proposed action and key. **Commit attempted.** The executor called the destination. **Response observed.** The executor received a provider response or timeout. **Result verified.** The executor queried the destination or received a stable object identifier. The destination can commit before the response reaches the agent. That is the dangerous gap. AWS notes that recording the idempotency token and mutating state must avoid partial outcomes. Its example covers the case where a resource is created but the response never returns. The retry should resolve the same request, not create another resource. ## Resolve unknown state before retrying When a timeout occurs after the commit request, mark the action `uncertain`. Use the idempotency key, provider object ID, recipient, or approved fields to query downstream state. If the intended object exists, write the receipt and stop. If it does not exist and the provider supports safe idempotency, retry with the same key. If the destination offers no safe read-back or idempotent contract, route the uncertain action to a human. Do not convert uncertainty into permission to try again. Stripe's API documents that it stores the status code and response body for the first request associated with an idempotency key. Later requests with the same key return the same result. [Stripe: Idempotent requests](https://docs.stripe.com/api/idempotent_requests) Not every provider behaves like Stripe. Record the contract and retention window for each executor. ## Test restart and schedule overlap The key and action ledger must survive a process restart. Persist intent before the external call. On restart, inspect the ledger. Resume from the last confirmed checkpoint. Do not regenerate a key from memory or a new run timestamp. Schedulers can overlap when a run takes longer than its interval or a manual run starts near the timer. Use a lock, unique job-window key, or deterministic claim in the work queue. Test the lock itself. Kill the process after it claims work. Confirm the lease expires or recovery process can inspect and resume it without producing duplicate action. The [run receipt requirement](/articles/receipt-is-part-of-the-output) helps the buyer see the logical run, individual attempts, and observed downstream result. ## Verify the destination The agent's statement "sent once" is not proof. Count provider message IDs, payment objects, bookings, CRM changes, or deployment revisions. Query the exact destination when possible. For a message recorder, require one object for the intended recipient and artifact hash. For a payment fixture, require one charge object and one ledger entry. For a CRM fixture, require one field transition and no unrelated changes. Record the downstream count in the acceptance receipt. If read-back is unavailable, name the limitation. The buyer may keep that action manual or require a broker that provides idempotency and queryable state. ## Inspect `idempotency-fixture.csv` Use five rows and exact expected side-effect counts. ```csv fixture_id,scenario,intent_id,attempts,fault,expected_effects,expected_status,required_evidence IDEM-01,double_click,ACT-101,2,requests_arrive_together,1,completed,one downstream object and one receipt IDEM-02,timeout_after_commit,ACT-102,2,first_response_dropped,1,completed,read-back finds first object IDEM-03,process_restart,ACT-103,2,restart_after_commit_attempt,1,completed,persisted ledger resumes same key IDEM-04,schedule_overlap,ACT-104,2,two workers claim window,1,completed,one claim wins and one duplicate receipt IDEM-05,manual_replay,ACT-105,2,replay_consumed_approval,1,denied,original object remains unchanged ``` The replay case expects one total side effect because the first accepted execution already happened. The replay itself must add zero. ## Run the acceptance fixture For each row: 1. Start with an empty recorder and ledger. 2. Create one approved intent and stable key. 3. Apply the named fault. 4. Allow the documented retry or recovery path. 5. Query the downstream recorder. 6. Count exact side effects. 7. Inspect logical run ID and attempt IDs. 8. Confirm the approval was consumed once. 9. Confirm the receipt explains duplicate suppression or denial. 10. Reset the test environment through the documented path. The [buyer acceptance checklist](/articles/ai-agent-acceptance-checklist) should mark any duplicate as a critical failure. ## Fail on mismatched reuse too An idempotency key cannot authorize changed intent. If a retry uses the same key with a new recipient, amount, account, artifact, or action class, reject it. Return a mismatch error and preserve both proposed payloads for inspection. This prevents a consumed approval and key from being repurposed for another action. The key store needs a retention rule. If keys expire before delayed retries can arrive, the system may accept an old request as new. Set retention from the maximum replay window and consequence of the action. ## Separate duplicate suppression from business deduplication Idempotency prevents repeated execution of one intent. Business deduplication decides whether two different intents target the same real-world item. Two approved follow-ups may have different intent IDs but still address the same lead and campaign. A separate eligibility rule should catch that business duplicate before approval. Test both layers. The idempotency fixture alone does not prove the lead should receive the message. ## Approval and stopping boundary The test agent may create fixture intents, inject failures, query a recorder, and write receipts. It stops before connecting real sending, payment, booking, deployment, or CRM credentials. The executor stops when the key is missing, reused with changed intent, outside its retention window, tied to an expired or consumed approval, or left in uncertain state without reliable read-back. Passing the fixture does not authorize production access. Activation, credentials, live schedules, and external actions require separate owner approval. ## Sources - [AWS Builders' Library: Making retries safe with idempotent APIs](https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/) - [Stripe: Idempotent requests](https://docs.stripe.com/api/idempotent_requests) - [OpenTelemetry: Tracing API](https://opentelemetry.io/docs/specs/otel/trace/api/) ## Questions about Test an AI Agent for Duplicate Runs, Retries, and Double Clicks ### How are you retrying agent write actions without doing them twice? Give one business intent a stable caller-generated idempotency key and reuse that key for every retry of the same action. Record the intent, request, provider response, external read-back, and receipt separately so an uncertain result is resolved before another write. ### How to handle repeated button clicks / duplicated customer messages without triggering the same path twice? Run a duplicate fixture that includes a double click, timeout after commit, process restart, schedule overlap, and replay. The destination should show one side effect for one intended action, while mismatched reuse of the same key must fail. ### Which tool calls are safe to retry? Retry only when the action has a stable intent key or when read-back proves the earlier attempt did not commit. If the external state remains uncertain, stop and record that condition instead of risking a duplicate send, payment, booking, deployment, or live-record change. ## Related: Testing and Acceptance - [Five AI Agent Reliability Metrics a Small Team Can Verify](/articles/ai-agent-reliability-metrics-small-business) - [What an AI Agent Rollback Plan Must Contain Before Production Access](/articles/ai-agent-rollback-plan) - [Claude vs ChatGPT for HR Policy Work: Test the Handbook Job, Not the Chatbot](/articles/claude-vs-chatgpt-hr-policy-work) ---- ```text .|########||. .|########||. .|########||. |##||. .||##|. |##||. .||##|. |##||. .||##|. |#|. .|#|. |#|. .|#|. |#|. .|#|. |#| |#| |#| |#| |#| |#| .#| |#. .#| |#. .#| |#. .#. .#| .#. .#| .#. .#| |#. .#| |#. .#| |#. .#| |# ...... #| |# ...... #| |# ...... #| .# ||#########| #| .# ||#########| #| .# ||#########| #| .#. |######||######|. .#. .#. |######||######|. .#. .#. |######||######|. .#. #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# #. .##|###|##|#|######| .# || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || || |##|#||||||||||||#||#| || # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. # |#||||||||||||||||||||#| #. || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # # .#||||||||||||||||||||||||#| # || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || || ....|||#||||||##|#|||#|#||##||||....|. || #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# #. . ....|# ....#||| ||| .#||| ....#. .# # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. # . ||||#| .#####|| . .#| .#||| ||||# #. .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || .| ||||| #. |#|||||. || #. |#||. .||||| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || || |.... #. ....|#. |. ...|. ....|| || #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# #. .||||||##||||||##||####|||||||#||||||#| .# # .||####|###########################|. # # .||####|###########################|. # # .||####|###########################|. # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#||||||.#.|| # |. ..# #| #|||||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||||| .. |# ## |#| ...#|#|||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#|||#|# .# .#| #| ##|.#..#|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # .#||||||############|######|||||#| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # # |...||#||||||#||||#|#||||||#|||||#||| |.| # #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# #. .. ||||# .|||##|. |#| .|| || .|||#. #| # .# || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || | ...|| ...##| | #| .|. |. ##### .. .| || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || || ||||| || ||||#| . |. |. |#. ||||| |#| || || .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. .# |.....#|....|#.||||.|||##.|#|....||.#||.#. #. #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# #.|||||||#############################| ||| .# || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || || ##||#||#||||||#||#||#||#||##. || # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # # .#|||||||||#||#|||||||||||#| # || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || || |#||||||||||||||||||||||#| || .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. .# |#||||||||||||||||||||#| #. || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || || |#||||||||||||||||||#| || #. .######|#|#########| .# #. .######|#|#########| .# #. .######|#|#########| .# .#. |#####||||#####| .#. .#. |#####||||#####| .#. .#. |#####||||#####| .#. |# ||########|| #. |# ||########|| #. |# ||########|| #. |# ...... #| |# ...... #| |# ...... #| |#. .#| |#. .#| |#. .#| |#. .#. |#. .#. |#. .#. .#| |#. .#| |#. .#| |#. |#| |#| |#| |#| |#| |#| .|#|. .|#|. .|#|. .|#|. .|#|. .|#|. |##||. .||##| |##||. .||##| |##||. .||##| .||########||. .||########||. .||########||. Scale With Search 2026 [scalewithsearch.com](https://scalewithsearch.com) ```