Run a Receivables Tone Experiment

This guide runs a controlled A/B comparison between two tone profiles. The experiment assigns eligible customers to one profile, keeps each customer's assignment stable, and reports recorded payment conversions by variant.

Prerequisites

• At least two reviewed tone profiles exist.
• A tone-enabled workflow sends payment communications from a template.
• The workflow has customer context and real eligible traffic.
• The workspace's AI and communication providers are configured.
• Invoice payment and recovery records are reliable.
• The team has defined the hypothesis, observation window, minimum sample, and decision owner.

An Auto-chase policy created only through Settings → Receivables Controls is not sufficient evidence that messages are tone-enabled, because that form does not attach a template or tone profile.

1. Define One Testable Hypothesis

Choose one deliberate difference, for example:

• empathetic versus neutral wording
• neutral versus firm wording
• lower versus higher urgency
• standard versus simplified reading level

Write down:

• expected direction of change
• target customer population
• start and end dates
• minimum assignment count per variant
• primary metric
• conditions that would stop the test

The page reports assignment and conversion rates. It does not calculate statistical significance or determine a safe sample size.

2. Prepare Variant A and B

1. Open Receivables → Tone Profiles.
2. Create or edit the two profiles.
3. Give each a clear experiment-safe name.
4. Review preset, formality, empathy, urgency, reading level, and brand voice.
5. Review blocked phrases, required disclaimers, and maximum length.
6. Save both profiles.

Do not make the variants differ in many unrelated ways if you need to understand why one performed differently.

3. Verify the Eligible Communication Path

Before creating the experiment, send an approved communication to a controlled test recipient through the same workflow.

Confirm:

• the workflow uses a communication template
• a template-specific or team-default tone profile is available
• customer context is present
• the AI provider transforms content when expected
• the provider sends the message
• the original template is safe if transformation fails

Tone failure falls back to the original content rather than blocking the send. Content rules help validate transformation but are not a substitute for reviewing the base template.

4. Create the Experiment

1. Open Receivables → Tone Experiments.
2. Select New Experiment.
3. Enter a name.
4. Add an optional description containing the hypothesis and planned window.
5. Choose the profile for Variant A.
6. Choose a different profile for Variant B.
7. Set the percentage assigned to A. The remainder is assigned to B.
8. Select Create experiment.
9. Verify the status is Draft.

Use a two-sided split such as 50/50 when both variants need comparable volume. A 0/100 or 100/0 split is allowed but cannot provide a useful two-variant comparison.

5. Start the Experiment

1. Select View details.
2. Recheck the variant names and traffic split.
3. Select Start experiment.
4. Verify the status is Running and the started time appears.

Only one experiment can run for the team. If the page names another running experiment, pause or complete that experiment before starting this one.

6. Let Assignments Accumulate

While the experiment is running, an eligible communication can assign its customer to A or B. The first assignment for that experiment is sticky: later eligible messages to the same customer use the same variant.

This means:

• the traffic split controls assignment probability, not exact counts
• repeat messages do not create a new assignment for the same customer
• the denominator represents assigned customers, not messages
• small samples can be visibly unbalanced

Open View details to inspect the A and B counts and customer IDs. If no assignments appear, do not assume the experiment is operating.

7. Preserve the Payment Observation Window

Keep the experiment Running while assigned customers have a reasonable opportunity to pay. The supported automatic outcome path looks up currently running experiments when it records a recovery payment conversion.

Pausing or completing too early can leave later payments unattributed. Manual invoice status changes and some alternative payment paths may also fail to populate an experiment outcome.

Before ending the test:

1. Verify assigned customers received the intended eligible communication.
2. Allow the planned payment window to pass.
3. Reconcile recorded payments and invoice status.
4. Confirm the experiment summary has updated outcomes.

8. Read the Results

The details view shows total and per-variant:

• assignments
• conversions
• conversion rate

Interpret conversion rate as:

assigned customers with a recorded true paid/converted outcome ÷ assigned customers

It is not:

• delivered messages divided by sent messages
• amount recovered divided by amount due
• time to payment
• proof that tone caused the difference

Compare sample sizes and source records before comparing percentages. One conversion in a very small group can produce a large but unstable rate.

9. Pause or Complete Deliberately

Use Pause when the test must temporarily stop accepting eligible traffic. Remember that automatic payment attribution can also stop while paused.

Use Mark completed only after the observation window closes.

1. Select Mark completed.
2. Verify the completed time.
3. Review both variant rates again.
4. Select Variant A wins or Variant B wins only when the decision owner approves.

The winner selection is a label on the experiment. It does not change live communication behavior by itself.

10. Roll Out the Winner

1. Open Receivables → Tone Profiles.
2. Select Set Default on the winning profile if it should become the fallback.
3. Update any template-specific profile that should use the winner.
4. Send a controlled message through each affected workflow.
5. Verify transformed content, required language, provider delivery, and analytics attribution.

Preserve the losing profile until all dependent templates, segment rules, and experiment records have been reviewed.

11. Preserve or Delete the Experiment

Keep the completed experiment when its assignment and conversion record is useful for audit or future comparison.

Use Delete only when permanent removal is intended. Deletion removes the experiment and its assignments and is not reversible. The experiment list does not show a separate confirmation dialog for this action.

Acceptance Checklist

The experiment is complete when:

• the hypothesis and observation window were defined before launch
• A and B are different, reviewed profiles
• the eligible workflow and base template were verified
• the experiment was the team's only running experiment
• both variants accumulated a useful assignment count
• payments and invoice records were reconciled
• result limitations were considered
• the winner was selected manually
• default and template configuration were updated separately
• post-rollout messages were verified

Troubleshooting

New Experiment is unavailable

Create tone profiles first. A meaningful experiment requires at least two different profiles.

Start experiment fails

Another experiment is already running. Open it, then pause or complete it before starting the new one.

No assignments appear

Confirm the status is Running and that template-based, tone-enabled workflow communications are being prepared with customer context. A saved experiment or a basic recovery policy does not generate traffic.

One variant has far fewer assignments

Assignment is random and sticky per customer. Wait for more unique eligible customers, and confirm the configured split is not near 0% or 100%.

Payments happened but conversions remain zero

Check whether the customers were assigned, whether the experiment was still running when the supported recovery outcome was recorded, and whether the payment path writes an experiment outcome. A manual paid status can leave the summary unchanged.

Tone Profiles delivery counts disagree with experiment assignments

The two views use different attribution. Tone analytics group tracked communications by the profile explicitly linked to the template, while the experiment view counts customer assignments and recorded outcomes.

Selecting a winner changed nothing

Open Tone Profiles, set the intended default, and update template-specific configuration. Winner selection does not deploy the profile.

Related Pages

• Tone Profiles and Experiments
• Receivables Controls
• Configure Receivables Controls
• Invoice Collection Workflow
• Receivables