If ‘1.4× conversion’ is the only reason to change your onboarding tool
When signups keep coming to a self-serve SaaS product but users drop off during initial setup, an AI that watches the screen and clicks on their behalf sounds compelling. Nimbia says it raised paid conversion by 1.4× for its customer WeMind.
But the number to bring up first in a buying meeting is neither 1.4× nor the prominently displayed 49%→70% on the website. It is the result that first-week activation changed from 45% to 68% in a one-month experiment that randomly assigned new signups. You need to distinguish the figure that is better suited to causal comparison from a simple before-and-after adoption observation.
One case study contains two different comparisons
The WeMind case includes both a randomized A/B test and a comparison from before and after full rollout. They show similar gains, but the nature of the evidence differs.
| Comparison | First-week activation rate | Calculated change | How to read it |
|---|---|---|---|
| One-month randomized A/B test | Existing onboarding 45% Nimbia 68% | +23 percentage points About +51.1% relative | An experimental result comparing the two approaches within the same signup flow |
| Before and after full rollout | Several months before rollout 49% All signups after rollout 70% | +21 percentage points About +42.9% relative | An observation where the product, marketing, seasonality, and signup mix may all have changed |
Therefore, you should not present 49%→70% as the control and treatment groups of an A/B test. The two values from the randomized experiment disclosed in the official customer story are 45% and 68%, while 49% and 70% are figures for the several months before full rollout and all signups after rollout, respectively.
The 1.4× paid-conversion figure means that, in that same one-month randomized experiment, the Nimbia group converted to paid more often than the existing onboarding-product group. But because the raw conversion rates and user counts for both groups were not disclosed, you cannot calculate the absolute lift, confidence interval, or revenue impact.
What the currently public information supports
The fact that this was a randomized comparison is an important addition that was not clear from earlier writing. However, the sample size for each group, the specific action counted as activation, the assignment implementation, and the paid-conversion observation period are still undisclosed. The official story also says marketing was not held constant during the experiment, so you cannot generalize 45%→68% as an effect that applies to every SaaS product.
If it worked, it likely reduced the ‘distance to execution’ more than the need for explanation
Nimbia is a frontend-embedded agent that converses with users by voice while viewing their screen, navigating within the product, clicking buttons, and filling fields. Its published flow is to provide documentation and recordings of existing onboarding calls or test accounts, create a walkthrough, then add a script and user context to the application to invoke it.
This approach can shorten the gap between reading help content and returning to the product to complete setup manually. It is especially worth considering for products where value appears only after users complete several actions, such as connecting data, creating a project, or producing a first output. Conversely, for products whose first value appears in one or two clicks, initiating a voice interaction may itself become new friction.
The functional boundary is also clear. According to a maker response on Product Hunt, if a user moves to another application, Nimbia cannot continue helping inside that external app. If steps outside the product, such as email verification or external payment, are central, break down and verify the full onboarding path before a demo.
The first step is not a sales call; it is separating two lines of metrics
Open the official WeMind case study and record the two items below separately. This distinction alone helps prevent you from incorrectly copying a vendor case study as your company’s experimental target.
- Separate experimental values from observational values.
Record the randomized experiment as “existing onboarding 45%, Nimbia 68%,” and the before-and-after full rollout comparison as “49%, 70%.” Next to 1.4× paid conversion, add “raw rates and sample sizes undisclosed.” - Define activation for your product in one sentence.
For example: “The percentage of new self-serve workspaces that connect real data and create their first output within seven days of signup.” Specify the population, action, and time limit. Also check whether this action is associated with 30-day retention in your existing cohorts. Amplitude likewise recommends agreeing on the value-experience action and using behavioral data to confirm its relationship with long-term retention. - Randomize 50:50 at a single unit.
For a B2B collaborative product, first consider assigning at the workspace level so members of the same company do not see different onboarding experiences. Give the control group the current flow, and give the treatment group the same signup flow plus a Nimbia entry point. The assignment unit and metric-analysis unit must match. - Set stopping conditions before you start.
Set the baseline activation rate, the minimum effect worth detecting, the significance level, and statistical power, then calculate the required sample size and duration. Ending early because results look favorable risks selecting a chance advantage. - Check behavioral boundaries in test accounts that resemble a real deployment.
Include not only the normal path but also insufficient permissions, empty data, slow screens, and transitions to external apps. Confirm with the vendor and documentation how personal-data exposure, incorrect clicks or entries, user interruptions, human handoff, and incident-response conditions are handled. NIST recommends pre-deployment evaluation and reviews of privacy, security, and vendor risks for third-party generative AI integrations.
The first success criterion is not simply a higher activation rate for the Nimbia group. You need to be able to calculate each group’s sample size and raw activation rate, absolute and relative differences, and uncertainty; paid conversion and 30-day repeat use should move in the same direction. Track incorrect actions, interruptions, human handoffs, and support requests separately as safety metrics.
1.4× is an experiment hypothesis, not an adoption conclusion
The WeMind case is more than a simple before-and-after rollout boast. Because it randomly assigned new users over one month within the same signup flow, it makes the possibility that screen-operating AI reduces the distance to first value more concrete than before.
Still, without sample sizes and raw paid-conversion rates, you cannot judge the precision of the effect or its business value. 45%→68% is a reason to try it; an experiment in your own product with each group’s denominator and long-term behavior is a reason to buy.
If you want to dig deeper
How WeMind Grew Trial-to-Paid Conversion 1.4× — You can separately verify the randomized experiment’s duration and comparison groups, the 45%→68% result, and the before-and-after full-rollout figures. nimbia.ai
Getting started: Driving product engagement by obsessing over activation — Explains how to define activation as a shared team language and assess its relationship with long-term retention. amplitude.com
Experiment health checks — Useful for checking conditions that can invalidate an experiment, including mismatches between randomization and analysis units and sample-ratio mismatches. launchdarkly.com
.png)
