pilot:SFA Pilot Evaluation: Completion, Not Logins

Judge SFA adoption in a small pilot by measuring completed, correct field tasks against scheduled work, logging observation notes and support requests, and agreeing acceptance criteria before the pilot starts.

Sales representative plans store visits with a street map and notebook.

Direct answer: Evaluate an SFA pilot by measuring completed, correct field tasks against scheduled tasks, then use observation notes and support requests to explain the gap between logins and completion. Counting logins is not the adoption verdict; agree acceptance criteria before the pilot starts.

SFA pilot evaluation diagram comparing completed, correct tasks with scheduled store visits
SFA pilot evaluation: completion is measured against scheduled work, not login activity.

The purpose of a sales force automation (SFA) pilot is not to prove that representatives can open an app. It is to prove that they can finish the daily work they already do — reach the right store, record what they found, take the agreed order, close the follow-up — accurately and without a manager standing beside them. Evaluate adoption by measuring completed, correct tasks against scheduled tasks, then use observation notes and support requests to explain the gap between logins and completion. Decide in advance what result counts as acceptance.

Why login activity is a weak adoption signal

Logins, check-ins and photo counts are activity signals. They are cheap to produce and easy to misread. A representative can check in at a store, photograph a shelf and leave without recording an out-of-stock item, a price issue or the follow-up the supervisor asked for. The activity looks healthy; the work is incomplete.

Completion signals are harder to fake and more useful to a buyer. For each workflow step, define the measure with a numerator, denominator, time period and exception policy. For example: scheduled store visits completed with all required fields populated, divided by scheduled visits for the pilot stores, over the pilot period, excluding visits cancelled with a documented reason. Suggested pilot measures include scheduled visits, completed visits, required-field completeness and follow-up completion — these are evaluation measures, not guaranteed improvements.

Ask a second question of every record: does it carry a time, a store, the specific task and a review status? A photo or check-in alone does not establish that a task was performed correctly.

How to evaluate SFA pilot adoption in five steps

Use the same task list for every participant and define each step before the evaluation starts. The evaluation should show whether work is completed correctly and independently, not just whether the app was opened.

  1. Define completion measures. Write the numerator, denominator, time period and exception policy for each workflow step you intend to judge.
  2. Fix the daily task list before day one. Include scheduled visits, mandatory audit fields, order entry rules, follow-up actions and cut-off times.
  3. Name the reviewer. Someone must check a sample of records daily during the evaluation, not only at the end.
  4. Record evidence daily. Log observation notes, support requests, training gaps and exception handling, then review the log weekly with the field supervisor.
  5. Apply written acceptance criteria. Agree the thresholds and sign-off owner before the evaluation begins, then choose one outcome: accept and expand, extend with specific fixes, fix and retest with a smaller group, or stop. Record the reason next to the decision.

Design the pilot around representative roles

Sample size and targets are company-specific. What matters is that the pilot covers the roles, routes and store types where the software will actually be used, and that the workload is close to a normal day rather than a demonstration.

  • Choose two or three representative roles — for example a van sales representative, a merchandiser or store auditor, and a supervisor who reviews the work. Each role should exercise different tasks.
  • Include difficult cases deliberately — a closed store, a stock-out, a credit hold, a store with an unclear address. A pilot with only clean cases tells you little about exception handling.
  • Be transparent with participants. Representatives should know what is being measured and why, or the observation data will reflect suspicion rather than the workflow.

Record four kinds of evidence every day

Completion rates show what happened. The evidence log shows why. Keep it short enough to fill in daily, and review it weekly with the field supervisor.

Daily evidence log for an SFA pilot
Evidence source What to capture Question it answers
Observation notes Where the representative hesitated, skipped a field, or did the task on paper first Is the workflow matched to how the job is really done?
Support requests Date, role, task, issue, who resolved it, how long it took Is the representative independent, or is the pilot dependent on a coach?
Training gaps Tasks that failed for knowledge reasons rather than design reasons Is this a software problem or a training problem?
Exception handling Closed stores, missing SKUs, rejected orders and how each was recorded Does the system accept real-world exceptions without breaking the record?

Keep observation notes factual. Write “rep opened the order screen, could not find the pack size, asked the supervisor, entered the order after eight minutes” rather than “order module is bad”. Facts can be reviewed; opinions cannot.

Turn the evidence into a clear acceptance decision

Write the acceptance criteria before the pilot, agree who signs them off, and set thresholds that reflect your own baseline rather than an external benchmark. A workable structure:

  1. Completion: a defined share of scheduled visits closed with all required fields and the correct follow-up status.
  2. Accuracy: a reviewer sample confirms that recorded values match what was observed in store.
  3. Independence: support requests per representative per day fall over the pilot period, and recurring requests have known causes.
  4. Exception coverage: every exception type encountered has a documented, repeatable handling route.
  5. Reviewability: a supervisor can reconstruct a visit — time, store, task, evidence and status — without calling the representative.

Be careful about what a small pilot can prove. It can show whether the workflow is completable and whether representatives can work independently. It cannot promise productivity gains, implementation timelines or downstream business results. A pilot that ends with “it went well” and no written criteria is not a pilot; it is an opinion with screenshots.

Where Airpalm fits, and what to verify

Airpalm describes mobile visits, route planning, store audits, sales orders, inventory and promotion management (source: airpalm.com, checked 2026-10-09). If those areas match your pilot scope, treat the public description as the starting point and ask for evidence against your own task list: a demonstration using sample stores and visits, the supported mobile environments, required data fields, user permissions and integration scope.

Confirm explicitly whether a task you intend to measure can be completed and reviewed in the product today. Do not assume offline operation, automatic tracking or a specific implementation timeline unless the vendor demonstrates it for your scenario. For related measurement detail, see how to measure store visit quality in FMCG field sales and how to test store audit data quality before buying SFA software.

A restrained next step

Pick three daily tasks, write the numerator, denominator and exception policy for each, and run them with a small representative group for one full work cycle. If you want to compare that task list against a live product walkthrough, start with the Airpalm product overview and ask for a demonstration built on your own store and visit examples.

Frequently asked questions (FAQ)

Is a login count ever useful during an SFA pilot?

Yes, as a participation and access check. If representatives cannot log in at all, no completion data will exist. Use logins to confirm the pilot population and supported devices, then report adoption with completion measures such as visits closed with all required fields and correct follow-up status.

How long should an SFA pilot run?

Long enough to cover at least one full cycle of every workflow you are measuring, including the follow-up step and a week in which exceptions occur. Set the duration with your own operating calendar rather than copying another company’s timeline, and state it in the acceptance criteria before the pilot starts.

What if completion looks acceptable but representatives keep asking for help?

Treat repeated support requests as an adoption finding, not a rounding error. Log the task, the issue and who resolved it. If the same request recurs for one task, it usually points to a design or training gap that should be fixed and retested before you expand the pilot.

Put this guide into practice

This general-purpose worksheet is a starting point. Adapt its fields, responsibilities and rules to your business process.