A mobile task does not begin when an agent makes its first tap. It begins with the state already present on the device and in the app: the screen that is open, the account that is signed in, the permissions that have been granted, and any data left by earlier activity.
That state can determine what the agent sees, which actions are available, and what a successful outcome means. If it is unknown, two runs that appear to use the same task and device may actually be testing different conditions.
Consider a simple evaluation: add one item to a shopping cart and confirm that the cart contains exactly one. The first attempt starts with an empty cart and succeeds. If the next attempt reuses the same signed-in account, the previous item may still be there. Adding the item again produces a quantity of two. The agent may have followed the instruction correctly, but the run no longer tests the same starting condition.
The issue is not that every run must begin from a perfectly clean phone. It is that the starting state must be intentional, observable enough for the question, and handled consistently across attempts.
State has more than one boundary
“Device state” is often used as if it were one thing. In practice, relevant state can live in several places, and a reset at one boundary may leave another untouched.
Process and interface state includes the foreground app, current screen, open dialog, navigation history, keyboard, and transient in-memory data. Restarting a process may change this layer without clearing information stored by the app.
App-local state includes preferences, databases, files, and other app-specific data. Android provides distinct storage mechanisms for these kinds of data. Clearing package data is a stronger intervention than closing or relaunching an app, but it acts on the app package’s data on the device.
System and device state includes permissions, locale, display settings, OS and app versions, network configuration, and other conditions that can affect what the app presents or how it behaves. Some are controlled inputs; others may change during execution and need to be observed.
Account and service state can outlive the local app session. A cart, saved address, account preference, or workflow record may be synchronized with a service. Clearing local app data does not, by itself, establish that remote account data has returned to a baseline.
These boundaries matter because a reset is an action, while a known state is a condition that must be checked. Android’s pm clear command deletes data associated with an app package. Android Test Orchestrator can use package-data clearing between test invocations to reduce shared app state, with the trade-off of restarting the app for each test. Those controls are useful, but neither is a universal proof that every relevant local, device, and service-side condition matches the intended baseline. Android Debug Bridge · AndroidJUnitRunner and Test Orchestrator
Reinstallation is not always a reliable shortcut either. Android Auto Backup can restore app data when an app is installed, depending on the app’s backup configuration and the restore path. A workflow that assumes “uninstalled means empty” can therefore begin with data it did not explicitly prepare. Back up user data with Auto Backup
Choose the state model that matches the question
A useful collection workflow distinguishes at least three practical starting patterns. They are options for different questions, not a universal taxonomy.
Known baseline. Prepare a defined starting point before each attempt. This supports comparison when the evaluation asks how an agent performs under controlled initial conditions. The baseline should say which app, account, permissions, and relevant device settings are required, and how each requirement is checked.
Known seeded state. Deliberately prepare a non-empty or partially completed state. This is useful when the task is to continue an existing workflow—for example, finish a saved draft or modify an existing cart. The seed should be described well enough that another run can begin from an equivalent condition.
Sequential state. Carry the result of one task into the next because the sequence itself is what is being evaluated. In this case, a reset between tasks would destroy the scenario. The workflow should preserve the predecessor relationship and record which prior result established the next starting point.
The cart example shows why this choice matters. If the question is whether an agent can add an item from an empty cart, each attempt needs an empty-cart baseline. If the question is whether it can add an item to an existing order, a seeded cart is the right start. If the question concerns a multi-step shopping journey, preserving state between steps is part of the task. Applying the same reset policy to all three would answer the wrong question for at least two of them.
Verify the baseline before the first action
A preparation command does not prove that preparation succeeded. A launch may restore a saved session; a reset may miss service-side data; a permission change may not take effect as expected; or the app may open on a screen different from the one the task assumes.
Before an attempt starts, define the state assertions that matter to the task. For the cart example, that could include the signed-in account, the expected cart contents, the app version, and the screen from which the task begins. It does not require recording every property of the phone. Record the variables that could change the interpretation of the result.
Then verify those assertions using an appropriate source. A visible screen may be enough to establish the cart count for a UI task. A task about durable state may require reopening the relevant view or using another authorized, reliable observation. If the baseline cannot be confirmed, mark the attempt as not comparable or unresolved rather than silently treating it as a clean run.
This creates a useful separation in the run record:
- the intended baseline and preparation steps;
- the observations that confirm the baseline;
- the actions and state changes during execution;
- the resulting state and evidence used for assessment.
Keeping these pieces distinct helps explain whether a surprising result came from the agent’s behavior, the initial conditions, or state left by an earlier attempt.
Decide what happens after the run
State management also needs an end-of-run policy. After evidence has been captured, the device can be restored to a baseline, preserved for a defined next step, or held out of the pool until its condition is understood.
Restoring a baseline is appropriate when the next attempt expects an independent start. Preserving state is appropriate when the next task intentionally depends on this result. Holding a device out of rotation is useful when a run ended partway through a change, a reset failed, or the resulting state is uncertain. These outcomes should be recorded; otherwise, “available” can be mistaken for “ready for this task.”
The policy should also protect evidence. Resetting too early can erase the state needed to assess an outcome. Capture the relevant observations and result first, then apply the cleanup or handoff step. When the state change itself is the result—for example, a preference that should persist after reopening—verify that condition before resetting it.
Measure state drift, not just task completion
For repeated collection, report how many attempts began from a verified baseline, how many required preparation again, how many carried state forward intentionally, and how many were excluded because their starting or ending condition was uncertain. These counts make it easier to distinguish task performance from state-management failures.
The right checks depend on the task. A short UI interaction may need only a known account and a verified starting screen. A multi-step workflow may need a state snapshot or seed between stages. A long-running evaluation may need to detect drift in app data, permissions, network conditions, or service-backed records. More instrumentation is useful only when it helps answer the question or diagnose a failure.
This builds on the run identity and evidence model in What Makes a Real-Device Data Collection System Programmable?. A run contract describes what should happen; state management makes its starting point and handoff between attempts explicit. It also extends the infrastructure concerns in Why 10 Phones on a Desk Are Not a Data Collection Infrastructure: a device pool is useful when each device’s readiness is tied to a defined task and observable condition.
Reliable mobile AI data does not require every device to be reset before every task. It requires a deliberate state model: define what must be true, prepare for that condition, verify it before execution, and account for every state change afterward. A clean slate is one possible starting point. A known state is the requirement.