Abstract

Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action. Most agent systems run on desktops or servers, which support tool use and task automation. Mobile devices are also important agent environments because they are widely accessible and contain users’ data, sensors, and daily-use applications. Existing mobile agents mainly operate smartphones through graphical user interface (GUI) actions such as tapping, swiping, and typing, which often form long, interface-dependent sequences, cannot directly access device capabilities, and make execution boundaries difficult to define. We present PalmClaw, an open-source agent framework that runs natively on mobile phones and manages the sessions, memory, skills, tools, and agent loop directly on the device. PalmClaw exposes device capabilities as device tools with explicit arguments, structured results, and clearly defined execution boundaries. This design enables agents to use mobile capabilities directly while keeping each action explicit and controlled. Experiments show an 11.5% relative improvement in task success and a 94.9% reduction in completion time over the strongest baseline, with lower setup burden and traces illustrating how execution boundaries are applied.

1 Introduction

Recent Large Language Model (LLM) agents have moved beyond generating responses to executing tasks in external environments [yao2023react, schick2023toolformer, li2026clawsbench]. Given a complex user instruction, the agent can decompose the task into multiple steps, call tools, observe the results, and iteratively decide what to do next until the task is completed [openclawGettingStarted, gao2025agentscope10developercentricframework]. This iterative interaction enables agents not only to offer advice, but more importantly, to autonomously execute tasks such as information seeking [zhu2026gisa], file management [xie2024osworld, li2026clawsbench], and workflow automation [xu2025theagentcompany].

An agent is essentially a framework upon a foundational LLM through the incorporation of instructions, persistent memory, reusable skills, executable tools, session management, and an iterative agent loop. Together, these components allow the agent to retrieve relevant memory and session history, load task-specific instructions and skills, invoke the LLM, execute tools, and feed the resulting observations into subsequent turns. Representative agent frameworks, such as AutoGen [wu2023autogen], AgentScope [gao2025agentscope10developercentricframework], and OpenClaw [openclawGettingStarted] host these components on a desktop computer or cloud server.

While desktop and server platforms provide a rich and practical environment for agents, we argue that supporting agents natively on mobile devices is also important. First, mobile phones are among the most widely available personal computing devices, with four out of five people aged 10 or older worldwide owning one [itu2024mobileownership]. This broad availability allows mobile-native agents to reach users who may not have continuous access to a personal computer. Second, smartphones store highly personal data, provide sensors such as cameras, microphones, and location, and host many of the applications used in everyday life [wang2024mobileagentbenchefficientuserfriendlybenchmark]. Running agents directly on smartphones can therefore place them closer to the users, their device resources, and the capabilities required for daily tasks.

Despite this potential, existing mobile agents primarily operate smartphones through GUI actions such as tapping, swiping, and typing (see Figure 1(a)) [wang2024mobileagentbenchefficientuserfriendlybenchmark, zhang-etal-2025-agentcpm, ye2025mobileagentv3fundamentalagentsgui]. Although broadly compatible with existing applications, this approach has three limitations: 1) high-level tasks become long action sequences that are sensitive to layout changes and complex interfaces; 2) it cannot directly access device resources and capabilities, such as local files and sensor data; and 3) GUI control grants broad access to reachable interfaces, without clear task-specific access limits or execution boundaries. GUI interaction is therefore useful for general screen operations, but insufficient as the sole action space for mobile agents. The recent rise of terminal-native agents such as Codex CLI [openaiCodexCLI], Claude Code [anthropicClaudeCode], and Gemini CLI [googleGeminiCLI] reflects a broader shift toward explicit and executable tool-based interaction, motivating a similar native on-device design for mobile agents.

We present PalmClaw, an open-source agent framework designed to run natively within the mobile environment¹¹1PalmClaw is released under the AGPLv3 license. Installable packages are available at https://github.com/ModalityDance/PalmClaw/releases/latest. As shown in Figure 1(b), PalmClaw runs the agent loop and manages its memory, skills, tools, and session state on the mobile device. It exposes device resources and capabilities through device tools: device operations with explicit arguments, structured results, and tool-specific execution boundaries. Together, our framework reduces dependence on external desktop or server devices for agent orchestration, allows agents to use mobile capabilities more directly, and keeps each mobile action explicit and bounded. Experiments on representative mobile tasks show that PalmClaw improves task completion success by 11.5% relative to the strongest baseline and reduces average task completion time by 94.9%. Deployment and trace analyses further show lower setup burden and illustrate how execution boundaries are applied.

The key contributions are as follows:

• We present PalmClaw, an open-source mobile-native agent framework that hosts the agent loop, memory, skills, tools, and session state directly on the mobile device.

• We design device tools that expose mobile resources and capabilities through explicit arguments, structured results, and tool-specific execution boundaries.

• We evaluate PalmClaw on mobile tasks, showing an 11.5% relative improvement in task success and a 94.9% reduction in completion time, with lower setup burden and a trace-based analysis of execution boundaries.

2 Related Work

General agent frameworks.

LLM agents extend language models with tool use, intermediate observations, and multi-step action loops. ReAct [yao2023react] connects reasoning traces with actions, and Toolformer [schick2023toolformer] studies API calls as part of language modeling. Frameworks such as AutoGen [wu2023autogen], AgentScope [gao2025agentscope10developercentricframework], and OpenClaw [openclawGettingStarted] package these mechanisms into systems for real tasks, hosted on computers or cloud servers. PalmClaw instead hosts the agent components directly on the mobile device.

Mobile agents.

Mobile agents study how LLM-based systems can complete tasks on smartphones. Existing work has developed mobile agents and benchmarks, including AndroidWorld [rawles2024androidworlddynamicbenchmarkingenvironment], MobileAgentBench [wang2024mobileagentbenchefficientuserfriendlybenchmark], Mobile-Bench [deng-etal-2024-mobile], AgentCPM-GUI [zhang-etal-2025-agentcpm], and Mobile-Agent-v3.5 [xu2026mobile]. Related systems such as ApkClaw [apkclawSoftware], MobileClaw [mobileclawSoftware], and ClawMobile [du2026clawmobile] also explore phone-side task execution with GUI actions. PalmClaw focuses on hosting the agent framework on the phone and exposing device capabilities through device tools.

3 Framework

This section describes the agent components in PalmClaw and how they work together during on-device execution.

3.1 Overview

PalmClaw runs the components needed for multi-step agent execution directly on the mobile device. As shown in Figure 2, sessions organize user interactions, memory preserves reusable information, skills provide task-specific instructions, and tools expose device resources and capabilities. The agent loop combines these components through repeated model and tool interactions. LLM inference is provided through a remote LLM API, while context management, tool execution, and session state remain on the mobile device. PalmClaw integrates these components into a mobile application, with representative interfaces shown in Figure 6.

3.2 Agent Components

PalmClaw maintains sessions, memory, skills, and device tools directly on the mobile device.

Sessions.

Sessions organize persistent agent interactions from three input sources: local chat messages, messages from connected remote channels, and automated messages generated by scheduled wakeups. Each session stores its conversation history, tool traces, attachments, and a dedicated workspace for documents, temporary files, and generated artifacts. PalmClaw routes each input to an existing or new session, allowing tasks to continue across multiple turns while keeping concurrent tasks separate.

Memory.

PalmClaw maintains two persistent memory layers: 1) shared long-term memory, which stores information available across sessions and is included in each agent’s turn, and 2) per-session summaries, which keep compact records of task progress and conclusions. When unconsolidated messages reach a configurable window, a separate LLM call summarizes older messages and updates both layers. Memory tools allow the agent to read or update long-term memory and to read or search the history of a selected session.

Skills.

Skills are reusable task instructions for applying tools and completing common routines, stored in SKILL.md files. PalmClaw provides built-in and user-installed skills. For each turn, it adds a summary of available skills to the context, loads the full instructions of skills marked as always active, and selects other relevant skills by matching recent user messages against their names, descriptions, and keywords. Only always-active and selected skills are loaded in full, providing task-specific guidance without loading every skill into the context. Figure 3 summarizes the built-in skills and tool groups.

Tools.

Device tools connect model-generated tool calls to resources and capabilities available on the phone. Each tool links a model-facing specification containing its name, description, and JSON argument schema and a device-facing function that invokes a mobile service or app-managed operation. PalmClaw sends the specification to the LLM, executes the associated function when called, and returns a structured result or error. Built-in device tools cover device status and permissions, media, Bluetooth, calendar, and contacts, and workspace files; other tool groups support web access, memory, communication, and scheduled automation.

3.3 Agent Execution

These components work together through context assembly, the agent loop, and device tool execution.

Context assembly.

After an input is assigned to a session, PalmClaw assembles the agent context from six sources: 1) system instructions from AGENT.md, SOUL.md, and USER.md; 2) runtime context, including the current time, timezone, session ID, and workspace paths; 3) shared long-term memory; 4) active skill instructions and a summary of available skills; 5) recent conversation history, attachments, and tool traces; and 6) available tool specifications, including their names, descriptions, and argument schemas. The first five sources form the model messages, while the tool specifications are provided alongside them to the remote LLM API. Figure 5 shows an example of this context.

Agent loop.

PalmClaw runs each agent turn as a bounded multi-round loop that connects remote LLM reasoning with on-device tool execution [yao2023react, schick2023toolformer]. In each round, PalmClaw rebuilds the context from the current session state, sends the model messages and available tool specifications to the remote LLM API, and stores the returned assistant message in the session. If the response contains tool calls, PalmClaw executes them, appends their results to the session, and starts the next round with the updated context. If the response contains no tool calls, it becomes the final reply. The loop also ends when a tool signals task completion or the round limit is reached.

Tool execution.

Tool calls from the agent loop enter the device tool execution path shown in Figure 2. PalmClaw first matches each call to the tool registry. Before the corresponding operation is executed, the call passes three checks: 1) Schema validates required arguments, types, and allowed ranges. 2) Permissions includes device authorization and user confirmation. Missing permissions invoke the system permission or settings flow, while actions requiring direct user involvement proceed only after the user confirms or completes them. 3) Workspace resolves file paths within the current session workspace or approved shared storage, rejects other session workspaces and unapproved global paths, and requires confirmation before external writes. A call that passes these checks is executed against the corresponding device resource or capability. Its structured result is returned to the session for the next round; if a check fails or the user declines, PalmClaw returns a structured error.

媒体内容 · 前往原文查看

Framework	Host	Action space	MobileTask				AssistantBench
			Actions	Tokens	Time	Success Rate	Accuracy
MobileClaw	Desktop	GUI actions	94.3	1.30M	1197.0s	60.0%	5.26%
ClawMobile	Termux	APIs and GUI actions	51.4	466.4K	451.1s	77.1%	10.49%
ApkClaw	Mobile app	GUI actions	103.9	2.06M	348.8s	87.1%	25.79%
PalmClaw	Mobile app	Device tools	2.8	50.4K	17.7s	97.1%	36.85%

4 Evaluation

We evaluate PalmClaw’s on-device agent framework through three questions:

• RQ1: Does PalmClaw support effective mobile task completion through its on-device agent framework and device tools?

• RQ2: Does hosting the agent framework on the mobile device reduce external deployment and operation requirements?

• RQ3: How are execution boundaries applied when PalmClaw’s device tools execute mobile actions?

4.1 Experimental Setup

Datasets.

We evaluate PalmClaw on two datasets. 1) MobileTask contains 70 mobile tasks adapted from AndroidWorld [rawles2024androidworlddynamicbenchmarkingenvironment], MobileAgentBench [wang2024mobileagentbenchefficientuserfriendlybenchmark], and Mobile-Bench [deng-etal-2024-mobile]. It covers calendar, weather, contacts, audio, notes, files, Bluetooth, and media tasks, and is evaluated by task success. We use deterministic final-state checks when possible, evidence-grounded judging for answer-based tasks, and manual review for permission- or device-state tasks. 2) AssistantBench [yoran2024assistantbenchwebagentssolve] evaluates realistic web information-seeking tasks with verifiable answers. We use a 19-task development-set subset because the local test split does not provide gold answers, and some development tasks contain unstable real-time values. AssistantBench is evaluated with the official accuracy metric. Appendix A.2 gives the construction procedure and dataset statistics.

Baselines.

We compare PalmClaw with three representative open-source mobile-agent baselines. 1) ApkClaw [apkclawSoftware] is an Android app for natural-language device control and acts through GUI operations such as tapping, swiping, typing, and opening apps. 2) MobileClaw [mobileclawSoftware] uses a phone-side client with an external Python runtime and controls the phone through screen observations and GUI actions. 3) ClawMobile [du2026clawmobile] places the agent runtime in Termux, an Android terminal environment that provides a Linux-like space on the phone, and executes actions through phone APIs, shell commands, and GUI control.

Implementation.

All mobile experiments are run on Android, using a Xiaomi Redmi 2312DRAABC phone with Android 13 (SDK 33). We use DeepSeek-V4-Flash [deepseekai2026deepseekv4highlyefficientmilliontoken] as the LLM backbone for all systems. MobileClaw additionally requires a visual GUI model for screenshot-based grounding, so we use qwen3.7-plus [qwen37]. For AssistantBench, PalmClaw uses a 30-round limit, and screen-based baselines use a 100-round limit, since GUI agents often spend extra rounds on observation and navigation before taking an action.

4.2 Task Completion (RQ1)

We first test whether PalmClaw can complete mobile tasks through its on-device agent framework and device tools. In addition to task success, we report the average number of actions, total tokens, and completion time per task. We count one LLM API call as one action. Token counts include all input and output tokens, and completion time is averaged over all tasks. These measures are essential because users are sensitive to response latency when completing tasks on a phone.

Results in Table 1 give two findings. 1) PalmClaw differs from the baselines in both framework placement and action space. It runs the agent components on the mobile device and acts through device tools, whereas the baselines rely on GUI actions, combine APIs with GUI control, or host the agent framework on a desktop. 2) On MobileTask, PalmClaw achieves an 11.5% relative improvement in task success and a 94.9% reduction in average completion time over the strongest baseline, while using fewer actions and tokens. It also obtains the highest accuracy on AssistantBench. These observations show that PalmClaw supports effective and efficient mobile task execution. A key reason for the efficiency gains is that device tools expose mobile resources and capabilities as structured operations, allowing many tasks to avoid long sequences of screen observation, navigation, and GUI actions.

4.3 Deployment and Operation (RQ2)

媒体内容 · 前往原文查看

Framework	External Requirements			Setup
	Computer	CLI	Bridge	# Steps	Time
OpenClaw	Yes	Yes	Optional	4	5 min
MobileClaw	Yes	Yes	Yes	8	15 min
ClawMobile	No	Yes	Optional	6	25 min
ApkClaw	No	No	No	3	5 min
PalmClaw	No	No	No	2	2 min

To examine whether PalmClaw reduces the external environment needed to deploy and operate a mobile agent, we compare two aspects of each framework: external requirements and setup burden. The comparison includes the mobile-agent baselines used above and OpenClaw [openclawGettingStarted], which serves as a representative desktop-hosted agent framework. Computer records whether deployment or operation requires a separate computer. CLI denotes setup or operation through command-line commands. Bridge denotes a required connection between the phone and another runtime or control service. We then manually deploy each framework and record the visible setup steps and time.

Table 2 gives two findings. 1) PalmClaw does not require a separate computer, CLI workflow, or bridge. The agent can therefore be started and used on the mobile device without attaching the task flow to another execution environment. This follows from PalmClaw’s on-device design: its agent components and execution flow are integrated within the same mobile application. 2) PalmClaw needs two setup steps and about two minutes before the first instruction, less than the compared frameworks. Because both the user entry point and agent framework are on the phone, setup does not require a separate computer or bridge.

4.4 Execution Boundaries (RQ3)

We use trace-based case analysis to illustrate how execution boundaries are applied during device tool execution. We collect records where PalmClaw stops, requests user involvement, or returns a failure because a tool call reaches a permission or confirmation boundary, a workspace boundary, or an unsupported operation.

The traces show how PalmClaw checks a tool call before it becomes a mobile operation. Each call is matched to the tool registry and then passes the relevant schema, permission or confirmation, and workspace checks before execution. Thus, the agent does not receive an open command channel to the phone. Figure 4 visualizes two representative cases: a calendar permission that can be granted through the platform permission flow, and a workspace boundary where a public path requires manual settings access. Table 3 in the appendix lists additional cases.

Case 1: Calendar permission.

In the calendar-permission case, the user asks PalmClaw to add a meeting, and the agent reaches the boundary when the calendar tool checks whether calendar access is available. As shown in Figure 4, PalmClaw does not continue with an unchecked calendar operation. It sends a mobile permission request to the user, then continues the same task if access is granted. If access is not granted, the agent reports the permission issue and explains what the user needs to do instead of claiming that the meeting was created.

Case 2: Workspace boundary.

In this case, the user asks PalmClaw to save a shopping list outside the current workspace, such as under /sdcard/list.txt. PalmClaw resolves the requested path before completing the file operation, and the file tool returns a boundary result because this public path requires access to all files. Unlike calendar permission, this access cannot be granted through a normal mobile permission prompt. PalmClaw opens the application settings page and tells the user to enable all files access manually.

5 Conclusion

This paper examined how agent frameworks can be designed for the mobile environment beyond operating phone interfaces. We presented PalmClaw, an open-source on-device agent framework that runs directly on the phone and exposes device capabilities through device tools with explicit arguments, structured results, and tool-specific execution boundaries. On MobileTask, PalmClaw achieves an 11.5% relative improvement in task success and a 94.9% reduction in completion time, while lowering setup burden. Trace-based cases further illustrate how its execution boundaries are applied. Together, these results show that mobile phones can serve as environments for agent frameworks, extending mobile-agent research beyond screen operation toward direct and controlled use of device capabilities.

Ethics Statement

PalmClaw is designed to keep mobile actions explicit and bounded: tools are registered, arguments are validated, sensitive actions rely on Android permissions or user confirmation, and file operations follow workspace boundaries. Deployments should still make provider-side data flow clear to users, especially when remote LLM providers receive prompts, tool results, or task context. The experiments in this paper use public or constructed evaluation tasks and do not require collecting private user data.

References

Appendix A Additional Details

A.1 Agent Context Composition

Figure 5 shows a shortened example of the context assembled for one PalmClaw agent turn. It includes system instructions, runtime context, shared long-term memory, active skill instructions, and an available-skill summary, recent conversation history, tool traces, and available tools.

A.2 Datasets and Metrics

MobileTask.

MobileTask adapts tasks from AndroidWorld [rawles2024androidworlddynamicbenchmarkingenvironment], MobileAgentBench [wang2024mobileagentbenchefficientuserfriendlybenchmark], and Mobile-Bench [deng-etal-2024-mobile]. Each candidate is rewritten as a user-goal instruction with a task identifier, source label, expected artifact, and evaluation note. We retain a task only if 1) it can be attempted without a fixed GUI path, 2) it does not depend on third-party app state, and 3) it does not favor GUI-only agents. We also remove unstable cases, including relative dates, live weather, and duplicated calendar variants.

AssistantBench.

AssistantBench [yoran2024assistantbenchwebagentssolve] is used as a secondary information-seeking set. From the 33-task dev set, we keep 19 tasks whose answers remain checkable after manual review. We exclude tasks with 1) unstable live-state answers, or 2) reference answers that cannot be checked reliably. Table 4 summarizes statistics for the two datasets used in the evaluation.

Metrics.

MobileTask reports the mean success rate over 70 tasks. A task is successful when its final state or answer matches the predefined oracle, using deterministic checks for directly inspectable outcomes. For answer-based tasks, the same DeepSeek-V4-Flash model serves as the judge. It receives the task instruction, the agent’s final answer, and fixed reference evidence, and returns a binary success-or-failure decision. AssistantBench reports official accuracy averaged over the subset, using word-level F1 for string answers, log-distance scoring for numeric answers, and aligned mean F1 for list answers.

媒体内容 · 前往原文查看

Boundary	User request	Boundary event	Agent outcome
Missing permission	“Add a meeting with Alex tomorrow at 10 a.m.”	Calendar permission missing	Requests permission
User confirmation	“Take a photo of my desk to check the setup.”	Manual confirmation needed	Sends confirmation request
Workspace boundary	“Save my shopping list to SDCard/list.txt.”	File path outside workspace	Tells user to enable access
Unsupported action	“Open Instagram to check updates from my friends.”	No matching tool	Reports unsupported operation

媒体内容 · 前往原文查看

Dataset	Category	Source	#
MobileTask	Calendar	AndroidWorld	32
		MobileAgentBench	3
	Weather	Mobile-Bench	10
	Contacts	AndroidWorld	3
		MobileAgentBench	4
	Audio	AndroidWorld	4
		MobileAgentBench	3
	Notes	MobileAgentBench	4
	Files	MobileAgentBench	3
	Bluetooth	AndroidWorld	2
	Media	AndroidWorld	2
	Total		70
AssistantBench	Medium	Dev set	10
	Hard	Dev set	9
	Total		19