Sees the real phone screen
Kickstarter story
Lumi AI Dock: The First Desktop AI That Physically Uses Your Phone
Your AI agent can browse the web. Now it can use your phone. Real phone. Real screen. Real taps.
tap
AI memory outputTodo created
Send investor update by Friday
Source: Phone chat 09:41observe_screen()tap target confirmedwrite Todo + trace
1
Your Phone Is Where Work Actually Happens
Important promises, links, files, screenshots, private chats, booking steps, and relationship context are trapped inside mobile apps. Lumi gives AI a supervised physical bridge into that reality.
2
Meet Lumi: See, Tap, Remember
See the real phone screenTap, swipe, and type physicallyRemember Todo, People, Notes, Calendar
3
Demo: Messages Become Todo
Messages -> Todo
Phone chat, physical tap, source-traced task.
4
Demo: People Memory
People memory
Waiting reply, promised quote, next meeting.
5
Demo: Notes and Calendar Memory
Notes and Calendar
Screenshots, files, links, and reminders organized.
6
Real Phone. Real Screen. Real Taps.
Positions for real taps and swipes
Physical interaction with the screen
Stable alignment for common phones
Session state, voice, and local controls
7
PC Phone Copilot
Confirmation requiredSend this message?
Lumi asks before sensitive or irreversible actions.
09:41 observed Messages09:42 opened source thread09:42 created Todo draft09:43 waiting for approval
8
Local-First Privacy
Local-first by defaultNo root or jailbreakVisible action logUser confirmationNo app hacking
Lumi does not hack apps. It sees and taps like a human, with your permission.
9
Phone Use SDK
session = phone_use.connect("lumi-dock")
screen = session.observe_screen()
target = screen.find("Confirm appointment")
session.confirm_action("Tap confirmation button?")
session.tap(target)
session.type("Friday works. Sending calendar invite.")
memory.write({
type: "todo",
source: session.screenshot(),
trace: session.action_log()
})10
Interactive AI Head and Voice
Lumi's head is a serious status surface for session state, voice input, permission prompts, and visible feedback. It is not designed as a toy companion.
11
Hardware and Specs
Phone accessReal screen observation with physical tap, swipe, and type
Mechanism4-axis physical control concept
PrivacyLocal-first memory and user-confirmed actions
MemoryTodo, People, Calendar, Notes
SDKobserve_screen(), tap(), swipe(), type(), screenshot(), confirm_action()
CompatibilityValidation in progress for common iOS and Android sizes
12
Rewards
Early Bird Lumi AI Dock
Best price for early Kickstarter backers.
Dock, Lumi app, core memory workflowsLumi AI Dock Pro / Creator Kit
For demo builders, creators, and power users.
Pro accessories, creator workflow templatesDeveloper Kit
Early hardware-level phone access for agent builders.
SDK preview, examples, developer updatesDuo / Team Pack
For founders, sales teams, and small studios.
Two docks, team onboarding, shared playbooks13
Timeline
Working prototype demo
EVT hardware validation
DVT + Lumi App Beta
PVT and pilot production
Mass production
Shipping begins
14
Risks and Limitations
Will do at launch
Real phone observation
Basic tap, swipe, type
User supervision and confirmation
Todo / People / Notes / Calendar basics
Basic Phone Use SDK
Won't promise
Full automation for every app
Unattended sending, payment, or deletion
Every phone model
Fully local large model for all tasks
Enterprise-grade RPA stability
App hacking or encryption bypassing
15
Back Lumi on Kickstarter
Follow now for launch notification, early bird rewards, and prototype progress.