HeyClicky Hands-on: A Screen-Aware AI That Goes Beyond Dictation

Introduction

Daily useful-tool sharing until military service, Day 72.

I found HeyClicky through founder Farza's demo post on X. It is not simple speech-to-text: when you use push-to-talk, it reads the current screen, understands what you are doing, and responds with voice guidance or cursor cues.

My first impression is that it turns Computer Use-style screen understanding into a desktop assistant you can simply talk to. It is a genuinely compelling idea—closer to an AI sitting next to you and helping with work.

Sources and download:

What HeyClicky Does

The official description calls it an AI buddy that lives next to your Mac cursor. You can ask about what is on screen and get spoken guidance, including visual pointers to interface elements. Saying heyclicky agent can hand background work—such as research or building a webpage—to an agent.

That makes it useful for screen-contextual tasks rather than just dictation:

Drafting a prompt in Claude Code based on the project or current screen
Handling a Gmail reply with the email's context visible
Asking what a control does in unfamiliar software
Getting help with the exact step currently blocking you

My Quick Test

I did not subscribe to Pro immediately. I tried the free allowance a few times first. The free tier currently gives 25 uses per month, so it runs out quickly; anyone wanting it as an everyday work companion will need Pro for a proper evaluation.

Still, the core experience shown in the demo was present in my test. It is not just a concept video. Putting voice, screen context, and cursor guidance into one interaction feels very different from a conventional dictation tool.

The Multi-Monitor Bug I Hit

I tested it on two monitors. While asking it to explain a task, its marker or click target sometimes appeared at the corresponding coordinates on my other display. Its multi-monitor coordinate handling clearly still needs work.

For now, I see it as a promising product being refined quickly: it is interesting for a single screen and for interface guidance, but I would still verify the cursor position myself for precision work across multiple monitors.

Privacy and Open-Source Status

With a screen-aware tool, it is important to know what it sees. The official privacy policy says HeyClicky captures screenshots and voice input locally on your Mac when you invoke push-to-talk, then sends them through its backend for AI processing. It names Anthropic, OpenAI, AssemblyAI, and ElevenLabs as providers. Do not leave sensitive material on screen when making a request.

The current HeyClicky app is not a fully open-source product. Farza has kept an earlier Clicky codebase public under the MIT license, but its README says that new features will remain private. For the latest app, use the official HeyClicky website.

Takeaway

The most interesting part of HeyClicky is not that it is another AI you can talk to. It brings screen understanding into real-time interaction.

It currently has a small free allowance and my multi-monitor positioning issue, but the direction feels right. If reliability improves, this is one of the AI tools I could see leaving on every day.