Skip to content

wip: Image Search, Bitmap and PNG savings, and test/build updates. #784

Draft
octalmage wants to merge 8 commits into
masterfrom
feat/imageSearch
Draft

wip: Image Search, Bitmap and PNG savings, and test/build updates. #784
octalmage wants to merge 8 commits into
masterfrom
feat/imageSearch

Conversation

@octalmage
Copy link
Copy Markdown
Owner

@octalmage octalmage commented Mar 14, 2026

It's wild looking at 10 year old code you wrote.

This is the first pass at the long, long awaited image searching. Find an image on the screen, or find an image in another image, it's basically the same thing.

const robot = require('robotjs');

const screen = robot.screen.capture();
const post = robot.image.load('./post.bmp');

screen.clickImage(post);

It looks something like that, I added a clickImage helper to simplify dealing with higher resolution displays where the captured images might be 2x the actual screen coords, but clickImage is kinda a weird concept. it's more like, click on screen, where the image is found in this image.

This is the long form example:

const robot = require('robotjs');

const screen = robot.screen.capture(); 
const post = robot.image.load('./post.bmp');

const match = screen.findImage(post);
if (match) {
  const point = screen.toScreenPoint(match, post);

  robot.moveMouse(point.x, point.y);
  robot.mouseClick();
}

Going to spend more time thinking about the public API, but this is a start!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant