[Cialug] Raspberry Pi UI Automation

[kristau]

For scripting mouse moves and clicks, and keyboard events take a look at
the xdotool package. I'm using that to "automate" a Web-based time clock
system so users only need to scan their QR code to clock in and out. The UI
requires buttons to be clicked with the mouse, but xdotool allowed me to
script moving the mouse over the button and clicking it. The commands look
like the following:

xdotool mousemove 500 500
xdotool click 1
xdotool key Tab

It's a bit of a kludge, but it works quite well. One gotcha is to make sure
your script has 'export DISPLAY=:0' before you kick off the xdotool
sequences.


On Mon, Aug 17, 2020 at 10:47 AM Brett Neese <brneese at brneese.com> wrote:

> Hello all,
>
> I have a Raspberry Pi connected to a pico projector in my kitchen, creating
> a smart display of sorts. Currently I’m running a web-based smart display
> slideshow thing (DakBoard), YouTube Music, and Solarium in different
> windows, and I alt-tab between them manually when I want.
>
> I would like, every night, for my Pi to automatically switch to Solarium
> from whatever is currently the active window — creating a virtual window
> onto the night sky — and switch back to the slideshow every morning. In
> addition, it would be nice to be able to do more advanced UI scripting if
> necessary.
>
> I know if this were non-graphical, I’d just run a CRON job that launches a
> bash script, and if this were Windows I’d use AutoHotKey or Automator on
> OSX, but I’m not sure if there’s anything similar for UI automation with
> Debian.
>
> Thoughts? Bonus points if I can schedule it from Home Assistant (
> https://homeassistant.io), which powers all my smart thingies and home
> automations.
>
> Brett
>
>
> --
> Brett Neese
> 563-210-3459
> _______________________________________________
> Cialug mailing list
> Cialug at cialug.org
> https://www.cialug.org/cgi-bin/mailman/listinfo/cialug
>


-- 
Tired programmer
Coding late into the night
The core dump follows