xonecas.io
56101
Df

dotfiles / files

dotfiles / .claude / skills / kde-computer-use / SKILL.md

2.6 KB
1---
2name: kde-computer-use
3description: Apply when you need to interact with a KDE Plasma desktop on Wayland as a user would. Use it to test and verify GUIs there. It finds and focuses windows through KWin, presses keys through uinput, and takes screenshots with Spectacle.
4---
5
6# Control a KDE Plasma desktop on Wayland
7
8`xdotool`, `wmctrl` and `maim` cannot reach native Wayland windows. Use KWin's D-Bus, a uinput keyboard and Spectacle instead. The user shares this desktop, so warn them first, and touch only the windows you opened.
9
10## Find a window
11
12Search by title. Each row starts with the window's match id, `0_{uuid}`:
13
14```bash
15busctl --user call org.kde.KWin /WindowsRunner org.kde.krunner1 Match s rats
16```
17
18Read a window's size, position, `pid` and `desktopFile`, which is its Wayland app id:
19
20```bash
21qdbus6 org.kde.KWin /KWin org.kde.KWin.getWindowInfo '{uuid}'
22```
23
24Two windows can share a title. Tell yours from the user's by `pid`.
25
26## Press keys
27
28`scripts/key.py` focuses the window, then presses keys and clicks. Pass the uuid with its braces. Coordinates are global, so add the window's `x` and `y`:
29
30```bash
31python3 ~/.claude/skills/kde-computer-use/scripts/key.py '{uuid}' click:1180,422 alt+o radio enter
32```
33
34It stops if the window loses the focus before a step. It needs write access to `/dev/uinput` and `python-evdev`. It moves the user's real pointer, and each run takes the focus from them.
35
36Count volume or list steps from a known state, and check the result in a screenshot. A miscount looks like a bug.
37
38## Take a screenshot
39
40The user keeps working, so the active window may be theirs by the time you capture. Focus yours in the same command, with no wait between. Read the image, and delete it at once if it shows anything but your window:
41
42```bash
43python3 ~/.claude/skills/kde-computer-use/scripts/key.py '{uuid}' && spectacle --background --nonotify --activewindow --output /path/to/shot.png
44```
45
46Spectacle takes a moment after it returns: wait for the file, then read it.
47
48## Hear what a window plays
49
50You cannot hear it, so measure it. Record the sink's monitor, which is what actually reaches the sink, before the sink's own volume. It holds every app's sound, so first check that only yours is playing:
51
52```bash
53timeout 4 pw-record -P '{ stream.capture.sink=true }' --target <sink-node-name> --rate 48000 --channels 2 --format s16 /path/to/out.wav
54```
55
56Prove the monitor reads with a quiet control tone from `pw-play`. Recording a stream's node directly misleads: it read sound from a stream the speakers never got. A stream in `running` state and moving bars do not prove sound.