56101
Df
dotfiles / files
dotfiles / .claude / skills / xfce-computer-use / SKILL.md
4.9 KB
1--- 2name: xfce-computer-use 3description: Apply when you need to interact with the desktop as a user would in a xfce desktop environment. Use it to test and verify GUIs or TUIs inside the Kitty terminal. 4--- 5 6# Control the XFCE desktop on X11 7 8Use `wmctrl`, `xdotool`, `xwininfo`, `maim`, and `kitten` to inspect and control the XFCE desktop on this machine. 9 10## Prepare the environment 11 12Non-GUI shells start with `XDG_SESSION_TYPE=tty` and an empty `DISPLAY` variable. Export `DISPLAY=:0` before you run any X11 command, and store screenshots or sockets in `/tmp/computer-use-shots`. 13 14```bash 15 export DISPLAY=:0 16``` 17 18The screen resolution is `1280x800`. The top `xfce4-panel` occupies `y=0` to `y=27`, and the bottom panel occupies `x=487` to `x=793` from `y=751` to `y=800`. Place windows between `y=27` and `y=750` so that the panels do not cover them. 19 20## Find, focus, move, and close windows 21 22Use `wmctrl` and `xdotool` to manage windows. Both tools, along with `xwininfo -id` and `maim -i`, accept hexadecimal or decimal window IDs. 23 241. List all open windows with their window ID, workspace, PID, geometry, window class, and title: 25 26```bash 27 DISPLAY=:0 wmctrl -lpGx 28``` 29 302. Launch a GUI application in the background and poll `xdotool` until its window maps: 31 32```bash 33 export DISPLAY=:0 34 nohup kitty >/dev/null 2>&1 & 35 APP_PID=$! 36 for i in {1..30}; do 37 WIN_ID=$(xdotool search --onlyvisible --pid "$APP_PID" 2>/dev/null | head -n1) 38 [ -n "$WIN_ID" ] && break 39 sleep 0.1 40 done 41``` 42 433. Focus a window and wait until it becomes active: 44 45```bash 46 DISPLAY=:0 xdotool windowactivate --sync "$WIN_ID" 47``` 48 494. Move and resize a window so that it fits inside the `1280x800` screen: 50 51```bash 52 DISPLAY=:0 xdotool windowmove --sync "$WIN_ID" 80 60 53 DISPLAY=:0 xdotool windowsize --sync "$WIN_ID" 800 500 54``` 55 56You can also move and resize a window in one command with `wmctrl`: 57 58```bash 59 DISPLAY=:0 wmctrl -ir "$WIN_ID" -e 0,80,60,800,500 60``` 61 625. Check the exact coordinates, size, and map state of a window: 63 64```bash 65 DISPLAY=:0 xwininfo -id "$WIN_ID" 66``` 67 686. Close a window cleanly through the window manager: 69 70```bash 71 DISPLAY=:0 wmctrl -ic "$WIN_ID" 72``` 73 74## Control the mouse and keyboard 75 76Use `xdotool` to move the pointer, click, drag, and send keystrokes. 77 781. Read the current pointer coordinates and the window under the pointer: 79 80```bash 81 DISPLAY=:0 xdotool getmouselocation --shell 82``` 83 842. Move the pointer and click a mouse button. Button `1` is left click, `2` is middle click, `3` is right click, `4` is scroll up, and `5` is scroll down: 85 86```bash 87 DISPLAY=:0 xdotool mousemove --sync 300 250 click 1 88``` 89 903. Drag the pointer from one point to another: 91 92```bash 93 DISPLAY=:0 xdotool mousemove --sync 300 250 mousedown 1 mousemove --sync 500 250 mouseup 1 94``` 95 964. Type text or press key combinations in the focused window: 97 98```bash 99 DISPLAY=:0 xdotool type --delay 20 "echo 'hello world'" 100 DISPLAY=:0 xdotool key Return 101 DISPLAY=:0 xdotool key ctrl+shift+enter 102``` 103 104## Capture screenshots with the mouse cursor visible 105 106`maim` draws the XFixes mouse cursor in screenshots by default. Do not pass `-u` when you need to verify the pointer location. After you save a screenshot, inspect it with the `read` tool. 107 1081. Capture the entire desktop with the mouse cursor visible: 109 110```bash 111 DISPLAY=:0 maim /tmp/opencode/desktop.png 112``` 113 1142. Capture a specific window by its window ID with the mouse cursor visible: 115 116```bash 117 DISPLAY=:0 maim -i "$WIN_ID" /tmp/opencode/window.png 118``` 119 1203. Capture a rectangular region of the screen: 121 122```bash 123 DISPLAY=:0 maim -g 800x500+80+60 /tmp/opencode/region.png 124``` 125 126## Automate the Kitty terminal with Kitten 127 128The user config at `~/.config/kitty/kitty.conf` sets two options that affect automation: 129 130- `initial_window_width 120c` and `initial_window_height 40c` open `kitty` at `1227x1067`, which extends past the bottom of the `1280x800` screen. Resize `kitty` after launch so the whole terminal is visible. 131- `hide_window_decorations yes` removes the titlebar. Move `kitty` with `xdotool windowmove` or `wmctrl -ir` instead of dragging a titlebar. 132 133Remote control is off by default in `~/.config/kitty/kitty.conf`. To drive `kitty` with `kitten @`, enable remote control and bind a UNIX socket when you start `kitty`: 134 135```bash 136 export DISPLAY=:0 137 rm -f /tmp/opencode/kitty.sock 138 nohup kitty -o allow_remote_control=yes --listen-on unix:/tmp/opencode/kitty.sock >/dev/null 2>&1 & 139 KITTY_PID=$! 140 for i in {1..30}; do 141 WIN_ID=$(xdotool search --onlyvisible --pid "$KITTY_PID" 2>/dev/null | head -n1) 142 [ -n "$WIN_ID" ] && [ -S /tmp/opencode/kitty.sock ] && break 143 sleep 0.1 144 done 145 wmctrl -ir "$WIN_ID" -e 0,240,90,850,520 146 kitten @ --to unix:/tmp/opencode/kitty.sock set-tab-title "Work" 147 kitten @ --to unix:/tmp/opencode/kitty.sock set-colors background=#1a1b26 foreground=#c0caf5 148 kitten @ --to unix:/tmp/opencode/kitty.sock launch --location=vsplit --cwd=current 149 kitten @ --to unix:/tmp/opencode/kitty.sock send-text "ls -la\r" 150```