“Make this chair wider” sounds straightforward until the chair has unapplied scale, a mirror modifier and a cushion parented to its frame.
An agent that immediately scales the selected object might technically follow the instruction and still damage the setup. I’d rather it inspect the objects, dimensions and modifier stack first. Sometimes the right change belongs in the mesh; sometimes there’s already a parameter for it.
This is the part of Blender tooling I’m interested in experimenting with: giving an agent enough access to understand what it is about to touch. Selection, collections, linked data and dependencies all need to be queryable. A screenshot alone won’t do it.
For procedural work, language could be a good starting point. “Scatter these stones but leave the path clear” could produce a Geometry Nodes setup with density, seed and an exclusion object exposed. Once the setup exists, I’d prefer to adjust those controls myself rather than negotiate every small change in chat.
And if I dislike the result, undo should work. Keeping the scene editable and the changes easy to inspect matters more to me than an impressive one-shot demonstration.