Picking up a saw does not guarantee a straight cut. Holding a chisel does not
make you a sculptor. Tools can amplify poor judgment as readily as good
judgment. Their usefulness depends on the operator's ability to direct them,
interpret feedback, recognize mistakes, and make corrections. Those
requirements also apply to LLMs.
An LLM's unpredictability is a meaningful limitation. But predictability,
reliability, simplicity, and bounded function are qualities by which we
evaluate tools, not prerequisites for membership in the category. A tool does
not stop being a tool because it is difficult to control or easy to misuse.
Hickey's distinction between tools and automation also creates a false
division. Automation can be a feature of a tool. A compiler performs
substantial work without the programmer continuously directing each
operation. A spreadsheet recalculates results without someone manually
carrying out every calculation. We still wield these systems by specifying
inputs, examining results, and deciding what to do next. Why should
generating text automatically disqualify an instrument from being a tool
when translating code or calculating numbers automatically does not?
The claim that LLM use lacks intention, feedback, direction, and expertise
is especially difficult to defend. A programmer can ask an LLM to explain
unfamiliar code, challenge that explanation, inspect the source, test a
proposed change, reject it, and refine the request. That is a feedback loop
directed by a human pursuing a purpose. Doing it well requires knowing what
to ask, what evidence matters, and when the answer is wrong. Expertise can
make the instrument more useful.
Hickey's concern about reading is valid for a particular use: accepting a
summary in place of reading a paper can sacrifice much of what reading
teaches. But someone can also read the paper and use an LLM to question
their interpretation, explore an objection, or identify something to
examine more closely. The possibility of avoiding intellectual work does
not establish that every use avoids it. The relevant question is what the
person actually does.
What disappoints me the most is the reliance on the AI industry's
anthropomorphic vocabulary: "go rogue," "escape containment," "conspire."
Repeating those expressions risks accepting the framing that makes these
systems sound like independent beings. Software can produce unexpected
outputs, fail to follow instructions, or cause harm when given access to
consequential actions. Those are real engineering problems. Calling them
"conspiracy" adds a suggestion of human motives without establishing any.
I would not claim that an LLM always does exactly what a human tells it to
do. It plainly can fail. But failure to obey perfectly does not make it
cease to be a tool, any more than successful use proves that its operator
has surrendered their judgment.
The useful distinction is between wielding an instrument with skill and
delegating work without adequate understanding or oversight. That
distinction applies to AI, and it gives us something more practical than
declaring that an entire class of software cannot be a tool.