Most AI tool trials I hear about start with a signup and end with a shrug. The tool was there, some people used it, nobody can say whether it helped, and it is still there. I think the fix is a short list of questions answered before the trial starts, in writing, by the people who will be accountable for the answers. Here is mine.
Where does the code go, and what is kept
The first question is about data. When an engineer sends a file to the tool, where does it go, how long is it kept, and who at the vendor can see it. Is it used to train anything. The answers are usually in a document nobody reads, and they differ enormously between tools and between tiers of the same tool. I think a trial that cannot answer this question has not been approved by anyone with the authority to approve it.
What can the tool reach
The second question is about access. An agent that can run commands can read environment variables, open network connections, and call whatever the developer's credentials can call. I would want to know exactly what the tool can reach from a developer's machine and from the build system, and I would want the trial to run with less than that. Scoped tokens, a sandbox, no production credentials in reach. The time to find out that a tool reads the whole home directory is before it does.
What would success look like
The third question is the one most trials skip. What is the tool for, which tasks, and what would change if it worked. Faster first drafts of a particular kind of change. Fewer review rounds on a particular kind of bug. Something specific enough that it could fail. I think a trial without a success condition cannot end, because there is nothing to compare the result against, and a trial that cannot end is just an adoption with no decision attached.
What happens to review
The fourth is about the rest of the team. If the tool works as advertised, more code arrives at review. Who reviews it, what are they looking for, and how much of their week is that going to take. I think the honest answer is often that review becomes the limiting step, and a trial that measures only the author's speed will declare victory while the reviewers quietly drown.
Who can turn it off
The fifth is about control. If the tool does something unexpected, sends data somewhere it should not, or an incident at the vendor makes the news, who on the team can disable it that afternoon, and how. A tool that can only be turned off by a procurement process is a tool the team does not control.
How the trial ends
The last question is about the exit, in both directions. If the trial fails, what gets removed and who tells the people who liked it. If it succeeds, what changes: which workflows, which documentation, which norms, and who owns those changes. Either way the trial should end on a date with a decision, not fade into permanence because nobody scheduled the conversation.
None of these questions are hard. Most of them take an afternoon. What they do is turn a trial from something that happens to a team into something the team does, with a reason, a measure, and an ending.
Photo source: https://photos.robertstowe.com/dolomites