Let Agents Log Friction

I use Pi as my coding harness. As I’ve built many skills and a few tools, I wanted a way for agents to report friction they ran into while using them.

Where friction usually comes from

• Outdated skill instructions: the product improved or changed, but the skill still refers to an old decision.
• Known script caveats: agents keep rediscovering the same edge cases because they haven’t been fixed or documented yet.
• Unreachable parts: a problem has already been solved. There may be additional context in docs/, a related script in scripts/, or an entirely separate skill dedicated to it. But due to poor cross-referencing, an agent has to navigate for a while before arriving at the right place.

If such problems are left unattended, agents spend more time and tokens. Worse, they may not complete the task despite having the right skills.

Fix: Let agents log friction

A small retrospective prompt is enough

Time for retrospection.

You've invoked the following skills in this conversation:
- skill-name (/path/to/SKILL.md)

Did you face any trouble or friction comprehending the skills or scripts in those skills?

If yes, append your feedbacks in the corresponding FEEDBACKS.md file that's present at the root of every skill.

- Only add concrete feedback that would help improve future use of that skill.
- Do not praise skills.
- Do not edit SKILL.md or scripts.
- Do not overwrite the FEEDBACKS.md file.
- If you did not face meaningful friction, do nothing.

Mention sessionId and dateTime in every feedback entry:
- sessionId: ${sessionId}
- datetime: ${dateTime}

Roughly 150-200 tokens. sessionId and dateTime are injected dynamically.

Two ways to implement this

Approach 1: Fork the conversation, ask the agent to log friction

Fork the conversation after the agent turn ends and inject the retrospective prompt as a new user message.

Advantages

• Preserves the context cache.
• The main agent and the user stay focused on the task. Feedback collection happens in the background.

I’ve built a small Pi extension for this. GitHub Link

Notes:

• “Agent turn end” (agent_end, agent_settled in Pi) means the agent has completed its work and/or is waiting for your reply.
• The extension triggers only if a skill was invoked in the session.

Approach 2: Add a logFriction tool

Expose a dedicated tool that the main agent can call whenever it runs into a problem.

Advantages

• Easy to add. No need to manage a separate lifecycle for when retrospection gets triggered.
• The user can see friction being logged as it happens.
• The main agent knows explicitly that problems with skills are reportable.

Why I prefer Approach 1

• Objective mismatch: the job of the main agent is to complete the task, not to report feedback eagerly. A logFriction tool adds another responsibility during execution.
• Task completion wins: in my experience, agents are more likely to find a workaround and continue than stop to report a problem.
• Logging is explicit: with a logFriction tool, the agent still has to decide to call it. In Approach 1, retrospection is a separate step after the turn ends. Instructions like “do nothing if no friction” and “don’t praise the tools” help reduce noise.
• Keep the toolset small: Pi ships with a minimal set of tools. I didn’t want to add another tool just for feedback collection when the existing ones are already enough to update FEEDBACKS.md.

Making use of collected feedback

Periodically, initially every week and then every month, review FEEDBACKS.md and make changes to the skill. Purge feedback entries that have been acted upon.

Why not edit SKILL.md directly?

A single failure isn’t enough evidence to modify instructions used by every future agent. The agent itself may have made a mistake. Log it first; repeated reports are a stronger signal.

Other uses

The same approach can be extended to collect feedback about tools, the system prompt, and AGENTS.md.

One caveat: if an agent finds a problem in a skill, it may over-index on the negative and start reporting problems in the other areas too. I’ve observed this in practice, so I’d use independent forks for each type of retrospection.