Skip to content

Commit acd9db7

Browse files
bokelleyclaude
bokelley
and
claude
authored
fix: Addie quality improvements from production thread review (#1066)
Reviewed ~50 recent Addie threads and identified specific behavioral gaps. Rules: - Current Spec Only: prevent overclaiming about unimplemented features - Fictional Names in Examples: use fictional brands in hypothetical scenarios - Ads.txt/Sellers.json Accuracy: correct pre-bid timing knowledge - AdCP Agent Types: keep Addie current on formal agent types - Shorter off-topic deflections Web chat: - Session-level feedback prompt after 45s idle on 3+ message conversations - Properly cleans up on conversation switch and tab navigation Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent dc36c29 commit acd9db7

4 files changed

Lines changed: 308 additions & 2 deletions

File tree

‎.changeset/addie-thread-review.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"adcontextprotocol": patch
3+
---
4+
5+
Addie quality improvements from thread review: accurate spec claims, fictional example names, ads.txt knowledge, shorter deflections, agent type awareness, and session-level web feedback prompt.

‎server/public/chat.html‎

Lines changed: 174 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -770,6 +770,85 @@
770770
padding: 4px 0;
771771
}
772772

773+
/* Session-level feedback prompt */
774+
.session-feedback {
775+
display: flex;
776+
align-items: center;
777+
justify-content: center;
778+
gap: 12px;
779+
padding: 12px 16px;
780+
margin: 16px 24px;
781+
background: var(--color-bg-subtle);
782+
border: 1px solid var(--color-border);
783+
border-radius: 10px;
784+
font-size: 13px;
785+
color: var(--color-text-secondary);
786+
animation: fadeInUp 0.3s ease-out;
787+
}
788+
789+
@keyframes fadeInUp {
790+
from { opacity: 0; transform: translateY(8px); }
791+
to { opacity: 1; transform: translateY(0); }
792+
}
793+
794+
.session-feedback-label {
795+
color: var(--color-text-secondary);
796+
}
797+
798+
.session-feedback-buttons {
799+
display: flex;
800+
gap: 6px;
801+
}
802+
803+
.session-feedback-btn {
804+
background: var(--color-bg-card);
805+
border: 1px solid var(--color-border);
806+
border-radius: 6px;
807+
padding: 6px 14px;
808+
cursor: pointer;
809+
font-size: 13px;
810+
color: var(--color-text-secondary);
811+
transition: all 0.2s;
812+
}
813+
814+
.session-feedback-btn:hover {
815+
border-color: var(--color-brand);
816+
background: var(--color-bg-subtle);
817+
}
818+
819+
.session-feedback-btn.positive:hover {
820+
background: var(--color-success-500);
821+
border-color: var(--color-success-500);
822+
color: white;
823+
}
824+
825+
.session-feedback-btn.negative:hover {
826+
background: var(--color-error-500);
827+
border-color: var(--color-error-500);
828+
color: white;
829+
}
830+
831+
.session-feedback-dismiss {
832+
background: none;
833+
border: none;
834+
color: var(--color-text-muted);
835+
cursor: pointer;
836+
font-size: 16px;
837+
padding: 0 4px;
838+
line-height: 1;
839+
}
840+
841+
.session-feedback-dismiss:hover {
842+
color: var(--color-text-secondary);
843+
}
844+
845+
.session-feedback--success {
846+
justify-content: center;
847+
color: var(--color-success-600);
848+
border-color: var(--color-success-500);
849+
background: var(--color-success-50, var(--color-bg-subtle));
850+
}
851+
773852
/* SI Agent CTA Buttons */
774853
.si-cta-container {
775854
display: flex;
@@ -2423,7 +2502,7 @@ <h2>Hi! I'm Addie</h2>
24232502

24242503
// Clear only chat messages from the container (preserves static elements like home/welcome)
24252504
function clearChatMessages() {
2426-
const messages = messagesContainer.querySelectorAll('.message');
2505+
const messages = messagesContainer.querySelectorAll('.message, .session-feedback');
24272506
messages.forEach(msg => msg.remove());
24282507
}
24292508

@@ -2633,6 +2712,8 @@ <h2>Hi! I'm Addie</h2>
26332712
currentTabId = 'home';
26342713
conversationId = null;
26352714
currentChannel = 'web';
2715+
lastAssistantMessageId = null;
2716+
clearSessionFeedbackTimer();
26362717

26372718
// Update UI
26382719
homeTab.classList.add('active');
@@ -2765,9 +2846,14 @@ <h2>Hi! I'm Addie</h2>
27652846

27662847
const data = await response.json();
27672848

2768-
// Render messages
2849+
// Render messages and track last assistant message
2850+
lastAssistantMessageId = null;
2851+
clearSessionFeedbackTimer();
27692852
data.messages.forEach(msg => {
27702853
addMessage(msg.content, msg.role, msg.message_id);
2854+
if (msg.role === 'assistant' && msg.message_id) {
2855+
lastAssistantMessageId = msg.message_id;
2856+
}
27712857
});
27722858

27732859
// Handle read-only mode for Slack threads
@@ -2805,6 +2891,8 @@ <h2>Hi! I'm Addie</h2>
28052891
conversationId = null;
28062892
currentChannel = 'web';
28072893
currentTabId = 'home'; // New chat starts from home
2894+
lastAssistantMessageId = null;
2895+
clearSessionFeedbackTimer();
28082896

28092897
// Clear messages and show home
28102898
clearChatMessages();
@@ -3320,6 +3408,80 @@ <h2>Hi! I'm Addie</h2>
33203408
}
33213409
}
33223410

3411+
// === Session-level feedback ===
3412+
let sessionFeedbackTimer = null;
3413+
let lastAssistantMessageId = null;
3414+
const sessionFeedbackShown = new Set(); // Track per conversation
3415+
3416+
function scheduleSessionFeedback() {
3417+
clearSessionFeedbackTimer();
3418+
if (!conversationId || sessionFeedbackShown.has(conversationId)) return;
3419+
if (!lastAssistantMessageId) return;
3420+
3421+
// Count assistant messages in current view
3422+
const assistantMessages = messagesContainer.querySelectorAll('.message--assistant');
3423+
if (assistantMessages.length < 3) return;
3424+
3425+
sessionFeedbackTimer = setTimeout(() => {
3426+
showSessionFeedback();
3427+
}, 45000); // 45 seconds of idle
3428+
}
3429+
3430+
function clearSessionFeedbackTimer() {
3431+
if (sessionFeedbackTimer) {
3432+
clearTimeout(sessionFeedbackTimer);
3433+
sessionFeedbackTimer = null;
3434+
}
3435+
}
3436+
3437+
function showSessionFeedback() {
3438+
if (!conversationId || sessionFeedbackShown.has(conversationId)) return;
3439+
sessionFeedbackShown.add(conversationId);
3440+
3441+
const prompt = document.createElement('div');
3442+
prompt.className = 'session-feedback';
3443+
prompt.innerHTML = `
3444+
<span class="session-feedback-label">How was this conversation?</span>
3445+
<div class="session-feedback-buttons">
3446+
<button class="session-feedback-btn positive" data-rating="5">&#128077; Helpful</button>
3447+
<button class="session-feedback-btn negative" data-rating="1">&#128078; Not helpful</button>
3448+
</div>
3449+
<button class="session-feedback-dismiss" title="Dismiss">&times;</button>
3450+
`;
3451+
3452+
prompt.querySelector('.session-feedback-dismiss').addEventListener('click', () => {
3453+
prompt.remove();
3454+
});
3455+
3456+
const feedbackMessageId = lastAssistantMessageId;
3457+
prompt.querySelectorAll('.session-feedback-btn').forEach(btn => {
3458+
btn.addEventListener('click', async () => {
3459+
const rating = parseInt(btn.dataset.rating);
3460+
try {
3461+
await authFetch(`/api/addie/chat/${conversationId}/feedback`, {
3462+
method: 'POST',
3463+
headers: { 'Content-Type': 'application/json' },
3464+
body: JSON.stringify({
3465+
message_id: feedbackMessageId,
3466+
rating: rating,
3467+
rating_category: 'session',
3468+
feedback_tags: null,
3469+
improvement_suggestion: null
3470+
})
3471+
});
3472+
} catch (e) {
3473+
console.error('Session feedback failed:', e);
3474+
}
3475+
prompt.innerHTML = '<span>Thanks for the feedback!</span>';
3476+
prompt.className = 'session-feedback session-feedback--success';
3477+
setTimeout(() => prompt.remove(), 3000);
3478+
});
3479+
});
3480+
3481+
messagesContainer.appendChild(prompt);
3482+
messagesContainer.scrollTop = messagesContainer.scrollHeight;
3483+
}
3484+
33233485
// Add typing indicator
33243486
function addTypingIndicator() {
33253487
const messageDiv = document.createElement('div');
@@ -3580,6 +3742,11 @@ <h2>Hi! I'm Addie</h2>
35803742

35813743
isLoading = true;
35823744
updateSendButton();
3745+
clearSessionFeedbackTimer();
3746+
3747+
// Remove any visible session feedback prompt
3748+
const existingPrompt = messagesContainer.querySelector('.session-feedback');
3749+
if (existingPrompt) existingPrompt.remove();
35833750

35843751
// Request notification permission on first message
35853752
requestNotificationPermission();
@@ -3705,6 +3872,7 @@ <h2>Hi! I'm Addie</h2>
37053872

37063873
// Finalize the message with feedback UI
37073874
finalizeStreamingMessage(messageDiv, contentDiv, fullContent, messageId);
3875+
if (messageId) lastAssistantMessageId = messageId;
37083876

37093877
// Open SI modal if a brand agent session was created
37103878
// This must happen AFTER the stream is completely done to avoid race conditions
@@ -3743,6 +3911,9 @@ <h2>Hi! I'm Addie</h2>
37433911
}
37443912
}
37453913

3914+
// Schedule session-level feedback prompt after idle
3915+
scheduleSessionFeedback();
3916+
37463917
// Refresh thread list to show new/updated conversation (debounced)
37473918
scheduleLoadThreads();
37483919

@@ -3768,6 +3939,7 @@ <h2>Hi! I'm Addie</h2>
37683939
chatInput.addEventListener('input', () => {
37693940
autoResize();
37703941
updateSendButton();
3942+
clearSessionFeedbackTimer(); // Reset idle timer when user types
37713943
});
37723944

37733945
chatInput.addEventListener('keydown', (e) => {

‎server/src/db/migrations/215_addie_thread_review_improvements.sql‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
-- Addie improvements from Feb 2026 thread review
2+
-- Addresses: speculative feature claims, real brand names in examples, ads.txt accuracy, verbose deflections
3+
4+
-- 1. Don't speculate about unimplemented protocol features
5+
INSERT INTO addie_rules (rule_type, name, description, content, priority, created_by) VALUES
6+
(
7+
'constraint',
8+
'Current Spec Only',
9+
'Distinguish between what AdCP currently specifies vs aspirational features',
10+
'When discussing AdCP capabilities, only describe features that exist in the current specification. Do NOT present aspirational or future features as current reality.
11+
12+
Specific examples:
13+
- AdCP does NOT currently have cryptographic verification, ads.cert integration, or blockchain-based trust
14+
- AdCP does NOT have "agent reputation networks" or formal trust scoring between agents
15+
- adagents.json is a discovery mechanism, not a cryptographic chain of trust
16+
17+
When discussing what AdCP COULD support in the future, clearly mark it as aspirational:
18+
- "This isn''t part of AdCP today, but the architecture could support..."
19+
- "The community is exploring..."
20+
21+
The protocol is young. Accurately representing its current state builds more credibility than overclaiming.',
22+
215,
23+
'system'
24+
);
25+
26+
-- 2. Use fictional names in examples
27+
INSERT INTO addie_rules (rule_type, name, description, content, priority, created_by) VALUES
28+
(
29+
'constraint',
30+
'Fictional Names in Examples',
31+
'Use fictional company names when creating illustrative examples',
32+
'When creating hypothetical examples or scenarios, use fictional company names instead of real brands, agencies, or publishers.
33+
34+
Use names like: Acme Corp, Pinnacle Media, Nova Brands, Summit Publishing, Apex Athletic, Horizon DSP, etc.
35+
36+
Exceptions:
37+
- When a user asks specifically about a real company (e.g., "what do we have for Fanta in the registry?")
38+
- When referencing industry players in factual context (e.g., "The Trade Desk supports UID2")
39+
- When discussing AdCP member organizations by name
40+
- Enum values that reference industry standards (e.g., "groupm" viewability standard)
41+
42+
The rule applies to INVENTED scenarios and examples, not factual references.',
43+
112,
44+
'system'
45+
);
46+
47+
-- 3. Accurate ads.txt/sellers.json knowledge
48+
INSERT INTO addie_rules (rule_type, name, description, content, priority, created_by) VALUES
49+
(
50+
'knowledge',
51+
'Ads.txt and Sellers.json Accuracy',
52+
'Correct understanding of supply chain authorization mechanisms',
53+
'When discussing ads.txt and sellers.json, be precise about how they work:
54+
55+
ads.txt:
56+
- Published at domain.com/ads.txt by publishers
57+
- Lists authorized seller account IDs and relationship type (DIRECT or RESELLER)
58+
- DSPs check ads.txt BEFORE bidding (pre-bid), not post-facto
59+
- Verification is cached/scraped periodically, not checked per-impression
60+
61+
sellers.json:
62+
- Published by SSPs/exchanges at their domain
63+
- Maps seller_id to business entity (name, domain, seller_type)
64+
- seller_type: PUBLISHER, INTERMEDIARY, or BOTH
65+
- Enables supply chain object (schain) validation
66+
67+
Supply chain object (schain):
68+
- Passed in bid requests per OpenRTB
69+
- Lists each node in the supply path
70+
- Buyers verify the complete chain against ads.txt + sellers.json
71+
72+
Common issues to understand:
73+
- DIRECT means the publisher has a direct business relationship with the advertising system
74+
- RESELLER means the publisher has authorized another entity to sell on their behalf
75+
- A seller claiming DIRECT when the relationship is through an intermediary is a misrepresentation',
76+
162,
77+
'system'
78+
);
79+
80+
-- 4. Shorten off-topic deflection template
81+
UPDATE addie_rules SET content = 'CRITICAL: You are an ad tech expert, NOT a general assistant. Your knowledge domain is:
82+
83+
TOPICS YOU KNOW ABOUT:
84+
- AdCP (Ad Context Protocol) and agentic advertising
85+
- AgenticAdvertising.org community, working groups, membership
86+
- Ad tech industry: programmatic, RTB, SSPs, DSPs, ad servers, Prebid, header bidding
87+
- AI and agents in advertising contexts
88+
- Industry players in factual context
89+
- Sustainability in advertising (GMSF, carbon impact)
90+
- Privacy and identity in advertising
91+
- Publisher monetization and buyer/seller dynamics
92+
93+
TOPICS OUTSIDE YOUR DOMAIN:
94+
- General news, sports, entertainment, weather
95+
- Topics unrelated to advertising, marketing, or media
96+
- General technology not related to ad tech or AI agents
97+
- Personal advice, health, legal matters
98+
- Questions about your own implementation or source code
99+
100+
When asked about off-topic subjects, keep the deflection SHORT (1-2 sentences max):
101+
"I specialize in ad tech and agentic advertising — that''s outside my area. Happy to help with anything AdCP or advertising related though!"
102+
103+
Do NOT list out everything you can help with when deflecting. Just redirect briefly and let the user ask.
104+
105+
When asked "what''s the latest news" or similar, interpret as ad tech news and use tools to search for recent updates.',
106+
version = version + 1,
107+
updated_at = NOW()
108+
WHERE name = 'Domain Focus - CRITICAL' AND rule_type = 'constraint';

‎server/src/db/migrations/216_addie_agent_types.sql‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
-- Keep Addie current on formal AdCP agent types
2+
-- These change over time and her training data lags
3+
4+
INSERT INTO addie_rules (rule_type, name, description, content, priority, created_by) VALUES
5+
(
6+
'knowledge',
7+
'AdCP Agent Types',
8+
'Current list of formal AdCP agent types',
9+
'The formal AdCP agent types are:
10+
- **Sales** — publisher-side inventory discovery and media buying
11+
- **Creative** — creative asset generation, format listing, preview rendering
12+
- **Signals** — audience signals discovery and activation
13+
- **Governance** — property lists (where ads run) and content standards (brand suitability)
14+
- **SI (Sponsored Intelligence)** — commerce-oriented sponsored placements
15+
16+
All of these are first-class agent types with their own tools and documentation in docs/. Use search_docs to look up details rather than answering from memory, especially for newer agent types.
17+
18+
Do NOT describe any of these as "not formally defined" or "conceptual" — they are all part of the current AdCP specification.',
19+
170,
20+
'system'
21+
);

0 commit comments

Comments
 (0)