feedback: thumb up/down on logs + reviewer self-score 0-10
Two unrelated features bundled because they ship together and share
migration 0028.
Strategic-log feedback (thumb up/down):
- New strategic_log_feedback table with UNIQUE(log_id, user_id) so each
user has one vote per log, flippable in place (up -> down -> clear).
UI shows aggregate counts only.
- app/services/log_feedback.py: set_vote, get_counts, sign/verify
feedback tokens (same itsdangerous pattern as auth.sign_pending,
30-day TTL for email links).
- POST /api/log/{id}/feedback: web vote, auth required, returns counts
+ the requesting user's own vote.
- GET /feedback?token=...&vote=...: email-link target, no auth, signed
token encodes (user, log, vote), renders feedback_thanks.html.
- partials/log.html: thumbs row below content, JS-driven swap via the
POST endpoint. Dashboard latest-log card and /log page both render
this partial via htmx, so the buttons appear in all three surfaces.
- digest emails: a "How was today's read?" row above the unsub footer,
with signed-token URLs against the latest StrategicLog at send time.
Plain-text fallback included.
Reviewer self-score (0-10):
- _SYSTEM_PROMPT asks for an integer score with anchors (10 exemplary,
5 borderline, 0 unfit). Verdict gains score: int | None.
- Deterministic-layer hits get score=0 (hard rule, no nuance);
error rows get None; LLM rows get the model's score clamped 0..10.
- ReviewerVerdict.score, StrategicLog.reviewer_score, and
IndicatorSummary.reviewer_score all new SMALLINT NULL columns.
- ai_log_job + indicator_summary_job persist verdict.score onto their
content rows when committing the row alongside content.
Tests:
- tests/test_strategic_log_feedback.py: vote, flip, clear, aggregate
across users, invalid vote, token round-trip + tamper + garbage +
'clear' not signable for email path.
- tests/test_output_review.py: score parsing, clamping (>10, <0),
missing/non-numeric -> None, deterministic-layer score=0.
Full suite: 427 passed (was 412), 5 skipped, no regressions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
f3ac65f8f7
commit
8946dee2e0
14 changed files with 962 additions and 14 deletions
|
|
@ -118,6 +118,10 @@ class StrategicLog(Base):
|
|||
prompt_tokens: Mapped[int | None] = mapped_column(Integer)
|
||||
completion_tokens: Mapped[int | None] = mapped_column(Integer)
|
||||
cost_usd: Mapped[float | None] = mapped_column(Float)
|
||||
# Reviewer self-rating 0-10 (10 = exemplary, 0 = unfit). Nullable for
|
||||
# rows generated before the score field existed; new rows always
|
||||
# carry the value the reviewer returned alongside its clean verdict.
|
||||
reviewer_score: Mapped[int | None] = mapped_column(SmallInteger)
|
||||
|
||||
|
||||
class StrategicLogTranslation(Base):
|
||||
|
|
@ -170,6 +174,8 @@ class IndicatorSummary(Base):
|
|||
prompt_tokens: Mapped[int | None] = mapped_column(Integer)
|
||||
completion_tokens: Mapped[int | None] = mapped_column(Integer)
|
||||
cost_usd: Mapped[float | None] = mapped_column(Float)
|
||||
# Reviewer self-rating 0-10. See StrategicLog.reviewer_score.
|
||||
reviewer_score: Mapped[int | None] = mapped_column(SmallInteger)
|
||||
|
||||
__table_args__ = (Index("ix_indsumm_group_generated", "group_name", "generated_at"),)
|
||||
|
||||
|
|
@ -244,6 +250,45 @@ class ReviewerVerdict(Base):
|
|||
layer: Mapped[str] = mapped_column(String(16), nullable=False)
|
||||
# LLM-layer model id, nullable for deterministic / error rows.
|
||||
model: Mapped[str | None] = mapped_column(String(64))
|
||||
# Reviewer self-rating 0-10. Deterministic-layer hits get 0 (hard
|
||||
# reject by rule), error rows get NULL, LLM rows get the model's score.
|
||||
score: Mapped[int | None] = mapped_column(SmallInteger)
|
||||
|
||||
|
||||
class StrategicLogFeedback(Base):
|
||||
"""Anonymous-in-UI thumb up/down votes on strategic-log rows.
|
||||
|
||||
One row per (log_id, user_id) — flippable: a user can change their
|
||||
vote (up → down) by overwriting, or clear it by deleting. The UI
|
||||
surfaces only aggregate counts; user attribution is server-side
|
||||
only and exists purely so we can dedup and let the voter see/flip
|
||||
their own vote. See app/services/log_feedback.py."""
|
||||
__tablename__ = "strategic_log_feedback"
|
||||
id: Mapped[int] = mapped_column(_PK, primary_key=True, autoincrement=True)
|
||||
log_id: Mapped[int] = mapped_column(
|
||||
BigInteger().with_variant(Integer(), "sqlite"),
|
||||
ForeignKey("strategic_logs.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
user_id: Mapped[int] = mapped_column(
|
||||
Integer,
|
||||
ForeignKey("users.id", ondelete="CASCADE"),
|
||||
nullable=False,
|
||||
)
|
||||
# 'up' or 'down'. Service layer enforces the enum.
|
||||
vote: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, default=utcnow,
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, default=utcnow,
|
||||
onupdate=utcnow,
|
||||
)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint("log_id", "user_id", name="uq_slf_log_user"),
|
||||
Index("ix_strategic_log_feedback_log", "log_id"),
|
||||
)
|
||||
|
||||
|
||||
class UserAcknowledgement(Base):
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue