A FastAPI app + plain HTML5 video page that replaces the matplotlib
picker. Browse to http://host:8000/, scrub through each video with
arrow keys (±5 s, ±1 s with Shift, ±0.1 s with Ctrl, ±1 frame with
,/.), and click one of three buttons:
- All barriers open — every ROI usable
- Upper barrier opens — ROIs 1,3,5 usable; lower row marked bad
- Lower barrier opens — ROIs 2,4,6 usable; upper row marked bad
The current playhead time is recorded as opening_s; bad_rois is set
accordingly. Also keyboard shortcuts (1/2/3 for the three modes,
s/u for skip/unusable). Refresh-safe: every submission persists to
data/metadata/barrier_opening.csv before advancing.
Server uses byte-range streaming so seeking inside long videos is
fast. Dockerfile + docker-compose.yml mount the data volume RO and
the metadata folder RW.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
pick_barrier.py loops over every tracked DB referenced by the merged
TSV, plots windowed mean inter-fly distance for all 6 ROIs in a single
figure, and lets the analyst click the moment the barrier opens. Saves
to data/metadata/barrier_opening.csv after each pick (crash-safe).
Auto-detector best-effort guess shown as orange dotted line — the
analyst always has the final say.
Output schema:
machine_name, session_date, session_time, opening_s, trim_first_s, notes
`trim_first_s` lets us record misframed starts so downstream code can
ignore the affected window. The 5 2025-07-15 entries are seeded from
the original legacy CSV so they're not re-picked.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Both videos have ~60-69 s of misframed data at the start (arena partially
out of frame). The original times (25 s, 20 s) were measured on
ffmpeg-trimmed copies that no longer exist; on the full untrimmed videos
the actual barriers open at 94 s and 89 s respectively. Confirmed by eye.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Tracked DBs now live at /mnt/data/projects/cupido/tracked/ (out of
ownCloud to avoid sync conflicts and bandwidth churn). config.py
TRACKING_OUTPUT_DIR points there; the docker-compose for ethoscope-lab
mounts it world-readable for JupyterHub users.
- New scripts/export_video_db_index.py joins all_video_info_merged.xlsx
with the video inventory and the on-disk DBs, producing a TSV that has
one row per fly/ROI plus training/testing video and DB paths. Handles
approximate xlsx times, cross-day training/testing, the 12 AM/PM
ambiguity, and date typos.
- scripts/load_roi_data.py rewritten as a TSV-driven loader returning a
single DataFrame with session and metadata columns. calculate_distances
and the two flies_analysis notebooks migrated to use it; downstream
trained/naive splits remain available via simple equality filters.
- Metadata vocabulary canonicalized: {naïve, niave, untrained, test} all
resolve to {trained, naive}. Normalization happens at the TSV-export
boundary (idempotent); the xlsx and the 2025-07-15 legacy CSV were
edited in place to remove the worst variants.
- scripts/monitor_tracking.py rate calculation fixed: with N parallel
workers, completions arrive in bursts; the old formula divided by burst
width and reported nonsense rates. Now uses a 6 h window denominator.
- scripts/track_videos.py: BGRMovieCamera retries cv2.read on transient
NFS hiccups and a post-tracking completeness gate (≥ 90 % of expected
duration via MAX(t) across all 6 ROIs) deletes silent partial DBs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reorganized flat 41-file directory into structured layout with:
- scripts/ for Python analysis code with shared config.py
- notebooks/ for Jupyter analysis notebooks
- data/ split into raw/, metadata/, processed/
- docs/ with analysis summary, experimental design, and bimodal hypothesis tutorial
- tasks/ with todo checklist and lessons learned
- Comprehensive README, PLANNING.md, and .gitignore
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
