Skip to content

Bias Audit Blocked: Need GPU Access for Florence-2 Inference (issue #531 continuation) #555

Description

@Mosss-OS

The Problem

The Fitzpatrick skin tone bias audit (issue #531) is blocked. All datasets and code are ready, but no GPU is available to run Florence-2 inference:

  • Local machine: no CUDA GPU
  • Kaggle API: cannot push kernels (403 Forbidden on SaveKernel)
  • Kaggle web UI: accelerators deactivated on this account

What We Have

Asset Location
Florence-2-base model Kaggle Dataset: mosesos/florence-2-base
SCIN image shards 0-4 Kaggle Dataset: mosesos/trij-scin-data
MSKCC images + metadata Kaggle Dataset: mosesos/trij-mskcc-data
Offline inference notebook docs/bias-audit/scripts/run_on_kaggle.ipynb
Audit pipeline script docs/bias-audit/scripts/run_comprehensive_audit.py

Resolution Paths (priority order)

Option A: Google Colab Pay As You Go (~$10)

  • Subscribe to Colab Pro/Pay As You Go (~$10/month)
  • Upload run_on_kaggle.ipynb (or adapt for Colab)
  • Runtime → Change runtime type → T4 GPU
  • Run inference on the 599 MSKCC images
  • Expected runtime: ~2-4 hours
  • Download inference_results.csv → run run_comprehensive_audit.py --csv /path/to/inference_results.csv
  • Pro: Cheapest, fastest path
  • Con: Need to re-download datasets to Colab

Option B: Borrow a GPU Machine

  • Any CUDA-capable machine
  • Check out repo, run run_on_kaggle.ipynb locally
  • Pro: Full control
  • Con: Need to find hardware

Option C: Kaggle API Fix

  • Try switching API token to owner-level token or different Kaggle account with GPU access
  • Re-attempt kernel push
  • Pro: No cost
  • Con: May not work — 403 may be account-level, not token-level

What Happens Next

  1. Get GPU access → run Florence-2 on all 599 MSKCC images
  2. Run run_comprehensive_audit.py on results
  3. Generate performance_gap_report.md
  4. Update issue Bias Audit: Run Real Inference on GPU & Package Results #531 with real (not simulated) bias metrics
  5. Use report in Horizon 1000 outreach as proof of equity readiness

Blockers

  • GPU access (Colab Pro ~$10 or borrowed GPU)
  • Cost approval if using paid Colab

References

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions