fix: support PyannoteAudioPretrainedSpeakerEmbedding in speaker mapping - #644
Open
wkochFPV wants to merge 2 commits into
Open
fix: support PyannoteAudioPretrainedSpeakerEmbedding in speaker mapping#644wkochFPV wants to merge 2 commits into
wkochFPV wants to merge 2 commits into
Conversation
The pyannote/speaker-diarization-community-1 model uses PyannoteAudioPretrainedSpeakerEmbedding as its embedding backend, which has no .eval() method and expects a 3D tensor [batch, channels, samples] instead of an audio dict. This caused _map_to_known_speakers to always fail with: AttributeError: 'PyannoteAudioPretrainedSpeakerEmbedding' has no attribute 'eval' ValueError: shapes (1,256) and (1,256) not aligned (missing .flatten()) Fix: - Add _to_3d() helper to normalize tensor dimensions - Add _embed() with fallback: try Inference() first, then direct call with 3D tensor - Add _embed_crop() with same fallback for per-turn crops - Always .flatten() the result to ensure 1D vector for cosine similarity
…dd speaker-count controls routers/diarization.py - Fix: call embedding model as callable instead of via Inference(...) to prevent crashes - Fix: properly resample reference and segment audio to 16 kHz; skip segments that are too short or produce NaN - Add `known_speaker_threshold` parameter (cosine similarity) — speakers below the threshold keep their anonymous SPEAKER_XX label instead of being mapped to a known name (default: 0.5) - Log cosine similarity scores per speaker for empirical threshold calibration - Add optional `num_speakers`, `min_speakers`, `max_speakers` fields to control speaker count and reduce speaker merging utils.py - `parse_data_url_to_audio` now returns `(data, sample_rate)` — real rate for WAV/FLAC, 16000 for raw PCM executors/pyannote_diarization.py - Allow clustering threshold to be overridden globally via `DIARIZATION_CLUSTERING_THRESHOLD` env var; supports both pyannote 3.x nested (`clustering.threshold`) and community-1 flat (`clustering_threshold`) parameter structures
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Known_speaker_names did not work for pyannote/speaker-diarization-community-1 using the new pyannote diarization.
Disclaimer: This is a fix that was entirely created using claude.ai Sonnet. It works perfectly for me, that is why I am sharing it. I did not personally review the changes, but performed a lot of successful tests with it.
The pyannote/speaker-diarization-community-1 model uses PyannoteAudioPretrainedSpeakerEmbedding as its embedding backend, which has no .eval() method and expects a 3D tensor [batch, channels, samples] instead of an audio dict.
This caused _map_to_known_speakers to always fail with:
AttributeError: 'PyannoteAudioPretrainedSpeakerEmbedding' has no attribute 'eval'
ValueError: shapes (1,256) and (1,256) not aligned (missing .flatten())
Fix: