feat(notch): show live voice and agent activity in macOS notch#3166
Draft
M3gA-Mind wants to merge 3 commits into
Draft
feat(notch): show live voice and agent activity in macOS notch#3166M3gA-Mind wants to merge 3 commits into
M3gA-Mind wants to merge 3 commits into
Conversation
…onnectors and settings
Production fix: Conversations.tsx sidebar now opens by default (useState(true))
— was causing all chat-harness and user-journey specs to fail with "Conversations
panel did not mount".
Test fixes by root cause:
Chat / harness (sidebar fix):
- Conversations.tsx: showSidebar initialises to true so the Threads heading is
visible on mount; was false, breaking every chat-harness spec.
Onboarding:
- onboarding-modes.spec.ts: walk all 5 custom wizard steps (inference, voice,
oauth, search, embeddings); guard RuntimeChoice block against local-E2E-session
redirect (VITE_OPENHUMAN_E2E_DEFAULT_CORE_MODE=local bypasses that step).
- runtime-picker-login.spec.ts: welcome text is t('welcome.title')
= "Welcome to OpenHuman" (login page), not the onboarding step greeting.
Intelligence / Skills:
- insights-dashboard.spec.ts: click the Memory tab before asserting
memory-workspace testid (default tab is tasks, not memory).
- skill-lifecycle.spec.ts: Skills page title/tabs changed to
Connections / Composio / MCP Servers.
Composio connectors (15 specs):
- composio-helpers.ts: openConnectorModal waits for tile status text before
clicking so connection data is loaded and modal opens in the correct phase;
assertModalPhase expired markers extended with "Reconnect to re-enable".
- telegram.ts: normalise snake_case has_credentials → hasCredentials.
- All connector specs: pass 'Auth expired' to openConnectorModal for the
expired-auth test so the modal opens in expired phase, not idle.
- composio-triggers-flow.spec.ts: formatTriggerLabel strips the leading toolkit
prefix → "New Gmail Message" not "Gmail New Gmail Message".
- accounts-provider-modal.spec.ts: dispatch Escape on document, not window
(useEscapeKey listens on document).
- telegram-channel-flow.spec.ts: remove invalid C.4 disconnection status check
after a failed connect attempt when a prior connection is live.
Telegram / mock helpers:
- mock-server.ts: getTelegramSentMessages unwraps { ok, messages: [...] }
→ array so sent.find() works.
- harness-cron-prompt-flow.spec.ts: use cron_add (not cron_create) with
{ kind:'cron', expr } schedule object; drop unsupported enabled param.
- harness-search/channel-bridge harness: sidebar fix resolves all cascades.
Settings:
- settings-channels-permissions.spec.ts, settings-feature-preferences.spec.ts:
Default Messaging Channel moved to /skills?tab=channels (removed
/settings/messaging route).
- settings-advanced-config.spec.ts: developer menu entry renamed from
"Composio Routing (Direct Mode)" → "Composio".
Adds a transparent, click-through NSPanel anchored to the top-centre of the primary screen that displays real-time status as an animated pill expanding from the physical notch (or as a top-centre HUD on older Macs). Implementation: - app/src-tauri/src/notch_window.rs — native NSPanel + WKWebView at NSStatusWindowLevel (25), above the menu bar, fully click-through (ignoresMouseEvents). Polls OPENHUMAN_CORE_RPC_URL (set by CoreProcessHandle when core is ready) and injects the Socket.IO base URL into the webview via evaluateJavaScript. - app/src-tauri/src/lib.rs — auto-shows notch panel at startup (macOS) alongside notch_window_show / notch_window_hide Tauri commands. - app/src/notch/NotchApp.tsx — React pill component that connects to the core Socket.IO and handles: dictation:toggle (waveform bars + Listening), dictation:transcription (spinner + text), companion:state_changed (thinking / speaking), overlay:attention (broadcast messages). - app/src/main.tsx — routes ?window=notch to NotchApp, skips Tauri IPC bootstrap (same pattern as mascot window). - app/src/index.css — three CSS keyframe animations (notch-pill-in, notch-bar, notch-dot) + transparent background rule for the notch window. - i18n — adds notch.listening / thinking / speaking / transcribing / executing keys to all 14 locale files with real translations.
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NSPanel+WKWebViewthat floats at the top-centre of the primary screen above the menu bar, visually anchored to the physical notch on MacBook Pros (or a top-centre floating HUD on older hardware)ignoresMouseEvents = true) — menu bar items remain clickable through the transparent regionsHow it works
dictation:togglepresseddictation:transcriptioncompanion:state_changedthinkingcompanion:state_changedspeakingoverlay:attentionThe pill animates in with a spring curve (
cubic-bezier(0.34, 1.56, 0.64, 1)) and auto-dismisses 1.8 s after the action ends.Architecture
The notch panel follows the same NSPanel + WKWebView pattern as the floating mascot (
mascot_native_window.rs) to work around the CEF transparency limitation. The WKWebView has no Tauri IPC bridge, soCoreProcessHandlesetsOPENHUMAN_CORE_RPC_URLin the process env when the embedded server is ready; a 1 Hz Foundation timer polls for this and injects the Socket.IO base URL into the webview viaevaluateJavaScriptonce. The React component (NotchApp.tsx) picks up the URL and connects to the core over Socket.IO — identical toOverlayApp.Files changed
app/src-tauri/src/notch_window.rsNSStatusWindowLevel(25), top-centre,ignoresMouseEvents, core URL injection timerapp/src-tauri/src/lib.rsmod notch_window,notch_window_show/hideTauri commands, auto-show on startupapp/src/notch/NotchApp.tsxapp/src/main.tsx?window=notchrouting, skip Tauri bootstrap (mirrors mascot pattern)app/src/index.cssnotch-pill-in,notch-bar,notch-dot) + transparent background ruleapp/src/lib/i18n/*.tsnotch.listening/thinking/speaking/transcribing/executing) in all 14 locale files with real translationsTest plan
pnpm dev:appon a Mac with a notch — confirm no pill visible at idlepnpm typecheck,pnpm format:check,pnpm i18n:check,cargo checkall pass