Skip to content

Commit e9693d9

Browse files
author
Alex J Lennon
committed
Add Android app (Option A): Kotlin + Compose, Gemini Live, OpenClaw, CI
- samples/CameraAccessAndroid: Gradle 8.5, Kotlin 1.9, Compose, Meta DAT (core+camera), OkHttp, CameraX - Gemini: GeminiConfig, GeminiLiveService (WebSocket), same protocol as iOS - OpenClaw: OpenClawBridge (checkConnection, delegateTask), SettingsProvider - UI: Home, Settings, Stream placeholder; NavGraph; theme - CI: android-build.yml (JDK 17, setup-gradle, assembleDebug, APK artifact; GH_PACKAGES_READ_TOKEN for Meta DAT) - docs/ANDROID_BUILD_PLAN.md, README Android section, .gitignore Android entries
1 parent e6831a2 commit e9693d9

29 files changed

Lines changed: 1280 additions & 0 deletions

‎.github/workflows/android-build.yml‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
name: Android Build & Validation
2+
3+
on:
4+
push:
5+
branches:
6+
- "**"
7+
pull_request:
8+
branches:
9+
- main
10+
11+
env:
12+
ANDROID_DIR: samples/CameraAccessAndroid
13+
14+
jobs:
15+
build:
16+
name: Build & Validate Android App
17+
runs-on: ubuntu-latest
18+
19+
steps:
20+
- name: Checkout Repository
21+
uses: actions/checkout@v4
22+
23+
- name: Set up JDK 17
24+
uses: actions/setup-java@v4
25+
with:
26+
distribution: "temurin"
27+
java-version: "17"
28+
29+
- name: Setup Gradle
30+
uses: gradle/actions/setup-gradle@v4
31+
with:
32+
gradle-version: "8.5"
33+
cache-read-only: false
34+
35+
- name: Create local.properties with GitHub token
36+
run: |
37+
# Meta DAT is on GitHub Packages (facebook/meta-wearables-dat-android). Default GITHUB_TOKEN
38+
# cannot read other orgs' packages; use GH_PACKAGES_READ_TOKEN secret (PAT with read:packages) if set.
39+
TOKEN="${{ secrets.GH_PACKAGES_READ_TOKEN || secrets.GITHUB_TOKEN }}"
40+
echo "github_token=$TOKEN" >> ${{ env.ANDROID_DIR }}/local.properties
41+
echo "Created local.properties for Meta DAT dependency resolution"
42+
43+
- name: Build Debug APK
44+
run: |
45+
cd ${{ env.ANDROID_DIR }}
46+
gradle assembleDebug --no-daemon --stacktrace
47+
env:
48+
GITHUB_TOKEN: ${{ secrets.GH_PACKAGES_READ_TOKEN || secrets.GITHUB_TOKEN }}
49+
50+
- name: Upload APK artifact
51+
uses: actions/upload-artifact@v4
52+
with:
53+
name: VisionClaw-Android-${{ github.sha }}
54+
path: ${{ env.ANDROID_DIR }}/app/build/outputs/apk/debug/app-debug.apk
55+
retention-days: 30
56+
57+
- name: Build Summary
58+
if: always()
59+
run: |
60+
echo "## Android Build Summary" >> $GITHUB_STEP_SUMMARY
61+
if [ -f "${{ env.ANDROID_DIR }}/app/build/outputs/apk/debug/app-debug.apk" ]; then
62+
echo "**Status**: SUCCESS" >> $GITHUB_STEP_SUMMARY
63+
echo "**APK**: VisionClaw-Android-${{ github.sha }}" >> $GITHUB_STEP_SUMMARY
64+
else
65+
echo "**Status**: FAILED" >> $GITHUB_STEP_SUMMARY
66+
fi

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,11 @@
1111
# Python
1212
__pycache__/
1313
*.pyc
14+
15+
# Android
16+
samples/CameraAccessAndroid/local.properties
17+
samples/CameraAccessAndroid/.gradle
18+
samples/CameraAccessAndroid/build
19+
samples/CameraAccessAndroid/app/build
20+
*.apk
21+
*.aab

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,10 @@ This will:
8080

8181
**See:** [docs/INSTALL_FROM_LINUX.md](docs/INSTALL_FROM_LINUX.md) for detailed guide.
8282

83+
### Android (Preview)
84+
85+
A native Android app (Kotlin + Jetpack Compose) is in **samples/CameraAccessAndroid**. It uses the same Gemini Live API and OpenClaw protocol. Build with Gradle (requires a GitHub token for the Meta Wearables DAT dependency). See [samples/CameraAccessAndroid/README.md](samples/CameraAccessAndroid/README.md) and [docs/ANDROID_BUILD_PLAN.md](docs/ANDROID_BUILD_PLAN.md).
86+
8387
---
8488

8589
### Development Setup (macOS Required)

‎docs/ANDROID_BUILD_PLAN.md‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
# Android Build Plan – VisionClaw
2+
3+
This document reviews the current iOS setup and outlines **options for creating an Android build** of VisionClaw, with effort, reuse, and trade-offs.
4+
5+
---
6+
7+
## 1. Current iOS Architecture (Summary)
8+
9+
| Layer | iOS implementation | Android equivalent |
10+
|-------|--------------------|--------------------|
11+
| **Glasses / camera** | Meta Wearables DAT (MWDATCore, MWDATCamera) – Swift | [Meta Wearables DAT Android](https://github.com/facebook/meta-wearables-dat-android) – Kotlin |
12+
| **AI / voice** | Gemini Live API over WebSocket | Same API (platform-agnostic) |
13+
| **Agentic actions** | OpenClaw gateway (HTTP + WebSocket) | Same (platform-agnostic) |
14+
| **Optional video call** | WebRTC (PiP, signaling) | WebRTC Android (e.g. webrtc.org) |
15+
| **UI** | SwiftUI | Jetpack Compose (or XML) |
16+
| **Phone camera fallback** | iPhone camera → frames to Gemini | CameraX / Camera2 → frames to Gemini |
17+
| **Settings / QR** | SwiftUI, AVFoundation QR | Compose, ML Kit / ZXing QR |
18+
19+
**Reusable as-is on Android**
20+
21+
- Gemini Live: same WebSocket URL, auth, JSON protocol.
22+
- OpenClaw: same host/port/tokens, same tool-call JSON.
23+
- Server (e.g. `samples/CameraAccess/server`): no change for Android.
24+
25+
**Platform-specific**
26+
27+
- Meta DAT (glasses streaming, registration, permissions): use [meta-wearables-dat-android](https://github.com/facebook/meta-wearables-dat-android) on Android.
28+
- UI, navigation, camera capture, audio: reimplement in Kotlin/Compose (or chosen stack).
29+
- Secrets/config: Android `BuildConfig` / `local.properties` / same env-based approach as CI.
30+
31+
---
32+
33+
## 2. Option A: Native Android (Kotlin + Jetpack Compose) – **Recommended**
34+
35+
**Idea:** Build an Android app that mirrors the iOS structure: same features (glasses + phone camera, Gemini Live, OpenClaw, optional WebRTC), using the official Meta Android DAT and the same backend contracts.
36+
37+
**Pros**
38+
39+
- Direct use of [Meta Wearables DAT Android](https://github.com/facebook/meta-wearables-dat-android) (video, audio, registration, permissions), aligned with current iOS DAT usage.
40+
- Same Gemini Live and OpenClaw integration (copy protocol and message shapes; reimplement only transport and threading in Kotlin).
41+
- No cross-platform framework lock-in; each platform can follow its own best practices and SDKs.
42+
- Fits existing CI style: add an `android-build.yml` (Gradle, APK/AAB), similar to `ios-build.yml`.
43+
- Clear path for “Android phone mode” (CameraX → send frames to Gemini) analogous to iPhone mode.
44+
45+
**Cons**
46+
47+
- Full reimplementation of UI and app flow in Kotlin/Compose (no code reuse from Swift).
48+
- Two codebases to maintain (iOS + Android); shared logic only in docs, API contracts, and server.
49+
50+
**Effort (rough)**
51+
52+
- **Core path (glasses + Gemini + OpenClaw):** ~3–6 weeks for one experienced Android dev (registration, streaming, Gemini WebSocket client, OpenClaw HTTP/WS, settings, QR).
53+
- **Phone camera mode + polish:** +1–2 weeks.
54+
- **WebRTC (if desired):** +1–2 weeks (signaling already exists; integrate webrtc.org on Android).
55+
56+
**Reuse**
57+
58+
- 100% of “backend” design: Gemini Live protocol, OpenClaw tool-call format, server.
59+
- 0% of UI/native code (by design).
60+
61+
---
62+
63+
## 3. Option B: Flutter (Dart)
64+
65+
**Idea:** One Flutter app for iOS and Android; use platform channels to call the native Meta DAT on each side (iOS Swift, Android Kotlin).
66+
67+
**Pros**
68+
69+
- Single UI codebase (Dart/Widgets).
70+
- Shared business logic in Dart (Gemini client, OpenClaw client, config, tool parsing).
71+
- One repo, one test suite for shared logic.
72+
73+
**Cons**
74+
75+
- Meta DAT is **native-only** (Swift + Kotlin). You must:
76+
- Maintain Flutter plugin(s) that wrap [meta-wearables-dat-ios](https://github.com/facebook/meta-wearables-dat-ios) and [meta-wearables-dat-android](https://github.com/facebook/meta-wearables-dat-android), or depend on a community one if it exists and is kept up to date.
77+
- Current iOS app is deeply Swift/SwiftUI; porting to Flutter is a full rewrite of the existing app, not an “add Android” step.
78+
- Debugging spans Dart + native (iOS/Android); CI needs both Xcode and Gradle.
79+
80+
**Effort**
81+
82+
- **Larger than Option A for “first Android-capable version”:** Flutter app from scratch + two native DAT plugins (or one dual-platform plugin) + Gemini/OpenClaw in Dart. Estimate ~2–3 months for parity with current iOS, then Android “for free” from the same app.
83+
84+
**Reuse**
85+
86+
- After rewrite: UI and app logic shared; only DAT and possibly camera/audio are native behind channels.
87+
88+
---
89+
90+
## 4. Option C: Kotlin Multiplatform (KMP)
91+
92+
**Idea:** Shared Kotlin code for Gemini Live client, OpenClaw client, and domain models; native UI on both platforms (SwiftUI on iOS, Compose on Android). Android consumes the shared module directly; iOS consumes it via a KMP framework (e.g. Kotlin/Native output).
93+
94+
**Pros**
95+
96+
- Real code reuse: WebSocket handling, JSON parsing, tool-call routing, config validation in one place.
97+
- iOS keeps SwiftUI; Android uses Compose; only “brain” is shared.
98+
99+
**Cons**
100+
101+
- iOS integration is non-trivial: Kotlin/Native, CocoaPods/SPM wrapper, and ensuring the Gemini/OpenClaw client runs correctly from Swift (threading, callbacks).
102+
- Current iOS codebase is 100% Swift; introducing KMP means a new build system and dependency story on both sides.
103+
- Overkill if the shared logic is “just” a few hundred lines of protocol and HTTP/WS glue.
104+
105+
**Effort**
106+
107+
- **Higher than Option A** for getting to a shippable Android app: KMP setup, shared module design, iOS consumption layer, then the Android app. Often 2–4 weeks extra before Android catches up to “Option A starting point.”
108+
109+
**Reuse**
110+
111+
- High for network/domain layer; UI remains native on both.
112+
113+
---
114+
115+
## 5. Option D: React Native
116+
117+
**Idea:** JavaScript/TypeScript app with native modules for Meta DAT (iOS + Android) and for camera; Gemini and OpenClaw called from JS (fetch/WebSocket).
118+
119+
**Pros**
120+
121+
- One JS codebase for UI and API calls.
122+
- Large ecosystem and hiring pool.
123+
124+
**Cons**
125+
126+
- Same as Flutter: Meta DAT is native-only; you need (or must build) RN native modules for both platforms.
127+
- Current app is Swift/SwiftUI; full rewrite to React Native.
128+
- Debugging and performance tuning across JS bridge and native; CI and release pipeline more involved.
129+
130+
**Effort**
131+
132+
- Similar in spirit to Flutter: full rewrite, then one codebase for both platforms. Typically ~2–3 months for feature parity.
133+
134+
**Reuse**
135+
136+
- After rewrite: UI and non-DAT logic in JS; DAT and possibly camera/audio in native modules.
137+
138+
---
139+
140+
## 6. CI/CD for Android (applies to any option)
141+
142+
Once you have an Android app (Option A recommended):
143+
144+
- **Workflow:** Add `.github/workflows/android-build.yml`:
145+
- Checkout, set up JDK (e.g. 17 or 21), run Gradle (assembleRelease or bundleRelease).
146+
- Optional: run static analysis (e.g. ktlint/Detekt) and fail on violations to keep “zero warnings” policy.
147+
- Produce **AAB** (Play Store) and/or **APK** (sideload/Testing), upload as artifacts (e.g. retention 30 days, mirroring iOS).
148+
- **Secrets:** Use GitHub secrets for signing (keystore, passwords) and inject into Gradle (e.g. `local.properties` or env); do not commit keystores.
149+
- **Runner:** `runs-on: ubuntu-latest` is enough for Gradle; no macOS required for the Android build itself.
150+
151+
This mirrors the structure of `ios-build.yml` (lint → build → artifact) and keeps both platforms consistent.
152+
153+
---
154+
155+
## 7. Recommendation Summary
156+
157+
| Option | Best for | First Android build | Long-term |
158+
|--------|----------|----------------------|-----------|
159+
| **A – Native Android (Kotlin + Compose)** | Fastest path to a real Android app, minimal risk, reuse of design only | **Recommended** | Two codebases; clear and maintainable |
160+
| **B – Flutter** | Single UI codebase and willingness to rewrite iOS | Rewrite then both platforms | One app, two native DAT integrations |
161+
| **C – KMP** | Strong need to share protocol/domain code and keep native UI | After KMP + iOS integration | Shared “brain”, native UI both sides |
162+
| **D – React Native** | Strong JS/React preference and willingness to rewrite | Rewrite then both platforms | One app, native DAT modules |
163+
164+
**Practical path:** Start with **Option A**. Implement an Android app in Kotlin + Compose that:
165+
166+
1. Uses Meta Wearables DAT Android for glasses streaming and registration.
167+
2. Reuses the same Gemini Live WebSocket protocol and OpenClaw tool-call format (reimplement in Kotlin).
168+
3. Adds “phone mode” with CameraX feeding frames into the same Gemini pipeline.
169+
4. Adds `android-build.yml` for build and artifacts.
170+
171+
Then, if you later want more code reuse, you can extract a small shared module (e.g. protocol constants, JSON shapes) into a KMP library or a separate repo consumed by both apps—without committing to a full KMP or Flutter rewrite up front.
172+
173+
---
174+
175+
## 8. References
176+
177+
- [Meta Wearables DAT Android](https://github.com/facebook/meta-wearables-dat-android) – Ray-Ban glasses on Android.
178+
- [Meta Wearables Developer – Android](https://wearables.developer.meta.com/docs/build-integration-android/) – Integration and permissions.
179+
- [Gemini Live API](https://ai.google.dev/gemini-api/docs/live) – Same for iOS and Android.
180+
- [OpenClaw](https://github.com/nichochar/openclaw) – Gateway and tool protocol (platform-agnostic).
181+
- Current iOS CI: `.github/workflows/ios-build.yml` – pattern for lint, build, artifact, and “zero warnings”.

‎samples/CameraAccessAndroid/README.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# VisionClaw Android
2+
3+
Native Android app (Kotlin + Jetpack Compose) for VisionClaw: AI assistant for Meta Ray-Ban smart glasses, using Gemini Live API and OpenClaw.
4+
5+
## Requirements
6+
7+
- Android Studio Ladybug (2024.2) or newer, or command-line SDK 34
8+
- JDK 17
9+
- **GitHub token** with `read:packages` scope (for Meta Wearables DAT dependency from GitHub Packages)
10+
11+
## Setup
12+
13+
1. **Clone and open**
14+
- Open `samples/CameraAccessAndroid` in Android Studio, or use the repo root and open this folder.
15+
16+
2. **Meta DAT dependency**
17+
- Create `local.properties` in `samples/CameraAccessAndroid/` with:
18+
```properties
19+
github_token=YOUR_GITHUB_PERSONAL_ACCESS_TOKEN
20+
```
21+
- Or set env var `GITHUB_TOKEN` when running Gradle.
22+
- Get a classic token from GitHub: Settings → Developer settings → Personal access tokens → `read:packages`.
23+
24+
3. **Run**
25+
- Connect a device or start an emulator (API 26+).
26+
- Run the `app` configuration.
27+
28+
## Structure
29+
30+
- **Gemini**: `GeminiConfig`, `GeminiLiveService` (WebSocket client for Gemini Live API).
31+
- **OpenClaw**: `OpenClawBridge` (session and task delegation to OpenClaw gateway).
32+
- **Settings**: `SettingsProvider` (Gemini API key, OpenClaw host/port/tokens).
33+
- **UI**: Compose screens (Home, Settings, Stream placeholder); navigation in `NavGraph`.
34+
35+
## CI
36+
37+
- `.github/workflows/android-build.yml` builds the app on push/PR.
38+
- Uses `GITHUB_TOKEN` to resolve the Meta DAT dependency; produces a debug APK artifact.
39+
40+
## Status
41+
42+
- ✅ Project scaffold, Gemini Live client, OpenClaw bridge, Settings, Home/Settings/Stream placeholder UI.
43+
- 🚧 Stream screen (glasses + phone camera mode) and full DAT integration to be completed.

‎samples/CameraAccessAndroid/app/build.gradle.kts‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
plugins {
2+
alias(libs.plugins.android.application)
3+
alias(libs.plugins.kotlin.android)
4+
}
5+
6+
android {
7+
namespace = "com.dynamicdevices.visionclaw"
8+
compileSdk = 34
9+
10+
defaultConfig {
11+
applicationId = "com.dynamicdevices.visionclaw"
12+
minSdk = 26
13+
targetSdk = 34
14+
versionCode = 1
15+
versionName = "1.0.0"
16+
}
17+
18+
buildTypes {
19+
release {
20+
isMinifyEnabled = false
21+
proguardFiles(
22+
getDefaultProguardFile("proguard-android-optimize.txt"),
23+
"proguard-rules.pro"
24+
)
25+
}
26+
}
27+
compileOptions {
28+
sourceCompatibility = JavaVersion.VERSION_17
29+
targetCompatibility = JavaVersion.VERSION_17
30+
}
31+
kotlinOptions {
32+
jvmTarget = "17"
33+
}
34+
buildFeatures {
35+
compose = true
36+
}
37+
composeOptions {
38+
kotlinCompilerExtensionVersion = "1.5.10"
39+
}
40+
}
41+
42+
dependencies {
43+
implementation(libs.androidx.core.ktx)
44+
implementation(libs.androidx.lifecycle.runtime.ktx)
45+
implementation(libs.androidx.lifecycle.viewmodel.compose)
46+
implementation(libs.androidx.activity.compose)
47+
implementation(platform(libs.androidx.compose.bom))
48+
implementation("androidx.compose.ui:ui")
49+
implementation("androidx.compose.ui:ui-graphics")
50+
implementation("androidx.compose.ui:ui-tooling-preview")
51+
implementation("androidx.compose.material3:material3")
52+
implementation("androidx.navigation:navigation-compose:2.7.7")
53+
54+
implementation(libs.okhttp)
55+
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.7.3")
56+
57+
implementation(libs.camerax.core)
58+
implementation(libs.camerax.camera2)
59+
implementation(libs.camerax.lifecycle)
60+
implementation(libs.camerax.view)
61+
62+
// Meta Wearables DAT – requires GITHUB_TOKEN or local.properties github_token
63+
implementation(libs.mwdat.core)
64+
implementation(libs.mwdat.camera)
65+
66+
debugImplementation("androidx.compose.ui:ui-tooling")
67+
debugImplementation("androidx.compose.ui:ui-test-manifest")
68+
}

‎samples/CameraAccessAndroid/app/proguard-rules.pro‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Keep OkHttp and WebSocket
2+
-dontwarn okhttp3.**
3+
-dontwarn org.conscrypt.**
4+
5+
# Keep Meta DAT (if obfuscated)
6+
-keep class com.meta.wearable.** { *; }

0 commit comments

Comments
 (0)