Skip to content

android-integrations.md

Latest commit

 

History

History
332 lines (295 loc) · 16.4 KB

android-integrations.md

File metadata and controls

332 lines (295 loc) · 16.4 KB
Raw

Android Integration for AI Personal Assistant

This document outlines key Android integration points for our AI assistant app, with detailed use cases and implementation considerations.

Calendar Integration

┌─────────────┐     ┌─────────────────┐     ┌───────────────┐
│ Assistant   │     │ Intent          │     │ Calendar App  │
│ "Schedule   │────►│ ACTION_INSERT   │────►│ New Event     │
│  meeting"   │     │ EXTRA_EVENT     │     │ Creation UI   │
└─────────────┘     └─────────────────┘     └───────────────┘

Use Case: Meeting Scheduling

  • User Input: "Schedule a meeting with John tomorrow at 2pm about the project timeline"
  • Assistant Action: Parses natural language for date, time, attendees, and subject
  • Integration Method: Intent with CalendarContract.Events.CONTENT_URI
  • Data Passed: 
    Intent intent = new Intent(Intent.ACTION_INSERT)
    .setData(CalendarContract.Events.CONTENT_URI)
    .putExtra(CalendarContract.EXTRA_EVENT_BEGIN_TIME, startMillis)
    .putExtra(CalendarContract.EXTRA_EVENT_END_TIME, endMillis)
    .putExtra(CalendarContract.Events.TITLE, "Project Timeline Discussion")
    .putExtra(CalendarContract.Events.DESCRIPTION, "Review current progress and next steps")
    .putExtra(Intent.EXTRA_EMAIL, "john@example.com");
  • User Experience: Calendar app opens with pre-filled event details for confirmation
  • Privacy Considerations: Access requires calendar permission grant

Document Access & Editing

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ DocumentsProvider│    │ Spreadsheet│
│ "Update sales │────►│ com.google.sheets│───►│ Data       │
│  spreadsheet" │     │ /sales_2023.xlsx │    │            │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Spreadsheet Updating

  • User Input: "Add today's sales figures to our tracking spreadsheet: $5,200 for electronics and $3,800 for appliances"
  • Assistant Action: Identifies target document, extracts data points, determines appropriate cells
  • Integration Method: DocumentsProvider or ContentProvider depending on app support
  • Implementation Details:
    1. Query document by name/path using DocumentsContract
    2. Open document as input stream
    3. Modify content (may require format-specific library)
    4. Save changes via output stream
  • Error Handling: Provide feedback if document format is unsupported or data cannot be inserted correctly
  • Advanced Feature: Maintain metadata about document structure for future edits

Message Composition

┌─────────────┐     ┌─────────────────┐     ┌───────────────┐
│ Assistant   │     │ Intent          │     │ Email/SMS App │
│ "Email John"│────►│ ACTION_SEND     │────►│ Composer with │
│             │     │ EXTRA_TEXT      │     │ Prefilled Text│
└─────────────┘     └─────────────────┘     └───────────────┘

Use Case: Email Composition

  • User Input: "Send an email to my team about the delayed shipment, explaining it will arrive next Tuesday"
  • Assistant Action: Drafts appropriate email text, identifies team recipients
  • Integration Methods:
    1. Simple: Intent with ACTION_SEND
    2. Gmail Specific: Gmail's ContentProvider (if available)
    3. Direct: JavaMail API with stored SMTP credentials (requires secure credential storage)
  • Sample Code: 
    Intent intent = new Intent(Intent.ACTION_SEND);
intent.setType("message/rfc822");
intent.putExtra(Intent.EXTRA_EMAIL, new String[]{"team@company.com"});
intent.putExtra(Intent.EXTRA_SUBJECT, "Shipment Delay Notification");
intent.putExtra(Intent.EXTRA_TEXT, "Team,\n\nI wanted to inform you that our shipment has been delayed. It is now expected to arrive next Tuesday.\n\nBest regards,\nYour Name");
  • Enhancement: Use contact groups to resolve "my team" to actual email addresses

Location-Based Reminders

┌─────────────┐     ┌─────────────────┐     ┌───────────────┐
│ Assistant   │     │ Geofencing API  │     │ BroadcastReceiver│
│ "Remind me  │────►│ Location Trigger│────►│ Notification  │
│ at grocery" │     │                 │     │ When Arriving │
└─────────────┘     └─────────────────┘     └───────────────┘

Use Case: Proximity Reminder

  • User Input: "Remind me to buy milk when I'm near Walmart"
  • Assistant Action: Creates geofence around all nearby Walmart locations
  • Integration Components:
    1. LocationManager + GeofencingClient: Set up proximity alert
    2. BroadcastReceiver: Handle geofence transitions
    3. NotificationManager: Display reminder when triggered
  • Implementation Challenges:
    • Power consumption balance with location accuracy
    • Handling multiple possible locations (all Walmart stores)
    • Reminder persistence across device reboots
  • Sample Geofence Setup: 
    Geofence geofence = new Geofence.Builder()
    .setRequestId("walmart_reminder")
    .setCircularRegion(latitude, longitude, 500) // 500m radius
    .setExpirationDuration(Geofence.NEVER_EXPIRE)
    .setTransitionTypes(Geofence.GEOFENCE_TRANSITION_ENTER)
    .build();
  • Privacy Considerations: Clear disclosure of background location usage

Content Sharing Between Apps

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ FileProvider   │     │ Social App │
│ "Share this   │────►│ content://...  │────►│ Share UI   │
│  chart"       │     │ /temp_chart.png│     │            │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Chart Sharing

  • User Input: "Share this expense chart with my finance group on WhatsApp"
  • Assistant Action: Generates visual representation of data, initiates share flow
  • Integration Components:
    1. FileProvider: Secure file URI generation
    2. Intent.ACTION_SEND: Sharing mechanism
  • Implementation Flow:
    1. Generate chart image (using chart library)
    2. Save to app's cache directory
    3. Get content:// URI via FileProvider
    4. Create share intent with specific target package (optional)
  • Setup Requirements:
    • FileProvider definition in AndroidManifest.xml
    • XML paths file specifying shareable directories
  • Sample Intent: 
    Intent shareIntent = new Intent();
shareIntent.setAction(Intent.ACTION_SEND);
shareIntent.putExtra(Intent.EXTRA_STREAM, fileUri);
shareIntent.setType("image/png");
shareIntent.setPackage("com.whatsapp"); // Optional - target specific app

Smart Home Control

┌─────────────┐     ┌─────────────────┐     ┌───────────────┐
│ Assistant   │     │ BroadcastReceiver│     │ Home App     │
│ "Turn on    │────►│ Custom Action   │────►│ Device Control│
│  lights"    │     │                 │     │ API           │
└─────────────┘     └─────────────────┘     └───────────────┘

Use Case: Home Automation

  • User Input: "Turn on the living room lights and set thermostat to 72 degrees"
  • Assistant Action: Identifies devices, desired states, and sends commands
  • Integration Options:
    1. Direct API Integration: Using smart home platform SDKs (Google Home, SmartThings, etc.)
    2. Intent-based: Launching companion apps with command parameters
    3. Custom Protocol: Broadcast intents for receiver apps
  • Security Considerations:
    • Authentication token management
    • Permission model for critical home controls
    • Activity logging for security-relevant actions
  • Implementation Approach:
    • Define standardized action schema for device operations
    • Support device discovery and capability querying
    • Implement state caching for faster response to queries
  • Error Handling: Graceful degradation when devices are unreachable

Contact Management

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ ContentProvider│     │ Contacts   │
│ "Add John's   │────►│ contacts://... │────►│ Database   │
│  new number"  │     │                │     │            │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Contact Updates

  • User Input: "Update Sarah Smith's contact with her new work email sarah.smith@newcompany.com"
  • Assistant Action: Searches contacts, identifies correct entry, updates specific field
  • Integration Method: ContactsContract ContentProvider
  • Operation Flow:
    1. Query contacts to find matching name
    2. If multiple matches, use disambiguation (e.g., "Sarah Smith from Marketing")
    3. Update specific field (Email, Phone, etc.)
    4. Provide confirmation of change
  • Sample Query: 
    Cursor cursor = contentResolver.query(
    ContactsContract.Contacts.CONTENT_URI,
    null,
    ContactsContract.Contacts.DISPLAY_NAME + " = ?",
    new String[]{"Sarah Smith"},
    null
);
  • Advanced Features:
    • Contact relationship understanding ("my boss", "my sister")
    • Contact merging suggestions
    • Business context awareness ("Sarah from the meeting yesterday")

Notification Management

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ NotificationMgr│     │ Status Bar │
│ "Summarize    │────►│ + Bundled      │────►│ Grouped    │
│  notifications"│    │ Notifications  │     │ Summary    │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Notification Summary

  • User Input: "Read me my important notifications" or "What notifications did I miss?"
  • Assistant Action: Accesses notification history, prioritizes, summarizes
  • Integration Requirements:
    • NotificationListenerService implementation
    • Special permission grant from user
    • Careful notification categorization
  • Implementation Considerations:
    1. Filter notifications by importance/category
    2. Apply NLP to extract key information
    3. Generate concise summaries grouping similar notifications
    4. Provide interaction options for each notification group
  • Sample Service Declaration: 
    <service
    android:name=".NotificationListener"
    android:label="Assistant Notification Access"
    android:permission="android.permission.BIND_NOTIFICATION_LISTENER_SERVICE">
    <intent-filter>
        <action android:name="android.service.notification.NotificationListenerService" />
    </intent-filter>
</service>
  • Privacy Features:
    • Content redaction options for sensitive apps
    • User-configurable app exclusions
    • Clear visual indicators when notification access is active

File Management Integration

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ StorageAccessFw│     │ File System│
│ "Find my tax  │────►│ DocumentsContract│───►│ Scoped     │
│  documents"   │     │                │     │ Access     │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Document Retrieval

  • User Input: "Find my tax documents from last year"
  • Assistant Action: Searches files using metadata and content patterns
  • Integration Components:
    1. Storage Access Framework: For user-directed directory access
    2. MediaStore API: For media file access
    3. DocumentsContract: For general document queries
  • Implementation Strategy:
    • Build query based on file type, naming patterns, and creation dates
    • Use content indexing for faster searches
    • Maintain user's common document locations
  • Advanced Features:
    • Content-based search (documents containing certain text)
    • OCR for document classification
    • Document suggestion based on current date/context
    • Learn from user interactions to improve future searches

System Settings Control

┌───────────────┐     ┌────────────────┐     ┌────────────┐
│ Assistant     │     │ Intent         │     │ Settings   │
│ "Turn on      │────►│ ACTION_WIFI_   │────►│ Panel      │
│  WiFi"        │     │ SETTINGS       │     │            │
└───────────────┘     └────────────────┘     └────────────┘

Use Case: Device Configuration

  • User Input: "Turn on Do Not Disturb until my meeting ends at 3 PM"
  • Assistant Action: Modifies system settings with time-based parameters
  • Integration Methods:
    1. Settings.Global/System/Secure: For direct settings manipulation (requires permissions)
    2. Intent Actions: For launching specific settings panels
    3. Notification Policy Access: For DND control
  • Implementation Permissions:
    • WRITE_SETTINGS permission for some controls
    • ACCESS_NOTIFICATION_POLICY for DND
  • Time-based Management:
    • Use AlarmManager for settings restoration
    • Context-aware triggers (location, calendar events)
  • Sample DND Code: 
    NotificationManager notificationManager = getSystemService(Context.NOTIFICATION_SERVICE);
if (notificationManager.isNotificationPolicyAccessGranted()) {
    notificationManager.setInterruptionFilter(
        NotificationManager.INTERRUPTION_FILTER_PRIORITY);
    
    // Schedule end time
    AlarmManager alarmManager = getSystemService(AlarmManager.class);
    // Set alarm to disable DND at meeting end time
}

Implementation Considerations

Security and Privacy

  • Clearly communicate permissions needed for each integration
  • Provide granular control over which integrations are enabled
  • Implement proper authentication for sensitive operations
  • Maintain audit logs of system-modifying actions

Performance Impact

  • Minimize background processes for location monitoring
  • Implement efficient ContentProvider queries
  • Use batched operations when possible
  • Consider battery impact of continuous monitoring

Error Handling

  • Graceful degradation when services are unavailable
  • Clear user communication about permission requirements
  • Fallback options when primary integration paths fail
  • Handle API variations across Android versions

User Experience

  • Maintain consistent interaction patterns across integrations
  • Provide visual confirmation of system-changing actions
  • Allow easy reversal of changes made by the assistant
  • Use progressive disclosure for complex operations