Apple Watch begins beta testing AI features for rewinding conversations

Apple has started rolling out its new AI-powered Audio Intelligence features for the Apple Watch into beta testing. Initially available to developers via the iOS 27.3 update, the tools were first teased during the company's hardware event in September.

• The suite includes four opt-in features: sound recognition, song identification via Shazam, Live Rewind, and Siri Recaps.
• Live Rewind allows users to double-press the digital crown to view a text snippet of what was said in the last 15 seconds.
• Siri Recaps generates high-level conversational summaries without saving raw audio or identifying individual speakers.
• Security architecture utilizes an on-chip secure enclave and encrypted Private Cloud Compute processing.

Live Rewind and Sound Recognition

The sound recognition feature assists deaf and hard-of-hearing users by identifying important alerts like sirens, alarms, or doorbells, while song identification leverages the Shazam app. Live Rewind lets wearers quickly check something they missed, such as verbal directions or restaurant specials, by displaying a temporary text snippet on the watch face after a quick gesture.

To address privacy concerns, recording is signaled through a combination of chimes, animations, and visual cues. Snippets automatically disappear after 30 seconds unless explicitly saved to the Siri or Notes app. Users can also tap 'Ask Siri' below the text to get immediate context on unfamiliar terms.

Conversation Notes with Siri Recaps

Siri Recaps takes ambient listening further by creating summarized bullet points from work meetings, parent-teacher conferences, or fitness sessions. Rather than providing word-for-word transcriptions or recognizing speakers, it highlights key dates, pricing, and tasks. Recaps can be saved to Notes or Journal apps, or they will automatically delete after seven days.

Users can manually enable recaps or set schedules, ensuring the feature only runs during work hours or stays off at night. The system pauses when headphones are active and is specifically designed to omit sensitive data such as government identifiers, financial information, and self-harm content.

Technical Architecture and Privacy

The features rely on Apple Watch Series 12 and Ultra 4 hardware, including the S11 chip, microphones, on-device iPhone models, and Private Cloud Compute. Raw audio is processed inside a secure enclave memory space that remains inaccessible to the rest of the operating system and Apple.

Lightweight AI models on the watch detect active conversations and send the audio to the iPhone, where it is condensed into text. This text is securely encrypted and sent to Apple's cloud infrastructure to generate the final summary before being immediately deleted. Apple states these features are optimized to maintain standard battery life.