"Alice, we should tidy up": Yandex's smart home has learned to understand natural speech

Yandex has updated its smart home: now the voice assistant "Alice" understands users' natural speech.

Users can give commands in free form, and the system automatically determines the required action. For example, the phrase "Alice, it's time to tidy up" starts the robot vacuum cleaner, "Alice, I need to get up at seven tomorrow" sets an alarm, and "Alice, it's too dark" adjusts the lighting.

The system recognizes synonyms and context: "turn on the light on the table" and "turn on the desk lamp" are interpreted identically; "hall" and "living room" are as well. It also processes colors correctly: if a user says "turquoise," the smart home turns on the light even if it is labeled as light blue in the palette.

Alice takes the dialogue context into account. After the question "At what temperature should black tea be brewed?" it is enough to say "Turn on the kettle to that temperature," and the device will execute the command. A single phrase can set several actions at once: "Alice, turn on yellow light at forty percent." The system understands constructions with exceptions: "Alice, turn off the lights everywhere except the kitchen."

Function-calling technology (tool calling) distributes requests among devices. The model analyzes the command, breaks it down into steps, and activates the corresponding devices. As it continues learning, the smart home will recognize conversational commands more accurately and carry out complex actions.