Yandex adds AI to smart IP camera

Yandex has introduced AI into Yandex’s smart IP camera: it describes specific events in the frame, lets you set “Home with Alice” scenarios in your own words, and sends a summary of the day once a day; the features are available in Standard and Extra subscriptions for RUB 299 and RUB 699 per month.

Yandex has incorporated artificial intelligence into Yandex’s smart IP camera. The built-in AI analyzes the video stream and reports on specific events in the frame. For example, “a cat appeared in the frame.” Previously, the notification could sound like “motion detected”. Now the meaning is more precise.

In addition, users can formulate conditions for smart home scenarios in their own words. It is enough to describe the situation in free form. For example, “the dog chews the sofa”. Then you have to choose what should happen in response. Also, once a day the AI summarizes everything that happened during the day and sends the user a summary of events. This is how Yandex’s smart IP camera helps you understand what’s going on at home and more accurately customize scenarios for everyday situations.

The new features are available as part of the Standard and Extra smart IP camera subscriptions. These plans offer additional benefits. These include cloud storage and motion detection archiving.

Notifications and archive search in Yandex’s smart IP camera

Now, when events in the frame occur, the user receives a notification with a specific description from the AI. The camera tells the user exactly what it has captured. If the camera is in a hallway and the kids are home from school, the notification won’t be about “the person in the frame.” It will come with the wording that the kids have gone home. This helps you understand the situation faster without having to open the app.

And AI descriptions have appeared in the archive of recordings as well. Each event is now accompanied by a text description of what the camera captured. This makes it easier for users to find what they’re looking for in the archive.

Scenarios “Home with Alice” by conditions specified by words

Smart home scenarios used to be tied to a limited set of camera signals. These included noise, motion, and a person in the frame. Now, the camera interprets what’s happening with AI and responds to specific events. The user can describe the condition in their own words. For example, “the cat jumped on the table”, “the dog appeared in the room”, “the child sat on the bed”. The condition can then be linked to a scenario in the Home with Alice app.

So you can turn on the nightlight in the nursery if the baby wakes up during the night. You can also activate the smart drinker when a dog comes into the picture. Another option is to send yourself a photo every time a pet jumps on the couch. As a result, the camera reacts not to an abstract “movement in the frame” but to an event that the user has described in words. This expands the range of scenarios for different life situations.

Summarize the Day and Alice AI VLM

Day summarization is enabled by default. At a time selected by the user, the neural network generates a summary report of the events that the camera has captured. When the summarization is ready, the Home with Alice app sends a notification. The user receives a summary of what happened at home during the day. The report indicates who and what appeared in the frame. It also notes how the behavior of people and pets or the position of objects changed. You can view the summarization in the device card.

To understand what’s happening in the frame, Yandex’s smart IP camera uses Alice AI VLM. This is Yandex’s visual-language model that analyzes the image. It is part of the Alice AI family of models along with the linguistic and generative models. Alice AI VLM is trained on a large amount of visual and textual data, which allows it to interpret what is happening in the frame. The same technology already works in live chat with Alice as part of the Alice Plus option. Now it’s coming to the smart IP camera as well.

Subscriptions, prices and what’s free

All new technologies are available with a subscription for additional features of the Extra smart IP camera. Use of AI-driven notifications, archive recordings, and day summarization is also available with the “Standard” subscription. The tariffs cost 699 rubles and 299 rubles respectively. For those who have not used paid options before, the first month of subscription is available for free.

At the same time, some of the features that were previously in the paid subscription are now available at no additional cost. Users can follow the person in the frame. A photo in a notification of motion or a person in the frame is also available. Another feature is sending a scripted photo, such as every day at 6:00 p.m.

Another feature is to send a photo on a scripted basis, such as every day at 6:00 p.m.