Meta launched a brand new synthetic intelligence (AI) mannequin on Monday that may carry out advanced laptop imaginative and prescient duties. Dubbed Phase Something Mannequin 2 (SAM 2), it follows after its predecessor that was launched final 12 months and was included in Instagram’s Backdrop and Cutouts instruments. The successor to the mannequin now comes with superior capabilities and the corporate stated it could carry out section identification and monitoring even on movies. Like most of Meta’s massive language fashions (LLMs), SAM 2 can be an open-source AI mannequin.
In a newsroom put up, Meta introduced the brand new AI mannequin which focuses on section evaluation on movies primarily, whereas enhancing its picture segmentation capabilities. Highlighting the accomplishments of its predecessor, Meta stated the AI mannequin was utilized in Instagram’s Backdrop and Cutouts options, whereas marine scientists used it to “section sonar pictures and analyse coral reefs, satellite tv for pc imagery evaluation for catastrophe reduction, and within the medical discipline, segmenting mobile pictures and aiding in detecting pores and skin most cancers”.
SAM 2 is able to object segmentation in a picture and video in addition to observe it throughout totally different frames of a video in real-time. The AI may observe and section objects in eventualities the place the objects transfer quick, change in look, or are hid by different objects or a completely totally different scene.
The muse mannequin for prompt-based visible segmentation is constructed on a easy transformer structure. It has a streaming reminiscence that enables it to course of movies in real-time. The corporate additionally claimed that the mannequin was skilled on its largest video segmentation dataset dubbed SA-V dataset.
Meta stated the AI mannequin can assist ease the method of video modifying or AI-based video technology, in addition to to energy new experiences within the firm’s mixed-reality ecosystem. The article monitoring functionality in movies may help in quicker annotation of visible knowledge to coach different laptop imaginative and prescient techniques, the corporate added.
Since it’s an open-source AI mannequin, the corporate has hosted its weights on its GitHub web page. people can obtain and take a look at out the AI mannequin. Notably, it’s licenced underneath the Apache 2.0 licence which permits for analysis, tutorial, and non-commercial utilization.
