Needle 2: A New Lightweight AI Model for Wearables and IoT Devices
Cactus has released Needle 2, a 14MB agentic large language model (LLM) designed for use in phones, wearables, smart homes, and small robots. This model is notable for its efficiency, operating within a power budget that is significantly lower than other models of similar performance. Needle 2 is capable of running on budget devices, such as sub-$200 phones and Raspberry Pis, making it accessible for a wide range of users. The model is based on Simple Attention Networks and is optimized for structured extraction tasks, allowing it to perform text classification and summarization efficiently. Needle 2's architecture allows it to decode at speeds of 500 tokens per second on a Raspberry Pi 5 and between 300-700 tokens per second on budget phones. This development is part of a broader trend towards integrating edge AI into a variety of consumer devices. 
