Earlier this year, at the I/O 2024, Google showcased a new Lens capability powered by multi-modal generative AI technology, which allows users to ask questions using a video called Google Lens "Ask With Video," and the same is now rolling out to Android and iOS devices, further expanding the capabilities of Google Lens. https://youtu.be/XEzRZ35urlk?list=PL590L5WQmH8doPo8OufXavO2Qu4ysZjyl&t=3040 According to Google, Lens' "Ask with video" feature will come in handy in troubleshooting gadgets. In a demo, Google showcased how the feature can be used to troubleshoot a faulty old record player without going through multiple complex web searches. Depending on the query, Google search results will give an AI overview with a possible answer and links with more details regarding the same. Lens' Ask with video capability is powered by Google's state-of-the-art speech model, deep visual understanding, and a custom Gemini model that runs on the cloud. When you ask a query, a custom Gemini model running on the cloud will analyze the video frame-by-frame and will come up with relevant search results. All this happens in just a few seconds, which when manually searched could take minutes to get the exact search results. We tested out quite a few scenarios, and the results from Google Lens were almost on point. This is an important feature for do-it-yourself (DIY) enthusiasts who want to troubleshoot or repair a gadget on their own. Google Lens is available as a dedicated app for Android smartphones, while there is no dedicated Lens app for iPhones; the same can be accessed within the Google app. In fact, Apple is also rolling out a similar feature for iPhones with Apple Intelligence called Visual Intelligence, which enables users to ask questions about anything by holding the camera capture button.
