This article may contain affiliate links. Learn more on our Partner Program page.
The Photos app on the iPhone is one of Apple's most popular apps. In addition to organizing and displaying photos, Apple has equipped it with practical features that help you search through your images more quickly and even learn more about their content. One of these features is the advanced visual search. But how does it actually work and how does Apple ensure that your data remains safe? In this article, you'll find out everything you need to know.
The enhanced visual search is not to be confused with the long-standing feature called Visual Look Up. Both are powerful tools based on machine learning to make it easier for you to work with your photos. While Visual Look Up has been known since iOS 15, the enhanced visual search goes a step further. It can search through your entire photo library, even if no location data is available. At the same time, Apple relies on complex technologies to ensure that your privacy remains protected.
What is the difference between Visual Look Up and Advanced Visual Search?
Visual Look Up is a tool that gives you information about individual images. When you look at a photo and swipe up on it, the app shows you information about the objects, places, or creatures in the image. For example, this could be dog breeds, plant species, landmarks, or even the symbols on your car dashboard. Advanced Visual Look Up, on the other hand, has a different purpose. It helps you search for specific places, objects, or landmarks across your entire photo library. For example, you can search for "Golden Gate Bridge" and the app will show you all the relevant photos in your collection, even if the landmark was taken with a blurred background or without geodata.
How does advanced visual search work?
The feature combines machine learning on your device with a global index maintained by Apple. Your iPhone first analyzes the photos to identify potential areas of interest, called regions of interest (ROI). When such a region is found, your device creates a mathematical representation - called a vector embedding - of the relevant part of the image. This data is then sent in encrypted form to a server to be compared against the global index. The actual photo or its pixels are never transmitted to Apple.
How does Apple protect your privacy?
Apple has built in several protections to ensure your data stays safe when using enhanced visual search:
- Homomorphic encryption: Before your iPhone sends a request to the server, it is fully encrypted. The server can process the data without decrypting the actual request. Only your iPhone can decrypt the response.
- Private Nearest Neighbour Search (PNNS): Your device queries the global index without Apple or the server being able to see your personal data. No photos are transferred, only encrypted mathematical representations.
- Differential privacy: Apple anonymizes each request. Additionally, fake requests are created and sent along with the real request, making it unclear which data is actually real. Additionally, the request is routed through an anonymization network so that the server cannot connect the requests to your device.
- No data storage: The servers delete all information immediately after processing and do not store any data long-term.
How to enable or disable the feature
Enhanced visual search is disabled by default. If you want to use it, you can enable it in the Photos app settings. To do so, go to the Settings app, select Photos, and scroll down to find the option. Apple recommends disabling the feature in data-poor areas to save on mobile data usage.
Search quickly, stay safe: The advanced visual search at a glance
Enhanced visual search makes the Photos app an even more powerful tool. It allows you to quickly and easily find specific locations or landmarks in your photo library without having to scroll for hours. At the same time, your privacy is protected by technologies like homomorphic encryption, PNNS, and differential privacy. Apple demonstrates how innovative features can be combined with a high level of data protection. If you choose to use this feature, you can rest assured that your data will remain protected. Looking for high-quality accessories? In our Amazon Storefront, you'll find numerous products from leading suppliers, ideal for HomeKit, and much more! (Image: Apple)
- How to take scrolling screenshots on your iPhone
- Quiet sounds from the AirPods Pro 2 charging case – what’s behind it
