> You are Apple. You want to make search work like magic in the Photos app, so the user can find all their “dog” pictures with ease.
What if you're a user and you don't care about searching for "dog" in your own photos, you might not even use the Photos app, but apple still scans all your photos and sends data off device without asking you?
Perhaps this complicated dance works, perhaps they have made no mistakes, perhaps no one hacks or coerces the relay host providers... they could still have just asked for consent the first time you open Photos (if you ever do) before starting the scan.
What if you're a user and you're fed up with all the "magic"? What if you want a device that works reliably, consistently, and in ways you can understand from empirical observation if you pay attention?
Apple, Google and Microsoft, they all seem to be tripping over each other in an effort to make this whole thing just as much ass-backwards as possible. Here is how it, IMHO, should work:
1) It scans stuff, detects faces and features. Locally or in the cloud or not at all, as governed by an explicit opt-in setting.
2) Fuck search. Search is not discoverable. I want to browse stuff. I want a list of objects/tags/concepts it recognized. I want a list of faces it recognized and the ability to manually retag them, and manually mark any that they missed.
3) If you insist on search, make it work. I type in a word, I want all photos tagged with it. I click on a face, I want all photos that have matching face on it. Simple as that. Not "eventual consistency", not "keep refreshing, every 5th refresh I may show you a result".
Don't know about Apple, but Google and Microsoft both refuse #2, and spectacularly fail at #3, and the way it works, I'm convinced it's intentional, as I can't even conceptualize a design that would exhibit such failures naturally.
EDIT:
4) (A cherry on a cake of making a sane product that works) Recognition data is stored in photo metadata, whether directly or in a sidecar file, in any of a bunch of formats sane people use, and is both exported along with the photos, and adhered to when importing new photos.
Exactly, I don't want my shit sent all across the internet without my explicit prior consent, period. No amount of explanation can erase Apple's fuck-up.
Not wrong, but it’s interesting that Apple gets so much flak for this when Google and Microsoft don’t even try. If anything they try to invade privacy as much as possible.
Of course maybe that question has its own answer. Apple markets itself as the last personal computing company where you are the customer not the product so they are held to a higher standard.
What they should do Is do the processing locally while the phone is plugged in, and just tell people they need a better phone for that feature if it’s too slow. Or do it on their Mac if they own one while that is plugged in.
Whataboutisms aren't all the great you know. Google and MS also get flak, and they also deserve it.
But now that we're talking about these differences, I'd say that Apple users are notoriously complacent and defend Apple and their practices. So, perhaps in some part it is in an attempt to compensate for that? I'm still surprised how we've now accepted that Apple receives information pretty much every time we run a process (or rather, if it ran more than 12 hours ago, or has been changed).
In other words: don't hate the player hate game, but the point still stands.
Android does this too. I don't really want all my photos indexed like that, I just want a linear timeline of photos, but I cant turn off their "memories" thing or all the analysis they do to them
I don't think Android does that. It's only Google Photo and only if you upload them to the cloud, if you don't sync/upload them, you can't search them with specific terms.
Android doesn't do this. Everything is opt-in.
Granted they require you to opt-in in order for the photos app to be usable & if you go out of your way to avoid opting in they make your photo-browsing life miserable with prompts & notifications. But you do still have to opt-in.
Google loves doing this.
If you dare turn off Play Protect for example, you will be asked to turn it on every time you install or update anything. Never mind that you said no the last thousand times it asked.
A number of good third-party photo-browsing apps make it non-miserable, even if you never open Google Photos or even uninstall it.
I've seen a lot of people saying this generally but no specific recommendations.
I've used Simple Gallery Pro before but it's not very good.
Currently using Immich but that's not really a general photo app - it's got a narrow use case - so I still use the Google Photos app alongside it quite often.
Specific alternative recommendations that aren't malware welcome.
Samsung at least does these "dog" cataloguing & searches entirely on-device, as trivially checked by disabling all network connectivity and taking a picture. It may ping home for several other reasons, though.
The "memories" part can be trivially done locally and probably is, it's really just reading the picture's "date taken", so it's conceptually as easy as a "sort by date". My old Android with whatever Photos app came with it (not Google's) shows this despite being disconnected for so long.
There's nothing stopping either Apple or Google from giving users an option to just disable these connected features, globally or per-app. Just allow a "no cloud services" toggle switch in the Photos app, get the warning that $FEATURES will stop working, and be done.
I know why Google isn't doing this, they're definitely monetizing every bit of that analyzed content. Not really sure about Apple though, might be that they consider their setup with HE as being on par with no cloud connectivity privacy wise.
"memories" constantly given me notifications about "similar shots" at random, so I'm assuming it is trying to analyse the content of the photos. I managed to disable the notifications, but not the actual analysis
No Android phone I've ever owned automatically uploaded your photos without asking. What exactly do you mean that it does too?
uninstall(disable) stock Google Photos app and install `gallery2.apk`. You can download one from sketchy github repos, or I think you can alternatively extract from Emulator image.
Why, install a non-sketchy open-source gallery app from F-Droid.
Uninstall Google photos and install a dumb photos app. I think most android phones don't even come with Google photos pre installed.
Dumb Photo App by Nefarious DataExfiltration Co & Son
Fossify gallery
This is what the "Allow Network permission" checkbox in the app installation dialog on GrapheneOS is for.
Well, not vouching for automated scanning or whatever, but the advantage of homomorphic encryption is that besides the power usage for the computation and the bandwidth to transmit the data, Apple doesn't learn anything about what's in your photos, only you can. So even if you don't use the feature, the impact is minimal for you
Its using Concrete from Zama.
I didn't like their license because it's BSD-3-Clause-Clear but then they state:
"Zama’s libraries are free to use under the BSD 3-Clause Clear license only for development, research, prototyping, and experimentation purposes. However, for any commercial use of Zama's open source code, companies must purchase Zama’s commercial patent license"
So Its not free, you need to pay for patent license, and they don't disclose how much.
I recommend OpenFHE as an alternative Free open source solution. I know its C++ and not Rust, but no patent license and it can do the same thing the blog post wants to do, it even has more features like proxy-reencryption that I think Concrete can't do.
How is this "BSD-licensed but only for research" not self-contradictory?
It's like saying: "FREE* candy! (Free to look at, eating is $6.95 / pound)"
There is another reason which I dislike this which is that now Apple has reason for "encrypted" data to be sent randomly or at least every time you take a picture. If in the future they silently change the photos app (a real risk that I have really emphasized in the past) they can now silently pass along a hash of the photo and noone would be the wiser.
If an iPhone was not sending any traffic whatsoever to the mothership, at least it would ring alarm bells if it suddenly started doing so.
> The two hops, the two companies, are already acting in partnership, so what is there technically in the relay setup to stop the two companies from getting together—either voluntarily or at the secret command of some government—to compare notes, as it were, and connect the dots?
The OHTTP scheme does not _technically_ prevent this. It increases the number parties need to cooperate to extract this information, hoping it would be caught somewhere in the pipeline.
and if a government say already forced ISPs to collect metadata about who connects to whom and when, I imagine they don't even need to bother getting data from the relay hosts
It's cool how neural networks, even convulutional ones, are one of the few applications that you can compute through homomorphic encryption without hitting a mountain of noise/bootstrapping costs. Minimal depth hurrhah!
I love homomorphic encryption, but why can't they just do a neural search locally?
- iOS Photos -> Vectors
- Search Query "Dog photos" -> Vectors
- Result (Cosine Similarity): Look some dog photos!
iPhones have plenty of local storage and compute power for doing this kind of thing when the phone is idle. And cosine similarity can work quickly at runtime.
Apparently they only do the cloud HE song and dance for landmarks, which is too big of a data set to realistically keep on-device.
That’s what they do
> One example of how we’re using this implementation in iOS 18, is the new Live Caller ID Lookup feature, which provides caller ID and spam blocking services. Live Caller ID Lookup uses homomorphic encryption to send an encrypted query to a server that can provide information about a phone number without the server knowing the specific phone number in the request.
Privacy by design is always nice to see.
I was going to make my usual comment of FHE being nice in theory but too slow in practice, and then the article points out that there’s now SHE (somewhat homomorphic encryption). I wasn’t aware that the security guarantees of FHE could be relaxed without sacrificing them. That’s pretty cool.
Is there any concrete info about noise budgets? It seems like that’s the critical concern, and I’d like to understand at what point precisely the security breaks down if you have too little (or too much?) noise.
SHE vs FHE has nothing to do with security. Instead, it’s about how many operations (eg homomorphic multiplications and additions) can be performed before the correctness of the scheme fails due to too much noise accumulating in the ciphertext. Indeed all FHE schemes are also SHE schemes.
What typically makes FHE expensive computationally is a “bootstrapping” step for removing the noise that accumulated after X operations and threatening correctness. After bootstrapping you can do another X operations. Rinse and repeat until you finish the computation to you want to perform.
Im not an expert on this, but my understanding is "noise" is less a security breakdown and more the entire system breaksdown. That is where the "somewhat" comes in, unlike "full" where the system can do (expensive) things to get rid of noise, in somewhat the noise just accumulates until the system stops working. (Definitely talking out of my hat here)
Interesting! Are there any security concerns with SHE? If not, it sounds like all of the benefits of FHE with none of the downsides, other than the noise overwhelming the system. If that’s true, and SHE can run at least somewhat inexpensively, then this could be big. I was once super hyped about FHE till I realized it couldn’t be practical, and this has my old excitement stirring again.
My impression is that SHE are still relatively expensive, not as crazy as FHE but still slow enough to preclude many usecases and the noise breakdown can happen relatively quickly making them not work for most algorithms people want to use FHE for.
Most FHE schemes are constructed out of SHE schemes. Also, there’s nothing preventing FHE from being practical, it’s just that existing constructions are not as fast we would like them to be
it's incredibly algorithm-dependent. if you look into the thesis that originates the 'bootstrapping' technique to transform SHE algorithms into FHE, they determine the noise limit of their specific algorithm in section 7.3 and then investigate expanding the noise limit in 8 and 10.
(written in 2009) http://crypto.stanford.edu/craig/craig-thesis.pdf
some newer FHE don't encounter a noise limit or don't use the bootstrapping technique.
All known FHE schemes use bootstrapping
i expected that, but a search turned up several things claiming to implement fhe without bootstrapping. i didn't investigate and i can't say i'm familiar so maybe they're bogus
SHE doesn’t relax security guarantees, it relaxes the class of supported computations
Is there any library for e.g. fhm hash table lookup or similar? I have seen papers but have not seen a consensus of what a useful implementatio is
OpenFHE could be what you looking for.
This would be even more exciting if there were some way to guarantee your phone, the servers, etc. are running untampered implementations, and that the proxies aren't colluding with Apple.
If someone or something can tamper with your phone, then nobody needs to collude with proxies or Apple. They can just ask your phone to send them exactly what they want, without all the homomorphic encryption dance.
The idea that Apple is going to use this feature to spy on you, completely misses the fact that they own the entire OS on your phone, and are quite capable of directly spying on you via your phone if they wanted to.
Is the Apple Photos feature mentioned actually implemented using Wally, or is that just speculation?
From a cursory glance, the computation of centroids done on the client device seems to obviate the need for sending embedded vectors of potentially sensitive photo details — is that incorrect?
I’d be curious to read a report of how on-device-only search (using latest hardware and software) is impacted by disabling the feature and/or network access…
According to this post on Apple's Machine Learning blog, yes, Wally is the method used for this feature.
https://machinelearning.apple.com/research/homomorphic-encry...
Thank you! This is exactly the information the OP seems to have missed. It seems to confirm my suspicion that the author’s concerns about server-side privacy are unfounded — I think:
> The client decrypts the reply to its PNNS query, which may contain multiple candidate landmarks. A specialized, lightweight on-device reranking model then predicts the best candidate…
[please correct me if I missed anything — this used to be my field, but I’ve been disabled for 10 years now, so grain of salt]
The devil is in the proprietary details though.
Sorry, what do you mean by “proprietary details”?
They are alluding to the fact that the implementation is closed source, and therefore "untrustworthy". It's a trite point, of course, but not without some merit.
You have to be quick if you want to disable the feature, as the scan starts on OS install, and disabling requires you to actively open the Photos app and turn it off.
I never even knew images could be searched this way on a phone and the iPhone users in my family don't either.
A huge privacy-bruising feature for nothing in our case.
I’ve been using it a lot recently. Multiple times even today while I’ve been trying to find just the right photos of my theater for a brochure I’m putting together. I have over 100,000 photos in Apple photos so even if I vaguely remember when I took a photo it’s still difficult to find it manually.
As a concrete example, someone on my team today asked me “can you send me that photo from the comedy festival a couple years ago that had the nice projections on the proscenium?”. I searched apple photos (on my phone, while hiking through a park) for “sketchfest theater projection”. It used the OCR to find Sketchfest and presumably the vector embeddings of theater and projection. The one photo she was referring to was the top result. It’s pretty impressive.
It can’t always find the exact photo I’m thinking of the first time, but I can generally find any vaguely-remembered photo from years ago without too much effort. It is pretty magical. You should get in the habit of trying it out, you’ll probably be pleasantly surprised.
Do you mean the ability to search in Apple Photos is “privacy-bruising”, or are you referring to landmark identification?
If the latter, please note that this feature doesn’t actually send a query to a server for a specific landmark — your device does the actual identification work. It’s a rather clever feature in that sense…
Don't they have TEE?
> This should be fine: vectorization is a lossy operation. But then you would know that Amy takes lots of pictures of golden retrievers, and that is a political disaster.
This downplays the issue. Knowing that Alice takes lots of screenshots of Winnie the Pooh memes means that Alice’s family gets put into Xinjiang concentration camps, not just a political disaster.
(This is a contrived example: iCloud Photos is already NOT e2ee and this is already possible now; but the point stands, as this would apply to people who have iCloud turned off, too.)
Agreed. And for a less contrived example, people may have photos of political protests that they attended (and the faces of others present), screenshots that include sensitive messages, subversive acts, etc.
It's worth noting though that it's now possible to opt in to iCloud Photo e2ee with "Advanced Data Protection". [0]
iCloud Photo e2ee still shares hashes of the plaintext with Apple, which means they can see the full list of everyone who has certain files, even if they all have e2ee enabled. They can see who had it first, and who got it later, and which sets of iCloud users have which unique files. It effectively leaks a social graph.
It’s also not available in China.
That's the joke/implication.
While this should been opt-in to avoid bad publicity... As usual HN commenters are out of touch (nothing changed since Dropbox launched, right?). Local AI-powered photo search is great! By far one of my favorite features introduced in smartphones lately. Even on Samsung which lags behind a bit. Text indexing, pet ID, quite detailed object classes. I use the camera as something of a diary and notepad and AI is amazing for organizing 10k+ photos.
This is about additional new photo search capabilities that are enabled by default and powered by sending (encrypted) data derived from your photos to the cloud, not locally powered AI search.
> There is no trust me bro element. > Barring some issue being found in the math or Apple’s implementation of it
Yes, is you bar the "trust me bro" element in your definition, you'll by definition have no such element.
Reality, though, doesn't care about your definition, so in reality this is exactly the "trust me bro" element that exists
> But we’re already living in a world where all our data is up there, not in our hands.
If that's your real view, then why do you care about all this fancy encryption at all? It doesn't help if everything is already lost
I mean if you'd like, you could reimplement the necessary client on an airgapped computer, produce an encrypted value, take that value to a networked computer, send it to the server, obtain an encrypted result that the server could not possibly know how to decrypt, and see if it has done the computation in question. No trust is required.
You could also observe all bits leaving the device from the moment you initialize it and determine that only encrypted bits leave and that no private keys leave, which only leaves the gap of some side channel at the factory, but you could perform the calculation to see that the bits are only encrypted with the key you expect them to be encrypted with.
How is this comment useful to the OPs valid arguments?