Microsoft Recall makes more sense when you realize its potential purpose

Posted by Hagai Bar-El on Sunday, March 9, 2025 | Categories: Categories: Discuss new tech, Security analysis

| Defined tags for this entry: privacy

When I first saw an analysis of Microsoft Recall, I was puzzled. Recall is the new feature that lets you recall anything you did on your Microsoft Windows workstation. You can get back to websites you visited or presentations you worked on by merely describing them. This feature is quite useful. I always thought Microsoft did not put smart enough search features; so now they did. What puzzled me still was not why the feature is needed, but why it is built the way it is. It just seems too inefficient.

Recall is based on taking snapshots, yes, screen-grabs, of your computer screen once every few minutes or so, while you’re working, storing them, processing them using AI, and using the resulting information to understand what you are working on, so you can recall it later. Really?! Is this the most efficient way to get what we want? What happened to good old indexing of data, like all search engines do? Does Google take graphic screen-shots of what web pages look like on the screen just to throw tons of expensive AI cycles on them to extract a textual description of what they are about?

It’s only when I started thinking of what could be the real motivation behind the Recall feature, that I could finally understand this design decision. No, it has nothing to do with security.

Text-to-Image-to-Text is awkward

Most of what you do on your computer can be reduced to text. Web pages you visit, documents you write, presentations, emails, all are mostly text. Even occasional pictures can often be reduced to text without much AI. When you have text in your system, it can readily be indexed as it is. Photographing it on the screen to run the resulting picture through AI to deduce what is written in it seems like a useless and expensive detour through the “analog hole”. It reminded me of a law firm I used to work with twenty years ago. When they needed to send me an e-mail, the secretary would type it on her computer, print it out on paper, scan the print-out, and e-mail me the scanned image. Not the most efficient way to send an e-mail around, but they must have had their legal reasons to do it, and certainly didn’t mind billing me an extra paralegal hour for this cumbersome process. I would expect Microsoft to know better.

The possible objective

A possible explanation occurred to me when I realized what the motivation for Recall might be. To be clear, this is purely my own personal speculation and opinion, which I cannot and will not even try to substantiate.

If Microsoft wanted to index and search their Microsoft Office files, they would have done that directly, and to some extent they already do. For M365 files it is even easier, as the files are on their servers already. The files they want to index with Recall are also the ever-increasing loads of files that are not theirs; all data that you dare to process on your computer not using Microsoft tools. All this data is only visible to Windows in an almost-usable form when it’s displayed on your screen.

The only sensible way to make all those images usable for Microsoft tools is by reverse-engineering their contents using AI, and the best way to legitimize such a process is by getting user consent for this practice in return for a history lookup service.

Lawfully reverse-engineering the contents of your screen allows Microsoft to also process not just the data of the few desktop applications that do not come as part of Microsoft Office, but also the ever increasing amount of data processed using cloud applications which are not M365. Microsoft can now help you (and itself) by indexing documents you write on Google Docs, or the emails you write and receive over GMail. (They could have scraped this data also from Microsoft Edge, if only you were using it.)

Recall is the first step towards making sense of everything that happens on your computer, even when it happens outside the Microsoft product lines and cloud services. It allows understanding what you read, write and do, for the sake of providing additional services. Today Recall is local, but I will not be surprised if in a year or two we will be offered to store and process Recall data on the Microsoft cloud as well, making it the only cloud service that understands everything you do on your workstation, even when using other competing products.