There’s increasingly little room for doubt when it comes to Facebook: the company serves ads first and customers second. We all know the brokerage of useful data is an industry unto itself, and one that Facebook helped pioneer. But at a time when the company can ill afford any further PR missteps, and when the company already raises eyebrows for its privacy practices, Facebook is not distancing itself from its core business.

In fact, if anything, a recent patent filing reveals the company is only doubling down on what made it a successful publicly traded company in the first place: new ways to tailor and serve ads to would-be customers.

The Latest Patent From Facebook

Facebook Family Targeting

The official filing with the U.S. patent office reveals a potential addition to Facebook’s family-targeting ad platform, which launched in 2017. The new patent describes an algorithm that would cross-reference various elements within uploaded photographs to help Facebook (and ad partners) build “ad profiles” for entire households.

With family-targeting, Facebook was already able to successfully triangulate which Facebook users were likely to cohabitate or belong to the same family. So why use photos if it achieves the same ends? Other data points Facebook used as clues to determine family living arrangements included relationship statuses, work locations, shared hometowns, live event check-ins, life events in common, and others.

What’s New About This Patent?

This new patent filing provides an even finer level of detail when it comes to targeting ads at families. In addition to the granular details Facebook already uses to determine family arrangements, the new photo-based algorithm would add IP address, photo metadata, descriptions, and as mentioned features and artifacts from within the photographs themselves like landmarks and any visible products.

To put it another way, Facebook will know — based on who’s uploading which photos, and how they’re describing those photos — who is whose daughter, who is the head of which households, and how many brothers and uncles we all have. We quite often connect these dots on our own, and Facebook thanks us for it, but this most recent patent filing describes a system that would probe and come to plausible, startlingly accurate, and ultimately profitable conclusions about who we all live with, how our families are structured, how our relationships are going, our likely socioeconomic status, our future likely gift purchases and, more than likely, how our quality of life and our “net worth” compares with the other households in our sales region.

The patent would be able to draw conclusions about our relationships based on individuals in our photographs who might not be registered on Facebook at all.

As many patents out of Silicon Valley tend to be, this one is broad in its possible applications. Facebook could use the technology described here to do everything from cross-reference locations and faces in photographs submitted by multiple users, to look deeper within photographs to determine family brand loyalty, if any, and other predictors for likely future spending.

Remember that it was Facebook which sought a patent on technology that would activate smartphone microphones automatically during TV programs to figure out who was watching what, and when. Taken together, Facebook’s recent output from the U.S. patent office — along with their recently launched Portal home video device — reveals the degree to which Facebook wants to mine each of our digital interactions. Now, it even wants a closer look at our interactions with loved ones conducted in the vicinity of home appliances.

But is it really all just to serve ads?

Bringing Families Together, Profitably

In Facebook’s defense, the elevator pitch for “Facebook Portal” is appealing: it’s aimed at helping family units stay in touch with younger or elderly loved ones, using an approachable and easy-to-use device. It’s billed as a “smart camera,” which, while not capable of “true” facial recognition, still centers on and tracks faces, automatically, using context and environment cues. The company calls it “2D Pose.”

It’s newsworthy that Facebook has filed a patent application for this photo-mining technology. But the news is not, at least not yet, that the company plans to deploy it today or in the near future. Tech companies often claim ideas, by invoking the patent process, which they know how to implement but for which they don’t actually have immediate plans to do so.

So, in the coming years, Facebook might well be helping its advertisers suggest more relevant holiday gifts for our loved ones, based on what it sees in our camera rolls. In the meantime, this idea is out there now — and it’s owned by a man who values privacy to such a degree that he bought the several homes surrounding his own just to get more of it.