Key Rack Research (KRR)

Home | Texts | SEO | Technical References | DAU
bc: index :: texts :: the poor man's facial recognition and profiling

The Poor Man’s Facial Recognition and Profiling

February 21, 2018

Not everyone can afford to purchase facial recognition solutions. Even navigating into the industry websites can be daunting, as one tries to determine industry lingo, what features each solution offers, how these systems can be brought into their own environment(s), and an unknown amount of other specific variables used to shop these.

We believe we have a solution for the poor man. Maybe you are just frugal, not necessarily poor. It’s a figure of speech, that fits our message here. Let’s move on.

The entire concept of facial recognition and profiling is quite “big brother.” So, when our solution includes some big brother steps, we will assume you are already on-board. We do not want to get into a debate about privacy, the lack of privacy, or any such other topic here.

When we are discussing “facial recognition” we are talking about a system that can recognize humans based on photographs or videos (a lot of quickly taken photographs). This is not our field, so there is probably a more elegant definition out there. We simply want a system that if we throw images at it, it can tell us the name of the person in the image.

When we are discussing “profiling,” in this instance, it is mostly the same concept as facial recognition, in that it is a system that can keep records of specific humans. Specifically, it is a system that will bundle or file all images of a single human so any future images may be added to the bundle, and thereby include that human, or they can be queried or reviewed. If this is a bit confusing, just press on, it will start to make more sense.

There may be a lot of options out there today. As we are looking for a low-cost solution, we are aiming for free. So, our first choice actually provides everything we need. Google Photos.

Google Photos offers unlimited storage of “high quality” images and video, using Google’s AI, Google Photos automatically identifies images of single humans, even cats and dogs now, and bundles them into a profile. This may not be the term they use, but it is what we call it.

So, what are some ways to use Google Photos as our Poor Man’s Facial Recognition and Profiling System?

Using a smartphone, such as a Google Pixel, take a lot of photographs and videos of our targets or data and use the Google Photos app to upload them.
Setup a camera to periodically, or based on motion, capture images of our targets or data to a computer that automatically uploads them to Google Photos.
Scrape the internet for images, upload the scraped images to Google Photos.
A hybrid of the above.

We chose the hybrid.

First, we created a new Google account. This is just to better manage the project.

Second, we used Scrapy to collect a lot of, perhaps too many, images. This was fairly easy for most websites, but some websites which use scripting posed some challenges. Also, websites that are membership based pose another level of challenges. Depending on your programming skills, and/or time, these may not be show stoppers. The trick here is to recursively scrape.

Let’s introduce a serious warning you will most likely disregard. Be very prepared to scrape images you may not want on your system(s). Within a few minutes of scraping a news website we had over 23,000 images on our hard disk drive. We may want to clarify here, that this news website is a respectable site, perhaps even family friendly. Well, our script followed links indiscriminately, and the results were definitely NOT family friendly. So, you have been warned. And now we shall continue forward ...

Third, we upload them to Google Photos. This we first did manually. The reason we did this manually the first go was because we just wanted to get some images uploaded for Google Photo’s to start chewing on.

Now, for a brief reason why we are using Google Photos. Google Photos offers unlimited image storage, for free. Yeah, reread that if you want, we will wait. Also, Google Photos includes a feature where Google uses their algorithms/AI to do facial recognition of images and they then bundle faces they believe to be the same human together. Google Photos does this for both still images and video images. It also uses meta information within the images and video to determine locations, dates, and so forth. Google Photos believes the meta information, so if your image has the wrong meta information, Google will base it’s findings on the metatags. As of the writing of this post, Google Photos does not have a mass metatag editing capability, so if you need to do mass edits, do so prior to uploading them.

After uploading your first bundle of images, you will need to exercise some patience, as Google Photos will take some time, as in anywhere from hours to days, before it starts to match faces. It seems that the more images Google has of one individual, the faster it can recognize images of that person. We have even been able to take a photograph of someone and by the time we move from the camera to Google Photos, then the Albums, then People, then select that person, the image is already being displayed there. (That is even with the upload happening somewhere in there!)

Fourth, name the people or animals Google Photos has recognized and bundled, which is our Profiling step. You have two options here, you can either name them straight from Google Photos, or you can use your Google Contacts to name them. We recommend you stick with whichever method you choose. Since this is a new Google account, you may have to import contacts, which is quite easy actually, but then you will have a whole new set of contacts to maintain. However, considering that this entire account is to be all big brother-ish, then perhaps maintaining a separate set of contacts if fine and dandy. We have not looked to see what Google Photos is doing exactly to the images, if anything, but we are guessing it is just adding some metadata to the images so it does not have to redo the facial recognition on the images it has already processed.

The process of going through the bundled or already recognized images within Google Photos is quite revealing. We have seen some very creepy things here, and some funny things as well. Want some examples of funny? No, you want creepy stuff. Sure. In one test, one of us was with their family at a family park, think Epcot Center type of place, and they were taking photographs of their family. They left that park, left that state, and returned back to their home state, hundreds and hundreds, maybe even a thousand miles away from the park. They continued taking photos and allowing Google Photos to do it’s magic. Using the naming convention we agreed to use, of “John Doe 000001” and “Jane Doe 000001” they had already named random strangers from the park trip accordingly. Then, Google Photos showed them these people in images taken near their home. Not creepy enough? A month or two later, they went on another family trip, in the opposite direction of the park, also in another state hundreds of miles away, and guess who appeared in their photographs taken there? Yep, the same Jane/John Doe.

Sometimes Google Photos is wrong on the facial recognition. The more images it has to work with, the more accurate it seems to be. Sometimes it is spot on target. In the case of the creepy people above, it was accurate. They went offline, packed up their belongings, moved across the country, and we have not heard from them since. (That last sentence is not true at all.)

Fifth, automation. How you decide to automate is up to your imagination and resources, but keeping this simple, we mentioned some options above. Let’s say you were an organization that kept photographs of all your members. You could put up a white background, have each member pose in front of the background, and using your telephone camera snap a picture, or ten … the more the merrier for our purposes, then allow your Google Photos app to naturally backup your images, and then do it’s thing.

Another option for automating this is to use one of the Google Photos desktop applications. The way it works is you download and install the application, and it watches a particular directory or a set of directories, and any image placed into the directories it is watching it automatically uploads to Google Photos. So, if you have a camera set to take video or photographs based on motion, or otherwise, and store them on a computer, all you do is add that storage directory to the Google Photos upload application, and there you go, full automation for obtaining images into Google Photos.

Some fun uses of our Poor Man’s Facial Recognition and Profiling System are:

Upload images of the FBI’s Most Wanted into your Poor Man’s Facial Recognition and Profiling System, then wait to see if they are spotted anywhere else on-the-line.
Upload images of your friends and see if they are spotted in any funny or weird images.
Upload images of your boss and see what happens.
Setup a camera that snaps images based on motion in your business and use it as an additional security system as well as Client tracking system.
Use multiple cameras, all connected to the same Google Account, and expand collection capabilities.

We know this is not the most feature rich solution. It is free, and causes us to think about the mind-boggling fact that facial recognition technology is available to the masses (again, for free). What does the future hold?