The Secrets Inside Your Smartphone Camera - A Look Behind the Scenes of Scanning

A Camera Is Made of "Light Sensors"

What a smartphone camera really is, is a chip lined with tens of millions of tiny light sensors. This chip is called an "image sensor." Light that comes in through the lens hits each individual sensor, and information such as "this spot is bright" and "this spot is dark" is converted into numbers. This collection of numbers is what a photo really is.

When reading a black-and-white pattern, the camera pays attention to the "difference in brightness" rather than to color. White areas reflect a lot of light, so their numbers become large, and black areas absorb light, so their numbers become small. The clearer this difference between light and dark, the higher the reading accuracy. The reason scanning does not work well in dark places is that this difference between light and dark becomes smaller.

How Focusing Works - What Autofocus Really Is

When you hold the camera up to a pattern, it comes into focus in an instant. This is thanks to a function called "autofocus." Many smartphones use a method called "phase detection AF," which calculates the distance at which the image will be in focus from how far the light reaching the sensor is shifted. It is the same principle as human eyes measuring distance from the parallax between the left and right eyes.

The time it takes to focus is about 0.03 seconds on a typical smartphone. That is faster than a blink. However, if the camera is too close to the pattern (5 cm or less), it can no longer focus because of the structure of the lens. Conversely, if it is too far away, the pattern becomes too small and the fine details cannot be read. 10-30 cm is just the right distance.

The Scanning Software Decodes the "Meaning of the Pattern"

The image taken by the camera is, as it is, just a black-and-white photo. This is where the scanning software (the decoder) comes in. The decoder first looks for three large square markers in the image. These markers are called "finder patterns," and they are the clues for identifying the position and tilt of the pattern.

Once the markers are found, the decoder divides the whole pattern into a grid like a Go board and determines whether each cell is white or black. White is converted to "0" and black to "1," and the sequence of 0s and 1s is restored into characters or a URL. This processing is completed inside the smartphone in less than a second.

Why It Can Be Read Even with Dirt or Scratches

Even if part of the pattern is dirty or scratched, it can sometimes still be read. This is thanks to a mechanism called "error correction." In addition to the original data, "spare data" is embedded in the pattern, so even if part of it is missing, the original data can be restored from the remaining information.

Error correction has four levels, L / M / Q / H, and at the strongest level, H, the data can be restored even if about 30% of the whole is missing. However, there is a trade-off: the stronger the error correction, the larger (and more complex) the pattern becomes. It is common to set it weaker when you want to print small, as on a business card, and stronger in places that get dirty easily, such as outdoor signs.

Tricks for Reading in Dark Places

The reason a QR code can be read even in a dim room or at night is that the camera automatically adjusts the amount of light it takes in to match the surrounding brightness. In scenes with little light, it tries to take in more light, and in scenes that are too bright, it reduces the amount it takes in. When there is still not enough light, the image becomes dark, the light and dark parts of the pattern become hard to tell apart, and scanning is more likely to fail. In such cases, raising the screen brightness or moving slightly so that light falls on the code will improve things. Helping the camera with its adjustments is the shortcut to a successful scan.

Why It Cannot Be Read When Too Close or Too Far

If you bring your smartphone too close to a QR code, it can actually fail to read it. This is because there is a limit to the distance at which the camera can focus. When it is too close, the camera cannot focus and the pattern becomes blurry. Conversely, when it is too far, each cell appears too small and the camera can no longer tell light from dark. When scanning does not go well, you adjust by moving the smartphone closer or farther away because you are searching for the distance at which the camera can focus just right. Finding a suitable distance leads to quick scanning.

Why It Can Be Read Faster Than a Barcode

The barcodes used at supermarket checkouts hold information only in the horizontal direction. It is one-dimensional information. On the other hand, the square black-and-white pattern is a "two-dimensional code" that holds information in two directions, vertical and horizontal. The amount of information it can store in the same area is greater by orders of magnitude, and while a barcode holds about 20 characters at most, a two-dimensional code can store up to 7,089 characters.

The reason the scanning speed is fast is that, thanks to the three markers, "the position can be identified in an instant from any angle." A barcode is read by sweeping a laser horizontally, so the angle needs to be aligned. A two-dimensional code is captured as a whole at once by the camera, so it can be read even at a slant or upside down.