Why Does Color Blindness Need a Real-Time Fix, Not a Filter?

Because reading a map, a transit schedule, or a cooked-meat color cue can't wait for a photo-editor round-trip — roughly 300 million people need correction as they look at the world, not after they've photographed it.

Roughly 8% of men and 0.5% of women worldwide experience some form of color vision deficiency (CVD). That translates to around 300 million people who struggle to distinguish certain colors in their daily lives. Red-green confusion is the most common, but blue-yellow deficiency exists too.

Most existing tools are photo editors. You snap a picture, the app processes it, and you see a corrected version a few seconds later. That works for identifying a shirt color, but it doesn't help when you're navigating a map, reading a transit schedule, or trying to tell if meat is cooked. For those situations, you need real-time correction — and that means running the correction algorithm on every single camera frame.

How Does Daltonization Actually Work?

Three steps: simulate what a color-blind viewer sees, compute which color information that simulation lost, then shift the lost information into a part of the spectrum they can still perceive.

The daltonization algorithm is a three-step process that's been well-studied in vision science since the early 2000s. The idea is straightforward: simulate what a color-blind person sees, figure out what color information is lost, and then shift the lost information into a part of the spectrum they can perceive.

The first step is converting from RGB to LMS color space. LMS represents the response of the three types of cone cells in the human eye: Long (red), Medium (green), and Short (blue). This is a linear transformation — a 3x3 matrix multiplication.

Next, we apply a simulation matrix that models the specific deficiency. For protanopia (reduced red sensitivity), we zero out the L channel and reconstruct it from M and S. For deuteranopia (reduced green sensitivity), we do the same with the M channel. Tritanopia affects the S channel.

Finally, we compute the difference between the original and simulated colors, and add that difference back into channels the viewer can perceive. A protanope can't see the red difference, but they can see it if we shift it into blue-green.

Why Does Daltonization Belong on the GPU, Not the CPU?

Because every pixel is processed independently with no dependency on its neighbors — embarrassingly parallel work is exactly what a GPU fragment shader is built for, and it costs the CPU nothing.

The key insight is that daltonization is embarrassingly parallel. Every pixel is processed independently, with no dependency on its neighbors. That makes it a perfect fit for a GPU fragment shader.

Our GLSL shader runs on every pixel of every camera frame. The core looks something like this: 

// Convert RGB to LMS
vec3 lms = rgb2lms * color.rgb;

// Simulate color deficiency
vec3 simulated = sim_matrix * lms;

// Convert back to RGB
vec3 sim_rgb = lms2rgb * simulated;

// Compute error and shift
vec3 error = color.rgb - sim_rgb;
vec3 correction = err_matrix * error;

// Apply correction
gl_FragColor = vec4(color.rgb + correction, 1.0);

The three matrices — rgb2lms, sim_matrix, and err_matrix — are different for each type of color vision deficiency. We pass the appropriate set as uniforms to the shader based on the user's selected correction mode.

The beauty of running this on the GPU is that it's free from the CPU's perspective. The fragment shader processes millions of pixels in parallel while the CPU handles camera management and UI rendering without breaking a sweat.

Why Is Feeding Camera Frames to the Shader the Hard Part?

Because iOS and Android hand you completely different pixel formats — BGRA is a trivial channel swizzle, but Android's YUV420 needs a real conversion step before it can reach an RGB-expecting shader.

Writing the shader is the easy part. Getting camera frames into the GPU in the right format is where the real complexity lives — especially on a cross-platform Flutter app.

On iOS, the camera plugin delivers frames in BGRA format. This is essentially RGB with the channels reordered and an alpha channel. Converting BGRA to RGB in a shader is trivial — just swizzle the channels. iOS was straightforward.

On Android, the camera delivers frames in YUV420 format. YUV420 is a completely different encoding — luminance (Y) in one plane, chrominance (U and V) in separate, subsampled planes. You can't just feed YUV420 into a shader that expects RGB. The data layout is different, the planes are different sizes, and the math to convert is non-trivial.

We use the yuv_converter package to handle the YUV-to-RGB conversion on Android before passing the frame to our daltonization shader. This adds a conversion step, but the GPU handles it efficiently enough that we stay within our frame budget.

Does the Shader Actually Hit 60fps in Practice?

Yes on modern hardware — under 2ms per frame on an iPhone 12 and newer with zero dropped frames; mid-range Android lands 30-60fps depending on the GPU, and the bottleneck is always camera frame delivery, never the shader math.

The target is 60 frames per second on modern devices. Each frame at 1080p has roughly 2 million pixels, and each pixel needs two matrix multiplications, a subtraction, another matrix multiplication, and an addition. That's a lot of math — but GPUs are designed for exactly this kind of workload.

On an iPhone 12 and newer, we consistently hit 60fps with no dropped frames. On mid-range Android devices, we typically see 30-60fps depending on the GPU. The shader itself runs in under 2ms per frame on most hardware. The bottleneck, when there is one, is always the camera frame delivery — never the shader computation.

We avoid any CPU-side pixel manipulation. The camera frame goes straight to a GPU texture, the shader processes it, and the result renders to screen. Zero copies through CPU memory on the hot path.

Does One Shader Handle All Types of Color Blindness?

Yes — protanopia, deuteranopia, and tritanopia share the exact same shader structure; only the simulation and error-correction matrices swap via uniforms, with no shader recompilation needed.

Color Vision Aid supports all three major types of color vision deficiency:

  • Protanopia — reduced sensitivity to red light. The most common form. The simulation matrix zeros the L cone response.
  • Deuteranopia — reduced sensitivity to green light. Nearly as common as protanopia. The simulation matrix zeros the M cone response.
  • Tritanopia — reduced sensitivity to blue light. Much rarer. The simulation matrix zeros the S cone response.

Each type uses a different set of simulation and error-correction matrices, but the shader structure is identical. We swap matrices via uniforms — no shader recompilation needed when the user switches modes.

What Does the Corrected Image Actually Look Like?

Subtle to normal vision but immediately noticeable to CVD users — a protanope can suddenly distinguish red from green, a deuteranope can read color-coded charts, a tritanope can tell blue from yellow.

The end result is a camera viewfinder that shows the world with corrected colors in real time. A protanope who can't distinguish red from green sees the difference clearly through the app. A deuteranope can read color-coded charts. A tritanope can tell blue from yellow.

This isn't a filter or an overlay. It's a mathematically rigorous correction that shifts lost color information into visible channels. The correction is subtle — the image doesn't look "wrong" to a person with normal vision — but to someone with CVD, the difference is immediately noticeable.

Color Vision Aid is free on the App Store, with no ads and no tracking. It's built to help people see, and the GPU does all the heavy lifting.