Zooming in on Pixelgate
Prosecutors in the Kyle Rittenhouse homicide trial introduced 11th-hour evidence on Friday, November 12 that some argue involves doctored video footage.
The question comes down to whether enhancing photos or videos for clarity modifies the data in a way that is unfaithful to the original.
With jurors absent on Friday, prosecutors Thomas Binger and James Kraus fought to move into evidence drone footage that attempts to enhance a (still incredibly blurry) video of Rittenhouse and two individuals—Joshua Ziminski and Joseph Rosenbaum—immediately prior to the chase that left Rosenbaum dead after he (according to other video evidence) pursued Rittenhouse and lunged for his gun.
Ground video shows Rittenhouse, his rifle in a “low ready” position, retreating from Ziminski, who is armed, after allegedly receiving verbal threats from Ziminski and his girlfriend. A flurry of activity ensues, in which Rosenbaum (who had allegedly threatened to kill Rittenhouse twice that night) pursues Rittenhouse. Ziminski fires his weapon, Rosenbaum reaches for Rittenhouse’s gun, and Rittenhouse fires three shots at Rosenbaum, killing him.
In the drone footage taken in the moments immediately prior to the chase, it’s possible to discern four Rittenhouse, Rosenbaum, Ziminski, and Ziminski’s-girlfriend-shaped blobs. The Rittenhouse-shaped blob is identifiable partly by the outline of his AR-15, attached to a chest strap.
In both the original and enhanced drone footage, the rifle’s position is unclear: we can only identify it based on the fact that in every photo or video in which Rittenhouse appears carrying the weapon—including the ground footage that picks up fractions of a second after the drone footage leaves off—the high point of Rittenhouse’s chest strap is his left shoulder, and Rittenhouse holds the rifle at low-ready, prepared to fire it right-handed.
In full disclosure, I see no tangible distinction between the original and ‘enhanced’ drone footage, meaning that both are absolutely worthless in isolation. Unless you have multiple people gesticulating wildly toward the various blobs in the video and swearing that this is Rittenhouse’s gun and that’s Rittenhouse’s arm and that’s Ziminski’s gun, you can’t even really tell that the drone footage involves human beings.
In fact, this very process of wild prosecutorial gesticulation played out in the Kenosha County courtroom on Friday, with Rittenhouse bracing his hands on a table feet away from Judge Bruce Schroeder, who had moved down from the bench and looked on, squinting, as Binger and Kraus did their damnedest to convince Schroeder that the enhanced footage showed Rittenhouse raising his gun at Ziminski, thus provoking the chase and nullifying his right to self-defense.
One problem with this theory is that, in order for it to be correct, Rittenhouse would have had to remove his harness, position it across the opposite side of his body, point the barrel of his rifle at high-ready, and brandish the weapon left-handed to threaten Ziminski; then—in the nanoseconds-long period between which the enhanced drone footage leaves off and the ground footage picks up—he would have needed to remove his harness again, position it back across the other side of his body, point the barrel of his rifle at low-ready, and hold the weapon right-handed—all while running away from a man with a gun and another man actively pursuing him.
As a juror, I would have to concede that, regardless of what the prosecutors or defense attorneys tried to tell me about this footage, a series of blobs do not remove reasonable doubt. The greater problem, legal sources have since alleged, is that this footage should never have been admitted into evidence to begin with, because it’s “doctored”.
Since the defense did not make this argument well when they introduced it, I was initially skeptical of their claim that enhancing a photo or video for clarity “adds pixels” using algorithmic guesswork. (It didn’t help that the defense kept saying “logarithm” instead of “algorithm”.) A quick dip into the technology rabbit hole, however, proved more edifying.
Simply put, enhancing an image adds data that weren’t there before. Usually, the implications of that are pretty inconsequential: if you take a photo of a sunset and want to enhance its definition, the AI in your phone or computer do a pretty good job of guessing that a particular stretch of pink sky is going to be composed of various gradations of pink. To get those more precise gradations, however, an AI algorithm breaks up the original pixels and, between them, generates new pixels whose colors and textures are midway between the colors and textures of the pixels around them. Those new pixels were not captured when the photograph was taken. They weren’t generated by the event, but by calculations based on the image of the event. They’re artificial data, inserted after the fact.
Likewise, if you’re analyzing drone footage of a nighttime street scene and want to enhance the definition of what’s either the (dark) barrel of a (dark) gun or the (dark) strap of a (dark) harness slung around a raised shoulder—set against a dark backdrop—your algorithm is going to generate data that will manufacture color and texture changes that did not exist in the original photo, and were not captured on the night in question.
By the way, this process is called “bicubic interpolation.” This is hilarious to me and exactly no one else, because it reminds me of Louis Althusser’s theory of interpellation, whereby ideological state apparatuses reconstruct your perceptions of identity and subjectivity by “concretizing” them through rituals of recognition, or “hailing”. Althusser illustrates this with an example of a police officer who shouts “hey, you there!” at a passerby, who reflexively turns around, affirming the recognition of his private, individual “self” as a concrete state subject. Althusser would be fascinated—or mortified—to know that using bicubic interpolation to enhance FBI drone footage has introduced us to an entirely new dimension of artificial subjectivity. With bicubic interpolation, state surveillance apparatuses can shout “hey, you there!” even if you’re not quite there, inserting new pixels to hail blurry individuals as concrete subjects.
I thought this joke was worth it, and I stand by that assessment.
All of this is to say that digitally enhanced footage is not a replica of the original footage. If you were to ask an expert witness whether the enhanced drone video is a true and accurate representation of the original footage or the event captured, their answer would be “no”. If you were to ask that witness whether the video contains images and data that did not exist the night the video was taken, their answer would be “yes”. This is perhaps why, when prompted by the judge and after saying they would do so, the prosecution ultimately did not summon any expert witnesses to testify to the validity of the enhanced footage.
Although the time to call new witnesses has passed for both the prosecution and the defense, the defense can certainly report on the credibility of the new evidence during their closing arguments. When they do, it’s best to keep it simple: all the jury needs to know is that the enhancement process adds data that were not present during, and do not perfectly replicate, the events captured by the original footage. The defense can also certainly use side-by-side footage comparing the still-difficult-to-discern drone video with the ground footage that follows immediately afterward and explain the impossibility of the gymnastics Rittenhouse would have to perform to shift from a left-handed up-ready stance to a right-handed down-ready one in that time frame (while running and removing and resituating his harness twice).
Even if Rittenhouse had managed that maneuver, however, what he did next completely nullifies the prosecution’s “provocation” claim: he ran. Under Wisconsin law, if a defendant provokes a violent encounter and stays to fight, they surrender their right to self-defense with a deadly weapon. If, on the other hand, they provoke, retreat, and are pursued, they reacquire that right.
In other words, the prosecution’s case against Kyle Rittenhouse rests on their interpretation of an interpolated pixel—and even then, it’s legally moot.
