Google's Attempt at Fixing Autoplay Videos Has Broken Countless Games


#1

“I cannot think of a comparable moment where one team of developers killed so much internet culture in one unilateral move.” That’s how Bennett Foddy, designer behind QWOP and Getting Over It, tried to describe how a seemingly minor quality-of-life fix for the Chrome web browser has had a potentially devastating impact on games.

Here’s what happened. Over the weekend, Google pushed out a series of changes to its enormously popular Chrome web browser, hoping to fix one of the Internet’s great annoyances: autoplaying videos with sound. A noble quest, but one with unexpected consequences; as soon as the feature went live, it broke all kinds of web content that relies on audio, including games. A key component of those games—sound—now doesn’t work, unless the designers intervene.

In trying to address the legitimate problem of autoplaying videos—a shitty marketing and advertising tactic to trick you into “viewing” a video—Google messed with and changed the open standards all sorts of other developers had been working with. Not every website has been banned from autoplaying videos, however; certain websites have been whitelisted, users can opt-in, and there’s a fuzzy metric whereby if someone’s “Media Engagement Index threshold has been crossed,” videos start autoplaying again.

“Currently, the product I am developing that has tens of thousands of paying customers where audio playback is a core feature is broken,” wrote one developer in the comments section of Google’s announcement.

The damage isn’t yet known, but that’s largely because the impact has been scattershot, and it will take time before developers realize if Google’s change actually impacted their work. In a statement to The Daily Dot, Google suggested the onus was on developers to keep up.

“With Chrome’s new autoplay policies, developers shouldn’t assume that audio can be played before a user gesture,” the company said. “With gaming in Chrome, this may affect Web Audio. We have shared details on how developers can do to address this, and the design for the policy was published last year.”

The fact that so few developers were aware remains on Google. It also suggests the company doesn’t have meaningful plans to address the issue facing game creators.

“We are losing a substantial amount of our cultural heritage, just because people find it annoying when ads play audio,” said Foddy, “and the Chrome folks couldn’t find a better workaround, like a browser mute setting or a code solution that respects the ways that existing games and art projects already have ‘click to play’ implemented.”

(“Click to play” means the user clicks on the game, letting it take control of your computer.)

It’s hard to argue Google’s wrong for trying to eliminate autoplaying videos, but in failing adequately research how audio/visual artists deploy their work on the web, they’ve hurt bystanders. Based on the way Google positioned their announcement, it’s possible they may not have even considered what impact this might have on games and other digital artists.

Isaac Cohen, a designer and artist who does experimental work on the web, woke up this weekend to discover “pretty much all of my online work is broken.”

He learned something when wrong when he tried to send one of his interactive speeches, something he’s given at places like Pixar, to a friend, and it wasn’t working. After digging around the code for a solution, he saw a friend post on Facebook about what’d changed.

“It is 'just the audio' that has broken, as a few Twitter users pointed out,” said Cohen, “However, when your pieces are specifically audio-visual, it no longer becomes just the audio, and rather destroys your entire piece.”

One part of the presentation, for example, is color defined by audio. It requires audio.

Thankfully, it does appear Cohen has found a relatively easy solution to make his online work functionable, but for a while, Cohen had made peace with saying goodbye.

“I think if you are making work because you want it to last forever, it's just another way from running from the fact that all things end,” said Cohen. “It's obviously tragic when it does end, and we want it to last as long as it can, but clinging onto a thing, instead of moving forward, is something that would probably destroy me (or at the very least my ability to create new content). I also can't imagine what it was like to watch the death of Flash.”

Flash is probably the last time web creators ran into a similar bloodbath. Flash, a bloated but serviceable way of letting people play games online, was due to die eventually. I wrote about that moment for Kotaku in 2015. Flash games are where designers like Edmund McMillen and others cut their teeth. If you enjoy their work now, you have Flash to thank. But at least with Flash, its transition into the Internet trash bin was over “security concerns,” even if it was mostly bullshit, a convenient cover for Apple declaring Flash too slow for the iPhone.

“The Flash thing is not comparable because it’s been such a slow change, and because it is based on a consensus, and because Flash was never an open standard,” said Foddy. “In this case the Chrome team is killing thousands upon thousands of art and game websites that they have no ownership of whatsoever. There is no good comparison.”

(Flash was made and maintained by a third party, Adobe.)

The real damage won’t be done to games that can be fixed, though, it’s the countless games that have been abandoned by their creators, or the underlying technology simply can’t be fixed with tinkering the code. It’s possible they may forever be in a semi-broken state.

“What you're looking at is a complicated and time-consuming testing period for any single audiovisual work on the web right now,” said Andi McClure, designer of numerous web curiosities, including Anti and 4 Cubes. “Those are the works that are going to get permanently broken or removed in response to these changes, and that's what frustrates me.”

Lots of web games are built using third-party tools, McClure pointed out, and there’s no assurance those companies will get around to addressing Google’s changes. Even if they do, it’s not as simple as rolling out an update for your game. The moment you tinker with the code, something else might break. If your game is based in Unity, for example, upgrading to a newer version of Unity might end up breaking older parts of your game. So, you’re stuck.

Games have always faced problems over being so closely tied to the progress of technology. Combined with gaming’s commercial desire to look forward, not back, and you have a medium that shows depressingly little interest in remembering, preserving its own history.

McClure doesn’t have high hopes for what will change as a result of this. Even if Google manages to clean this up, the same dynamics that serve the powerful will come back.

“Nothing,” said McClure. “Same thing as when Flash got killed, or when the iPhone App Store mass deprecates old games. A bunch of artists most people haven't heard of get really frustrated. Media consumers look at new stuff and don't realize there's a block of several years worth of people's work that isn't there anymore. Nothing changes. The web is a commercial juggernaut and it exists for MSNBC, not me or any other small artist.”

Follow Patrick on Twitter. If you have a tip or a story idea, drop him an email: patrick.klepek@vice.com.

Have thoughts? Swing by Waypoints forums to share them!


This is a companion discussion topic for the original entry at https://waypoint.vice.com/en_us/article/xwmqdk/googles-attempt-at-fixing-autoplay-videos-has-broken-countless-games

#2

Not every website has been banned from autoplaying videos, however; certain websites have been whitelisted, users can opt-in, and there’s a fuzzy metric whereby if someone’s “Media Engagement Index threshold has been crossed,” videos start autoplaying again.

i feel like this bit wasn’t emphasized enough. google, the creator of the world’s most popular web browser, gets to casually sweep entire chunks of culture into the bin while blaming developers who might not be able to do anything (or, indeed, might not even be alive, because art is supposed to last long after one’s existence), but they set up something that still ensures that their auto-playing video website gets to work just fine, even if nobody else’s can?

this is the COI to end all COIs. no matter what, anything else they say on this subject should be listened while keeping in mind that they have the ability to choose what sites other than theirs should be allowed to have sound on by default, and we’re expected to just trust them on that


#3

so… Google’s changes amount to recognizing a pre-existing API convention that people didn’t pay attention to? The worst!

Also, before you get your knickers in a twist over Google making decisions on who gets on the autoplay whitelist, do yourself a favor and read about how it actually works. It’s based on your browsing habits, not what Google thinks.


#4

I know this point is tangential to the article, but Flash’s security problems were so, so far from bullshit. Check out this list of vulnerabilities in Flash. Every arbitrary code execution vulnerability on that list could allow any web page you visit to compromise your computer, letting an attacker install software that could track your activity, steal your information and accounts, or even hold your files for ransom. Until browsers moved Flash to click-to-play as part of the deprecation process it was one of the biggest computer security risks any user faced.

Look, I like Flash culture a lot. It was a powerful tool that enabled a lot of fantastic work, and I’m watching efforts to save that work with a degree of anxiety. Adobe’s lack of a preservation plan now that Flash isn’t profitable is terrible. But Flash as a default part of the web platform really, really needed to go.


#5

except people did pay attention, they asked google for clarification that never came for months, and then they quietly added “oh btw we’re muting everything” with zero announcement at all.

…which ignores the more important part, which is that nobody should have to do any of this garbage. this doesn’t happen anywhere else except for games, because rather than actual pieces of art that can stand the test of time, games are treated in the same way my fucking word processor is treated, where it must be updated at all times, and if someone else breaks it, it’s somehow the creator’s fault.

if i make a video, i can get that video back ten years later and still play it back with ease. if i write a book, i don’t have to get worried that my BookOS will update and render it unreadable. if i made a game online, that game is going to be unrecognizable within ten years, because games aren’t art to browser developers, they’re web apps, and if they break, then it’s just up to them to fix it, because actually it’s now your fault i smashed your sculpture with a hammer


#6

Firefox Quantum kicks so much ass

please, everybody

just use firefox now. really. it’s great

Look for real, if users are frustrated that Google - an advertising company - is unilaterally deciding what kinds of advertisements should and should not work on the internet, thereby dictating its own terms in how it may best profit from its most profitable (by a massive fucking margin) business venture and steamrolling everyone else tangentially affected by these business decisions, consider using a different browser.


#7

Google actually changed the API spec as part of this move. The current draft of the Web Audio API spec has this paragraph:

An AudioContext is said to be allowed to start if the user agent and the system allow audio output in the current context. In other words, if the AudioContext control thread state is allowed to transition from suspended to running.

That text is absent from the previous version of the spec. But that earlier version had this to say about the AudioContextState of an AudioContext:

The context state MUST begin in “suspended”, and transitions to “running” when system resources are acquired and audio has begun processing.

A reasonable reading of this version of the spec is that the state is “running” as soon as the browser has initialized the audio playback system. In fact, I would argue that this version of the spec requires that autoplay audio is allowed, and this change to Chrome is a backwards-incompatible violation of the previous draft, which had been stable for years.


#8

Unpopular opinion: Destroying all of this art would be worth it if it meant I never have another video autoplay when I don’t want it to ever again.

It’s very sad, and I feel for the artists, but I would be more than happy to take an absolute scorched earth approach to this.

Unfortunately, that’s not what’s going on. This is just another fairly transparent attempt for Google to leverage their market position to promote their own services. I hope they get smacked down with antitrust suits just like Microsoft did in the 00s

The problem with this, is having checked my MEI scores at chrome://media-engagement/ literally only Youtube gets marked as Yes.

Not Giantbomb, not iPlayer, not My5 or TVcatchup or any of the other dozen sites I go to almost exclusively to watch video. Just Youtube.

It’s blatantly a play to promote Youtube at the expense of other services.


#9

lol, have you watched a movie on your Cinematograph lately? Mediums change all the time, and the art that was recorded in that media is either adapted, or lost. Art isn’t some cultural constant that stands outside of commerce or technology, it is often quite contingent on these things. Unless you are into happenings

Meanwhile, the number of people whose experience on the web has been improved, both in terms of quality of life and data consumption, by the acknowledgement that media should require consent before playback far out-numbers the amount of artists who lost their audience due to changes in the open platform (API is controlled by W3C non-profit, with engineers from Mozilla and Google co-signing the recent API edit.)


#10

We’re not talking about the march of progress replacing old technologies with better ones. This is a change by one vendor to their implementation of a technology. If you want a movie analogy, it’s like if Sony pushed an update to all their Blu-Ray players so that most discs pressed before this year would play back without sound.

Writing automatic media playback into web standards was an awful, short-sighted, user-hostile decision. But Chrome is a platform used by millions of developers, and Google has a responsibility to those developers to be cautious not to haphazardly break working code. Google’s communication on this issue has been terrible from beginning to end. This post on Twitter from @mcclure111 let me know that the initial announcement last September only described the changes to audio and video elements. Requests for clarification about how WebAudio would be affected went unanswered. Google revealed that WebAudio would be affected in a silent edit to the months-old blog post in February of this year. 3 months isn’t nearly long enough to go from hidden announcement to breaking change. Especially when there are plenty of UI solutions to mitigate the breakage that they decided not to implement, as detailed in @mcclure111’s post linked above.


#11

Meanwhile, the number of people whose experience on the web has been improved, both in terms of quality of life and data consumption, by the acknowledgement that media should require consent before playback far out-numbers the amount of artists who lost their audience due to changes in the open platform (API is controlled by W3C non-profit, with engineers from Mozilla and Google co-signing the recent API edit.)

it’s okay if we burn an entire art museum to the ground, because you see, passersby didn’t like looking at advertisements, and that means it’s actually the museum’s fault! fantastic, glad we cleared this up


#12

I’m not familiar with the tech behind Google’s updates, so maybe this question is naive. But is this the sort of thing that a browser extension could fix for select webpages? It would be an imperfect solution, but then developers and publishers could place a message on their sites directing Chrome users to install the extension and be able to access the games. Google seems pretty opaque and slow-moving when it comes to fixing problems for audiences it sees as niche. Could be one work-around.


#13

There’s already code for a workaround, and it wouldn’t be too tricky to wrap that code in a Chrome extension. What it does is add an “unmute” button to a page that the user can click to enable WebAudio playback. That’s a fairly clumsy solution, though, and it could be handled much more elegantly if Google would add a UI for controlling autoplay preference directly in Chrome.


#14

Yo Adobe is a really gross company when it comes to security and I would strongly suggest looking more into that kind of thing before doing air quotes around “security concerns”.

For a recent example see NTLM Credentials Theft via PDF Files

Just a few days after it was reported that malicious actors can exploit a vulnerability in MS outlook using OLE to steal a Windows user’s NTLM hashes, the Check Point research team can also reveal that NTLM hash leaks can also be achieved via PDF files with no user interaction or exploitation.

Adobe responded as shown below, without assigning any CVE or fix for the vulnerability:
“Thank you for checking in on this case. Microsoft issued an optional security enhancement late last year that provides customers with the ability to disable NTLM SSO authentication as a method for public resources. With this mitigation available to customers, we are not planning to make changes in Acrobat.“

Emphasis is mine on that “optional security enhancement”.

In other words Adobe knows there is an exploit that can be achieved with PDF files and their “solution” is to quietly say that you that you should go download a patch from Microsoft and it really isn’t their concern despite the fact that PDFs shouldn’t be able to do that in the first place.


#16

lol https://bugs.chromium.org/p/chromium/issues/detail?id=840866