Accessibility hacks! adventures in hands-free gaming

I’ve been distracted from figuring out first and third person games after getting into Caves of Qud recently. I had a bit of a minor breakthrough playing it this afternoon with Talon and the dance pad, so I figured that’s an excuse to post here.

I had never played CoQ before, but it’s been on my radar ever since it came up on 3MA ages ago. A dance pad is basically just a big numpad so I decided to give it a try. You could play the whole thing with Talon alone, since there’s big user interface hints for every keyboard shortcut so it’s very easy play out of the box with Talon’s phonetic alphabet. In fact the word for the letter L in the alphabet is “look” which is just perfect, since that’s the command in CoQ to look around the world and read descriptions. But it’s so easy to bind a new word or phrase to a key press in Talon that was easy to quickly build up a file full of commands that match the actions in the game, here’s a sample, to give you an idea how easy it is. This just goes in a plain text file in Talon’s user directory and it gets picked up automatically

overlay on: key(alt:down)
overlay off: key(alt:up)
abilities: key(a)
move: key(w)
character: key(x)
inventory: key(i)
equipment: key(e)
quests: key(q)
tinker: key(k)
powers: key(p)

But what I’m doing is combining it with the dance pad bound to the numpad keys, and using that for movement and menu navigation. You can just tap, or in my case step, repeatedly in the direction you want to move but there’s also an automatic movement mode. That works by first pressing the walk key, and then pressing on the numpad the direction you want to move, after which your character will proceed in that direction until running into a wall, seeing an enemy, or until you press the walk key again. So I bound that key to the word “move” (instead of “walk” because that gets confused for “look”) and have been roaming around learning the game with that.

But today I had an idea after seeing some discussion on the Talon Slack. It turns out that it’s not too hard to turn Talon’s pop to click into a context-sensitive action. I’ve set up my Talon such that in every application except for CoQ the pop noise is bound to mouse clicks (or activating the eye tracker zoom mouse mode if the eye tracker is on). But in CoQ the pop is bound to the W key. So now I can pop, then step on the pad to pick a direction, and pop again to stop moving.

(for posterity here’s the code: 1 2)

It’s pretty minor victory but I was still excited after figuring it out. It’s really amazing the difference that having a common action bound to a low latency noise makes. And of course this is a testament to the flexibility of Talon. I think the dev was very wise to design it such that even though there is a closed source core at the heart of the system, everything else is hackable

The rest of this post touches a little bit on the medical side of my situation and others like it, so I’m going to put it behind a spoiler tag. Content warning for discussions of chronic pain and related issues

This experience reminded me of this talk by Naomi Saphra that I watched a few weeks ago: https://www.youtube.com/watch?v=BKYWt8B9hgs

The whole talk is one I highly recommend for anyone interested in this subject and I think the Waypoint community would appreciate the attitude that Naomi brings to the topic, to put it not so lightly the talk has real “fuck capitalism, go home” energy.

In the talk they discuss Talon specifically and relate it to several situations where other communities of disabled folks have had to come up with their own solutions because they weren’t being served well by existing institutions. However one bit specifically stood out to me and I thought of it this afternoon:

When you’re in this much pain it is tempting to think of yourself as in a waiting room, that the pain is something you just have to wait out until a doctor fixes you and you can resume your life. I don’t care if you find that miracle cure, there’s no resuming your life because this is your life, this whole bit that you’re living in. So do not let a year go by just trapped in the metaphorical waiting room. It took me a long time to realize that it was time for me to do things that made my life enjoyable in the moment like learning to dictate code, rather than waiting for someone to fix me.

To be honest hearing this brought me to tears. My condition is not even so severe as some people I know, but it has not been easy. However it is easy into falling into this trap of the waiting room. A minor victory like figuring out how to make it easier to play Caves of Qud can sometimes feel a bit silly but this quote reminds me that there’s value in this kind of tinkering.

I would be remiss if I didn’t include a link to this article from the verge, which was published on Friday: When games are hard on their hands, some players turn their voices into controllers - The Verge It’s not really specifically related to anything in this post, but the developer behind Parrot.py was interviewed and mentioned Talon. There’s a lot of great stuff in this article and some of the other pieces in the series seem pretty interesting to

7 Likes

I haven’t really been playing that many video games recently, although I did find a neat hack to make Shadow Tactics easier to play hands free, which I’ll repost here for posterity:

I’ve spent the past year playing a lot of Go online, probably my difficulty playing video games contributed to my renewed interest in that game. but I don’t actually play hands free, I just use a touchscreen tablet. Playing with a head mouse or an eye tracker is just a little bit too stressful, I am already someone who has trouble with ladder anxiety, so adding in a big risk of misclicking it is not a good recipe

Reviewing games afterwards is a big part of Go culture, but that’s kind of a pain the ass to do on a tablet. So I made a custom Talon mode which allows me to step forward and backward through a game using the “pop” sound, while still allowing me to switch back to the zoom mouse mode.

It’s not really that exciting, just another example of Talon’s power and flexibility

2 Likes

content warning for chronic illness, ableism and existential dread I guess?

the “waiting room” metaphor reminded me of this article about chronic fatigue by Elizabeth K. Hinnant, particularly this bit:

When I first got sick, the negative-trait interpretation of my illness would have fit like a glove. So many of the things I loved and identified with—exploring new cities on foot, riding my bike to work, working in an office—became impossible due to my reduced stamina and strength. But once I realized there was no quick fix for my severe energy limitations, I knew I had to find a way to build a life for myself. I had to learn to exist in this body as a complete person, not a “regular” person with reduced stats.

It’s hard to explain in a world full of disability narratives dominated by limitation and loss, but even though I spend 23 hours of most days in bed, I’m not missing out on life. This is my life. It’s a part of the human experience, even if it’s considered less valuable by a system that demands 110% of our energy in exchange for survival.

this also hit me pretty hard emotionally. I think that part of Saphra’s talk you shared sorta extends this idea to include “temporary” conditions.

it’s often a confusing matter to me. I feel I’m not just sitting in the “waiting room”, I’m being held prisoner in it. Others put me in this space, locked the door, and labeled it “waiting room”. I guess I believe in that label sometimes. I still think I deserve more space, or better spaces, so I might ask for help or fight to escape. But I’m gonna be here for a while. This is where I live, at least right now. So I may also reject the “waiting room” label and try making this space my own.

then some douche will come by and say “if you don’t call this a waiting room, why are you complaining about being locked in a waiting room?” or maybe “idk looks to me like you’re doing fine” or “this other person lives in a waiting room and they are doing fine”. Or maybe “poor guy, he lives in a waiting room”. Then I will get pissed because they aren’t helping me unlock the door and they will be like “wow stop censoring/harassing us” or something. idk I’ve stretched the metaphor waaay too far but hopefully this makes some sense lol

3 Likes

On the subject of Roguelikes, there was a talk at Roguelike Celebration by Alexei Pepers about Roguelike accessibility for visually impaired players, and making NetHack more accessible. While it’s not super relevant to hands-free stuff, I think it’s really neat. I think the fact most RLs are open-source opens them up to these hacks; according to Pepers, NetHack’s code was already full of useful stuff that they were able to use.

(I am using all of my willpower resisting the urge to gush about Caves of Qud)

6 Likes

Thanks for reminding me of that article, it’s a really thought provoking one

It’s a cool game! I kinda stopped playing it because dragging out the DDR pad is a bit of a chore, but I want to revisit it.

For what it’s worth, the current build has some mouse support, so moving and stuff is probably manageable through that if you can use a mouse to click. I’d also be curious if there’s much you can do with the keybinds; the game lets you set up keybinds for specific abilities. I’m curious if that could be expanded to even more options. Also worth mentioning the game also has an auto-explore button, bound to numpad 0 by default!

Also, worth mentioning: Crypt of the Necrodancer is literally designed to be played on a dancepad! :smiley:

Something I’m also wondering now is how playable Dwarf Fortress would be, and what you could do to improve it. The most frustrating part would probably be the cursor stuff, but since the game can be controlled entirely with key commands, I imagine it would be pretty manageable without even hacking. You could probably set it up to the point that you could set up commands like “build carpenter’s workshop”. I bet you could get to the point that the voice commands are easier to use than using the keys!

1 Like

I don’t have anything really new to post about, but I want to spend some time to praise a couple of games I’ve been playing, and highlight some things about them that make them more accessible for me.

navel gazing background

A lot of my experience figuring out how to play games hands-free has been frustrating. I’m neither a game developer nor an accessibility expert, so I’m sure that I underestimate the work involved in designing a game’s user interface to be accessible. The frustration usually comes from the desire to play a game and being unable to for whatever reason. That frustration sometimes leads me to feel some animosity towards the developers, or even to the entire industry and art form of video games themselves. I hope those feelings haven’t soured this thread too much, hopefully I’ve been able to keep them to myself. But that’s one of the reasons I want to make this post, to counteract some of those feelings with something more positive

First, let’s talk about what makes Invisible, Inc.'s UI so great.

This is a game I’ve been playing with my head mouse (still the same Remote+ from the first post, rubber banded to headphones. the hair clip sees less use these days) along with foot pedals for right and left clicking (Kinesis Savant Elite2, relatively expensive but I mostly recommend these). After all the things I’ve tried, this combination is still the most convenient, because I can just pick it up and play for a little bit without setting much up, and I don’t have to use my voice.

There’s two things about the camera controls that make this game stand out. The first one is that you can disable edge panning in the game’s settings, by zero-ing out the values:

Invisible Inc is not the only game I’ve seen with this setting, but it’s surprising how few strategy games have this as an option, I would guess easily half or more of the games I’ve played don’t support disabling it. (A while ago I wrote about why edge panning is so annoying while using a head mouse. I can’t remember if I ever posted about this bit of C# code I found that works as an external workaround to disable it in many games, unfortunately it does not work universally)

The second thing Invisible Inc gets right is that it allows you to hold the left mouse button down and drag the camera around. This is another thing that some other games support in one form or another (often with the middle mouse button) but again it’s surprising but this is not more common. It’s especially surprising to me after thinking about it more because this is exactly how Google Maps and most other online maps work. Left-clicking still selects your characters, but if you left click and hold any empty space on the map, you can drag the camera.

In my opinion these two things should be industry-standard in any of these isometric, turn-based tactical games. I want to shout out Cantata, the demo of which supports dragging the camera around and got me thinking about this, before I started playing Invisible Inc again.

A few of the lesser things Invisible Inc gets right, but I still want to shout out, are putting all of these things on the screen at all times:

This is definitely less noteworthy than the camera controls, since most turn-based tactical games follow a similar pattern. It’s still worth highlighting because I think sometimes it’s small, seemingly minor things that make a game more accessible.

Invisible Inc also has these context-sensitive menus that will show up depending on where your characters are located in the environment:

I think this is just great design, in addition to being more accessible for pure mouse driven controls.

All of this stuff adds up to Invisible Inc being eminently more playable for me then some of the other games in the genre. That’s not to say it’s not without its faults…

nit picking

There are two specific issues that I want to mention, the prevent it from being flawless (at least with respect to being playable with this particular combination of hardware)

One problem is the “tactical view” (a stark filter on the camera which makes things easy to decipher) which is turned on by either holding the left mouse button down at the top of the screen or holding down Alt. I think this should be a toggle, or it should be an option to be toggleable. Although I’m not an expert in accessibility, this to me seems like a fairly obvious issue, so it’s an unfortunate oversight. Thankfully this view isn’t critical to playing the game

The other problem pertains to the “action cam” when certain things happen like a guard spotting one of your agents. You can disable the automatic panning in the settings, but the camera will still zoom almost all the way in. This must not be a big deal if you’re using a normal mouse with a scroll wheel, but in my case I need to have Talon running and occasionally issue a “wheel downer” command to reset the zoom.

The only reason I’m including this nit picking section is because after playing this for close to 20 hours over the past few weeks, these issues stand out as really the only complaints I have, which just underscores how well it works otherwise.

The other game I want to talk about briefly is Black Book, a “card battler” (aka Slay-the-Spire-like) which Austin W mentioned on the pod a while back.

Any time a comparison to Slay the Spire is made my ears perk up a little bit, not because I’m super enamored with the genre (don’t get me wrong, I certainly like them) but it’s almost the perfect game for my setup. Besides being turn-based, there’s no scrolling or movement, each battle takes place in a single screen with a fixed viewpoint, and any other interactions like shops, the overworld map are purely mouse driven. Other games in the genre I’ve played, like Griftlands, adopt a similar 2D style.

Black Book was the first game in the genre that I was interested in playing which deviated from this: it’s rendered in a three-dimensional style with a third person perspective, and so scenes like this one from the trailer had me worried before I downloaded the demo. It seemed possible or even very likely that the game was going to require typical third person controls (ie dual stick or WASD) which would have made it more trouble than it’s worth for me.

Thankfully, I was wrong. Black Book does allow you to use WASD to control the character outside of combat, but you can also just move and interact in mode like a point-and-click adventure:

I was genuinely relieved and surprised to discover this. I think it’s telling that that my expectations were so low, speaking to some of my frustrations with other games. It also highlights an issue with Steam: there is often crucial information about a game’s control scheme buried in the user-defined tags system. A game like Black Book might have the tag “mouse only”, but I couldn’t tell you because it’s not present in the list on its Steam page, so either it does not have the tag or the tag is not popular enough to be visible.

I guess the premise in the title of this thread is that it takes hacks to work around limitations of some games, but it’s nice to think about and appreciate when games are actually meeting you more than halfway.

6 Likes

I want to talk about playing Inscryption hands-free, and eventually use that as a jumping-off point to talk a little bit about first person games, specifically figuring out how to play Paradise Killer hands-free.

Regarding Inscryption, everything I’m talking about is shown to the player within the first 15 minutes or so, and the video below was taken right after the tutorial section, so hopefully not spoiler territory. I’m not talking about anything past this initial gameplay, and not talking about anything other than how it controls.

So Inscryption is one of these deck-like rogue-builders, right? It should be right up my alley as far as easy to play hands-free games, as I was talking about in my last post: draw cards, play cards, use items, end turn, click click click, pure head-mouse + foot pedals…

At least that’s what it looked like while I was watching Rob and Natalie play it on Twitch, before I tried the game myself. It turns out I wasn’t paying close enough attention and my preconceived notions about the game immediately ran into this wall (highlighted in pink on the right):

Every time you play a card to the board, you have to use the keyboard to go back to your hand. If you’re looking at your hand and you want to look at the board state, you need to use the keyboard. If you’re looking at the board state and you want to look at your items or draw card you need to use the keyboard…

This posed a challenge to me, because although I can easily use Talon’s radio alphabet to press these keys, I could tell that having to say “whale, sun, whale, sun, …” over and over again every turn was going to be frustrating.

I’m happy with the solution I came up with. I ended up using Talon but based this on a similar feature from Parrot.py that I used to play Hades. I started with the context sensitive “pop” that I got working for Caves of Qud and Go reviews, but added code to use the current mouse cursor position to determine which key to press. That code splits the screen into roughly 6 regions: top and bottom along the vertical axis; left, middle, and right along the horizontal. Then if Talon hears a “pop” and the mouse cursor is in the top middle region, the code will press “w”, if the cursor is anywhere in the right region, press “d”, etc. Because of course Inscryption uses the keyboard for more than just looking up and down, you also get up from the table and walk around:

(In this video I’m saying “click” to highlight the mouse clicks, but in practice I used my foot pedal to click) This system ended up working great, and I basically played the whole game using it.

Because you can get up from the table and walk around, I started thinking a bit about first person games. My system only worked was because Inscryption uses “Myst-like” movement. In other words instead of mouse look and free movement, your view is always fixed and movement only happens in discreet steps.

I feel like this is kind of unusual, at least I haven’t seen it much recently. If Inscryption was like most other first person games, the game would actually have been unplayable for me, at least I have yet to figure out how to play a traditional first person game hands free (but more on this below)

What I found kind of interesting was that my first reaction was to compare this to Myst, and yet that game is playable only with a mouse: you click near the top of the screen to walk forward, click near the sides of the screen to turn left and right, etc. When you want to interact with things in the world your mouse cursor changes to let you know did you can manipulate something. In other words it’s exactly like Inscryption, except Inscryption forces you to use WASD

Maybe it’s because Myst and Riven (and Gadget) left a strong impression on me as a kid, because they were some of the first games I played, but this control scheme seems so natural and obvious to me.

I’ve tried to avoid making any kind of big claims in this thread, because I’m not an expert in UX design, accessibility, or much of anything to be honest. But I have started to come around to a general idea about game control design: if a game uses keyboard and mouse, then it’s worth trying to design it to be playable with only one or the other, ie playable with only a mouse or only a keyboard (I think Inscryption could be designed to be keyboard-only, but it would be a harder design problem than making it mouse-only).

Obviously this is not always possible, or even often possible. But there’s enough games that are close to this that I think it’s safe to make this kind of general statement, that game developers ought to at least consider it.

Ok I’ve already written way too much today, but since Inscryption got me thinking about it, I’m going to talk a little bit about walking simulators, specifically Paradise Killer, which I’ve been trying to figure out how to play for a while.

I thought if I’m ever going to be able to figure out how to play first person games hands-free, something like Paradise Killer without combat or a lot of platforming would be a good start. To the game’s credit it has a bunch of accessibility options, including things like “auto walk” which do make it easier for me.

I’ve tried a bunch of things, a while ago I tried to remap my DDR pad to act like the twin analog sticks of a controller, but it didn’t really work and I can’t remember why. I started thinking about Paradise Killer again after thinking about Myst and walking simulators in general, wondering if there wasn’t some way to hack Myst like controls. The problem is that a game using traditional first person controls will hide the mouse cursor, of course. The player’s camera is attached to the mouse, so there’s no way to implement something like what I did for Inscryption, with context sensitive mouse actions.

I started the game up anyway and was playing around with the head mouse, trying to see if cranking mouse sensitivity all the way up or down would make things easier. The problem with the head mouse is it’s like a VR headset, except the screen is still in front of you instead of attached to your eyes. So you can turn right and left to rotate the view, but since the screen doesn’t move with your head, it’s impossible to actually turn around in the game. (I’ve never actually used VR so maybe this analogy is off…)

I think that maxing out mouse sensitivity helps a lot all by itself, you can kind of whip your view around with a twitch of your head, but you still kind of end up in the annoying adjustment loop that makes edge panning so annoying: tiny movements of your head shift the view set out of alignment with your “center” and there’s no real way to fix it, aside from opening the menu to reset things.

While I was messing around I somehow glitched the game, and my mouse got unlocked. The mouse cursor was visible, the camera was no longer tracking it, but clicking and dragging the mouse panned the camera around. Basically imagine Google street view, it was exactly like that. I was only able to glitch it like this once more, and I have no idea how, something about the way I was clicking.

I’m not really sure what got me to think about console commands. I think I had an hope that maybe there was a command that would unlock the mouse. Of course the Unreal Engine console is disabled in the Steam build, which necessitated using a tool called Universal Unreal Unlocker, which seems to be mostly used for super powered photo modes.

What I discovered is that opening the console just unlocks the mouse automatically. Any key commands of course get submitted to the console, but if you click and drag your mouse around then the camera moves around. Since the console is fairly unobtrusive, I decided to try just playing the game like that:

The challenge here was writing Talon code which toggles the console on and off when I need to press keys. For instance interacting with an object requires pressing “E”, but if you press that while the console is open then the character is just submitted. so the console needs to be closed, the key pressed, and then the console reopened. Talon itself is not stateful, at least there’s not really a built-in way to implement this kind of toggle directly, it has to be done in Python. And of course the Talon code isn’t actually interacting with the Paradise Killer process state, so it can easily get out of sync. You can see this happen right at the start of the video.

After setting all that up, I just bound the trusty context-sensitive pop to the most frequently used key, the auto walk toggle, and also bound a context-sensitive hiss to jump (the only other noise that Talon supports out of the box)

It’s pretty janky, especially since it depends on the behavior of the Unreal console. I would much prefer a hack which was more general purpose, and could be used in other first person games. But it’s usable enough for me to try it for now and get a better idea of whether this is really playable

8 Likes

yeah…so this is even less reliable than I thought. I was kind of excited about being able to play first person games and so have been experimenting more the past couple of days (y’know, instead of actually playing Paradise Killer)

First, I started looking around trying to see if there was a way to do something like this in Unity (the primary motivator for that being I’ve never played Umurangi Generation). As far as I can tell, Unity games don’t have an easily unlockable dev console, and who knows if the behavior is the same anyway. If anyone knows anything about modding Unity, and can comment about it I would be very curious, given the popularity of the engine

Second, since I’ve got it working with at least one Unreal game, I tried to get it working in others. This proved to be a little bit disappointing: in at least one game (Observer) even though I was able to unlock the console, doing that didn’t free the mouse to enable the street view style camera. in another game (Vanishing of Ethan Carter) the console couldn’t be turned on. I did find a couple of games where it worked: Call of Cthulhu and Conarium. none of these games are really my first choices but :man_shrugging:t2: I’ll take what I can get.

Something interesting playing those was renewed appreciation for Paradise Killer’s auto walk feature; it’s definitely possible to play games without it (just script Talon to hold W down) but it’s less elegant (can’t use the street view camera while walking)

The Paradise Killer developers posted a tweet about accessibility recently so I made a more detailed demo to send to them. I’ll repost it here for posterity and in case it was not really clear how this works from the other video:

I’ve also been thinking about what it would take to generalize this, in other words make it work in any first person game. What I need is some way to disable mouse input on the fly, or set things up such that mouse input is only submitted while a button is being pressed. I briefly experimented with some Python code (using pywinhook) which was easy to integrate with Talon, but unfortunately it didn’t actually disable the mouse, just slowed it down.

If anyone has any ideas I would be curious…The only thing I can really think of is to start hardware hacking and build my own head mouse, which isn’t outside the realm of possibility but I’m not sure that’s really what I want to spend my time on.

I genuinely think that this “street view style” (for lack of a better term) has a lot of potential, after playing about 3-4 hours across a few games. I was a little bit worried about motion sickness (which was a problem for me before I dealing with RSI) but I think the more deliberate style of camera movement helps mitigate that a little bit. At the same time I don’t really know if this is something that would be useful for anybody else…

3 Likes

Any .NET debugger should work. dnSpy and the one by JetBrains are popular choices I know but I’ve personally never done it. My understanding is you don’t even require a debug build you can just get it to spill the entire source code.

As an aside there’s quite a mystery around dnSpy. The original creator not only archived all projects on GitHub a day after the latest release and wiped all of their presence online but also uploaded to archive.org at the same time seemingly out of fear that GitHub would take it down. No advance notice was given and no reason was ever given. Just vanished. No one even has an idea as to why as there’s never been a formal complaint against the original project or the forked dnSpy Revive version. Any time someone would post an issue inquiring on any of the original projects they would get deleted shortly after until finally issues were disabled entirely. So someone is still in charge of the original GitHub account and watching. Normally if someone is just done working on something like this they would transfer ownership of the project and move on. Whole thing is a modern day spooky campfire story.

3 Likes

So one thing I was thinking about was somehow setting up something akin to the Roll’n’Rocker for the NES. I figured you could maybe get something that could function on one foot, kind of like a joystick almost.

image

But as I was thinking about it, I just started to think “man, that sounds tiring”. Like, having to rest your foot on that mechanism and pilot it that way sounds like it would get kind of tiring pretty fast. I’m also thinking about how you mentioned using a DDR pad, which also sounds useful, but again, sounds like it would get exhausting. After all, a lot of us play video games to veg out and relax, and having some kind of mechanism like this kind would get in the way. At the same time, these peripherals offer a level of control fidelity that is really useful.

So I guess my question is like, how do you personally balance the tensions between comfort and control, since so many games seem not to offer ways to negotiate them?

3 Likes

I’ve never heard of this device, thanks for sharing! It actually looks kinda similar to another device I’ve been thinking about trying to reimplement with some new hacks but it will be a few weeks before I can actually test my idea…

I think this question gets at the heart of my experience, but I don’t have a satisfying answer. There is definitely a tension but for me the balance has been fairly binary in reality: either I’m able to figure out how to play a game with minimal friction (eg Invisible Inc or Inscryption) or it’s too much of a hassle to play and I end up drifting away from the game entirely and stop playing. So in practice there is less distinction between something like Hades played with Parrotpy and my head mouse (which was the pinnacle for me of very effective and fine grained control, but suffered from the problem of vocal strain) and something like any recent AAA first/third person console game (completely unplayable due to inability to customize, real time nature, complicated controls, etc)

I was initially thinking that this isn’t really how I play games, because I mostly seek out the Tactical Tuesday experience of agonizing over turns in strategy games, but that’s not always been true. I used to play way more video games in general and would play a lot that fall into this category (thinking of the big bucket of AAA first/third person console game again).

But now I just play fewer games and my experience of gaming these days definitely lacks the more casual laid back feel of picking up a controller and messing about, because basically playing anything takes a little more effort than it used to, so I have to be much more committed to wanting to play and “not relaxing”, in order to be engaging with the controls. Because of that I think I’ve gravitated towards playing more strategy games and other turn based things because they’re easier for me to manage and that experience rewards feeling engaged, and I’m just fortunate that I enjoyed these games already anyway

2 Likes

I wanted to plug two games that I think end up being more accessible and hackable kind of incidentally.

First, Vampire Survivors. This game blew up and has been on the forefront of conversations for a bit so I wont bother explaining the specifics, but the combat itself only requires either 4 buttons, and can be played with the mouse. You can also check out a little demo thing on the itch page.

Secondly. SNKRX. This was recommended to me during a conversation about the above game, and is similar in some ways, but it’s more inspired by auto-chess mechanics and kinda like Snake. It only uses two buttons for the combat phase!

It occurs to me now that auto-battlers in general would typically require a lot less modification to get them to a playable state.

3 Likes

I finally got around to playing this last weekend. I had totally missed any hype or conversations about it, having only seen it played on Twitch once or twice, so actually I didn’t realize its whole deal is abilities being passively triggered and that you could play it with just a mouse (my own fault for skimming the Steam page).

It’s very playable for me with the head mouse and foot pedal combo, and pretty fun so thanks for the recommendation. Bookmarking SNKRX for later…

5 Likes

I’ve had this idea for a while for playing Nintendo Switch games using voice controls, but only recently had the time to sit down and try it. I’ve owned a Switch for a while, but haven’t really played it in the past two years since dealing with RSI. There were a couple of games I played using the touchscreen (like the adorable Dobutsu Shogi) but it seems like so few games support it, and actually touchscreens aren’t great for me anyway, so my Switch has been collecting dust. But a couple of Switch-exclusive or console-only games gave me the motivation to figure something out.

I actually looked into the reverse a while ago: using the Ring Fit Adventure controller to play PC games, inspired by Super Louis 64. This lead me to purchase a Titan Two on eBay. However, I discovered after fussing around with it that the Ringcon controller doesn’t send the signal from squeezing and pulling in such a way that the Titan Two can read it (Super Louis 64’s scripts do something clever which is look for spikes in the gyro axes’ signal and assume that a squeeze or pull is happening but I found this a little bit flaky in practice). This ended up limiting the usefulness of it somewhat, although I still think there’s a lot of potential just using the gyro controls. In fact I dusted it off earlier this year to try using the Ringcon controller as a janky wheel to play Art of Rally (inspired by Rob’s Year of Motorsport), along with my foot pedals, which actually was very successful, however sadly I discovered that holding the Ringcon up and twisting it back and forth was still triggering my symptoms (I don’t drive IRL).

There’s at least four different games I’m interested in being released on Switch this spring that are either exclusives (Triangle Strategy, Mario Rabbids sequel), console only (13 Sentinels), or region locked on Steam (remaster of Taiko Risshiden 5). That was basically enough justification for me to spend money and time on my idea. I’m not exactly sure why I first started thinking about this, but because the primary interaction for playing games these days is my head mouse* I probably was just thinking about how to port that interaction model to other games.

The idea was to get a capture card so that I would have video from the Switch streaming to my PC, where the mouse cursor would be visible on top of the game. Then the mouse cursor’s position would dictate which buttons were pressed. Definitely this was inspired by my successful experiences playing Inscryption and Total War** using systems like this. After quite a bit of experimentation and refinement, I’m pretty happy with how it turned out:

The Triangle Strategy demo was a great candidate for this experimentation: asides from being free, because its controls are fairly simple I was able to get something working pretty quickly. I did spend a while refining it, refactoring the code that I had been copying and pasting for all these games, and then testing with two different capture cards. For the too-long-didn’t-watch version, the main idea is to do something similar that I did with Inscryption. After capturing the video from the Switch, the screen is divided into regions, and each region corresponds to a direction on the D-pad or analog sticks.

I used Talon’s trusty pop noise for the D-pad, so popping while the mouse cursor is in the upper left region will press up on the D-pad, and popping while the mouse cursor is in the lower right will press down. I’ve done something similar with the analog sticks, just programming Talon to hold those buttons down and change directions when the mouse cursor moves between regions. All this is possible because the software that that you use to configure the Titan Two (GTunerIV) can be configured to capture input from the PC, and it’s able to also capture key presses coming from Talon. That software then sends the inputs to the actual Titan Two hardware, which sends it to the Switch:

This is a picture I took to explain this to a friend, because my desk surface is black it’s challenging to actually take a decent video showing this setup in action, but just imagine the PC is off to the right. Interestingly, the guide that I found for setting up the Titan Two using a PS4 suggested that you need to have proper Dualshock plugged into the Titan Two, but in my testing this isn’t true for the Switch, despite the pro controller in this picture. The Switch seems perfectly happy to take input from the Titan Two without a pro controller plugged in.

The capture card I ended up using is not cheap, although I did try a cheaper one to start with, the experience was a bit subpar: surprisingly latency in OBS’ preview was never a problem with either card, the problem was constant glitchiness with the cheap one, which was a pretty common complaint in the reviews for it. At the end of the day it’s simply not worth my time to debug something like that, when buying a more expensive capture card fixed the problem immediately. That being said, this set up is definitely not cheap.

I probably wouldn’t bother doing this just for Triangle Strategy, since I assume that’s going to come out on PC at some point, but experimenting with this system has given me some other ideas which I’m excited to try. At some point I guess I should actually play these games too…

* for posterity, my beloved Remote+ started behaving flaky recently, after two years of a lot of use, I guess it doesn’t owe me too much, although I would be dismayed if it turns out that these cheap gyro remotes just become e-trash after a while. anyway I switched to this. ymmv for anyone who happens to try this hardware

** I’ll probably follow up with another post and video about Total War soon, because that turns out to be one of the more playable games with my setup, which is a bit surprising

6 Likes

I had a bit of a breakthrough with this today, although I got there in a bit of a roundabout way. After experimenting with playing Switch games through the OBS preview window, it occurred to me there might be something there for first person games. At first what I was imagining was capturing a console game through OBS and using the mouse cursor position to drive analog sticks. Then it occurred to me that you could maybe just capture a PC game through OBS and do the same thing. My thought process was that by capturing a game like this you decouple the mouse from the game which might enable implementing something like I did with Paradise Killer.

As I was experimenting with this, I dusted off a piece of software that I bought a long time ago: ReWASD. I can’t remember exactly what I was trying to do with it before, and why it didn’t work for my situation at the time, but it’s a quite powerful controller remapping software, that also lets you do things with mouse and keyboards.

What I discovered today is that you can create multiple layers for a single input, switch between those layers with a keypress toggle, and one of the options for a layer is to completely unbind an input. Together with my foot pedal this allowed me to toggle head mouse input, which solves the problem that the unlocked Unreal console was solving, but in a general purpose way. In the end you don’t need to bother with capturing software at all, because ReWASD does all the work of decoupling the mouse.

I don’t want to get ahead of myself, but I think these ideas might open up a bunch of possibilities for me. Throughout my experiments with hands free gaming, there have been a couple of “white whales”. One of them is WASD camera movement in strategy games and RPGs. The other one is what I call the “twin stick problem”. There are just so many games which expect you to be able to move a character in 3D space, while also controlling a camera, either with two analog sticks or with both a mouse and keyboard. First person games, even slow paced games like walking simulators, fall into this category.

I don’t think anything I’m doing is really revolutionary, I mean what I figured out today is literally the most basic usage of this software. And it’s a bit funny to me that I only figured this out 13 months after buying ReWASD, but I guess it took a while for me to even formulate the problem. But I’m looking forward to exploring what else is possible, and also finally going back and re-listening to WPR discussions about Umurangi Generation.

2 Likes

New one-button game on the horizon!! I hope it’s alright to come in and drop links to hands-free friendly games in this thread

3 Likes

I bought an Xbox Adaptive Controller along with the Logitech Adaptive Gaming Kit and have been experimenting with playing games using my feet. so far I’ve had some success playing Rain World, which I mentioned in the other thread, but I’m still definitely figuring out how best to use it. I’ll probably have more to say about it after more experience but last night I came up with this hack which I thought I’d share:

The Logitech Kit comes with two velcro pads to attach switches to, which are neat. however, setting them up on the floor means they slide around a lot and collect dust which is an issue in my apartment, not to mention the cat hair.

Rather than buy something to solve this problem, I flattened out one of the cardboard boxes this stuff came in, taped the underside to make a rigid platform, and tied everything down with old bicycle inner tubes. Not only does this keep everything in place while I’m actually using it, this allows me to unplug the USB from the back of the XAC and pick the whole mess up off the floor to clean and sweep every morning (seriously my apartment is extremely dusty)

Buying hundreds of dollars (yeah this XAC stuff is not cheap…) of electronics is hardly a hack, but repurposing a bunch of actual trash to solve my specific problem is something I love doing.

3 Likes

Haven’t picked this up but it looks very cool. Horror game about crawling through a space lab. Everything is controlled by the mouse, so it should be controllable with any mouse replacement! Though, worth saying it’s not technically hands free since the hand is the only thing you control in the game! :stuck_out_tongue:

1 Like

lmao :joy:

this looks like my jam, thanks for the rec! always on the lookout for non-first-person horror games as they’re somewhat uncommon