Home

July 06, 2024: Accidental Malware

Accidental Malware

After more-or-less finalizing several web-based clicking games to measure motor information throughput, I figured I should make it possible to interface to the tasks using more than just a mouse, trackpad, or touchscreen. My idea was to write a compiled c/c++ executable, which I could easily run from any Windows-11 device (i.e. most of my normal lab workstations), that would run in the background as a “controller emulator server.” I got to work writing an Input_Utilities code repository. You may have guessed where this is headed.

It works

I started out with a server utility that opens port 6053 and accepts websocket TCP/IP connections from any IP address that the program has access to. Once a socket connection is made, it indicates in the console that "XBOX controller connected." After that, it pretends to create Windows Gamepad events based on TCP char packets consisting of 'a1' (for “A-button” release) or '40' (for d-pad left press; full table here). So it lets you issue commands as if you were connected via an XBOX controller, simply by connecting to the socket and issuing the correct 2-char newline-terminated sequence. I thought this was pretty nifty since it allows me to, for example, do some decoding in MATLAB or Python, then from within that environment create a TCP/IP socket connection to the emulator and proceed to convert any decoder output into the appropriate character sequence. In turn, this allows me to interact with ALL the numerous types of applications that accept XBOX Gamepad inputs! Not bad. To keep it relatively “safe” I made sure the program exits once the socket has been closed. Of course, this gets annoying if you have to restart the server every time, so the next steps were:

1. Add other forms of input emulation, and
2. Add “server persistence” because I’m lazy.

Mouse Input Emulation

In the second version, I added mouse input emulation. This basically uses the same approach as before, but opens an additional socket connection on port 6054. Both of the sockets need to be connected for the main loop to run on the server application. The mouse inputs can emulate mouse movement events, left/right clicks, press and release, and mouse wheel scroll-up and scroll-down. The full API description for the mouse events is in the table here.

Adding Persistence

The final version, it basically just wraps the application in a while loop that allows it to cycle back into waiting for both of the client socket connections to complete so that the server application no longer closes when the socket connections are closed. I realized that, basically what I had made was an application that runs in the background and allows some other machine to take over the server application’s mouse, which sounds a little bit like malware but it’s for a good purpose. So I added some caveats about security and how it should be deployed along with some default .bat scripts to modify firewall rules for the ports used by the application in an attempt to make sure if anybody other than me uses this, hopefully “you’ve been warned.” Please read the safety concerns here, if you’re considering using it.

June 30, 2024: Fitts Law Tasks

Fitts Law

Fitts’ Law is a way of measuring motor throughput in point-to-point movements. The framework has been extended to evaluating human-computer interactions, but may need refinement depending on the nature of the task and what type of information we care about specifically.

Fitts Law Tasks

Recently, there has been a fair amount of hype about the throughput achieved by the first publicly visible participant in the Neuralink BCI trials. After going to their website and playing the task variant they’re using to quantify information throughput, my immediate reaction was that this is not a throughput task that derives the metric for information transfer from Fitts’ Law. Based on the size of the grid, distance traveled, and the total time it took me to click through (you get to play for 60 seconds total, as many correct clicks as you can), there’s no way based on how high my score was that this is how they’re doing it.

So, I decided to take a stab at making some web-based clicking games to measure motor information throughput. My idea was to make some tweaks and adjustments to our existing Fitts’ Law variants to see if we could make them “fuzzy” and allow participants to score even when they are having a very hard time clicking the target correctly at all. I did the same thing for my best-guess as to how the Neuralink task works, and I’m pretty happy with how it turned out!

February 29, 2024: NML Behavioral Tasks

Behavioral Task Docs

I have been working to upgrade the documentation and make more modular the codebase running behavioral tasks. Specifically, I wanted to make a networked protocol compiled in c/c++ that allowed for very-high fidelity in logging from devices across several different operating systems/distributions. I’ve included the relevant doxygen documentation on the NML-WTF website.

GPT and Image-Generation

I tried to have GPT help me out with modifying the site layout, but image generation still isn’t great when words are included. I’m using the weird garbled “worded” images it has generated based on my posts to-date, since I find it endearing.

March 21, 2021: Neuro-Activities

Neuro-Activities

One of the graduate students at our lab (Monica) made this really cool website with interactive neuroscience activities for teaching key concepts related to some of the more abstract parts of research going on in motor systems neuroscience.

Center-Out Task

I made a fork of her repository and added some minor adjustments to simulate some parts of the NHP task I’ll be using for my research. This state-coding demo is still a work-in-progress, but the objective is to illustrate how we can play on the expected movements of the virtual cursor to test hypotheses about how individuals control movement. Currently, there is a jitter component that will eventually include a slider to scale the magnitude of the cursor jitter in order to inhibit feedforward prediction of where the cursor will be as the mouse moves. After a limited number of trials, the cursor movements are perturbed using a fixed angular offset, which the brain realizes relatively quickly and corrects on ensuing trials. Eventually, I would also like to add functionality that gradually weans reliance on visual position of the cursor in order to collect pilot data on how well uninstructed individuals are able to correct for the perturbation in the absence of visual feedback (I suspect that it will be quite difficult, but may be facilitated by allowing a very brief indication of how the perturbation is influencing motion).

January 15, 2021: Active Inference Model-Stream

Active Inference Model-Stream

I’ll be participating in the Active Inference ModelStream #001.0: “A Step-by-Step Tutorial on Active Inference” today at 11am CST. Please join me here if you would like to watch live!

About ModelStream

The ModelStream channel will aim to provide tutorials for the use of Active Inference or comparable models in describing different kinds of data. The videos for a given topic will most likely be multi-part. For today’s video, we will be covering Sections 2, 3, and 5 of this tutorial paper describing how Active Inference could be used to describe Partially Observable Markov Decision Processes (POMDP), such as may occur when an individual agent is choosing to make a binary decision while using or excluding information that is associated with risk mitigation. Importantly, the authors have provided corresponding Matlab code that directly implements the model simulations described with great detail in the tutorial paper.

As someone who does not study the neuroscience of decision-making (as a neuroscientist, I consider myself to study motor systems), I can say that this is still an excellent way to learn about how you might incorporate a process model to provide possible explanations regarding why certain data were observed. This is not a mutually exclusive process to more traditional statistical methods that use variance models to study how certain data might have arisen given the experiment and resulting observations. The two are related and both important to understand as neuroscientists seeking to assess and explain biological phenomena in a parsimonious sense.

November 22, 2020: Thoughts on TCOE

Thoughts on TCOE

I am an avid fan of Dungeons & Dragons (5e), which means I run games for some of my friends (more frequently the case in the past). Recently, Wizards of the Coast released a new supplement (Tasha’s Cauldron of Everything; TCOE), which introduces new optional class features. I had an immediate dislike to one of the new Fighter class features, so I decided to do some simulations.

Argument

I initially thought that Superior Technique was over-powered as a Fighting Style and added too much damage in addition to the utility it affords due to access to Battle Maneuvers previously reserved for access through feats (which are costly to spend on this) or by sub-classing Fighter to the Battle Master class (which was usually how Maneuvers would be obtained). However, simulations show that Great Weapon Fighter fighting style still has an increased damage expectation compared to baseline damage expected to be added from Superior Technique. Further analysis is required to include the utility effects that are peculiar to each Maneuver that could be selected when choosing this Fighting Style.

November 19, 2020: Thoughts on Peer Review

First Post: On Academic Service

Check out my thoughts on modern peer review. I see peer review as the main way in which I can currently provide a service to my academic peers. However, I have a real bone to pick with the details of how peer review is conducted in practice.

Argument

The act of publishing articles in journals other than Nature or comparable is unethical. Publication in other journals (with few exceptions) dilutes knowledge and fundamentally threatens the true institution of peer review. Unfortunately, the current traditional peer review model perpetuated by predatory publishers such as IOP, Elsevier, Taylor and Francis, and Springer only serves to encourage the aforementioned knowledge dilution. Worse, these syndicates waste the time of well-meaning scientists by advocating erroneous arguments for how the traditional peer review model serves science while in reality it is simply a source of free, expert labor. In this post, I argue that modern peer review supplants this traditional model, because technologically the services of the journals is no longer required. I concede that a few journals remain viable and provide useful services, but ultimately we will need a better way to recognize academic service with respect to knowledge curation and dissemination because the traditional “currency” of the academic system–publications–is no longer a tenable solution for the next generation of up-and-coming scientists.

November 10, 2020: Carnegie Mellon talk

Virtual presentation: Carnegie Mellon University

I presented a modified version of my thesis to the Weber lab and affiliate faculty, going into a little more technical detail about the results from Aim 1 of my dissertation.

Aftermath

It was so nice to meet everyone there and hear about their work. I was so impressed with all the interesting projects that are ongoing and look forward to hearing more in the future.

October 09, 2020: Successful Dissertation Defense

Successful Defense!

I successfully defended my doctoral thesis (Neurophysiological mechanisms of sensorimotor recovery from stroke), with honors!

Aftermath

I celebrated with friends by playing Dungeons and Dragons, in which a long-running (2-year) campaign arc reached its satisfying conclusion.