In the blog post I mention using RS485 (or CANBUS) instead of the solution we went with, but on a 5 day timeline, and building on a set where I wouldn't be present (initially I was not planning on being there at all during the shoot), I didn't want to put in a solution someone else on staff couldn't at least wrap their mind around.
In hindsight... there's always a lot that can be done different :)
And regarding the debounce—I don't mention this in the video explicitly (for time), but there were three other 'modes' for the buttons that were implemented that were never ultimately used, for games like Simon (press colors to match a pattern), and timing-based games, so the debounce was actually critical to allow varying ages to press buttons sometimes in tandem and sometimes in sequence with certain timings.
Because of the issues we had (not the least of which was a weird bit of a drain on the 3.3v as you held down a button), I recommended against some of the challenge types. In the end, the only things that made it to the final cut of the 1-100 video was the 'yes or no' style votes on a challenge. Which, yes, are incredibly easy to debounce. Lock in the first button press and you're done ;)
Yeah, 3.3v isn't really viable as a signalling voltage for anything that leaves an enclosure unless you take steps to mitigate interference such as differential signalling, shielded cable or some sort of protocol rather than presence/absence of voltage. It's just too easy to induce that voltage - albeit very temporarily - through capacitive or inductive coupling on even 10m of wire; as you experienced it's also too easy to have that voltage mysteriously leak out into what a multimeter will tell you is a definitively open circuit.
If you're thinking of doing something like this again, I'd suggest:
1. Definitely use PLCs, their inputs are inherently hardened and you can use 24/48/240v signalling voltages - and by extension a whole universe of very robust industrial indicators and switches.
2. Don't use data cable for non-data transmission, the twisted pair coupling will give you issues you don't want and the stuff is fragile as anything. If you want a good cable which supports multiple non-data channels the term 'pilot cable' or 'control cable' will see you right, that's a multi-core cable which usually has 4 thick cores and a bunch of thin pairs, it's traditionally used for carrying 4-20ma signalling loops in factories and oil rigs etc, it's well shielded.
3. Do use RS485 to loop your PLCs together.
4. Don't use weak voltages on stuff that's being handmade and screwed together onsite. You run into all sorts of transient issues, 3.3v will balk at a loose terminal whereas 48v will romp home if there's even a hint of contact.
That was what my Dad mentioned too—he's built a lot of signaling systems for tower sites and radio studios and was nervous about me running 3.3v over any length of Ethernet. But he assumed we'd have shielded Cat5e at least... that would've helped somewhat.
Either pullup or pulldown resistors would have helped - you see this a lot in PCB design, even short traces that are used for signalling will be pulled low or high via a relatively high resistance. That ensures that you have a sharp transition when the switch or whatever is closed or opened and that a floating voltage can't appear over time on your line and cause false triggers.
You...had current limiting resistors in series for each buttons, right?
edit: I think each buttons could be made to register the last pressed UNIX time rather than sending out button event, so the host can collect reports and sort out time sensitive aspects "later" in the loop to account for delays and reordering type of issues
How much accuracy do you want? Now you have issues syncing all the SBCs to a timeserver and how much drift their internal clocks have. Some of those things have very vague timing.
If timing is critical I'd run all the buttons back to a dedicated device but where you'd find a PLC-alike with 200 inputs polled 'simultaneously' (FSVO simultaneously) I don't entirely know.
This is the type of project where very particular issues become annoyingly apparent only after you've chosen a hardware platform.
Quite true. And a few of the hard requirements became a bit softer after we got on-site and ran into the issues that couldn't be solved for lack of time.
There are modules where you just have ethernet and few IO ports, for example WISE-40X0LAN series.
It's pretty much what you've built but industralized. And, as it is for industrial stuff, pretty pricy, 150 EUR a pop. But works with via both HTTP and MQTT
In hindsight... there's always a lot that can be done different :)
And regarding the debounce—I don't mention this in the video explicitly (for time), but there were three other 'modes' for the buttons that were implemented that were never ultimately used, for games like Simon (press colors to match a pattern), and timing-based games, so the debounce was actually critical to allow varying ages to press buttons sometimes in tandem and sometimes in sequence with certain timings.
Because of the issues we had (not the least of which was a weird bit of a drain on the 3.3v as you held down a button), I recommended against some of the challenge types. In the end, the only things that made it to the final cut of the 1-100 video was the 'yes or no' style votes on a challenge. Which, yes, are incredibly easy to debounce. Lock in the first button press and you're done ;)