I lost that poor little machine when I moved across the country; not sure where it could have ended up. Now my only "system to screw around with" is my RasPi.
The first PC I had with X installed was a 486SX/25MHz with 20 MB of RAM. On the rare occasion that I "needed" to use X, I would start it, do whatever I needed to, kill X, and return to the console.
I still spend most of my time in a terminal window, but it's on a machine with ~1120x as much CPU horsepower and ~1638x as much RAM. Just calculating that made me go "wow" and realize how far we've come in the last 20 years or so.
I remember doing some intro comp.sci homework on a hand-me-down Pentium 2 laptop. It sort of managed to run X11 (in 8bit greyscale!) -- but the jdk took literally 5 minutes to start up (compilation time was effectively bounded by the star-up time: It took 5 minutes until I had my .class files, or my list of syntax errors...).
I did a similar thing. My P2-233 (at the time this was the top end) blew a PSU and I did my engineering writeup on a P66 (pulled from a campus skip) with 32Mb of RAM using heirloom nroff, eqn, pic and vi on FreeBSD because it was all I had access to and you couldn't compile TeX on that box in a week. I have a lot of respect for the troff family of tools and still use them now.
Interesting question. I think the problem was the 4(or 8?) MBs of ram, and javac swapping out while loading. I can't recall that I did any compiling with gcc on that laptop -- at the time gcj certainly wasn't mature enough. I don't think I tried with jikes either, for some reason -- can't remember why now.
It still feels really fast if you don't use X.