wvenable's idea is good but a cheat. You referenced Wirth, who simplified assembly (eg P-code). Do something similar: extend BF with a macro language that simplifies it, build macros for BF-like programming constructs, use them to simplify programming the interpreter/compiler, and then host with that. If you want to do real BF, the very act of implementing those macros might help you mentally understand how to... one piece at a time lol... implement the real thing in vanilla BF.
Which you can hand-implement in machine code if you want. Choose your level of masochism carefully. :)
The part I'm most curious about would how to make the JIT itself self hosting. If brainfuck's only method of I/O is getchar/putchar, the only way I can think of calling mmap is doing a stack buffer overflow attack based on the misrepresentation of tape.
I remember hacking on Rust being a mind bending experience (the Rust compiler has been self hosting for a long time). Thinking in another level of abstraction is hurting my head.
I think it would have to be able to output an macho64 executable. Oh man.
That does sound difficult. Maybe do it bare-metal on a simple, non-MMAP architecture* with basic routines coded in assembler. Both main interpreter and JIT'd code might reference those core functions. I know it's not directly solving the problem but I sneak around the impossible parts wherever possible.
* Come to think of it, one of those microcontrollers or Java processors might come in handy here as they supply basic libs or RTOS's with primitive I/O functionality.
Which you can hand-implement in machine code if you want. Choose your level of masochism carefully. :)