It's pretty easy to quit the Finder if you don't need to mess with files graphically [1]. Once you've set that you can also use cmd-q to do that when you're cmd-tabbing through apps (if something else re-opens it).

1: https://www.howtogeek.com/259378/how-to-quit-the-finder-in-o...