It might be helpful to be a bit more precise here.
The expression "first-order logic" (FOL) is a bit vague. Do you mean
pure FOL (i.e. all axioms are about equality) or FOL plus additional
axioms? Pure FOL (without additional axioms) is indeed rather weak
and cannot be used for much. That's why mathematicians build a
foundation of mathematics on top of FOL by assuming the truth of
additional axioms.
The most well known foundation of mathematics built on top of FOL is
ZF-set theory which adds about a dozen additional axioms about how to
construct new sets from old, and when two sets are equal.
There are other FOL-based axiomatisations of mathematics such as Quine's NF, or
category theory (which can be expressed as a theory on top of
FOL). There are also foundations of mathematics that are not using FOL
(Martin-Loef type theory and successors such as HoTT).
Goedel's first incompleteness theorem forces all those axiomatisations
of mathematics to be incomplete, meaning there are formulae A such
that neither A nor the negation of A can be derived.
A statement like "99.9% of the mathematics anyone ever does" is about
FOL extended with foundations of mathematics like ZF, or a
type-theoretic equivalent.
However, mathematicians are getting ever better at producing
interesting formulae whose truth (or otherwise) cannot be settled in
ZF (or similar) [1]. In order to deal with such expressivity gaps,
mathematicians investigate more powerful axioms, called "large
cardinals", which claim the existence of certain extremely large sets.
You can also think of them as more powerful induction principles.
The problem with inventing new large cardinals (i.e. more powerful
induction principles) is to navigate between the Scylla of expressive
weakness (the axiom is a consequence of other axioms) and the
Charybdis of inconsistency (adding the axiom allows us to derive
"false").
The expression "first-order logic" (FOL) is a bit vague. Do you mean pure FOL (i.e. all axioms are about equality) or FOL plus additional axioms? Pure FOL (without additional axioms) is indeed rather weak and cannot be used for much. That's why mathematicians build a foundation of mathematics on top of FOL by assuming the truth of additional axioms.
The most well known foundation of mathematics built on top of FOL is ZF-set theory which adds about a dozen additional axioms about how to construct new sets from old, and when two sets are equal.
There are other FOL-based axiomatisations of mathematics such as Quine's NF, or category theory (which can be expressed as a theory on top of FOL). There are also foundations of mathematics that are not using FOL (Martin-Loef type theory and successors such as HoTT).
Goedel's first incompleteness theorem forces all those axiomatisations of mathematics to be incomplete, meaning there are formulae A such that neither A nor the negation of A can be derived.
A statement like "99.9% of the mathematics anyone ever does" is about FOL extended with foundations of mathematics like ZF, or a type-theoretic equivalent.
However, mathematicians are getting ever better at producing interesting formulae whose truth (or otherwise) cannot be settled in ZF (or similar) [1]. In order to deal with such expressivity gaps, mathematicians investigate more powerful axioms, called "large cardinals", which claim the existence of certain extremely large sets. You can also think of them as more powerful induction principles.
The problem with inventing new large cardinals (i.e. more powerful induction principles) is to navigate between the Scylla of expressive weakness (the axiom is a consequence of other axioms) and the Charybdis of inconsistency (adding the axiom allows us to derive "false").
[1] https://u.osu.edu/friedman.8/