Race policies in colleges have nothing to do with Black or Hispanic poverty, to any reasonable approximation. The people who are affected, one way or the other, are squarely middle-class.
This may be expected given policies of the previous several decades seeking out minority students and the correlation in the US between minority membership and poverty (though there's certainly correlation / causation concerns with an assumption like that).