Thanks! First thank you for all the work to develop a math LLM. Those prompts are helpful. I shrank the weights down to f16 to fit on my system from f32 using the procedure at ggerganov/llama.cpp, then tried some of those prompts. From line 34:
> Problem:
> If $\det \mathbf{A} = 2$ and $\det \mathbf{B} = 12,$ then find $\det (\mathbf{A \mathbf{B}).$
> Solution:
> We know that for a matrix \mathbf{M}, the determinant of its inverse is given by $\frac{1}{\det \mathbf{M}}.$ We also know that the determinant of a product of matrices is equal to the product of their respective determinants. Therefore, we have:
\begin{align} \det (\mathbf{A}^{-1}\mathbf{B}) &= \frac{\det(\mathbf{AB})}{\det\mathbf{A}}\\ &= \frac{\det\mathbf{A}}{\det\mathbf{B}} \end{align}
> Therefore, we have:
> $\det(\mathbf{AB}) = 24
The f32 answer:
> We have that $\det (\mathbf{A} \mathbf{B}) = (\det \mathbf{A})(\det \mathbf{B}) = (2)(12) = \boxed{24}.$
> Final Answer: The final answer is $24$. I hope it is correct.
Final answer is the same, which is encouraging for quantization to expand hardware options.
You must ask yourself why the government is able to retain word-class managers and jurists despite stiff competition from the private sector. James Comey was an executive at the world's largest hedge fund, why did he quit for a government job where he was making 100x less? Any federal judge could've made a fortune in corporate law, why didn't they choose to? Because there is a sense of prestige and national duty about their posts. Creating a similar status for technologists in government would go a long way towards recruiting and retaining top talent.
If we want to convince people that being a government engineer is worthwhile and respected, starting with students seems reasonable.
I think it's more complicated than that, all of these roles come with substantial amounts of near unilateral executive power that can impact the way the country is run for decades to come.
Even all of the existing US government tech posts like the USDS and 18F are ultimately there purely to service the needs of a bureaucratic agency and their particular management.
For a technology specific role to have the same level of prestige as someone on the federal bench or leading the FBI, they'd need to be giving them unilateral regulatory powers over the broader technology industry in the country.
I'd just push back - technology is a tool to accomplish the missions of government.
The positions that run those missions - enforcing the law, ruling on the law, running Medicare, etc etc - should be more prestigious. That said, if the government wants to have prestigious technical work, it should be more open to taking big swings (in an agile way) on hard problems, but that requires investment and taking risks.
I think there would be comparable prestige if the government did something like roll out public broadband or made some sort of publicly owned social media.
Why? For some the reasons you list are certainly it. For many more, it's the revolving door. You make money in private industry, change the legal or regulatory structure and then go back to private industry to make even more.
My understanding is that you can sell stock (that might pose a conflict of interest) tax-free when you take a government role. I presume that a hedge fund manager such as James Comey would have significant unrealized capital gains that he could sell prior to taking his government position.
Gary Cohn, Trump's chief economic advisor did exactly this[0].
"The chief economic adviser was required to sell his Goldman Sachs shares -- and he did so tax-free and with perfect timing"
Where does the author get the idea that sustainable growth is no longer possible? Even setting aside how unfair it is to tell the global poor they've had enough growth when we've had 10x more, the claim isn't even true for developed countries. Rich countries are comfortably able to grow at rates higher than 1% a year (which, thinking long-term, is 2.7x in 100 years) and there is no reason to suspect that decarbonization will put a serious dent in that. The author seems to believe that growth is predicated solely on physical inputs, when in reality institutions and human capital are the real drivers.
You can check out our evaluation library (https://github.com/wellecks/lm-evaluation-harness) for the exact benchmark implementations we used, including prompting.
In particular, the prompt that starts at line 27 in this file (https://github.com/wellecks/lm-evaluation-harness/blob/maste...) is quite good for high school/olympiad problems. We took this prompt from Google's Minerva paper.