Too bad OpenAI has less than a handful of people working on safety and only one safety paper (to be published in a few months) throughout these two years of OpenAI's existence. Actions speak louder than words.

This is Dario Amodei, head of the OpenAI safety team. We are devoting a substantial fraction of the organization’s bandwidth to safety, both on the side of technical research (where we have several people currently working on it full-time, and are very actively hiring for more: https://openai.com/jobs/), and in terms of what we think the right policy actions are (several of us, including me, have been doing a lot of talking to people in government about what we can expect from AI and how the government can help). Beyond that it’s just a very central part of our strategy — it’s important for our organization to be on the cutting edge of AI, but as time goes on, safety will become more and more central, particularly as we have more concrete and powerful systems that need to be made safe.

Past examples of our papers include this: https://blog.openai.com/deep-reinforcement-learning-from-hum... (with the associated system released here: https://blog.openai.com/gathering_human_feedback/) and this https://arxiv.org/abs/1606.06565 (done while I was still at Google, but in collaboration with OpenAI people right before I joined OpenAI). Based on the current projects going on in my group, I am hopeful we’ll have several more papers out soon.

> substantial fraction of the organization’s bandwidth to safety

Although OpenAI's group ethos has a strong safety bent, there are only three research scientists working on technical safety research full-time, including yourself and a very recent hire. Before this summer, while you focused on policy and preventing arms races, there was only one person focusing solely on technical safety research full-time, despite the hundreds of millions donated for safety research. The team and effort should be larger.

> it’s important for our organization to be on the cutting edge of AI

I agree that OpenAI needs to be at the cutting edge, though always pushing the edge of AI to work on safety is needless when there is a significant backlog of research that can be done in ML (not just in RL). It's true capabilities and safety are intertwined goals, but, to use your analogy, the safety meter is not even a percent full. Topics outside of value learning using trendy Deep RL that OpenAI should pioneer or advance include data poisoning, adversarial and natural distortion robustness, calibration, anomaly and error detection, interpretability, and other topics that are ripe for attack but unearthed. There is no need to hasten AI development, and doing so does not represent the goals of the EAs or utilitarians who depend on you --- notwithstanding the approval of advising EAs with whom you have significant COIs.

OpenAI's safety strategy should be developed openly since, as of now, OpenAI has no open dialogue with even the EA community.

A huge part of how they are working towards safer AI is making the tools accessible to everyone. There code is open source, which helps democratize AI instead of keeping it in the hands of a few corporations. IMO, that's a pretty sound strategy.

Not necessarily. You wouldn't want to open source nuclear weapon blueprints, for example. Nick Bostrom examines the details of particular types of openness and how they might help (or not) here: https://nickbostrom.com/papers/openness.pdf