Here at EleutherAI, we are probably most well known for our ongoing project to produce a GPT⁠-⁠3-like very large language model and release it as open source. Reasonable safety concerns about this project have been raised many times. We take AI safety extremely seriously, and consider it one of the, if not the most important problem to be working on today. We have discussed extensively the risk-benefit tradeoff (it's always a tradeoff), and are by now quite certain that the construction and release of such a model is net good for society, because it will enable more safety-relevant research to be done on such models.

While this is a genuinely nuanced issue whose full subtlety cannot be captured in a single short blogpost, we have tried to summarize the most important reasons we believe this is the best course of action for us:

There is significant, important safety research that can only be done with access to large, pretrained models. We would like to make such research possible and easy for low-resource researchers (and participate in such research ourselves). We take the possibility that the first TAI ends up being effectively a scaled up transformer without any radically new scientific insights in its architecture extremely seriously. We feel that the ways in which future scaled-up LMs could be dangerously powerful are not sufficiently well understood. Meanwhile, since GPT⁠-⁠3 already exists, and we have not yet been taken over by some form of malicious AGI, we are quite confident that models of this scale are not world-endingly dangerous. We think that this means we have the opportunity to do safety critical research before such models become truly dangerous. In order to do so though, we need access to large models to do the best research. Access to the actual underlying model is critical for work on model interpretability (a field especially useful to safety), and it seems certain relevant capabilities worth studying only start to emerge at larger scales (such as few-shot improvements only becoming noticeable for large models). It is very unclear if and when such models will start to exhibit far more powerful and dangerous capabilities. If we had access to a truly unprecedentedly large model (say one quadrillion parameters), we would not release it, as no one could know what such a system might be capable of.