The notoriously secretive Meta has set a milestone for transparency.
The company this week offered the entire research community access to a fully-trained large language model (LLM).
Named the Open Pretrained Transformer (OPT), the system mirrors the performance and size of OpenAI’s vaunted GPT-3 model.
This mimicry is deliberate. While GPT-3 has a stunning ability to produce human-like text, it also has a powerful capacity for biases, bigotry, and disinformation.
OPT’s creators said their system can reduce these risks:
Our aim in developing this suite of OPT models is to enable reproducible and responsible research at scale, and to bring more voices to the table in studying the impact of these LLMs.
In addition to sharing OPT for non-commercial use, Meta has released its pre-trained models, their underlying code, and a logbook of their development. No other company has ever provided this level of access to an LLM.
Such openness may appear uncharacteristic.
After all, Meta is often accused of concealing its algorithms and their harmful impacts. Yet the move may not be entirely altruistic.