We evaluate different fewshot prompts on GPT-3 to see how it changes performance.

We tuned GPT-Neo on eval harness tasks to see how it would change its performance.

An ablation of activation functions in GPT-like autoregressive language models.

We’re proud to announce that the 2021 class of OpenAI Scholars has completed our six-month mentorship program and have produced an open-source research project with stipends and support from OpenAI.

OpenAI is committed to developing general-purpose artificial intelligence that benefits all humanity, and we believe that achieving our goal requires expertise in public policy as well as technology. So, we’re delighted to announce that Congressman Will Hurd has joined our board of directors.

Rotary Positional Embedding (RoPE) is a new type of position encoding that unifies absolute and relative approaches. We put it to the test.

Over 300 applications are delivering GPT-3–powered search, conversation, text completion, and other advanced AI features through our API.

We’ve discovered neurons in CLIP that respond to the same concept whether presented literally, symbolically, or conceptually. This may explain CLIP’s accuracy in classifying surprising visual renditions of concepts, and is also an important step toward understanding the associations and biases that CLIP and similar models learn.

We’ve scaled Kubernetes clusters to 7,500 nodes, producing a scalable infrastructure for large models like GPT-3, CLIP, and DALL·E, but also for rapid small-scale iterative research such as Scaling Laws for Neural Language Models.

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the “zero-shot” capabilities of GPT-2 and GPT-3.

We’ve trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language.

It’s been a year of dramatic change and growth at OpenAI.

OpenAI has agreed to license GPT-3 to Microsoft for their own products and services.

We’ve applied reinforcement learning from human feedback to train language models that are better at summarization.

Our third class of OpenAI Scholars presented their final projects at virtual Demo Day, showcasing their research results from over the past five months.

We’re excited to announce that OpenAI is co-organizing two NeurIPS 2020 competitions with AIcrowd, Carnegie Mellon University, and DeepMind, using Procgen Benchmark and MineRL.

We find that, just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. By establishing a correlation between sample quality and image classification accuracy, we show that our best generative model also contains features competitive with top convolutional nets in the unsupervised setting.

We’re releasing an API for accessing new AI models developed by OpenAI.