Text Generation using GPT-J with Hugging Face 🤗 and Segmind

Code Generation using GPT-J

Modified Image from Source

Text generation is the task of automatically generating text using a machine learning system. A good text generation system can make it really hard to distinguish between human and machine-written text pieces. Some of the popular real-world use cases where text generation systems can be seen in action are Machine translation, Text summarization, Auto-completion, etc.

Recent advancements in Machine Learning, especially in Natural Language Processing, have produced some really good language models with good zero-shot generalization capabilities. Some of the models you might want to check out are BERT, GPT-3, GPT-J, T5, etc. As a part of this blog, we will look into how we can use the pre-trained GPT-J model for the task of text generation using Hugging Face pipelines on Segmind.

GPT-J

GPT-J, a 6 billion parameter model released by Eleuther AI is one of the largest, open-sourced, and best-performing text generation models out there that’s trained on the Pile Dataset (The Pile is an 825 GB diverse, open-source language modeling data set that consists of 22 smaller, high-quality datasets combined together). It is seen to show pretty good performance on Zero-shot tasks as well, For example — GPT-J is seen to outperform…