Abstractive summarization using the Hosted Inference API on Hugging Face - Hugging Face Tutorial

We've seen that when we use abstractive text summarization models to generate a summary of longer text by creating new sentences that capture the essence of the original content. Abstractive summarization involves generating novel sentences that convey the key points in the original text in a more humanlike and concise manner. We've also discussed that Abstractive summaries are usually produced by large language sequence to sequence models, typically transformer models. Hugging Face has a collection of transformer models that perform text summarization. And here, let's see how we can use the hosted inference API for these models to generate summaries. Click on the models link here on Hugging Face and you'll be taken to a page where you'll have access to all of the available models in this community. We are interested in summarization, select the summarization task under natural language processing. Now here are all of the models available on Hugging Face for text summarization. Let's pick the first one, the Bart model from Facebook. Every model available on Hugging Face has a model card, and you can see that we have the model card tab selected. This is a document that provides important information and details about the pre-trained model that you're looking at. Bart is a sequence-to-sequence transformer model, so it uses attention. It has a bidirectional encoder. A bidirectional encoder looks at all of the words in the input simultaneously, and not in sequence. The model card gives us information about the key characteristics of the model, its intended use cases, its performance, limitations, and ethical considerations. Observe here on the bottom right that we have the ROUGE scores for this summarization model on different data sets. This is a fairly good model for summarizing news articles. Now here, off to the top right, notice that this model offers a hosted inference API. The hosted inference API on Hugging Face allows developers to easily use and integrate pre-trained models from the Hugging Face library without the need to set up and manage the models themselves. What we'll do here is simply use this hosted inference API to summarize the text that we are interested in. The text that I've chosen is a little longer than the Empire State article. This is an article on the Panama Canal from the history.com site. This is the article for which we'll generate a summary. So copy this over and let's switch back to Hugging Face and paste this in into this text box which will hit the hosted inference API for this Bart model. We have the article that we want to summarize. I simply click on the "Compute" button here just below the input text box, and this is what will generate the summary using this Bart transformer model. And here's what the summary looks like. You can see the summary produced here in green. It's just a few sentences, but it's an abstractive summary. None of these sentences are likely to be in the original text. Now, most of the NLP models available here on Hugging Face offer a hosted inference API. Let's try another summarization model. I click on summarization here so that I get the models that specialize in summarization. And let's scroll down and pick one of these. How about a model from Google? We had one from Facebook. I'm now going to pick a model from Google. This is the bigbird-pegasus-large model. It's once again a transformer model and it uses a kind of attention described as sparse attention. Other details about this model are available here on this model card. And here at the bottom, you can find the ROUGE scores for this model on different datasets. Look at the first three ROUGE scores on scientific_papers. Those ROUGE scores are rather high. But look at the ROUGE scores on the cnn_dailymail datasets, those ROUGE scores are much lower. This model is likely to perform better on scientific papers than, say, news articles, just by looking at this model card. Let's use its hosted inference API. Here is the same article on the Panama Canal. I'm going to click on "Compute", and let's take a look at the summary that this model produces. And this summary is definitely not a great one. It talks much more about the locks and the canal then an overall description of the canal. Well, here are two models for which we use the hosted inference API and we saw how they've performed. There's one more model that we can try. Let's go back to summarization and let's pick the Google's pegasus-cnn_dailymail model. This model seems to be fine-tuned on the cnn_dailymail dataset, which is a news article dataset. This model will likely perform better with our Panama Canal article. Let's go ahead and click on |"Compute" and see what the summary looks like. And you have to admit that this summary is a lot better than the BigBird Archive model. That model was likely trained on scientific papers. It's probably good for scientific summaries, but did not do well on regular articles. So when you're picking a summarization model for your use case, make sure the model has been trained or fine-tuned on a dataset similar to the one that you plan to use.