Find insights in text using Amazon Comprehend

Amazon Comprehend is a natural language processing (NLP) service that uses machine learning to find insights and relationships in text. No machine learning experience required.

In this lab, you will use Amazon Comprehend to understand the sentiment of three book reviews to help you determine if you want to buy the book.

Create project

  1. Create a new .NET Core console application project.

Comprehend

  1. Add AWSSDK.Comprehend Nuget package to the project:

Nuget

  1. Add the following import statements to Program.cs:
using System;
using System.IO;
using System.Text;
    
using System.Threading.Tasks;

using Amazon.Comprehend;
using Amazon.Comprehend.Model;
  1. Replace the Main method in Program.cs with the following async version:
static async Task Main(string[] args)
{
    if (args.Length != 1)
    {
        Console.WriteLine("Please provide text file.");

        return;
    }

    var filename = args[0];

    await ComprehendInputFile(filename);
}
  1. Add new method ComprehendInputFile. This method is the main one used by the Main to process the job. It reads the input text file and starts calling the other methods to analyze the input text.

Please note that when you initialize AWS SDK’s AmazonComprehendClient, you need to pass the RegionEndpoint of the region you are making labs in. The code below initializes AmazonComprehendClient in the EUWest1 region.

static async Task ComprehendInputFile(string fileName)
{
    var inputText = File.ReadAllText(fileName);

    var comprehendClient = new AmazonComprehendClient(Amazon.RegionEndpoint.EUWest1);

    Console.WriteLine("Calling DetectDominantLanguage\n");
    var text1 = await DetectDominantLanguage(inputText, comprehendClient);

    Console.WriteLine("Calling DetectEntities\n");
    var text2 = await DetectEntities(inputText, comprehendClient);

    Console.WriteLine("Calling DetectKeyPhrases\n");
    var text3 = await DetectKeyPhrases(inputText, comprehendClient);

    Console.WriteLine("Calling DetectSentiment\n");
    var text4 = await DetectSentiment(inputText, comprehendClient);

    var outputBuilder = new StringBuilder();
    outputBuilder.AppendLine(text1);
    outputBuilder.AppendLine(text2);
    outputBuilder.AppendLine(text3);
    outputBuilder.AppendLine(text4);

    var outputFileName = fileName.Replace(Path.GetExtension(fileName), "_comprehend.txt");
    File.WriteAllText(outputFileName, outputBuilder.ToString());
}
  1. Add method DetectDominantLanguage that creates an instance of DetectDominantLanguageRequest and calls method DetectDominantLanguageAsync:
private static async Task<string> DetectDominantLanguage(string text, AmazonComprehendClient comprehendClient)
{
    var stringBuilder = new StringBuilder();

    var detectDominantLanguageRequest = new DetectDominantLanguageRequest()
    {
        Text = text
    };

    var detectDominantLanguageResponse = await comprehendClient.DetectDominantLanguageAsync(detectDominantLanguageRequest);

    stringBuilder.AppendLine("Detect Dominant Language:");
    stringBuilder.AppendLine("==========================");

    foreach (var dominantLanguage in detectDominantLanguageResponse.Languages)
    {
        stringBuilder.AppendLine(string.Format("Language Code: {0}, Score: {1}",
            dominantLanguage.LanguageCode, dominantLanguage.Score));
    }

    Console.WriteLine("DetectDominantLanguage => Done\n");

    return stringBuilder.ToString();
}
  1. Add method DetectEntities that would create an instance of DetectEntitiesRequest and call the method DetectEntitiesAsync:
private static async Task<string> DetectEntities(string text, AmazonComprehendClient comprehendClient)
{
    var stringBuilder = new StringBuilder();

    var detectEntitiesRequest = new DetectEntitiesRequest()
    {
        Text = text,
        LanguageCode = "en"
    };

    var detectEntitiesResponse = await comprehendClient.DetectEntitiesAsync(detectEntitiesRequest);

    stringBuilder.AppendLine("Detect Entities:");
    stringBuilder.AppendLine("==========================");

    foreach (var entity in detectEntitiesResponse.Entities)
    {
        stringBuilder.AppendLine(string.Format("Text: {0}, Type: {1}, Score: {2}, BeginOffset: {3}, EndOffset: {4}",
            entity.Text, entity.Type, entity.Score, entity.BeginOffset, entity.EndOffset));
    }

    Console.WriteLine("DetectEntities => Done\n");

    return stringBuilder.ToString();
}
  1. Add method DetectKeyPhrases that creates an instance of DetectKeyPhrasesRequest and calls the method DetectKeyPhrasesSync:
private static async Task<string> DetectKeyPhrases(string text, AmazonComprehendClient comprehendClient)
{
    var stringBuilder = new StringBuilder();

    var detectKeyPhrasesRequest = new DetectKeyPhrasesRequest()
    {
        Text = text,
        LanguageCode = "en"
    };

    var detectKeyPhrasesResponse = await comprehendClient.DetectKeyPhrasesAsync(detectKeyPhrasesRequest);

    stringBuilder.AppendLine("Detect Key Phrases:");
    stringBuilder.AppendLine("==========================");

    foreach (var keyPhrase in detectKeyPhrasesResponse.KeyPhrases)
    {
        stringBuilder.AppendLine(string.Format("Text: {0}, Score: {1}, BeginOffset: {2}, EndOffset: {3}",
            keyPhrase.Text, keyPhrase.Score, keyPhrase.BeginOffset, keyPhrase.EndOffset));
    }

    Console.WriteLine("DetectKeyPhrases => Done\n");

    return stringBuilder.ToString();
}
  1. Add method DetectSentiment that creates an instance of DetectSentimentRequest and calls the method DetectSentimentAsync:
private static async Task<string> DetectSentiment(string text, AmazonComprehendClient comprehendClient)
{
    var stringBuilder = new StringBuilder();

    var detectSentimentRequest = new DetectSentimentRequest()
    {
        Text = text,
        LanguageCode = "en"
    };

    var detectSentimentResponse = await comprehendClient.DetectSentimentAsync(detectSentimentRequest);

    stringBuilder.AppendLine("Detect Sentiment:");
    stringBuilder.AppendLine("==========================");

    stringBuilder.AppendLine(detectSentimentResponse.Sentiment);

    Console.WriteLine("DetectSentiment => Done\n");

    return stringBuilder.ToString();
}

Run application

Download the three reviews using the following links, and make sure to save them locally:

book-review-01.txt

book-review-02.txt

book-review-03.txt

Now you can build the application and run it by passing the path to the sample text files:

Comprehend.exe C:\projects\book-review-01.txt

Comprehend.exe C:\projects\book-review-02.txt

Comprehend.exe C:\projects\book-review-03.txt

Check the folder with original text file, you should see the following new files:

book-review-01_comprehend.txt
book-review-02_comprehend.txt
book-review-03_comprehend.txt

Open the files and check the content, you will see if the review is positive or negative at the bottom.