Photo by Niclas Illg on Unsplash

Google Assistant — Customizing Template Action

Google Assistant is a personal voice assistant that offers a host of actions and integrations. From sending texts and setting reminders, to ordering coffee and playing music, the 1 million+ actions available suit a wide range of voice command needs. Google Assistant is offered on Android and iOS, but it can even be integrated with other devices like smartwatches, Google Homes, and Android TVs.

Actions templates allow you to build Google Assistant applications like trivia games or flashcard generators without writing a single line of code. You can integrate Speech Synthesis Markup Language (SSML) to give your application more character and make responses seem more life-like. Google Assistant has also added multilingual capabilities, which allows you to expand your application to a much a wider audience.

We will build a trivia Assistant application from Actions template that utilizes SSML responses and multilingual capabilities. This will give your Assistant application unique sound effects and an option to be used in another language. You will add these features by leveraging GCP services like Cloud Storage and the Cloud Translation API and by the end you will have built a robust Assistant application.

What you will Learn

  • Build an Assistant application with the flash cards template.
  • Host custom MP3 files in Google Cloud Storage and integrate them with your Assistant application.
  • Generate flash card questions and answers in different languages with the Cloud Translation API and add them to your template.

Prerequisites

  • Google Account
  • Google Cloud (You should have enabled billing)
  • Headphones (Optional to listen)

Let’s Start ……………

Create an Action’s Project

Open a new tab in your browser and go to the Actions on Google Developer Console.

Click on Add/import project and then enter your project name. Once the project is created you will see a welcome screen like below

Scroll down and find the “More options” header. Then click on the Templates card.

Add the trivia Actions template

From the three options of templates, select Flash cards. You have now added a flash card template as an Action to your project.

You should see the flash cards Action instantiated and your page should resemble the following:

In the Personality section, listen to the personality options. Select the one you like the most and click Next.

Now that you’re in the content section, select UPLOAD SHEET > Make a copy of the pre-filled Google Sheet for editing > Make a copy. You should now be on the following Sheets page:

Now add the following questions, answers, and hints to the bottom of this spreadsheet(Later you can add your own)

These will be rows 39–41 in the spreadsheet. Now that you’ve added some new flashcard questions, you will learn how to integrate custom sounds with your Actions template. Keep this Google Sheet tab open.

Create a Cloud Storage bucket

The flashcard template has additional configuration parameters that we can use to make our application more personalized. In this section, we will be working on integrating our own sounds for AudioGameIntro, AudioGameOutro, and AudioDing parameters.

Once there, open the navigation menu and click Storage > Browser > Create bucket. click Enable Billing

Enter in a unique name for your bucket and keep all the other default settings the way they are. Then click Create:

Copy MP3 files to your Cloud Storage bucket

The Youtube Audio Library lets you browse and download royalty free music and sound effects. Learn more about the YouTube Audio Library and read the terms of service below:

Three MP3 sample files have been downloaded from the Youtube Audio Library and are stored in a Cloud Storage bucket. You will now copy these files to the Cloud Storage bucket you just created.

Start a new session in Cloud Shell and run the following commands to copy and unzip the MP3 file folder into your home directory:

gsutil cp gs://spls/gsp280/gsp280-mp3-files.zip . 
unzip gsp280-mp3-files.zip

Now copy the unzipped folder to your Cloud Storage bucket by running the following command, replacing <YOUR_BUCKET_NAME> with the name of your Cloud Storage bucket:

gsutil cp -r gsp280-mp3-files gs://<YOUR_BUCKET_NAME>

// is important without that you will get an error

You should receive the following output:

Now examine the contents of your Cloud Storage bucket. Refresh your bucket and click on the newly added gsp280-mp3-files folder. Banjo_Short.mp3 will be used for the AudioGameIntro, Country_Cue_1.mp3 as the AudioGameOutro, and Beep_Short.mp3 as the AudioDing.

Add MP3 files to the template

Go back to your Copy of Flash Cards Game sheet (where you previously added three new trivia questions and answers.) At the bottom of the page find and click the Configuration tab. Now add the following values to the Key column:

  • AudioGameIntro
  • AudioGameOutro
  • AudioDing

For the values, you will be using the MP3 files that you uploaded to Cloud Storage. Run through the following steps to add the MP3 files as configuration parameter values:

  • In your Cloud Storage bucket click on the name Banjo_Short.mp3. This will open an audio player in a new tab.
  • Copy the URL.
  • Paste the URL in for the AudioGameIntro value in Copy of Flash Cards Game.

Build and test your Action

Once you’re finished modifying the sheet, copy the URL of Copy of Flash Cards Game and return to the Actions console. You should have left off on the following page:

Click Next and paste in the URL for the Copy of Flash Cards Game Sheet that you’ve been editing. Then click Upload. Your sheet will be validated and you should see the following output:

Click on the CREATE APP button. Actions are now building an Assistant application from the trivia spreadsheet. It might take a couple of minutes for the application to build. You can move on to the next step in the meantime.

Check your Google permission settings

To test the Action, you need to enable the necessary permissions.

Go to the Activity Controls page and sign in.

Ensure that the following permissions are enabled by sliding the toggles and selecting TURN ON for the following cards:

  • Web & App Activity
  • Device Information
  • Voice & Audio Activity

Now close the Activity Controls page.

Test the application with the Actions simulator

Return to the Actions console and verify that your application has been built. Your page should resemble the following:

Click on the TEST YOUR ACTION button. This will bring you to the Actions Simulator page, where you can test your Assistant application without the need for a device like a Google Home.

Be sure to use headphones or turn up the volume on your speakers to hear the new MP3 samples you just added!

To invoke the Action, hit enter in the Talk to my test app box near the bottom of the simulator console. You will be asked if you want to play a round of flashcards:

You should hear Banjo_Short.mp3 play after "Alright. Getting the test version of my test app". Now enter Yes when prompted if you are ready to play. You will then be given a baby animal flashcard:

You should have heard Beep_Short.mp3 play after the "What's the baby animal name for ?" Go through the short Flashcard game, answering the questions to the best of your abilities.

At the end, you will be prompted with “Would you like to go forth to another set?” Enter in No and hit enter. After some brief dialogue, you will hear the AudioGameOutro loop Country_Cue_1.mp3.

By using the Youtube Audio Library and Cloud Storage you were able to add custom MP3 files to your Assistant application, which gives it a unique, personalized touch.

Add a flashcard template in another language

Now that you’ve gotten hands-on practice integrating audio files with your Assistant application, you will take it to the next level by adding a version in another language.

In the Actions console, select Build > Actions from the left-hand menu.

Click the Modify languages button in the top right corner. Select Spanish and then click Save in the top right corner.

Now click Actions from the left-hand menu. Your flash card Action should now resemble the following (note the Spanish tab):

Click on the Spanish language tab and select Beeps the Robot as your personality. Then click Next.

Now that you are in the content section, click UPLOAD SHEET > Make a copy of the pre-filled Google Sheet for editing > Make a copy.

You will now have a copy of the flash card sheet in Spanish:

Keep this tab open — you will be adding content to it over the rest of the lab.

Enable the Translation API

To keep our application versions consistent with one another, we will need to translate the three trivia questions we added to the English template to Spanish. You will use the Cloud Translation API to accomplish this.

Return to the GCP console. From the Navigation menu select APIs & Services > Library.

Then search for the Cloud Translation API and select the API from the results. If the API is disabled, enable it. Your page should resemble the following:

Create a service account key

Before we can use the Cloud Translation API, we will need to create a service account key.

Open the Navigation menu and select APIs & Services > Credentials. Then click the Create credentials dropdown menu and select Service account key.

From the Service account drop down menu select New service account. For the service account name, type in a translation-api. For the role field, select Project > Editor.

Once filled in, click Create:

Select Service account key and fill the details

Click create a JSON file will be downloaded.

Open up a Cloud Shell session and in the right-hand corner select the menu item (three dots) and click Upload file.

Select the JSON service account file that you downloaded in the previous step (name should resemble gcp-xxxxxxxxxxxxxxxx-xxxxxxxxxx.json) and click Open. You now have your service account key in the right place and you are ready to use it with the translation API.

Run the following command in Cloud Shell to get the complete path to your service account file, replacing <YOUR-SERVICE-ACCT-KEY> with the file's name:

readlink -f <YOUR-SERVICE-ACCT-KEY>

Your output should resemble the following:

/home/nidhinkumar/customize-gcp-6dd23d7d3eeec7e9-509abe66b702.json

Now run the following command to set the environment variableGOOGLE_APPLICATION_CREDENTIALS to the path of the service account key. Replace [PATH] with the output of the previous step:

export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

Use the Translation API

Now that your service account credentials are in the proper place, you are ready to make translation API requests. You will be translating the English flashcard questions we added earlier into Spanish.

Be sure to keep your Copy of Flash Cards Game (es) sheet handy since you will be copying text that is outputted from the translation API and adding it to the sheet. Here are the questions and answers that we added before:

Run the following command in Cloud Shell to translate the flash card questions into Spanish using the Cloud Translation API:

curl -X POST \
-H "Authorization: Bearer "$(gcloud auth application-default print-access-token) \
-H "Content-Type: application/json; charset=utf-8" \
--data "{
'q': 'What\'s the baby animal name for bat?',
'q': 'What\'s the baby animal name for cow?',
'q': 'What\'s the baby animal name for snake?',
'target': 'es'
}" "https://translation.googleapis.com/language/translate/v2"

You should receive the following output:

{
"data": {
"translations": [
{
"translatedText": "¿Cuál es el nombre del animalito para el murciélago?",
"detectedSourceLanguage": "en"
},
{
"translatedText": "¿Cuál es el nombre del animalito para la vaca?",
"detectedSourceLanguage": "en"
},
{
"translatedText": "¿Cuál es el nombre del animalito para serpiente?",
"detectedSourceLanguage": "en"
}
]
}
}

Copy the translated text and paste it in your sheet

By leveraging the Translation API, you were able to convert your questions, answers, and hints into another language (and all from Cloud Shell!) As you continue building more intricate Assistant applications, you will begin to see just how important and necessary APIs are for accessing external services and data.

Now you can build your application as we did previously and test the app in the Spanish language

If you don’t want to use translation API there is a simple approach

Create a new Google Sheet and place the text which you want to convert

=GOOGLETRANSLATE(A2, "en", "es")

once done your text will be converted to Spanish you can just copy and paste in the copy of the flash card.

Congratulations!

Great job on building a Google Assistant application with templates! You are well on your way to becoming a Google Assistant Developer.

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Conditional Statements 101

8 Popular frameworks to help you build websites fast

Php framworks

A Complete Guide to Principal Component Analysis — PCA in Machine Learning

Extensible telematics data pipeline — for time-series data visualization

Flutter Deep Dive Part 1: “RenderFlex children have non-zero flex…

Prototyping Interactive Experiences — Week 2

Citizen Development — What is it?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Nidhin kumar

Nidhin kumar

More from Medium

Choosing the right tools for your backend

Integrating Tauk into your Appium Python tests

Getting Started with Contract Test using Spring Cloud Contract

REST API, Swagger, Postman, and REST API Playground