[2016] Deciding the best companion for TGS2016 using Microsoft's Emotion API [Review]

Hello.
I'm Mandai, in charge of Wild on the development team.

My wife told me to cut and bevel a pumpkin because she was going to make a pumpkin stew, so I cut a bevel for the first time in a while.
We must learn from the delicacy of Japanese cuisine.

Today I wanted to use Microsoft's Emotion API, so I was looking for a story, and the image I had on hand that seemed like a good story was from a companion from Tokyo Game Show 2016 (hereinafter referred to as TGS2016). About the image.
That's why we would like to hold a "2016 Review Project! TGS2016 Best Companion Competition"!

Preparing to use Emotion API with a Microsoft account

You will need an API Key to access the Emotion API, so first prepare to use the Emotion API from the Microsoft Cognitive Services

Proceed to the login screen from the "My Account" link in the upper right corner.

• Microsoft account
• Github account
• LinkdIn account

It seems that you can log in from these three.
It was a mystery why LinkedIn existed, but now that I think about it, it was acquired by Microsoft.

Once you have successfully logged in, enable the API you want to use from "Get started for free" next to "Sign Out" in the upper right.

This time, we will enable Emotion API.
Check the select box where "Product Name" is "Emotion", and also check "I agree to the Microsoft Cognitive Services Terms and Microsoft Privacy Statement" at the bottom.
After that, just press "Subscribe" and you're ready to go.

When the screen changes, you will see a list of currently enabled APIs.

Currently (December 27, 2016), the Emotion API is in preview, so you can use it for free up to 30,000 times a month.
The pricing structure is slightly different from GCP's Vision API, but up to 1000 units per month are free.
The term unit is a little difficult to understand, but when it comes to analysis items, it is possible to obtain data for multiple items in one image, such as 1 unit for face recognition, and 1 more unit for text recognition. I can do things.

Going a little deeper, Vision API is characterized by a larger amount of information.
Even with facial recognition, it is possible to obtain detailed information on the positions of facial parts, which may be useful for later processing.

Back to the topic, once you enable Emotion API, you need to obtain an API Key.
When you enable the API, two API keys are enabled, so you can use either one.

In the list, the API key is hidden with an "X" in the Keys field, so you can copy it by pressing Show or by clicking the Copy link.

Once you have successfully obtained the API Key, your work here is complete!

 

If you want to try it out, curl seems like a good option.

If you have a Linux environment at hand, you can quickly try it out using curl.

The Emotion API documentation includes a sample that sends the URL of a file on the Internet in JSON format, but this time we will try sending a local binary file directly from curl.

curl -v -X POST "https://api.projectoxford.ai/emotion/v1.0/recognize" \ -H "Content-Type: application/octet-stream" \ -H "Ocp-Apim-Subscription-Key : [API Key]" \ --data-binary "@[/path/to/image]"

If you want to send the file directly

• Content-Type to "application/octet-stream"
• --data-binary option, image path

Enter.

If you receive response number 200 and JSON data like the following, it is a success.
(It's hard to read, so it's formatted with line breaks and tabs, but it's one line of data.)

[ { "faceRectangle":{ "height":184, "left":223, "top":217, "width":184 }, "scores":{ "anger":2.41070044E-08, "contempt": 4.531843E-06, "disgust":7.3893716E-07, "fear":1.44139625E-08, "happiness":0.9999242, "neutral":6.80201556E-05, "sadness":3.14932123E-07, "surprise" :2.19046137E-06 } } ]

If you can do this far, you can have a lot of fun.

 

Try rewriting it to PHP

You can do a lot of interesting things with the acquired analysis data, such as processing it and creating and arranging web pages.
This time, I want to play around with the analysis data in the DB, so I'll rewrite it to PHP.
However, just run the above command.

$url = 'https://api.projectoxford.ai/emotion/v1.0/recognize'; $subscription_key = 'your api key'; $path = '/path/to/image'; $ch = curl_init($ url); curl_setopt_array($ch, [ CURLOPT_HTTPHEADER => [ 'Content-Type: application/octet-stream', 'Ocp-Apim-Subscription-Key: '. $subscription_key, ], CURLOPT_POST => true, CURLOPT_RETURNTRANSFER => true , CURLOPT_HEADER => true, CURLOPT_VERBOSE => true, CURLOPT_POSTFIELDS => file_get_contents($path), ]); $res = curl_exec($ch);

All you have to do is create a list of files, loop it, and analyze a large number of images.
However, there is a limit of 20 times/minute, so don't forget that you will need to adjust the time using sleep, etc.

I tried analyzing about 200 images, and the results were good.
It would be a waste to hit it over and over again due to the limit on the number of uses, so it would be a good idea to store the acquired data in a DB.

 

The data is complete. Let's go to the examination...!

I felt like I had made a lot of things and was done with them, so I thought it was time to wrap things up, but there's still the judging to decide the most important number one!

The analyzed data includes data on what emotions are included in facial expressions and in what proportion.
The emotions classified are as follows.

• anger
• contempt
• disgust
• fear
• happiness
• neutral
• sadness
• surprise

This classification is a preview version as of December 27, 2016, so it may increase or decrease in the future.
It is also possible that the data format changes to something completely incompatible.

Also, these data are provided so that the total is 1.
In other words, you can multiply each number by 100 and see it as a percentage of the total.

Out of the above emotions, the only one that seems to be usable this time is ``happiness'', so we will judge based on this data.
(However, the analysis has been completed at this point, so it is safe to say that the review has been completed.)

 

The result of shock... and.

Perhaps because of her job, she is used to smiling, and her happiness score was quite high, making this examination difficult.

The reason is that there were 21 pieces of data that only included happiness!

In other words, "Happiness 100%".

I am filled with the desire to commend all the companions who achieved that score.

However, there is one person who is by far the most amazing and is reflected in all 6 photos and I am 100% happy in all of them.

I was afraid.

That's this image! ! !

There are five of us, and I'm the one in the middle.

Akaranger in Goranger.

Ikariya Chosuke in The Drifters.

Ginyu in the Ginyu Special Sentai.

That's what it means.
You are in the middle as much as you want to be.

wonderful.
I was impressed.

I hope to see you at next year's game show.

That's it.

 

Who said there would be a grand prize?

But.
There was an image of pushing aside such a strong force that deeply pierced my heart.

That is this image.

Of course, the score is 100% Happiness.
Data doesn't lie.

Are you a Beyond person?

However, even if I am called a nepotistic person, I have no regrets about this decision.

I am wearing the Little Red Riding Hood costume that I borrowed from IIJ, so this photo was taken before the start of the second day.
She must have been tired from standing all day the day before, but she had a wonderful smile on her face.
I think there may have been some jobs that were successful as a result.

As one of the people involved, I felt deeply touched by the results, so I would like to award this image the grand prize.

Well, I'm glad it all fit together!

Lastly, I would like to close with a close-up of the grand prize winner.

It's you.

That's it.