Cats or Croissants

As the very cool picture below shows, an orange tabby cat can look surprisingly similar to a croissant. This naturally raises the important question: If a human (such as myself) can be so fooled, what about an image prediction API?

(I want to point out that I am perfectly capable of figuring out which picture is of a cat, and which is of a croissant.)

To answer the question in question—that is, "Can we use an image prediction API to distinguish cats from crossaints"—I used the Google Vision API. Each of the images in the above image was cut using GIMP and then fed into the box on the Google Vision API website labelled "try the API". The table below shows the result of doing so:

#	True label	Predicted label (% confidence)
1	Cat	Cat (99%)
2	Croissant	Croissant (89%)
3	Two cats (very cute)	Cat (%87)
4	Two croissants (very delicious)	Croissant x2 (80%)
5	Croissant	Croissant (%88)
6	Cat on a mat	Cat (98%)
7	(Two?) Croissant(s) on a plate	Croissant (93%)
8	Cat wit a blue collar	Cat (89%)
9	Small cat	Cat (98%)
10	Croissant with nuts	Bread (64%)
11	Cat	Cat (98%)
12	Croissant with powdered sugar	Croissant (94%)
13	Croissant	Croissant (92%)
14	Cat	Cat (99%)
15	Croissant with background croissants	Croissant x2 (92%, 79%)
16	Cat	Cat (99%)

Interestingly, the API can indentify multiple objects, which is why pictures with mutliple croissants get multiple labels. Perhaps just as interestingly, this didn't apply for pictures with more than one cat. (Perhaps Google discriminates against cats?) Note also that the croissant with nuts (picture # 10) was identified as bread. A disucssion on whether or not a croissant with toppings is in fact still a croissant is out of scope for this article though.

So, in conclusion, if you unsure about whether to comment "cute!" or "delicious!" the next time someone uploads a picture of something orange on their social media, you can ask google for help.