Classifying tanks with the IBM Visual Recognition Service

The IBM Visual Recognition service just released (December 2, 2015) its  2nd beta version. The significant functionality released with this beta release is the ability to create your own image classifiers! Additionally, you can identify the classifiers you want to use when submitting an image to classify. So, this release allows you to create your own classifiers and leverage filtering to only see classifications you care about!

I had to test the ability to create my own classifiers. And I also wanted to test the quality of the classifiers on similar subjects of analysis. So I picked tanks! I thought of two tanks that can look very similar depending on their configuration and angles. So, I picked the M1A1 Abrams and the British Chieftain! Here is what I did…

Create the Visual Recognition Service and credentials

I have the Cloud Foundry tools installed on my MacBook Pro, so I simply executed the following commands to create a Visual Recognition Service and a set of credentials to use.

##Create the Visual Recognition service 
cf create-service visual_recognition free vrTanks

##Create credentials for the VR service instance
cf create-service-key vrTanks tanksCreds

##Dump the credentials so that I may use them
cf service-key vrTanksCreds

I now have a service instance and credentials…let’s get to classifying!

The data set

I obtain 100 images for each of the tanks I was interested in classifying from Google Images! That was 200 images in total. Why did I grab that many pictures of tanks? Here is the basic guidance from the Visual Recognition service documentation for creating a classifier:

  • supported format: jpg, png, gif
  • minimum size: 200×200
  • minimum number of positive images: 50
  • minimum number of negative images: 50

Additionally, the documentation states:

If you upload less than 50 images or an unequal number of positive and negative example images, the quality of the trained classifier declines.

My strategy was to select an even number of images for both tanks. I just doubled the minimum. More data, better classifiers….right?? The positive images are the images sent to train a classifier are just the images you want to classify…the negative images are those that are something else.

So when I create the m1 classifier, I sent the m1 images as the positive images and the chieftain images as the negative images. For the chieftain classifier, I reversed the images.

Creating the classifiers

Here is how I created the M1A1 Abrams classifier:

curl -u "{service uid}":"{service pwd}" \
 -X POST \
 -F "" \
 -F "" \
 -F "name=m1" \

The Service UID and PWD are the service credentials created earlier.  The name is the name of the classifier.

Similarly, this is how I created the Chieftain classifier:

curl -u "{service uid}":"{service pwd}" \
 -X POST \
 -F "" \
 -F "" \
 -F "name=chief" \

Once a classifier is trained based on the input data, the response contains the classifier id, name, and other information, similar to:

   "name": "tiger", 
   "classifier_id": "tiger_1234", 
   "created": "2015-11-23 17:43:11+00:00", 
   "owner": "dkfj32-al543-324js-382js" 

I now have two classifiers – one for classifying m1 tanks and the other chieftain tanks!

Testing it

I created a simple script that accepts an image and sends it to the Visual Recognition service for classification. Here is the script I used:

curl -u "{service uid}":"{service pwd}" \
 -X POST \
 -F "classifier_ids=<classifierlist.json;type=application/json" \
 -F "images_file=@$1" \

Note that I am also sending the contents of classifierlist.json. This is my filter and contains the classifier ids that I want to operate on my data. The contents are:

   "classifier_ids" : [ "m1_591027136", "chief_1987416538"]

Here are two results:

$ ./ test/m1a1test0.jpeg
"scores":[ {"classifier_id":"m1_591027136","name":"m1","score":0.704989}]}]

$ ./ test/ctest0.jpeg

And there you go! Each test case was classified correctly! The custom classifiers are high quality and can distinguish between two similar armored vehicles.

The next step is to add additional tanks to the solution.


You can find my scripts and data at github here: blog-tanks-visual_recognition

Cognitive Catalyst’s rr-bash-tools and the Watson Retrieve and Rank Service

In this post I’m going to leverage the rr-bash-tools from the Cognitive Catalyst project to interact with a Retrieve and Rank service on Bluemix. Basically, I’m just going to create a R&R service, load it with some content and conduct a quick search. So, this is a tutorial of sorts on how to use the rr-bash-tools.

First, though, what is the Cognitive Catalyst project? Well, Project Cognitive Catalyst is an open source community created by the IBM Watson Ecosystem. The purpose of which is to create a set of tools for use by the community to build cognitive applications! If you are interested in building cognitive applications check it out and see if any of the tools are useful to you. Or if you want to create or improve a tool used by the community, stop by and see where you might fit in….or if you have ideas there is even a wish list of projects.

Great, now what are these rr-bash-tools and how do they relate to the Watson Retrieve and Rank service? Ok, so, I created the rr-bash-tools initially to just wrap the curl calls required to interact with Retrieve and Rank. Also, I wanted to prototype some functionality for a set of nodejs tools that I will write next. Because I’m a firm supporter of the OSS community, I thought I might as well give access to the tools and, hopefully, make someone’s life a little easier. To the point, the rr-bash-tools are a set of bash scripts that invoke a Retrieve and Rank service by curl. And I’m sure the scripts could use some TCL from some shell hackers out there!

Before I start with how to use the rr-bash-tools, I assume the following are true:

  1. You have established an account on Bluemix
  2. You have installed the Cloud Foundry CLI tools

Once you’ve meet those requirements, you can follow along with the rest of the this post….or you can just read it too.

The first step is to create a Retrieve and Rank service, using the Cloud Foundry CLI tools:

mb:blog cmadison$ cf create-service retrieve_and_rank standard blogRR
Creating service instance blogRR in org Madison / space blog as

Attention: The plan `standard` of service `retrieve_and_rank` is not free. The instance `blogRR` will incur a cost. Contact your administrator if you think this is in error.

I provided the create-service command the service name (retrieve_and_rank), the service plan (standard) and the R&R instance name (blogRR). Your service plan will vary depending on if you have a partnership agreement with IBM or not. Note that the standard plan will incur a cost if you exceed the free tier!

Now that we have a service, we can create the service credentials that will allows us to authenticate with the service:

mb:blog cmadison$ cf create-service-key blogRR blogkey
Creating service key blogkey for service instance blogRR as

mb:blog cmadison$ cf service-key blogRR blogkey
Getting key blogkey for service instance blogRR as
 "password": "olTpcGdZh7C7",
 "url": "",
 "username": "1559a1dc-67c9-4a0c-9414-089ae6f09665"

Here I executed two commands, the first creates the service key (credentials) for the blogRR Retrieve and Rank instance. The second command retrieves the credentials. Don’t worry I will remove this set of credentials before I publish this post!

Now, I want to get the rr-bash-tools so that I can start working with my instance. To do that, I just use git and clone the repository:

mb:blog cmadison$ git clone
Cloning into 'rr-bash-tools'...
remote: Counting objects: 36, done.
remote: Total 36 (delta 0), reused 0 (delta 0), pack-reused 36
Unpacking objects: 100% (36/36), done.
Checking connectivity... done.

Now that we have the rr-bash-tools locally, let’s see what is there.

mb:rr-bash-tools cmadison$ ls
LICENSE    bin     data    lib     samples  config  deploy  output

I have the tools and the credentials, so let’s set the configuration file and get started! The only configuration, at this time, is to create a credentials file in the the configuration directory.

mb:rr-bash-tools cmadison$ echo 1559a1dc-67c9-4a0c-9414-089ae6f09665:olTpcGdZh7C7 > config/creds

That’s it! I just create a simple credentials file in the configuration directory. Curl will use this information to interact with your Retrieve and Rank service. Now let’s create our SOLR cluster.

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster
 % Total % Received % Xferd Average Speed Time Time Time Current
 Dload Upload Total Spent Left Speed
100 177 0 145 100 32 340 75 --:--:-- --:--:-- --:--:-- 340
copying solr config to ../lib

mb:bin cmadison$ ls ../lib

When I execute the script, I am prompted for the cluster name. The return from the request to create the cluster is the SOLR configuration – blogCluster_solr_config in this case.

You can check the status of cluster by using the script, like so:

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster

When you cluster’s status is READY your cluster is ready for configuration and use!

The rr-bash-tools have sample data for use that will allow you get up and running and help you learn how to interact with the Retrieve and Rank service. To configure SOLR with the sample’s configuration, execute the follow:

mb:bin cmadison$ cp -r ../samples/base_config/* ../lib/blogCluster_solr_config

That will copy the sample’s schema into your cluster’s configuration. Now load the configuration and create a collection.

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster
 adding: currency.xml (deflated 62%)
 adding: lang/ (stored 0%)
 adding: lang/stopwords_en.txt (deflated 44%)
 adding: protwords.txt (deflated 45%)
 adding: schema.xml (deflated 74%)
 adding: solrconfig.xml (deflated 66%)
 adding: stopwords.txt (deflated 42%)
 adding: synonyms.txt (deflated 44%)

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster
<?xml version="1.0" encoding="UTF-8"?>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1974</int></lst><lst name="success"><lst><lst name="responseHeader"><int name="status">0</int><int name="QTime">1706</int></lst><str name="core">blogCluster-collection_shard1_replica1</str></lst></lst>

The script, will zip up the local configuration files and send them to your cluster. The script creates a collection for our content.

Let’s load the sample content into Retrieve and Rank!

mb:bin cmadison$ x2sj -i ../samples/nimh.xlsx -o ../output/nimh.json

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster
Enter the collection's path : ../output/nimh.json
[snip...HTTP info removed]
* Connection #0 to host left intact

The first command, x2sj, is a nodejs utility to transforms an Excel spreadsheet into the JSON expected by the Retrieve and Rank service. The x2sj utility is also a Project Cognitive Catalyst module.

Once we have an uploaded set of documents, we can load the collection with the script.

Now we can conduct a quick test using the script.

mb:bin cmadison$ ./
Enter the cluster's name : blogCluster
Ask question : anorexia&wt=json&fl=id,title
{"responseHeader":{"status":0,"QTime":50,"params":{"q":"anorexia","fl":"id,title","wt":"json"}},"response":{"numFound":7,"start":0,"docs":[{"id":"ed9","title":["Eating Disorders : Treatment : Anorexia nervosa"]},{"id":"ed3","title":["Eating Disorders : Signs and Symptoms : Anorexia nervosa"]},{"id":"ed7","title":["Eating Disorders : Diagnosis"]},{"id":"ed1","title":["Eating Disorders : What are Eating Disorders?"]},{"id":"ed4","title":["Eating Disorders : Signs and Symptoms : Bulimia nervosa"]},{"id":"ed6","title":["Eating Disorders : Who is at risk?"]},{"id":"ed10","title":["Eating Disorders : Treatment : Bulimia nervosa"]}]}}

Perhaps, ‘ask’ and ‘ask question’ isn’t really the appropriate terms. I am really just searching on keywords, in this case, anorexia. I also directed Retrieve and Rank to return the results in JSON format (wt=json) and requested that only the id and title fields (fl=id,title) be returned.

That’s it! A quick tutorial on using the rr-bash-tool and Retrieve and Rank to get up and running on Bluemix FAST! And, yes, I did remove the credentials I displayed earlier!

Introducing IBM’s Retrieve and Rank Service


IBM recently released the Retrieve and Rank service on Bluemix. Retrieve and Rank is the final release of the BlueGoat services (Dialog, Natural Language Classifier, and Retrieve and Rank). The BlueGoat services represent IBM Watson’s commitment to migrating the Watson Advisor products onto the cloud. BlueGoat targets the migration of the Watson Engagement Advisor onto the Watson Developer Cloud.

For some trivia, the BlueGoat name is derived from:

  • Blue = IBM
  • Goat (also translated as Ram or Sheep) = Chinese year of the release of the services

As one might guess, the Retrieve and Rank service is composed of two primary components. The first is…Retrieve! The Retrieve component wraps Apache SOLR and provides search capabilities across content. A majority of the SOLR functionality and APIs are provided through the Retrieve service. An added benefit is that there is a wealth of information on the Internet to support your Retrieve endeavors!

Additionally, using SOLR is much easier to get started on the Bluemix platform using the Retrieve and Rank service. When you create a Retrieve and Rank service in Bluemix, an entire SOLR cluster is allocated for you to use. That’s it! You do not need to install SOLR. You create your service, invoke the API and you have an Apache SOLR cluster available for you to configure, load and use.

The other component – and the component that adds cognitive capability – is Rank. The Rank component uses several learning-to-rank machine learning algorithms to reorder the Retrieve search results. Learning-to-Rank is a supervised machine learning approach to rank search results. You train the Rank component by creating a CSV of which each row contains a question and a set of answer ids and relevancy rankings. In the future, I’ll explore the training aspect in a little more detail. In any case, once the a ranker is trained, you may identify the ranker in the search request to obtain a ranked order or search results.

Process Overview

So, great, IBM has this cool service that allows me to use SOLR for searching and a cognitive component for the ranking of results….but….there is more to it than spinning up a Retrieve and Rank service, what does the end-to-end process look like?

That is a great question! Generally, there are four steps or processes to consider: Curation, Configuration, Training and Use and feedback. The steps are not mutually exclusive and somewhat depend on each other. For example, curation may impact configuration as new content types are added, which requires a schema update. And new content may impact the trained model necessitating additional training. The point is that the processes are somewhat dependent on each other. Let’s briefly go through each of the processes.

Curation: The curation step is simply deciding which content to use in your application! This might come from your proprietary data sources, 3rd parties, open information or a combination of those sources. Once you have your hands on the content you want to leverage, there may be formatting, conversion and modification to take into account. Curation is one area of developing a cognitive solution that seems to be overlooked in planning. Ensure that you think through how you are going to acquire the content and make it useful to your application.

Configuration: Configuration focuses on configuring SOLR! You focus on a couple of things here. The two main (there are others, most notably synonyms.txt) configuration files to consider are solrconfig.xml and schema.xml. The configuration file, solrconfig.xml, is the basic SOLR configuration file that configures the core functionality provided by the service. The schema file, schema.xml, defines the metadata that is applied to each document. The schema also impacts indexing, document identification and faceted search.

Training: Here you focus on collecting relevant questions for your domain and providing ranking information. Ranking information includes the set of answers and their relevancy to the question. This collection of information is referred to as the ‘ground truth’ of your domain. The ground truth is submitted to the ranker for training. Once the ranker converges on a solution it becomes ready to rank search results!

Use and feedback: Simply having users use your system and determine if improvements in answers (training) and coverage (content) can be made!


So that’s it in a nutshell…Retrieve and Rank is an exciting new service offered by the Watson Developer Cloud. It provides searching (SOLR) and ranking (machine learning) functionality in a simplified deployment scenario in the cloud. In the future, I’ll explore other aspects of Retrieve and Rank and post my findings here!