Why Is No One Making A Chatbot?

This is the creation of yet another category on my blog, because I’ve developed yet another interest: artificial intelligence! Or, rather, I’ve decided to start writing on an interest I’ve held for a long time. Thus, while I’ve always had an interest in computer programming and specifically artificial intelligence, I’ve never before had an interest in writing about it. That is, not until this day! Woo!

Anyways, I’m sure most everyone is familiar with IBM’s Watson program having recent dominance in the show “Jeopardy!”. If you are not, or have not seen it directly, or just want a link to the videos, you really should watch them — they are quite entertaining, at least: Day 1 (Part 1 | Part 2), Day 2 (Part 1 | Part 2), Day 3 (Part 1 | Part 2). There also is a cool Nova documentary on the making of Watson, as well as an interview by some of the team.

However, my blog post was not just to share cool links. My point is that Watson is only the beginning. Watson was designed to answer Jeopardy questions through an interpretation of the natural language involved in creating the questions. A lot of these questions are extremely complicated and punny, on a depth that one would not expect a computer to understand. However, there is a much bigger field out there for computers to achieve — the entire field of natural language. It’s quite simple: forget Jeopardy questions — the next challenge is a computer that can understand any input a human can.

 

 

Intelligence and the Turing Test

Perhaps this is the “holy grail” of computational understanding, but there are simpler implementations. The most basic is still ridiculously hard to master — the Turing test put forth by Alan Turing in 1950. In this test, a judge will talk to a human and a computer via a text-only chat interface, and have to determine which is which. If the computer can successfully fool the judge into thinking the computer program is in fact the human 50% of the time (de facto indecision), the computer is said to have “mastered human conversation”.

Of course, what level of mastery this truly represents remains an open question. As Russel and Norvig mention:

AI researchers have devoted little effort to passing the Turing test, believing that it is more important to study the underlying principles of intelligence than to duplicate an exemplar. The quest for “artificial flight” succeeded when the Wright brothers and others stopped imitating birds and started using wind tunnels and learning about aerodynamics. Aeronautical engineering texts do not define the goal of their field as making “machines that fly so exactly like pigeons that they can fool even other pigeons.”

 

Perhaps the ability to replicate human conversation (machines that talk so exactly like humans that they can fool even other humans) should not be the goal of AI, but it still would be a very good measurement — if a machine can pass a Turing test, it very likely is capable of interpreting quite a lot of natural language input. For example, consider the wide variety of responses a machine would have to interpret:

  • If you are human, you should be able to add two and two together and tell me the result.
  • If you are human, you should be able to tell me which is bigger: Scotland or a grapefruit?
  • If you are human, you should be able to assign x the value 4+y, assign y the value 6, assign z the value x+2, calculate y+x, and tell me the result.
  • If you are human, you should be able to tell me which of these is not a fruit: apple, banana, chair, orange.
  • If you are human, you should be able to tell me: how many fingers does Barack Obama have?

In addition to to this, thousands, perhaps millions, of other possible questions could be created that would test simple, common sense.

 

Currently, just ask these five questions and you can tell a computer apart from a human without fail. The famous WolframAlpha (which is really neat and indeed listening) gets caught up on the “If you are human” conditional and is unable to answer the question. It doesn’t even understand the input “add two and two together”, though it does know “two plus two”. WolframAlpha also skips out entirely on size comparisons, knowledge of fruit vs. non-fruit, fingers, and algebra, though it can answer a simplified “x = 4+y, y = 6, z = x+2, calculate y+x”.

If we take this trivia quiz to the most famous chatbot, Cleverbot, who is supposed to have some actual intelligence, we get:

Me: Are you human?
Cleverbot: Yes I’m human.

Me: If you are human, you should be able to add two and two together and tell me the result.
Cleverbot: For God so loved the world that he gave his only begotten son so that whoever believes in Him will have eternal life.

Me: If you are human, you should be able to tell me which is bigger: Scotland or a grapefruit?
Cleverbot: I like talking though and was not forced to come here, I wanted to. Oh you misspelled own.

Me: If you are human, you should be able to assign x the value 4+y, assign y the value 6, assign z the value x+2, calculate y+x, and tell me the result.
Cleverbot: I am sorry for having offended thee.

Me: If you are human, you should be able to tell me which of these is not a fruit: apple, banana, chair, orange.
Cleverbot: No, not at all. There will be some similarities, but your intelligence will never be a match for a fully developed human brain.

Me: If you are human, you should be able to tell me: how many fingers does Barack Obama have?
Cleverbot: Yes, what are you getting at?

Me: I don’t think you are human.
Cleverbot: Yes I am I am just… Intelligent.

 

We then move on to TrueKnowledge, a program specifically designed to answer questions similar to these, and is truly capable of some remarkable answers in some areas. However, it flubs on all five questions, not even being able to answer the super-simplified “What is 2+2?”. (Though it does correctly answer “Are you human” with “No”.)

Clearly no computer program is going to be able to fool any person into thinking they are human any time soon, let alone pass this simple five question challenge.

 

 

Understanding Natural Language

However, it seems like computers should be able to answer these five questions, or rather, answering these questions would be a very good indication of a computer programmed with an exceptional understanding of natural language and common sense knowledge.

For example, an ideal bot could preform like this:

Me: If you are human, you should be able to add two and two together and tell me the result.
Bot: I know that 2+2 = 4.

Me: If you are human, you should be able to tell me which is bigger: Scotland or a grapefruit?
Bot: I know that Scotland is 30414 square miles and grapefruits are 2 inches. Therefore, Scotland is bigger.

Me: If you are human, you should be able to assign x the value 4+y, assign y the value 6, assign z the value x+2, calculate y+x, and tell me the result.
Bot: To solve x+y, I need to know the value of x and the value of y. If y = 6, then x = 4+6 = 10, and therefore y+x = 10+6 = 16.

Me: If you are human, you should be able to tell me which of these is not a fruit: apple, banana, chair, orange.
Bot: Since apple, banana, and orange all have the subset fruit and chair does not, the chair is the object you mentioned which is not a fruit.

Me: If you are human, you should be able to tell me: how many fingers does Barack Obama have?
Bot: Since Barack Obama is human and all humans have two hands with five fingers each, Barack Obama has ten fingers.

But of course, this is perhaps the hugest case of easier said than done in my blogging history.

 

Understanding Basic Logic

Perhaps a good first step is the implementation of simple syllogistic logic. For example:

Me: Socrates is a man.
Bot: Understood. I now know that Socrates is a man.

Me: All men are mortal.
Bot: Understood. I now know that all men are mortal.

Me: Is Socrates mortal?
Bot: Since Socrates is a man and all men are mortal, Socrates is therefore mortal.

Me: Will Socrates die?
Bot: I don’t know. I don’t have any information about death.

Me: All mortals will die eventually.
Bot: Understood. I now know that anything that is mortal will die eventually.

Me: Will Socrates die?
Bot: Since Socrates is a man, all men are mortal, and anything that is mortal will die, Socrates will die.

Me: When will Socrates die?
Bot: Since Socrates is a man, all men are mortal, and anything that is mortal will die eventually, I know that Socrates will die within an unspecified but finite timeframe.

Me: How will Socrates die?
Bot: I don’t know. The only thing I know about death is that all mortals will die eventually.

 

 

Implementing the Basic Logic

This system actually seems pretty easy to implement. The computer would just need a database of information that is added to when people say things. As soon as someone says “Socrates is a man”, the computer program could interpret that and place “Socrates = man” in its database. Then when asked “Is Socrates a man?”, the computer program would look at its database and see if “Socrates = man” is contained within it. “Is Socrates a fish?” would see if “Socrates = fish” is contained within the database.

The remaining step is the logical inference. The computer would have to know that if “Socrates = man” and “man = mortal” is in the database, then “Socrates = mortal”, because the connection can be made: “[Socrates = <man] = mortal>”. In order to accomplish this, the computer would check everything Socrates is, and then check everything those qualities are, and see if they end up making a path between Socrates and mortal.

For example, if the database had these facts:

  • Socrates = philosopher
  • Socrates = man
  • man = human
  • human = person
  • human = mortal

 

And it was to see if Socrates is mortal, then it would do all sorts of tests. It would start with “Socrates”. First it would notice that Socrates is philosopher, but it wouldn’t have anywhere else to go, so it would abandon the connection. Then it would see that Socrates is man, and continue the tree. Man is human, and human is person. However, the tree ends here without any notion of “mortal”. Socrates is mand, man is human, and human is mortal, however does end up where we need to go. So yes, Socrates is mortal.

 

 

The Benefits of Understanding

While this appears all well and good, why would we want to spend the time doing this?

Now, of course, this concept of syllogistic logic is not brand new or even that remarkable. In fact, it has been implemented in some systems, most notably in Wikipedia articles, or any other large internet database. Imagine a computer with access to the entire sum of human knowledge and the ability to actually make use of it.

Thirdly, such a machine that can reason would be immensely valuable to the current data overload we all face. An ideal computer could analyze thousands of gigabytes of data and make unbiased determinations and identify trends, figure out what works and what doesn’t, and just generally solve problems greater than humans can. Such a machine doesn’t necessarily need to be Skynet or even generally manipulative; it could actually benefit humanity. It could turn out that not building advanced AI is what kills humanity, rather than a robot apocalypse.

Lastly, it’s a worthwhile academic pursuit. Even if devoid of any actual significance in the field of intelligence, the Turing test remains an intellectual curiosity, and a computer that is able to not only pass any trivia quiz but hold a conversation with the user would be the next step in mechanical awesome. Perhaps eventually there might be a computer that can not only play Chess or win Jepoardy, but learn the game ahead of time without needing the strategy pre-programmed.

 

All we need is a computer that can preform logic on a wide variety of natural language statements, on a level perhaps tenfold stronger than the level provided by WolframAlpha or TrueKnowledge. And it seems like this task might not be as difficult as it first seems.

I may not know why no one else is making this kind of chatbot, but I personally would like to. I accept the challenge.

Followed up by: A Complete List of Syllogisms

 

-

I now blog at EverydayUtilitarian.com. I hope you'll join me at my new blog! This page has been left as an archive.

On 23 Mar 2011 in All, Computer Science. No Comments.

Leave a Reply

You must be logged in to post a comment.