Hidden human H26 Matt Whitby writes of his experience in Reading University's Turing test contest on the 100th anniversary of Alan Turing's birth, 23 June 2012 at Bletchley Park:
Matt can be followed at these links:
"Turing 100.Alan Turing is one of those people of whom my first introduction has
become lost in memory. I do however remember that my first visit to
Bletchley Park would have been before my son was born which would put
it at over fifteen years ago and I distinctly remember being excited at
seeing the room Turing worked in (albeit from the outside).
Whilst being a complete layperson I’ve always had a fascination with
Artificial Intelligence and have had entering the Loebner Contest on my
bucket list for over a decade I saw on Twitter that Reading University was holding a Turing Test at Bletchley Park to celebrate Turing’s 100th anniversary. I tweeted back that it sounded really interesting and I’d do my best to come along. Almost
immediately I got a reply saying there was a place free for being a human in
the upcoming tests if I was free. Well, who would turn down such an
offer?
So early on Saturday morning I drove from Berkshire up to Bletchley Park
to take a small part in the test. It would, I thought be fun and would also
potentially give me a better insight into the competition in the event that I
did actually try and enter one day.
I was session four of five and wasn’t due to sit down at a terminal until 1:45
and it was only 9:30 but that was fine because their room was full of super
smart people (well, it’s all relative but they all seemed super smart to me).
There was always someone interesting to talk to. I hung round developers
most of the time but rarely with people who try and give computers the
ability to read a person’s lips, or provide power sources for robots, or
wireless electricity, make swarms of autonomous self-driving cars or talk
about multi-dimensional space.
After coming back from looking at the Bombe (an electromechanical device
used by British cryptologists to help decipher German Enigma-machineencrypted
signals during World War II.) I eagerly sat down for my session.
There were three groups: humans, judges and bots. The judges – as you
would expect – have conversations with both the humans and bots and try
and differentiate between the two. Turing said (and I’m paraphrasing to
bear with me) that if someone could converse with something on a
terminal and be unable to distinguish whether it was a human or a piece of
software then the thing they were communicating with could be said to be
exhibiting intelligent behaviour.
Five developers were pitting their bots against each other; Rollo Carpenter,
Robby Garner, Robert Medeksza, Fred Roberts and Vladimir Veselov.
In each slot the humans spoke to five judges for five minutes each. The
judges always initiated the conversation and only one entry could be
submitted before receiving a response and allowing you to continue with
your next submission. Whether it was due to the slow typing of the judges
or whether the system was deliberately slowing the responses down so the
judges couldn’t tell – by speed of response – between bot or human I
wasn’t sure. In the first round I only got about three responses in before
the screen disappeared which seemed like a pretty small sample of
conversation with which the judge could determine whether it was human
or bot.
You could see some deliberate traps set by the judges. One said to me:
> “I eat too many crisps. Should I see a fireman?”
I would imagine it would be relatively tricky for a bot to have determined
that the question didn’t really make sense and to formulate a response. My
reply was:
> “Not unless you set fire to the saucepan whilst making the crisps.”
My poor joke made me wonder whether a key factor in determining the
humanness of the humans was down to their sense of humour and how
hard that would be to build into a bot’s responses.
Another line of questioning that came up frequently was to refer to
something local to the environment. Was it sunny? Were the rooms too
cold? Information the bot wouldn’t know unless it had been specifically
prompted before the test began.
As soon as the test had begun we’d finished our round of testing and it
was onto the people in round five.
The winning bot was ‘Eugene’ by Vladimir Veselov.
It was an interesting peek behind the curtain at how these tests happen© Matt Whitby June 2012
and I feel slightly more armed at my future entry. Well, if it ever happens.
Thanks to all involved for a great day."
 |
| 'Hidden humans' preparing for Judges' questions in Reading University's Turing tests - picture taken in the Ballroom of the mansion at Bletchley Park: Saturday 23 June 2012 |
Matt can be followed at these links:
Website http://www.mattwhitby.net
Music http://www.last.fm/user/mattwhitby
Blogging http://matt-whitby.blogspot.com
Twitter http://www.twitter.como/mattwhitby
Facebook http://www.facebook.com/whitby
Skype matt.whitby
Spotify matt.whitby
No comments:
Post a Comment