Tag: Google

Machine Translation

Background

In 1933, Soviet scientist Peter Troyanskii presented “the machine for the selection and printing of words when translating from one language to another” to the Academy of Sciences of the USSR. Soviet aparchnicks during the Stalin era declared the invention “useless” but allowed Troyanskii to continue his work. He died of natural causes in 1950 – a noteworthy accomplishment for a professor during the Stalin era – but never finished his translation machine.

Early IBM Machine Translation

In the US, during the Cold War, Americans had a different problem: there were few Russian speakers. Whereas the Anglophone countries pushed out countless media to learn English, the Soviet Union produced far less. Furthermore, spoken Russian was different than the more formalized written Russian. As the saying goes, even Tolstoy didn’t speak like Tolstoy.

In response, the US decided the burgeoning computer field might be helpful. On January 7, 1954, at IBM headquarters in New York, an IBM 701 automatically translated 60 Russian sentences into English.

“A girl who didn’t understand a word of the language of the Soviets punched out the Russian messages on IBM cards. The “brain” dashed off its English translations on an automatic printer at the breakneck speed of two and a half lines per second.

“‘Mi pyeryedayem mislyi posryedstvom ryechyi,’ the girl punched. And the 701 responded: We transmit thoughts by means of speech.’

“‘Vyelyichyina ugla opryedyelyayetsya otnoshyenyiyem dlyini dugi k radyiusu,’ the punch rattled. The ‘brain’ came back: ‘Magnitude of angle is determined by the relation of length of arc to radius.'”

IBM Press Release

Georgetown’s Leon Dostert led the team that created the program.

Blyat

Even IBM notes that the computer cannot think for itself, limiting the usefulness of the program for vague sentences. Apparently, nobody at Georgetown or IBM ever heard real Russians speak or they’d know that vague is an understatement with a language that has dozens of ways to say the same word. Furthermore, the need to transliterate the Russian into Latin letters, rather than typing in Cyrillic, no doubt further introduced room for enormous error.

In 1966, the Automatic Language Processing Advisory Committee, a group of seven scientists, released a more somber report. They found that machine translation is “expensive, inaccurate, and unpromising.” The message was clear: the best way to translate to and from Russian, or any other language, is to learn the language.

Progress continued, usually yielding abysmal results. Computers would substitute dictionary words in one language for comparable words in another, with results oftentimes more amusing than informative.

Towards Less Terrible Translations

One breakthrough came from Japan in 1984, which favored machine learning because few Japanese people learned English. Researcher Mankoto Nagao came up with the idea of searching for and substituting phrases rather than words. This yielded far better, but still generally terrible results.

Eventually, in the early 1990s, IBM built on Nagao’s method by running accurate manual translations and building an enormous database analyzing word frequency. The translations became slightly less horrible. This led to “statistical translation” that was significantly less terrible.

As the World Wide Web shrunk the world the need for automated translations grew and the vast majority of these were some type of statistical translation. Subsequently, they continually improved to the point where Google Translate could pretty much help decipher, say, a bill.

Modern Translating

Finally, in 2016, neural networks and machine learning (artificial intelligence) started to produce vastly superior machine translations. All the sudden, translations were actually readable. As of 2019, the best online translation engine, German-based DeepL, is entirely AI-powered.

Speech Recognition

Speech recognition is the ability of a computer to recognize the spoken word.

“Alexa: read me something interesting from Innowiki.”

“Duh human, everything on Innowiki is interesting or it wouldn’t be there.”

Today, inexpensive pocket-sized phones connect to centralized servers and understand the spoken word in countless languages. Not so long ago, that was science fiction.

Background

Star Trek in 1966, The HAL 9000 of 2001: A Space Odyssey of 1968, Westworld in 1973, and Star Wars in 1977 all assumed computers will understand the spoken word. What they missed is that people would become so fast at using other input devices, especially keyboards, that speaking is viewed as an inefficient input method.

The first real speech recognition actually predates science fiction ones. In 1952, three Bell Labs scientists created a system, “Audrey,” which recognized a voice speaking digits. A decade later, IBM researchers launched “Shoebox” that recognized 16 English words.

In 1971, DARPA intervened with the “Speech Understanding Research” (SUR) program aimed at a system which could understand 1,000 English words. Researchers at Carnegie Mellon created “Harpy” which understood a vocabulary comparable to a three-year-old child.

Researched continued. In the 1980s the “Hidden Markov Model” (HMM) proved a major breakthrough. Computer scientists realized computers need not understand what a person was saying but, rather, just to listen to sounds and look for patterns. By the 1990’s faster and less expensive CPUs brought speech recognition to the masses with software like Dragon Dictate. Bell South created the voice portal phone-tree system which, unfortunately, frustrates and annoys people to this day.

DARPA stepped back in during the 2000s, sponsoring multi-language speech recognition systems.

Rapid Advancement

However, a major breakthrough came from the private sector. Google released a service called “Google 411” allowing people to dial Google and lookup telephone numbers for free. People would speak to a computer that would guess what they said then an operator would answer, check the computer’s accuracy, and delivered the phone number. The real purpose of the system was to better train computers with a myriad of voices, including difficult-to-decipher names. Eventually, this evolved into Google’s voice recognition software still in use today.

Speech recognition continues to advance in countless languages. Especially for English, the systems are nearing perfection. They are fast, accurate, and require relatively little computer processing power.

In 2019 anybody can speak to a computer though unless their hands are busy doing something else, most prefer not to.

Autonomous Vehicles (Self-Driving Cars)

DARPA, the US government agency that invented the internet (among other things) created a contest to build a self-driving car.

The first DARPA Grand Challenge, in 2004, was a 150 mile (240 km. route). The robot-car that drove the furthest before breaking down, built by Carnegie-Mellon University (CMU), lasted 11.78km.

Undeterred, DARPA tried again. Subsequently, the 2005 DARPA Grand Challenge involved driving about 132 miles (212km) autonomously. Five cars finished, with Stanford coming in first. By 2007, DARPA issued their third and final challenge, to navigate the streets of a fake city. Carnegie Mellon won.

Sebastian Thrun, Stanford’s team lead, and Red Whittaker, of CMU, were former colleagues and friendly rivals. Autonomous cars built by their students repeatedly came in first or second in the various challenges.

Google/Waymo

Google co-founders Larry Page and Sergey Brin attended the 2005 DARPA Grand Challenge in disguise. They soon after hired Thrun. Initially, most analysts assumed Google would lean on his expertise in artificial intelligence – the core of a self-driving car – to improve the core Google search engine. However, the company eventually built out a separate business for self-driving cars. In late 2016 Google spun the self-driving car division into its own company, Waymo.

By 2018 Waymo was testing self-driving cars, albeit with safety drivers, around Phoenix. By late 2018, they commercialized the service. In 2019, Waymo announced plans to build an auto plant in Michigan to convert ordinary cars to autonomous vehicles to scale up their AV taxi service.

Today, every automaker is working furiously to perfect self-driving technology for cars, busses, and trucks.

Web Search Engine

Noteworthy early search engines include Archie, from 1990, that searched filenames, and Gopher, from 1991, that organized files.

Early Search Engines

In March 1994, Stanford students David Filo and Jerry Yang created “Jerry and David’s Guide to the World Wide Web.” Their website contained lists arranged by category of the burgeoning
World Wide Web. Sites were added by hand, with short snippets written by site creators. Initially, there was no charge to list a site. In January 1995 they renamed their website Yahoo.

In December 1995, to showcase the power of Digital Equipment Corporation (DEC) hardware, engineers designed a computer program to read and search (index) the entire World Wide Web. Originally meant as a hardware demo their website, Alta Vista, became popular. Alta Vista was the earliest full-text search engine.

Alta Vista merely matched words a user searched for and verbiage on websites. It was extremely primitive technology that did prioritize the significance or quality of websites. Yahoo was hand curated so did a better job, but the curation process did not scale well and, eventually, they started charging a fee for inclusion. Neither site did an especially good job searching. A third search engine, Excite, founded in 1994 rounded out the top search engines of the era. There were other smaller but still popular web search engines including Lycos (1994), Ask Jeeves (1996), and LookSmart (1995).

Google

In 1996 Stanford students Larry Page and Sergey Brin worked on a computer program to determine context. They decided to read the entire web, the same way that Alta Vista did, except to rank the importance of websites. Initially, their primary criteria for importance was the number of links from other websites and the rank of those sites. This metric, called “Page Rank” (a pun on Larry Page’s last name and the utility of the technology), yielded vastly better search results than either Yahoo or Alta Vista. In late August 1996, Larry Page noted Google downloaded and indexed 207GB of content storing it in a 28GB database.

In September 1997 Page and Brin moved towards commercializing their search engine, registering the domain name google.com, a play on the word googol (a one with a hundred zeros after it).

Wish to return to their academic lives Page and Brin tried to sell their young company. They offered it to the owners of Alta Vista and Excite for $1 million. Both passed. They lowered the offer to Excite to $750,000. The company still passed. Page and Brin were all but forced to build out their budding search engine, eventually selling plain-text ads based on the search request.

In March 2005 IAC/InterActiveCorp purchased Excite, which still had significant traffic, for $1.9 billion. As of 2019 Excite has no significant search traffic. Excite was shuttered August 2013. Google parent Alphabet is worth just over $800 billion. Other search engines exist, most notably Microsoft’s Bing, but none have nearly the same number of users as Google.