Artificial Intelligence (AI)

Welcome into the universe of Artificial Intelligence from Courses Buddy!

Whether we notice it or not, artificial intelligence is EVERYWHERE. It’s guiding critical decisions in every field, from finance to health.

Artificial intelligence, commonly known as AI, is driving the algorithms of technology giants like Google, Amazon, Facebook, and Netflix. It’s what puts the “smart” in our smart devices like Siri, Alexa, and Google Assistant.

Artificially intelligent systems are analyzing huge amounts of data that we’re generating every day, to infer patterns and predict likely futures. For example, predicting the amount of yogurt people might buy next week at the grocery store, so the store knows how much to stock.

Everyone seems to be talking about AI and how it’s going to revolutionize the world or create some dystopian future. While we don’t know what the future holds, we believe that creating a better outcome starts with a better understanding.

The goal of this course is to teach the logic behind AI by tracing its history, examining how it’s being used today, and showing some practical examples.

We’ll study a hot area of AI known as machine learning. Using the brain as inspiration, these AI can learn from examples. We’ve taught machines to recognize people in photos, predict the next word we’re going to type on our phones, and recommend videos for us on YouTube!

AI is broader than just machine learning. So we’ll talk about how AI systems can act based on plans, rules, and logical representations of the world. These are the kinds of algorithms that are used to plan the path of a delivery-drone or determine the best move in a game of chess.

Now, I want to make sure we’re starting on the same page. Artificial intelligence is everywhere. It’s helping banks make loan decisions, and helping doctors diagnose patients. It’s on our cell phones, autocompleting texts, and it’s the algorithm recommending YouTube videos to watch after this one!

AI already has a pretty huge impact on all of our lives. So people, understandably, have some polarized feelings about it.

Perceptions of AI

Some of us imagine that AI will change the world in positive ways. It could end car accidents because we have self-driving cars, or it could give the elderly great, personalized care. Others worry that AI will lead to constant surveillance by a Big Brother government.

Some say that automation will take all our jobs. Or the robots might try and kill us all. No, we’re not worried about you John-Bot. But when we interact with AI that’s currently available like Siri…

Hey Siri. Is AI going to kill us all?”

Siri: “I don’t understand ‘Is AI going to kill us all.’”

… it’s clear that those are still distant futures. Now to understand where artificial intelligence might be headed, and our role in the AI revolution, we have to understand how we got to where we are today.

AI is still a new field, so there are a lot of problems to solve but not that many people trying to solve them. We also have to think about what the future of AI means for us as humans, and we believe it’s in our best interest to have as many people as possible who understand the field of AI so that the conversation can expand.

The AI revolution itself isn’t even that old. The term artificial intelligence didn’t even exist a century ago. It was coined in 1956 by a computer scientist named John McCarthy. He used it to name the “Dartmouth Summer Research Project on Artificial Intelligence.”

Most people call it the “Dartmouth Conference” for short. Now, this was way more than a weekend where you listen to a few talks, and maybe go to a networking dinner. Back in the day, academics just got together to think for a while.

The Dartmouth Conference lasted eight weeks and got a bunch of computer scientists, cognitive psychologists, and mathematicians to join forces.

Many of the concepts that we’ll talk about in this AI course, like artificial neural networks, were dreamed up and developed during this conference and in the few years that followed. But because these excited academics were really optimistic about artificial intelligence, they may have oversold it a bit.

Overestimations and Reality

Marvin Minsky was a talented cognitive scientist who was part of the Dartmouth Conference. But he also had some ridiculously wrong predictions about technology, and specifically AI.

In 1970, he claimed that in “three to eight years we will have a machine with the general intelligence of an average human being.” And, uh, sorry Marvin. We’re not even close to that now.

Scientists at the Dartmouth Conference seriously underestimated how much data and computing power an AI would need to solve complex, real world problems.

An artificial intelligence doesn’t really “know” anything when it’s first created, kind of like a human baby.

Now, most kinds of artificial intelligence don’t have things like senses, a body, or a brain that can automatically judge a lot of different things like a human baby does. Modern AI systems are just programs in machines.

So we need to give AI a lot of data.

Also, we have to label the data with whatever information the AI is trying to learn, like whether food tastes good to humans. And then, the AI needs a powerful enough computer to make sense of all the data. All of this just wasn’t available in 1956.

So until about 2010 or so, the field was basically frozen in what’s called the AI Winter. Still there were a lot of changes in the last half a century that led us to the AI Revolution.

As a friend once said: “History reminds us that revolutions are not so much events as they are processes.”

The AI Revolution didn’t begin with a single event, idea, or invention. We got to where we are today because of lots of small decisions, and two big developments in computing.

Computational Power

The first development was a huge increase in computing power and how fast computers could process data. To see just how huge, let’s deep dive into the history.

During the Dartmouth Conference in 1956, the most advanced computer was the IBM 7090. It filled a whole room, stored data on basically giant cassette tapes, and took instructions using paper punch cards.

Every second, the IBM 7090 could do about 200,000 operations. But if you tried to do that it would take you 55 and a half hours! Assuming you did one operation per second, and took no breaks. That’s right.

At the time, that was enough computing power to help with the U.S. Air Force’s Ballistic Missile Warning System. But AI needs to do a lot more computations with a lot more data. The speed of a computer is linked to the number of transistors it has to do operations.

Every two years or so since 1956, engineers have doubled the number of transistors that can fit in the same amount of space. So computers have gotten much faster.

When the first iPhone was released in 2007, it could do about 400 million operations per second.

But ten years later, Apple says the iPhone X’s processor can do about 600 billion operations per second.

That’s like having the computing power of over a thousand original iPhones in your pocket. (For all the nerds out there, listen you’re right, it’s not quite that simple – we’re just talking about FLOPS here).

And a modern supercomputer, which does computational functions like the IBM 7090 did, can do over 30 quadrillion operations per second. To put it another way, a program that would take a modern supercomputer one second to compute, would have taken the IBM 7090 4,753 years.

So computers started to have enough computing power to mimic certain brain functions with artificial intelligence around 2005, and that’s when the AI winter started to show signs of thawing. But it doesn’t really matter if you have a powerful computer unless you also have a lot of data for it to munch on.

The second development that kicked off the AI revolution is something that you’re using right now: the Internet and social media.

In the past 20 years, our world has become much more interconnected.

Whether you livestream from your phone, or just use a credit card, we’re all participating in the modern world.

Every time we upload a photo, click a link, tweet a hashtag, tweet without a hashtag, like a YouTube video, tag a friend on Facebook, argue on Reddit, post on TikTok [R.I.P. Vine], support a Kickstarter campaign, buy snacks on Amazon, call an Uber from a party, and basically ANYTHING, that generates data.

Even when we do something that seems like it’s offline, like applying for a loan to buy a new car or using a passport at the airport, those datasets end up in a bigger system.

The AI revolution is happening now, because we have this wealth of data and the computing power to make sense of it. And I get it. The idea that we’re generating a bunch of data but don’t always know how, why, or if it’s being used by computer programs can be kind of overwhelming.

But through this free AI course with Courses Buddy, we want you to learn how artificial intelligence works because it’s impacting our lives in huge ways. And that impact will only continue to grow.

With knowledge, we can make small decisions that will help guide the AI revolution, instead of feeling like we’re riding a rollercoaster we didn’t sign up for.

We’re creating the future of artificial intelligence together, every single day. Which I think is pretty cool.

Humans aren’t born with many skills, and we need to learn how to sort mail, land airplanes, and have friendly conversations. So computer scientists have tried to help computers learn like we do, with a process called supervised learning.

Understanding Learning

The process of learning is how anything can make decisions, like humans, animals, or AI systems. They can adapt their behavior based on their experiences. We’ll talk about three main types of learning:

Reinforcement Learning, Unsupervised Learning, and Supervised Learning.

Supervised Learning

Supervised learning is the process of learning with training labels, where a supervisor points out mistakes during the learning process. This is akin to a teacher correcting a student’s work. It involves training an AI to classify data based on labeled examples.

Inspiration from Human Brains: The Perceptron

To understand how early computer scientists approached supervised learning, we delve into the concept of the Perceptron. In 1958, psychologist Frank Rosenblatt aimed to create an artificial neuron, inspired by the Dartmouth Conference. His goal was to teach this AI to classify images as “triangles” or “not-triangles” with supervision.

Rosenblatt’s Perceptron received inputs from a camera, multiplied by weights, and processed through a threshold function. Through supervised training, the Perceptron adjusted its weights based on feedback, improving its classification accuracy over time.

Update Rule: Learning from Feedback

The update rule allowed the Perceptron to learn from mistakes. If it correctly classified an input, the weights remained unchanged. However, if it made an error, the weights were adjusted to improve future classification accuracy.

Challenges and Future Directions

While supervised learning offers powerful capabilities, challenges such as feature selection and balancing precision and recall remain. As AI continues to evolve, addressing these challenges will be crucial in building more sophisticated systems capable of handling complex tasks.

Do you know our brains make decisions with 100 billion neurons, which have trillions of connections between them!

We can actually do a lot more with AI if we connect a bunch of perceptrons together, to create what’s called an artificial neural network.

The Need for Image Recognition

Not that long ago, a big challenge in AI was real-world image recognition, like recognizing a dog from a cat, and a car from a plane from a boat. Even though we do it every day, it’s really hard for computers. That’s because computers are good at literal comparisons, like matching 0s and 1s, one at a time.

It’s easy for a computer to tell that these images are the same by matching the pixels. But before AI, a computer couldn’t tell that these images are of the same dog, and had no hope of telling that all of these different images are dogs.

So, a professor named Fei-Fei Li and a group of other machine learning and computer vision researchers wanted to help the research community develop AI that could recognize images.

The first step was to create a huge public dataset of labeled real-world photos. That way, computer scientists around the world could come up with and test different algorithms.

They called this dataset ImageNet. It has 3.2 million labeled images, sorted into 5,247 nested categories of nouns.

The Rise of AlexNet

Alex Krizhevsky, who was a graduate student at the University of Toronto. In 2012, he decided to apply a neural network to ImageNet, even though similar solutions hadn’t been successful in the past. His neural network, called AlexNet, had a couple of innovations that set it apart. He also used faster computation hardware to handle all the math that neural networks do. AlexNet outperformed the next best approaches by over 10%.

Understanding Neural Network Architecture

To understand how neural networks can be used for these classification problems, we have to understand their architecture first. All neural networks are made up of an input layer, an output layer, and any number of hidden layers in between.

There are many different arrangements but we’ll use the classic multi-layer perceptron as an example.

The Power of Hidden Layers

The input layer is where the neural network receives data represented as numbers. Each input neuron represents a single feature, which is some characteristic of the data. Once the features have data, each one sends its number to every neuron in the next layer, called the hidden layer.

The hidden layer is where the magic happens.

Each hidden layer neuron mathematically combines all the numbers it gets, measuring whether the input data has certain components. These components may include specific shapes, textures, or patterns.

Processing with Hidden Layers

When all of these neurons pass their estimates onto the next hidden layer, those neurons may be trained to look for more complex components. After a few more hidden layers, we reach the output layer with one neuron!

So after one more weighted addition of the previous layer’s data, which happens in the output neuron, the network should have a good estimate if this image is a dog.

Challenges and Considerations

Neural networks don’t actually understand ideas like “nose” or “eyelid.” Each neuron is doing a calculation on the data it’s given and just flagging specific patterns of light and dark. As networks get deeper, they become more abstract, making it harder for humans to interpret their decisions.

However, understanding how neural networks operate is crucial as they are increasingly used in decision-making processes in various domains.

The neurons already had algorithms that weighted inputs. Understanding the fundamentals of neural networks provides insights into their capabilities and limitations, paving the way for further advancements in AI technology.

One way to make an artificial brain is by creating a neural network, which can have millions of neurons and billions (or trillions) of connections between them.

Nowadays, some neural networks are fast and big enough to do some tasks even better than humans can, like playing chess or predicting the weather. But as we’ve talked about in the AI course, neural networks don’t just work on their own.

They need to learn to solve problems by making mistakes. Sounds kind of like us, right?

Neural Network Components

Neural networks handle mistakes using an algorithm called backpropagation to make sure all the neurons that contributed to an error get their math adjusted.

Neural networks consist of two main parts: the architecture and the weights. The architecture includes neurons and their connections, while the weights are numbers that fine-tune how the neurons do their math to get an output.

Linear Regression Example

A simple way computers predict outcomes is through linear regression. This involves drawing a random straight line on a graph to fit data points and then adjusting the line to minimize the error, ultimately creating the best fit line.

As more features are considered, such as temperature, humidity, and rain, the optimization problem becomes more complex, requiring a neural network’s multidimensional capabilities.

Training Neural Networks

Neural networks learn to solve complicated problems by connecting simple neurons with weights. These weights are adjusted through training data to minimize errors in predictions.

This process involves backpropagation, where errors are traced back through the network to adjust neuron weights accordingly.

Backpropagation: Learning from Mistakes

Backpropagation involves assigning blame to neurons in previous layers based on the error calculated at the output layer. This blame is used to adjust weights, aiming to minimize error and optimize performance.

Backpropagation is essential for teaching neural networks from their mistakes and improving their accuracy over time.

Preventing Overfitting

Neural networks are susceptible to overfitting, where they fit too closely to training data and struggle with new, unseen data. Preventing overfitting involves keeping the neural network simple and avoiding unnecessary features that may introduce noise or irrelevant correlations.

Training a neural network involves more than just mathematical adjustments; it requires careful consideration of features, potential mistakes, and strategies to optimize performance.

By understanding the mechanics of neural networks and their training process, we can harness their power for solving complex problems effectively.

Note: This topic is a hands-on practice, which is impossible in writing, but we will guide somehow on it in writing.

So, we embark on a journey to program a neural network capable of recognizing handwritten letters. This exercise will guide you through the process of building, training, and testing a neural network using Python and the Google Collaboratory platform.

Acquiring the EMNIST Dataset

To kickstart our project, we utilize the Extended Modified National Institute of Standards and Technology (EMNIST) dataset, which contains tens of thousands of labeled images of handwritten letters. This dataset serves as the foundation for training our neural network.

We divide the dataset into training and testing sets, normalize pixel values, and ensure that each image is appropriately labeled. This preprocessing step lays the groundwork for training our neural network effectively.

Constructing the Neural Network

Using the Sci Kit Learn (SKLearn) library, we create a multi-layer perceptron neural network (MLP) with specific input and output layers tailored to our task. The structure includes an input layer of 784 neurons representing pixel values and an output layer of 26 neurons corresponding to letters in the English alphabet.

We define the architecture of the neural network, experiment with hidden layers and neuron configurations, and train the model over multiple epochs to improve its accuracy.

Evaluating Performance

Through rigorous testing, we assess the accuracy of our trained neural network on both the training and testing datasets. We analyze error rates, visualize confusion matrices, and iterate on the network’s structure to enhance performance.

Applying the Neural Network

In the final step, we apply our trained neural network to digitize handwritten text. We preprocess scanned images, apply image filters, and resize them to match the EMNIST dataset’s format.

Analyzing Results

Upon processing the handwritten text, we examine the output to identify any inaccuracies or areas for improvement. Despite achieving a high accuracy rate, we fine-tune the neural network to handle nuances and variations in handwriting more effectively.

Through this hands-on writing, we’ve gained valuable insights into the process of developing and training neural networks for real-world applications. While supervised machine learning offers powerful capabilities, it’s essential to continue exploring other machine learning paradigms, such as unsupervised and reinforcement learning, to unlock new possibilities in artificial intelligence.

So far we’ve focused on artificial intelligence that uses Supervised Learning. These programs need a teacher to use labeled data to tell them “right” from “wrong.”

And we humans have places where supervised learning happens, like classrooms with teachers, but that’s not the only way we learn. We can also learn lots of things on our own by finding patterns in the world.

We can look at dogs and elephants and know they’re different animals without anyone telling us. Or we can even figure out the rules of a sport just by watching people play.

This kind of learning without a teacher is called Unsupervised Learning and, in some cases, computers can do it too.

The key difference between supervised and unsupervised learning is what we’re trying to predict. In supervised learning, we’re trying to build a model to predict an answer or label provided by a teacher.

Differentiating Supervised and Unsupervised Learning

In unsupervised learning, instead of a teacher, the world around us is basically providing training labels. For example, if I freeze this video of a tennis ball RIGHT NOW, can you draw what could be the next frame?

Unsupervised learning is about modeling the world by guessing like this, and it’s useful because we don’t need labels provided by a teacher. Babies do a lot of unsupervised learning by watching and imitating people, and we’d like computers to be able to learn like this as well.

This lets us utilize lots of freely available data in the world or on the internet.

In many cases, one of the easiest ways to understand how AI can use unsupervised learning is by doing it ourselves. The most basic way to model the world is to assume that it’s made up of distinct groups of objects that share properties.

Immediately recognizing different properties and creating categories is called unsupervised clustering.

We don’t have labels provided by a teacher, but we do have a key assumption about the world that we’re modeling: certain objects are more similar to each other than others.

In supervised learning, models predict labels based on provided data, whereas unsupervised learning involves modeling the world by identifying patterns without explicit guidance from a teacher. We explore how unsupervised learning mirrors human learning through observation and inference.

Modeling the World

We introduce the concept of unsupervised clustering, where data is grouped into clusters based on inherent similarities. This approach allows computers to categorize objects without predefined labels, akin to how humans recognize patterns in the absence of explicit instructions.

How do AI Predict?

Remember, unsupervised learning is about modeling the world, so our algorithm will have two steps:

First, our AI will predict.

What does the model expect the world to look like? In other words, which flowers should be clustered together because they’re the same species?

Second, our AI will correct or learn.

The model will update its beliefs to agree with its observation of the world. To start the process, we have to specify how many clusters the model should look for.

The K-Means Clustering Algorithm

Using the K-means clustering algorithm as an example, we illustrate how computers can autonomously identify clusters within data. By iteratively predicting and refining cluster assignments, the algorithm iterates towards a model that accurately represents underlying patterns in the data.

We examine the output of the K-means algorithm to determine if meaningful patterns have been identified. Through unsupervised learning, computers can uncover inherent structures in the data, such as distinct species of flowers, without explicit labeling.

Representation Learning in Unsupervised Learning

Representation learning enables computers to extract meaningful features from data without human intervention. By discerning abstract patterns across multiple observations, computers can develop representations that facilitate comparison and analysis.

Through an experiment involving rapid image recognition and reconstruction, we illustrate the concept of representation learning. Just as humans build mental representations of observed objects, computers learn to encode and reconstruct data to identify meaningful patterns.

Challenges in Unsupervised Learning

Despite its potential, unsupervised learning presents challenges in replicating the nuanced learning processes of the human brain. While humans effortlessly learn through observation and imitation, designing AI systems to autonomously discern meaningful patterns remains a complex endeavor.

Future of Unsupervised Learning

As researchers continue to explore unsupervised learning, advancements in AI hold the promise of unlocking deeper insights into the complexities of the world. By harnessing the power of unsupervised learning, AI systems can emulate human-like learning processes and gain a deeper understanding of the world around them.

Unsupervised learning represents a frontier in artificial intelligence, offering a pathway to autonomous learning and discovery. By leveraging unsupervised learning algorithms, computers can uncover hidden patterns, generate meaningful representations, and expand our understanding of the world.

As we journey further into the realm of AI, the exploration of unsupervised learning promises to unveil new insights and capabilities, propelling us towards a future of enhanced intelligence and understanding.

Unsupervised learning is a huge area of active research.

The human brain is specially designed for this kind of learning and has different parts for vision, language, movement, and so on. These structures and what kinds of patterns our brains look for were developed over billions of years of evolution.

But it’s really tricky to build an AI that does unsupervised learning well because AI systems can’t learn exactly like humans often do, just by watching and imitating.

Someone, like us, has to design the models and tell them how to look for patterns before letting them loose.

Language is one of the most impressive things humans do. It’s how I’m transferring knowledge from my brain to yours right this second!

Languages come in many shapes and sizes, they can be spoken or written, and are made up of different components like sentences, words, and characters that vary across cultures. For instance, English has 26 letters and Chinese has tens-of-thousands of characters.

So far, a lot of the problems we’ve been solving with AI and machine learning technologies have involved processing images, but the most common way that most of us interact with computers is through language.

We type questions into search engines, we talk to our smartphones to set alarms, and sometimes we even get a little help with our Spanish homework from Google Translate.

So, we’re going to explore the field of Natural Language Processing.

Understanding The Power of Natural Language

Natural Language Processing, or NLP, mainly explores two big ideas. First, there’s Natural Language Understanding, or how we get meaning out of combinations of letters. These are AI that filter your spam emails, figure out if that Amazon search for “apple” was grocery or computer shopping, or instruct your self-driving car how to get to a friend’s house.

And second, there’s Natural Language Generation, or how to generate language from knowledge. These are AI that perform translations, summarize documents, or chat with you. The key to both problems is understanding the meaning of a word, which is tricky because words have no meaning on their own.

We assign meaning to symbols.

To make things even harder, in many cases, language can be ambiguous and the meaning of a word depends on the context it’s used in. If I tell you to meet me at the bank, without any context, I could mean the river bank or the place where I’m grabbing some cash.

If I say “This fridge is great!”, that’s a totally different meaning from “This fridge was *great*, it lasted a whole week before breaking.” So, how did we learn to attach meaning to sounds?

How do we know great [enthusiastic] means something different from great [sarcastic]?

Well, even though there’s nothing inherent in the word “cat” that tells us it’s soft, purrs, and chases mice… when we were kids, someone probably told us “this is a cat.” Or a gato, māo, billee, qut.

When we’re solving a natural language processing problem, whether it’s natural language understanding or natural language generation, we have to think about how our AI is going to learn the meaning of words and understand our potential mistakes.

Sometimes we can compare words by looking at the letters they share. This works well if a word has morphology. Take the root word “swim” for example. We can modify it with rules so if someone’s doing it right now, they’re swimming, or the person doing the action is the swimmer.

Drinking, drinker, thinking, thinker, … you get the idea. But we can’t use morphology for all words, like how knowing that a van is a vehicle doesn’t let us know that a vandal smashed in a car window. Many words that are really similar, like cat and car, are completely unrelated.

And on the other hand, cat and Felidae (the word for the scientific family of cats) mean very similar things and only share one letter!

Distributional Semantics

One common way to guess that words have similar meaning is using distributional semantics, or seeing which words appear in the same sentences a lot. This is one of many cases where NLP relies on insights from the field of linguistics.

As the linguist John Firth once said, “You shall know a word by the company it keeps.”

But to make computers understand distributional semantics, we have to express the concept in math. One simple technique is to use count vectors. A count vector is the number of times a word appears in the same article or sentence as other common words.

If two words show up in the same sentence, they probably have pretty similar meanings.

Count Vectors

So let’s say we asked an algorithm to compare three words, car, cat, and Felidae, using count vectors to guess which ones have similar meaning. We could download the beginning of the Wikipedia pages for each word to see which other words show up.

Here’s what we got: And a lot of the top words are all the same: the, and, of, in. These are all function words or stop words, which help define the structure of language, and help convey precise meaning.

Like how “an apple” means any apple, but “the apple” specifies one in particular. But, because they change the meaning of another word, they don’t have much meaning by themselves, so we’ll remove them for now, and simplify plurals and conjugations.

Let’s try it again: Based on this, it looks like cat and Felidae mean almost the same thing, because they both show up with lots of the same words in their Wikipedia articles! And neither of them mean the same thing as a car. But this is also a really simplified example.

One of the problems with count vectors is that we have to store a LOT of data. To compare a bunch of words using counts like this, we’d need a massive list of every words we’ve ever seen in the same sentence, and that’s unmanageable.

So, we’d like to learn a representation for words that captures all the same relationships and similarities as count vectors but is much more compact. In the unsupervised learning topic, we talked about how to compare images by building representations of those images.

We needed a model that could build internal representations and that could generate predictions. And we can do the same thing for words.

Decoder Models

This is called an encoder-decoder model: the encoder tells us what we should think and remember about what we just read… and the decoder uses that thought to decide what we want to say or do.

We’re going to start with a simple version of this framework.

Let’s create a little game of fill in the blank to see what basic pieces we need to train an unsupervised learning model. This is a simple task called language modeling. If I have the sentence: I’m kinda hungry, I think I’d like some chocolate _____ . What are the most likely words that can go in that spot? And how might we train a model to encode the sentence and decode a guess for the blank?

In this example, I can guess the answer might be “cake” or “milk” but probably not something like “potatoes,” because I’ve never heard of “chocolate potatoes” so they probably don’t exist. Definitely don’t exist. That should not be a thing.

The group of words that can fill in that blank is an unsupervised cluster that an AI could use. So for this sentence, our encoder might only need to focus on the word chocolate so the decoder has a cluster of “chocolate food words” to pull from to fill in the blank.

Now let’s try a harder example: Dianna, a friend of mine from San Diego who really loves physics, is having a birthday party next week, so I want to find a present for ____. When I read this sentence, my brain identifies and remembers two things: First, that we’re talking about Dianna from 27 words ago! And second, that my friend Dianna uses the pronoun “her.” That means we want our encoder to build a representation that captures all these pieces of information from the sentence, so the decoder can choose the right word for the blank.

And if we keep the sentence going: Dianna, a friend of mine from San Diego who really loves physics, is having a birthday party next week, so I want to find a present for her that has to do with _____ .

Now, I can remember that Dianna likes physics from earlier in the sentence. So we’d like our encoder to remember that too, so that the decoder can use that information to guess the answer. So we can see how the representation the model builds really has to remember key details of what we’ve said or heard.

And there’s a limit to how much a model can remember.

Professor Ray Mooney has famously said that we’ll “never fit the whole meaning of a sentence into a single vector” and we still don’t know if we can.

Professor Mooney may be right, but that doesn’t mean we can’t make something useful. So far we’ve been using words. But computers don’t work words quite like this.

So let’s step away from our high level view of language modeling and try to predict the next word in a sentence anyway with a neural network.

To do this, our data will be lots of sentences we collect from things like someone speaking or text from books. Then, for each word in every sentence, we’ll play a game of fill-in-the-blank.

We’ll train a model to encode up to that blank and then predict the word that should go there. And since we have the whole sentence, we know the correct answer.

Recurrent Neural Network ( RNN)

First, we need to define the encoder. We need a model that can read the input, which in this case is a sentence. To do this, we’ll use a type of neural network called a Recurrent Neural Network or RNN. RNNs have a loop in them that lets them reuse a single hidden layer, which gets updated as the model reads one word at a time.

Slowly, the model builds up an understanding of the whole sentence, including which words came first or last, which words are modifying other words, and a whole bunch of other grammatical properties that are linked to meaning.

Now, we can’t just directly put words inside a network. But we also don’t have features we can easily measure and give the model either. Unlike images, we can’t even measure pixel values. So we’re going to ask the model to learn the right representation for a word on its own (this is where the unsupervised learning comes in).

To do this, we’ll start off by assigning each word a random representation — in this case a random list of numbers called a vector. Next, our encoder will take in each of those representations and combine them into a single shared representation for the whole sentence.

At this point, our representation might be gibberish, but in order to train the RNN, we need it to make predictions.

For this particular problem, we’ll consider a very simple decoder, a single layer network that takes in the sentence representation vector, and then outputs a score for every possible word in our vocabulary.

We can then interpret the highest scored word as our model’s prediction. Then, we can use backpropagation to train the RNN, like we’ve done before with neural networks in free AI Courses from Courses Buddy.

So by training the model on which word to predict next, the model learns weights for the encoder RNN and the decoder prediction layer. Also, the model changes those random representations we gave every word at the beginning. Specifically, if two words mean something similar, the model makes their vectors more similar.

Using the vectors to help make a plot, we can actually visualize word representations. For example, earlier we talked about chocolate and physics, so let’s look at some word representations that researchers at Google trained.

Near “chocolate,” we have lots of foods like cocoa and candy: By comparison, words with similar representations to “physics” are newton and universe. This whole process has used unsupervised learning, and it’s given us a basic way to learn some pretty interesting linguistic representations and word clusters.

But taking in part of a sentence and predicting the next word is just the tip of the iceberg for NLP. If our model took in English and produced Spanish, we’d have a translation system.

Our model could read questions and produce answers, like Siri or Alexa try to do. Or our model could convert instructions into actions to control a household robot. But the representations of words that our model learns for one kind of task might not work for others.

Acquiring, encoding, and using written or spoken knowledge to help people is a huge and exciting task, because we use language for so many things! Every time you type or talk to a computer, phone or other gadget, NLP is there.

Now that we understand the basics, next week we’ll dive in and build a language model together in our COMING TOPICS!

Reinforcement Learning

We learn lots of things by trial-and-error, and this kind of “learning by doing” to achieve complicated goals is called Reinforcement Learning.

So far, we’ve talked about two types of learning in this AI Course: Supervised Learning, where a teacher gives an AI answers to learn from, and Unsupervised Learning, where an AI tries to find patterns in the world.

Reinforcement Learning is particularly useful for situations where we want to train AIs to have certain skills we don’t fully understand ourselves. For example, I’m pretty good at walking, but trying to explain the process of walking is kind of difficult.

What angle should your femur be relative to your foot? And should you move it with an average angular velocity of… yeah, never mind… It’s really difficult.

With reinforcement learning, we can train AIs to perform complicated tasks. But unlike other techniques, we only have to tell them at the very end of the task if they succeeded, and then ask them to tell us how they did it.

Understanding REWARD

So if we want an AI to learn to walk, we give them a reward if they’re both standing up and moving forward, and then figure out what steps they took to get to that point. The longer the AI stands up and moves forward, the longer it’s walking, and the more reward it gets.

So you can kind of see how the key to reinforcement learning is just trial-and-error, again and again. For humans, a reward might be a cookie or the joy of winning a board game. But for an AI system, a reward is just a small positive signal that basically tells it “good job” and “do that again”!

Google Deepmind got some pretty impressive results when they used reinforcement learning to teach virtual AI systems to walk, jump, and even duck under obstacles. It looks kinda silly, but works pretty well!

Other researchers have even helped real life robots learn to walk. So seeing the end result is pretty fun and can help us understand the goals of reinforcement learning. But to really understand how reinforcement learning works, we have to learn a new language to talk about these AI and what they’re doing.

We have an AI (or Agent) as our loyal subject that’s going to learn. An agent makes predictions or performs Actions, like moving a tiny bit forward, or picking the next best move in a game. And it performs actions based on its current inputs, which we call the State.

In supervised learning, after each action, we would have a training label that tells our AI whether it did the right thing or not. We can’t do that here with reinforcement learning, because we don’t know what the “right thing” actually is until it’s completely done with the task.

This difference actually highlights one of the hardest parts of reinforcement learning called credit assignment.

Credit Assignment

It’s hard to know which actions helped us get to the reward (and should get credit) and which actions slowed down our AI when we don’t pause to think after every action.

So the agent ends up interacting with its Environment for a while, whether that’s a game board, a virtual maze, or real life kitchen. And the agent takes many actions until it gets a Reward, which we give out when it wins a game or gets that cookie jar from that really tall shelf.

Then, every time the agent wins (or succeeds at its task), we can look back on the actions it took and slowly figure out which game states were helpful and which weren’t.

During this reflection, we’re assigning Value to those different game states and deciding on a Policy for which actions work best. We need Values and Policies to get anything done in reinforcement learning.

Let’s say I see some food in the kitchen: a box, a small bag, and a plate with a donut. So my brain can assign each of these a value, a numerical yummy-ness value. The box probably has 6 donuts in it, the bag probably has 2, and the plate just has 1… so the values I assign are 6, 2, and 1.

Now that I’ve assigned each of them a value, I can decide on a policy to plan what action to take! The simplest policy is to go to the highest value (that box of possibly 6 donuts). But I can’t see inside of it, and that could be a box of bagels, so it’s high reward but high risk.

Another policy could be low reward but low risk, going with the plate with 1 guaranteed delicious donut. Personally, I’d pick a middle-ground policy, and go for the bag because I have a better chance of guessing that there are donuts inside than the box, and a value of 1 donut isn’t enough.

That’s a lot of vocab, so let’s see these concepts in action to help us remember everything. Our example is going to focus on a mathematical framework that could be used with different underlying machine learning techniques.

Let’s say John-bot (imaginary name) wants to go to the charging station to recharge his batteries. In this example, John-bot is a brand new Agent, and the room is the Environment he needs to learn about. From where he is now in the room, he has four possible Actions: moving up, down, left, or right. And his State is a couple of different inputs: where he is, where he came from, and what he sees.

For this example, we’ll assume John-bot can see the whole room. So when he moves up (or any direction), his state changes. But he doesn’t know yet if moving up was a good idea, because he hasn’t reached a goal. He found the battery, so he got a Reward (that little plus one).

Now, we can look back at the path he took and give all the cells he walked through a Value — specifically, a higher value for those near the goal, and lower for those farther away.

These higher and lower values help with the trial-and-error of reinforcement learning, and they give our agent more information about better actions to take when he tries again!

So if we put John-bot back at the start, he’ll want to decide on a Policy that maximizes reward. Since he already knows a path to the battery, he’ll walk along that path, and he’s guaranteed another +1. But that’s… too easy. And kind of boring if John-bot just takes the same long and winding path every time.

So another important concept in reinforcement learning is the trade-off between exploitation and exploration.

Now that John-bot knows one way to get to the battery, he could just exploit this knowledge by always taking the same 10 actions. It’s not a terrible idea — he knows he won’t get lost and he’ll definitely get a reward. But this 10-action path is also pretty inefficient, and there are probably more efficient paths out there.

So exploitation may not be the best strategy. It’s usually worth trying lots of different actions to see what happens, which is a strategy called exploration.

Exploration

Every new path John-bot takes will give him a bit more data about the best way to get a reward. So let’s let John-bot explore for 100 actions, and after he completes a path, we’ll update the values of the cells he’s been to.

Now we can look at all these new values!

During exploration, John-bot found a short-cut, so now he knows a path that only takes 4 actions to get to the goal. This means our new policy (which always chooses the best value for the next action) will take John-bot down this faster path to the target.

That’s much better than before, but we paid a cost, because during those 100 actions of exploration, he took some paths that were even more inefficient than the first 10-action try and only got a total of 6 points.

If John-bot had just exploited his knowledge of the first path he took for those 100 actions, he could have made it to the battery 10 times and gotten 10 points. So you could say that exploration was a waste of time.

But if we started a new competition between the new John-bot (who knows a 4-action path) and his younger, more foolish self (who knows a 10-action path), over 100 actions, the new John-bot would be able to get 25 points because his path is much faster.

So should we explore more to try and find an even better path? Or should we just use exploitation right away to collect more points? In many reinforcement learning problems, we need a balance of exploitation and exploration, and people are actively researching this tradeoff.

These kinds of problems can get even more complicated if we add different kinds of rewards, like a +1 battery and a +3 bigger battery. Or there could even be Negative Rewards that John-Bot needs to learn to avoid, like this black hole.

If we let John-Bot explore this new environment using reinforcement learning, sometimes he falls into the black hole. So the cells will end up having different values than the earlier environment, and there could be a different best policy. Also, the whole environment could change in many of these problems.

If we have an AI in our car helping us drive home, the same road will have different people, bicycles, cars, and black holes on it every day. There might even be construction that completely reroutes us.

This is where reinforcement learning problems get more fun, but much harder.

Value Function

When John-bot was learning how to navigate on that small grid, cells closer to the battery had higher values than those far away. But for many problems, we’ll want to use a value function to think about what we’ve done so far, and decide on the next move using math.

For example, in this situation where an AI is helping us drive home, if we’re optimizing safety and we see the brake lights of the car in front of us, it’s probably time to slow down, but if we saw a bag of donuts in the street, we would want to stop.

So reinforcement learning is a powerful tool that’s been around for decades, but a lot of problems need a ton of data and a ton of time to solve. There have been really impressive results recently thanks to deep reinforcement learning on large-scale computing.

These systems can explore massive environments and a huge number of states, leading to results like AIs learning to play games.

At the core of a lot of these problems are discrete symbols, like a command for forward or the squares on a game board, so how to reason and plan in these spaces is a key part of AI.

Back in 1959, three AI pioneers set out to build a computer program that simulated how a human thinks to solve problems.

Allen Newell was a psychologist who was interested in simulating how humans think, and Herbert Simon was an economist, who later won the Nobel prize for showing that humans aren’t all that good at thinking. They teamed up with Cliff Shaw, who was a programmer at the RAND corporation, to build a program called the General Problem Solver.

To keep things simple, Newell, Simon, and Shaw decided it was best to think about the content of a problem separately from the problem-solving technique. And that’s a really important insight.

For example, my brain would use the same basic reasoning strategies to plan the best route to work, school, or wherever I need to go, no matter where I start. Computers are logical machines that use math to do calculations, so logic was an obvious choice for the General Problem Solver’s problem-solving technique.

Representing the problem itself was less straightforward. But Newell, Simon, and Shaw wanted to simulate humans, and human brains are really good at recognizing objects in the world around us. So in a computer program, they represented real-world objects as symbols. That’s where the term Symbolic AI comes from, and it’s how certain AI systems make decisions, generate plans, and appear to “think.”

Symbolic AI

If you’ve ever applied for a credit card, purchased auto insurance, or played a computer game newer than something like PacMan, then you’ve interacted with an AI system that uses Symbolic AI.

Modern neural networks train a model on lots of data and predict answers using best guesses and probabilities. But Symbolic AI, or “good old-fashioned AI” as it’s sometimes called, is hugely different.

Symbolic AI requires no training, no massive amounts of data, and no guesswork. It represents problems using symbols and then uses logic to search for solutions, so all we have to do is represent the entire universe we care about as symbols in a computer… no big deal.

To recap, logic is our problem-solving technique and symbols are how we’re going to represent the problem in a computer.

Symbols can be anything in the universe: numbers, letters, words, bagels, donuts, toasters, John-bots. One way we can visualize this is by writing symbols surrounded by parentheses, like (donut) or (John).

A relation can be an adjective that describes a symbol, and we write it in front of the symbol that’s in parentheses. So, for example, if we wanted to represent a chocolate donut, we can write that as chocolate(donut). Relations can also be verbs that describe how symbols interact with other symbols.

So, for example, I can eat a donut, which we would write as eat(John, donut) because the relation describes how one symbol is related to the other. Or we could represent John-bot’s relation to me, using sidekick(John-bot, John).

A symbol can be part of lots of relations depending on what we want our AI system to do, so we can write others like is(John-bot, robot) or wears(John-bot, polo). All of our examples in this topic will include a max of two symbols for simplicity, but you can have any number of symbols described by one relation.

A simple way to remember the difference between symbols and relations is to think of symbols as nouns and relations as adjectives or verbs that describe how symbols play nicely together.

This way of thinking about symbols and their relations lets us capture pieces of our universe in a way that computers can understand. And then they can use their superior logic powers to help us solve problems.

How does AI Symbolic Work?

The collection of all true things about our universe is called a knowledge base, and we can use logic to carefully examine our knowledge bases in order to answer questions and discover new things with AI. This is basically how Siri works.

Siri maintains a huge knowledge base of symbols, so when we ask her a question, she recognizes the nouns and verbs, turns the nouns into symbols and verbs into relations, and then looks for them in the knowledge base.

Let’s try an example of converting a sentence into symbols and relations, and using logic to solve questions. Let’s say that “John-bot drives a smelly, old, car.” I could represent this statement in a computer with the symbols John-Green-bot and car, and the relations between drives, smelly, and old.

Using logical connectives like AND and OR, we can combine these symbols to make sentences called propositions. And then, we can use a computer to figure out whether these propositions are true or not using the rules of propositional logic and a tool called a truth table.

Propositional logic is basically a fancy name for Boolean Logic.

And the truth table helps us decide what’s true and what’s not. So, in this example, if the car is actually smelly, and actually old, and if John-bot actually drives the car… then the proposition, “Smelly car AND old car AND John-bot drives the car.” is true.

We can understand that sort of logic with our brains: if all three things are true, then the whole proposition is true. But for an AI to understand that, it needs to use some math. With a computer, we can think of a false relation as 0 and true relations as any number that’s not 0.

We can also think of ANDs as multiplication and ORs as addition. But let’s look at what happens to the math if the car is not actually old. Again, our brains might be able to jump to the conclusion that if one of the three things isn’t true, then the whole proposition must be false.

But to do the math like an AI would, we can translate this proposition as true times false times true, which is 1 times 0 times 1. That equals 0, which means the whole proposition is false. So that’s the basics of how to solve propositions that involve AND.

But what if we want to know if John-bot drives a car and that the car is either smelly OR old? Like I mentioned earlier, OR can be translated as addition. So, using our math rules, we can fill out this new, bigger truth table.

The proposition we’re dealing with now is “Smelly car OR old car AND John-bot drives the car.” For the first row, this translates as (true plus true), then that result times true, which we calculate as (1 plus 1) times 1. That equals 2 times 1, which is 2, which means the whole proposition is true!

Remember, any answer that isn’t 0 is true.

The second row translates as (true plus false), then that result times true, which we calculate as (1 plus 0) times 1. That equals 1 times 1, which is 1, which means the whole proposition is true again.

And we can fill out the rest of the truth table the same way!

Another logical connective besides AND and OR, is NOT, which switches true things to false and false things to true. And there are a handful of other logical connectives that are based on ANDs, ORs, and NOTs.

Symbolic AI Implications

One of the most important ones is called implication, which connects two different propositions. Basically, what it means is that IF the left proposition is true, THEN the right proposition must also be true. Implications are also called if/then statements.

We make thousands of tiny if/then decisions every hour (like, for example, IF tired THEN take nap or IF hungry THEN eat snacks). And modern Symbolic AI systems can simulate billions of if/then statements every second!

To understand the implications, how about we use a new example: IF I’m cold THEN I wear a jacket. This is saying that if I’m definitely cold then I must be wearing my jacket, but if I’m not cold, I can wear whatever I want. So if cold is true and jacket is true, both sides of the implication are true. Even if I’m not cold and I wear my jacket, then the statement still holds up.

Same if I’m not cold and I decide to not wear my jacket. I can do whatever since I’m not cold. BUT if I am cold and I decide not to wear my jacket, then the statement no longer works. The implication is false. Simply put, An implication is true if the THEN-side is true or the IF-side is false.

Using the basic rules of propositional logic, we can start building a knowledge base of all of the propositions that are true about our universe.

After that knowledge base is built, we can use Symbolic AI to answer questions and discover new things! So, for example, if I were to help John-bot start building a knowledge base, I’d tell him a bunch of true propositions.

Oh John Bot?

Alright, you ready John Bot?

Smith is a person.

John-bot is a machine.

Car is a machine.

Car is old.

Car is smelly.

John Bot is not a person.

Smith isn’t a machine.

Toaster is a machine.

You getting all this John Bot?

Clearly, at this pace, John-bot would never be able to build a knowledge base with all the possible relations and symbols that exist in the universe. There are just too many.

Fortunately, computers are really good at solving logic problems. So if we populate a knowledge base with some propositions, then a program can find new propositions that fit with the logic of the knowledge base without humans telling it every single one.

This process of coming up with new propositions and checking whether they fit with the logic of a knowledge base is called inference. For example, the knowledge base of a grocery store might have a proposition that sandwich implies Between(Meat, Bread), or “IF sandwich THEN between(meat, bread)”.

Meat and Bread are the symbols, and Between is the relation that defines them.

So basically, this proposition is defining a sandwich as a symbol with meat between bread. Simple enough.

There might be other rules in the grocery knowledge base. Like, for example, a hotdog also implies Between(Meat, Bread), or “IF hotdog THEN between(meat, bread).” Now, if the grocery store is having a sale on sandwiches, should the hot dogs also be on sale?

Well, with inference, the grocery store’s AI system can apply the following logic: because sandwiches and hotdogs are both symbols that imply meat between bread, then hot dogs are inferred to be sandwiches, and the discount applies!

Over the years, we’ve created knowledge bases for grocery stores, banks, insurance companies, and other industries to make important decisions. These AI systems are called expert systems, because they basically replace an expert like an insurance agent or a loan officer.

Symbolic AI expert systems have some advantages over other types of AI that we’ve talked about, like neural networks.

First, a human expert can easily define and redefine the propositional logic in an expert system. If a bank wants to give out more loans, for example, then they can change propositions involving credit score or account balance rules in their AI’s knowledge base.

If a grocery store decides that they don’t want to discount hotdogs during the sandwich-sale, then they might redefine what it means to be a sandwich or a hotdog.

Hey Siri, is a hotdog a sandwich?

Siri: Of course not John.

Do not waste my time with foolish questions.

Second, expert systems make conclusions based on logic and reason, not just trial-and-error guesses like a neural network. And third, an expert system can explain its decisions by showing which parts were evaluated as true or false.

A Symbolic AI can show a doctor why it chose one diagnosis over another or explain why an auto loan was denied. The hidden layers in neural networks just can’t do that… at least, not yet.

This, so-called, “good old-fashioned AI” has been really helpful in situations where the rules are obvious and can be explicitly entered as symbols into a knowledge base. But this isn’t always as easy as it sounds.

How would you describe a hand-drawn number 2 as symbols in a number knowledge base? It’s not that easy. Also, lots of scenarios are not just true or false, the real world is fuzzy and uncertain.

As we grow up, our brains learn intuition about these fuzzy things, and this kind of human-intuition is difficult or maybe impossible to program with symbols and propositional logic.

Finally, the universe is more than just a collection of symbols.

The universe has time, and over time, facts change and actions have consequences.

Robots aren’t like humans who can do a lot of different things.

Robotics is a broad topic, because it’s the science of building a computer that moves and interacts with the world (or even beyond the world, in space).

What are Robots?

Some of the most exciting AIs are robots that move through the world with us, gathering data and taking actions! Robots can have wings to fly, fins to swim, wheels to drive, or legs to walk. And they can explore environments that humans can’t even survive in.

Unlike humans, who can do many different things. Robots are built to perform specific tasks, with different requirements for hardware and for learning.

Challenges for Robots

Curiosity is a pretty amazing robot who spent 7 years exploring Mars for us, but it wouldn’t be able to build cars like industrial robots or clean your apartment like a Roomba.

Robotics is such a huge topic that it’s also part of Computer Science, Engineering, and other fields. In the field of AI, robotics is full of huge challenges.

In some cases, what’s easy for computers (like doing millions of computations per second) is hard for humans. But with robotics, what’s easy for humans, like making sense of a bunch of diverse data and complex environments, is really hard for computers.

Problems for Robots

Like, for example, in the reinforcement learning topic, we talked about walking, and how hard it would be to precisely describe all the joints and small movements involved in a single step. But if we’re going to build robots to explore the stars or, get me a snack, we have to figure out all those details, from how to build an arm to how to use it to grab things.

So we’re going to focus on three core problems in robotics: Localization, Planning and Manipulation. The most basic feature of a robot is that it interacts with the world.

To do that, it needs to know where it is (which is localization) and how to get somewhere else (which is planning). So localization and planning go hand-in-hand. We humans do localization and planning all the time.

Let’s say you go to a new mall and you want to find some shoes. What do you do? You start to build a map of the mall in your head by looking around at all the walls, escalators, shops, and doors. As you move around, you can update your mental map and keep track of how you got there — that’s localization.

And once you have a mental map and know the way to the shoe store, you can get there more quickly next time.

For example, you can plan that the escalator is faster than the elevator. The most common way we input that data is with our eyes through perception. Our two slightly different views of the world allows us to see how far away objects are in space. This is called stereoscopic vision.

And this mental map is the key to what many robots do too, if they need to move around the world. As they explore, they need to simultaneously track their position and update their mental map of what they see.

This process is called Simultaneous Localization And Mapping, which goes by the cool nickname SLAM. But instead of eyes, robots use all kinds of different cameras.

Many robots use RGB cameras for perception, which gather color images of the world. Some robots, like John-bot, use two cameras to achieve stereoscopic vision. But robots can also have sensors to help them see the world in ways that humans can’t.

One example is infrared depth cameras. These cameras measure distances by shooting out infrared light (which is invisible to our eyes) and then seeing how long it takes to bounce back.

Infrared depth cameras are how some video game motion sensors work, like how the Microsoft Kinect could figure out where a player is and how they’re gesturing.

This is also a part of how many self-driving cars work, using a technology called LiDAR, which emits over 100,000 laser pulses a second and measures when they bounce back.

This generates a map of the world that marks out flat surfaces and the rough placement of 3D objects, like a streetlamp, a mailbox, or a tree on the side of the road.

Once robots know how close or far away things are, they can build maps of what they think the world looks like, and navigate around objects more safely. With each observation and by keeping track of its own path, a robot can update its mental map. Just keep in mind, most environments change, and no sensor is perfect.

So a lot goes into localization, but after a robot learns about the world, it can plan paths to navigate through it.

Planning

Planning is when an AI strings together a sequence of events to achieve some goal, and this is where robotics can tie into Symbolic AI from the last topic.

For example, let’s say John-bot had been trained to learn a map of this office, and I wanted him to grab me a snack from the kitchen. He has localization covered, and now it’s time to plan.

To plan, we need to define actions, or things that John-bot can do. Actions require preconditions, or how objects currently exist in the world. And actions have effects on those objects in the world to change how they exist.

So if John Bot’s mental map has a door between his current location and the kitchen, he might want to use an “open door” action to go through it. This action requires a precondition of the door being closed, and the effect is that the door will be open so that John Bot can go through it.

John Bot’s AI would need to consider different possible sequences of actions (including their preconditions and effects) to reason through all the routes to the kitchen in this building and choose one to take.

Manipulation

Searching through all these possibilities can be really challenging, and there are lots of different approaches we can use to help AIs plan. Anyway, during planning we run into the third core problem of robotics: manipulation.

What can John Bot’s mechanical parts actually do? Can he actually reach out his arms and interact with objects in the world? Many humans can become great at manipulating things (and I’m talking about objects, not that force powers stuff).

Like, for example, I can do this but it took me a while to get good at it. Just look at babies, they’re really clumsy by comparison. Two traits that help us with manipulation, and can help our robots, are proprioception and closed loop control. Proprioception is how we know where our body is and how it’s moving, even if we can’t see our limbs.

Let’s try an experiment: I’m going to close my eyes, stretch my arms out wide, and point with both hands. Now, I’m going to try to touch my index fingers without looking. Almost perfect!

And I wasn’t able to leave because of proprioception. Our nervous system and muscles help our body’s sense of proprioception. But most robots have motors and need sensors to figure out if their machine parts are moving and how quickly.

The second piece of the puzzle is closed loop control or control with feedback. The “loop” we’re talking about involves the sensors that perceive what’s going on, and whatever mechanical pieces control what’s going on. If I tried that experiment again with my eyes open instead of closed, it would go even better.

As my fingers get closer to each other, I can see their positions and make tiny adjustments. I use my eyes to perceive, and I control my arms and fingers with my muscles, and there’s a closed loop between them — they’re all part of my body and connected to my brain.

It’d be a totally different problem if there was an open loop or control without feedback, like, for example, if I closed my eyes and tried to touch my finger to someone else’s.

My brain can’t perceive with their eyes or control their muscles, so I don’t get any feedback and basically have to keep doing whatever I start doing. We use closed loop control in lots of situations without even thinking about it.

If a box we’re picking up is heavier than expected, we feel it pull the skin on our fingers or arms so we tighten our grip. If it’s EVEN heavier than expected, we’d might try & involve our other hand, & if it’s too heavy, well, we’ll call over my open-loop-example buddy. But this process has to be programmed when it comes to building robots.

Interaction

Manipulation can look different depending on a robot’s hardware and programming. But with enough work we can get robots to perform specific tasks like removing the cream from an oreo cookie.

Beyond building capable robots that work on their own, we also have to consider how robots interact and coordinate with other robots and even humans.

In fact, there’s a whole field of Human-Robot Interaction that studies how to have robots work with or learn from humans. This means they have to understand our body and spoken language commands.

What’s so exciting about Robotics is that it brings together every area of AI into one machine. And in the future, it could bring us super powers, help with disabilities, and even make the world a little more convenient by delivering snacks. But… we’re still a long way from household robots that can do all these things.

And when we’re building and training robots, we’re working in test spaces rather than the real world. For instance, a LOT of work gets done on self-driving car AI, before it even gets close to a real road. We don’t want a flawed system to accidentally hurt humans.

These test spaces for AI can be anything from warehouses, where robots can practice walking, to virtual mazes that can help an AI model learn to navigate.

In fact, some of the common virtual test spaces are programmed for human entertainment: games.

AI researchers spend a LOT of time trying to teach AI how to beat humans at games, and this isn’t just because games are fun.

Games provide constrained scenarios for testing new AI algorithms and approaches. In a game, it’s easy to know whether the AI is winning or losing, because there’s usually a score or some objective measure of “winning.”

This is great because AI learns from examples, trying things out, and slowly improving. Games basically provide their own training data, which is a big relief because AI systems need lots of training data to get really good.

An AI can even play against itself to generate training data and evolve better game strategies. Or an AI can be programmed to look at previous games (even games played by expert humans) for strategies that lead to victory.

Comparing AIs against expert human gamers can also help us figure out how an AI is improving over time. This comparison also gives us a sense of the difficulty of problems an AI can solve.

In other words, if I can teach John-Bot to beat me at battleship, I can probably also teach him to beat me at any game that’s simpler than battleship. Finally, games are cool (and that’s important too!).

Sometimes AI programming can feel a bit difficult or dry because of all the math and troubleshooting, and games can provide a fun motivation to learn all this stuff.

This is the reason why some of my first AI demos were games. For all these reasons, games and computing have a rich history that’s worth diving into.

History of Chess

Humans have always been fascinated by the idea that machines could beat us at our own strategy games. In 1770, the inventor Wolfgang von Kempelen revealed an invention to the Empress Maria Theresa of Austria.

The Mechanical Turk was a clock-like contraption that appeared to play chess. Chess pieces were attached to rods that were hooked into a bathtub-sized box.

After Empress Maria made a move on the board, she turned a crank that activated the machine, which would move chess pieces mechanically. To her surprise, the machine was able to beat most challengers.

However, it was an elaborate hoax and The Mechanical Turk was actually controlled by a person hidden inside! Getting a machine to play chess is actually really complicated.

So when AI researchers first tried to tackle the problem in the late 1940s and early 1950s, they focused on simpler chess situations. Like, games with only a few pieces remaining on the board or full games played on a small 6×6 board without any bishops.

At the same time, researchers worked on an AI that could play checkers, because checkers looked easier… although it was really almost as complicated.

The first program to play a legitimate game of checkers was built by IBM in 1956. And, in a classic cold war move, two programs that could play a full chess game were developed in parallel by the US and Russia in 1957. But these programs didn’t get good for another 40 years.

Checkers was first, with a program called Chinook which started dominating masters in 1995. Chess followed when a computer called Deep Blue beat the chessmaster Garry Kasparov in 1997.

Since then, strategy games have been mastered one-by-one, with the most recent victories over humans at Go in 2017, DOTA 2 in 2018, and Starcraft II in 2019.

Okay, so the best way to understand the difficulty of teaching AI to play games is through an example.

Minimax Algorithm

So let’s start with a really simple goal: teaching John-bot to play tic-tac-toe. One of the ways that we can think about playing tic-tac-toe is as a tree with all the possible moves from any given state of what the game board looks like.

For example, if this is the current game state, it’s John-bot’s turn, and he’s using Xs… there are three places he can go. We can draw a tree representing possible outcomes for each of these options, and all of the options his opponent (me, or anyone else) can take:

Because computers think with numbers, each outcome can be assigned a reward — a number like a 1 for a win, and -1 for a loss or tie. Basically, John-bot will need to search through the tree of possibilities to find his win.

To decide which choice to make, John-bot will assume that in each tree, both he AND his opponent will make the best possible choices. In other words, his policy (or his framework for making decisions) will alternate between choosing the branch that will maximize the outcome of winning on his turn, and minimize the outcome of his opponent winning on their turn. This is called the minimax algorithm.

Then, each game state can be assigned a value based on how likely it leads to John-bot winning, and he can decide which move to make based on his policy. Looking at this tree, John-bot will always pick option 1.0, and win the game!

Of course, this was a pretty small tree because we were looking at a game in progress. To draw the whole tic-tac-toe tree from beginning to end, we would need to represent about 250,000 boards.

That seems like a lot, but it would take like a half a second for a powerful modern computer to compute this many options. By laying out all the possibilities and taking the paths that led to a win, John-bot can solve tic-tac-toe.

This means that John-bot will always achieve the best possible outcome, either a win or a tie, no matter how his opponent plays. But we can’t solve all games this way.

Checkers, for example, has about 10 to the 20th power board states… or 10 followed by 20 zeros. That’s more board states than there are grains of sand on Earth! Chess has 10 to the 50th power board states. And Go has 10 to the 250th power board states.

To put those huge numbers into perspective, there are only 10 to the 80th atoms in the entire known universe! Computer scientists have theorized that it would be impossible for conventional computers to calculate this many states due to the laws of physics.

Like, for example, if you combined all planets and stars and everything in the whole universe into a single supercomputer, it still wouldn’t be powerful enough to solve the game of Go. But some people have hope that quantum computers may be able to get there one day…

Monte Carlo

So, if figuring out all of the board states could be mathematically impossible, how did computers beat the number one ranked human masters in Chess and Go?

Many modern systems, including Google’s AlphaGo computer that beat a human master in Go in 2017, use an algorithm called Monte Carlo Tree Search. Monte Carlo is a famous casino, so whenever you see the term “monte carlo,” it’s a good bet that the algorithm will be using randomness and chance to solve a problem.

Combining Monte Carlo randomness and regular tree search like minimax, modern game AIs decide which part of the huge tree to search by guessing at odds. Basically, they want higher odds that the part of the game tree they search will lead to a win. But these aren’t just random guesses like we would make in many casino games, AI systems can simulate millions of “what-if” scenarios and use math to estimate the likelihood of winning if they choose one path or another.

In each “what-if” scenario, the AI considers making one particular move and then simulates playing a large number of (but not all) possible games, where the next moves are chosen at random.

By averaging these possible outcomes, the AI estimates how “good” that particular move is. It’s so much faster to estimate a handful of choices than exhaustively calculate each branch of the game tree. And some computers can even do this estimation in real time.

One example of this is Google’s DeepMind which defeated human professional players at Starcraft II in 2019 — where time is very critical. Of course, Starcraft II, Go, and Tic-Tac-Toe aren’t all the types of games that humans play.

Other games require other strategies and have other computational challenges:

IBM’s Watson question-answering system was able to beat human Jeopardy! champions in two televised matches in 2011. Watson listened for keywords in the clue and tried to use a knowledge graph to figure out responses.

And we’ll talk more about knowledge graphs in a coming topic. Watson wasn’t perfect and struggled a bit with context.

For example, it famously guessed “What is Toronto?” on something in the category “US Cities.” But Watson was still able to do better than human contestants overall. Evolutionary neural networks use the environment as an input, like reinforcement learning.

Evolutionary Neural Networks

But this approach introduces multiple agents who try multiple neural network structures, and then build on successful ones for the next generation. Sort of like animals, the ones that are better at surviving get to pass on their genes.

For example, the AI MarI/O can learn how to play a Super Mario World level by telling MarI/O what buttons it can push and that getting farther to the “right” in the level is good.

This AI will start by basically mashing buttons at random, but as some button mashes get it farther to the right, it remembers and learns from those successful attempts.

So, are there any games that are safe to play, where humans will always have an edge and AI won’t be able to beat us?

Computers definitely seem to struggle with parts of language like humor, irony, metaphor, and wordplay. Computers also aren’t great at understanding and predicting real people, who don’t always act “optimally,” so social games could be more difficult too. But AI systems are finding some success in bluffing games like online poker, so it’s important not to underestimate them.

Computers might also struggle with creativity or surprise, because there’s not a really clear way to assign values to states. It’s difficult to assign a number to “how creative” something is, compared to saying “go as far right as you can in the Mario level” or “achieve a winning state in a chess game.”

So, considering all of that, maybe games like charades would be pretty stacked for a human victory. Or… what about pictionary? Hide-and seek???

It seems like every time we look at the news, there’s a new article about how AI and automation is going to take everybody’s jobs!

You know there is a way for AI and humans to work together, besides competing for the same jobs, resources, and game championships. Human-AI teams can use our strengths to help each other, and collaborate to fill in each other’s weaknesses.

Together, we can make better diagnoses, brainstorm new inventions, or imagine a future where humans and robots are working side-by-side.

AIs and humans have skills that can complement each other, AI can be good at searching through lots of possibilities and making some intelligent guesses at which one to pick. Also, AI systems are consistent, and won’t make mistakes because they’re tired or hungry, like we sometimes do.

On the other hand, humans can be good at insight, creativity, and understanding the nuances of language and behavior. We’ve learned from living in the world and interacting with each other, so we’re better than AI systems at interpreting social signals.

There are many ways that AI could support us, but a big one is that AI could amplify our decision-making activities with the right information. Let’s understand this through an example of Human-AI collaboration.

Humans and computers can play games against each other, but when they join forces, they basically become a superhero dynamic duo. In chess, this particular kind of Human-AI team game is called advanced chess, cyborg chess, or centaur chess!

Cyborg Chess

In centaur chess, the computer does what it’s great at: it looks several moves ahead and estimates the most promising next moves. And the human does what they’re great at: they choose among uncertain possibilities based on experience, intuition, and even what they know about the opponent.

Usually, the computer program is in the driver’s seat for the early game, which is relatively mindless because there aren’t too many move choices. But humans step in to guide the strategy in the middle game when things get more complicated.

The first centaur chess tournament was held in 2007, with the winning team led by Anson Williams. Anson’s team, Intagrand, consists of him and the programs written by the Computer Science and Math experts Yingheng Chen and Nelson Hernandez.

And the cool thing about this team is that it’s currently considered to be the best chess player in the world, among humans and AI!

Intagrand was able to win 2-0 against chess grandmasters, even though none of the team members are grandmasters.

And in 2014, multiple AIs played against multiple centaur chess teams in competitions. Pure AI won 42 games, while centaur teams won 53 games!

It seems like Human-AI collaboration is working out for chess, so that means it could be promising in other parts of our lives too.

Games provide a great constrained environment to demonstrate the possibilities of AI and, in this case, human-AI collaboration. Chess victories might not seem that significant, but similar Human-AI collaboration can be applied to other problems.

AI takes on the parts that require memorization, rote response, and following rules. Humans focus on aspects that require nuance, social understanding, and intuition.

For one, AI could help humans make decisions when we’re dealing with large amounts of complicated information.

When a doctor is trying to make a diagnosis, they try to use their medical experience and intuition to make sense of their patient’s symptoms and all the published clinical data.

AI could help wade through all that data and highlight the most probable diagnoses, so the doctor can focus their experience and intuition on choosing from those which is where they’ll be most helpful.

Second, AI could help when humans are trying to come up with new inventions or designs, like a new engineered structure. An AI could apply predetermined physical constraints, like, for example, how much something should weigh or how much force it should be able to withstand.

This lets the human experts think about the most practical designs, and could spark new creative ideas.

Third, AI could also support and scale-up interaction between people. It could save people from doing rote mental tasks, so that they have more time and energy to help.

For example, in customer support, virtual assistants can help answer easy questions about checking on an order or starting a return… or at least that’s the goal. If you’ve ever tried these systems, you know that they can sometimes fail in spectacular ways, leaving you mashing the 0 button on your phone to try and get a representative on the line.

And fourth, robots that are AI-enabled but guided by humans could give people more strength, endurance, or precision to do certain kinds of work. Some examples of this may be an exosuit worn by a construction worker, or a remotely-guided search and rescue robot.

These devices still need some AI to apply the right amount of force or navigate effectively, but all of the real decision-making is done by humans.

There are entire research fields, like Human-Computer Interaction and Human-Robot Interaction, dedicated to investigating and building new AI, Machine Learning, and Robotics systems to complement and enhance human capabilities.

But it’s not just that humans can become more effective with AI, AI also needs humans to succeed!

Human AI Interaction

This whole topic we’ve been talking about how we can program AI to help them exist and learn, but a lot of human-AI interaction is more subtle. You could’ve supported an AI without even recognizing it.

First, humans can provide AI with meaning and labels, because we have so much more experience with living in the real world. For example, if you’ve ever edited Wikipedia, you’ve contributed to Wikipedia-based algorithms such as WikiBrain.

Because Wikipedia puts articles into nesting structures (like how an elephant is a mammal) and because articles link each other, algorithms can use this structure to understand the meaning-based connection between topics.

In fact, when we interact with digital technology, whether it’s posting content, giving something a thumbs up, following driving directions on a phone, or typing a text message, we’re often providing training data to help make AI systems more effective.

Without our data, there wouldn’t be recommended YouTube videos, predictive text messages, or traffic data for the GPS to use in suggesting a route.

Humans and AI Working Together

Sometimes providing personal data can be a double-edged sword which we’ll get to in the Algorithmic Bias topic. Secondly, humans can also try to explain an AI system’s predictions, outputs, and even possible mistakes to other humans, who aren’t as familiar with AI.

As we mentioned in the topic about artificial neural networks, the reasons for an AI producing particular results can be tough to understand. We can see what data goes into the program and which results fit the data well, but it can be hard to know what the hidden layers are doing to get those results!

For example, an algorithm might recommend denying a customer’s loan request. So a loan officer needs to be able to look at the input data and algorithm, and then communicate what factors might have led to denial. Many European countries are now making it a legal right to receive these kinds of explanations.

Third, human experts can also inspect algorithms for fairness to different kinds of people, rather than producing biased results.

Bias is a very complicated topic, so we’ll dive deeper into the nuances in an upcoming topic.

And finally, AI doesn’t understand things like the potential consequences of its mistakes or the moral implications of its decisions.

That’s beyond the scope of its programming. It’s a common Sci-Fi trope that an AI built to minimize suffering might choose to eliminate all life on Earth, because if there’s no life, there’s no suffering!

That’s why humans may want to moderate and filter AI actions in the world, so we can make sure they line up with societal values, morals, and thoughtful intentions.

The bottom line is: organic and artificial brains may be better together, and through this Free AI Course with Courses Buddy, you could be on the way to becoming one of those experts that works on the helpfulness and fairness of AI systems.

In this topic, we didn’t focus on explaining one specific algorithm or AI technology. Instead, it’s more about where our world might be going from the AI revolution that’s happening now… besides just “automation replacing jobs.”

We should recognize what data humans are providing to algorithms.

What would it mean if we could claim some credit for the ways that our data have allowed algorithms to change lives for the better? Or how do we claim more power in cases where data is being used in potentially harmful or problematic ways?

Second, we should think about if and how our human jobs could be made easier by working with AI — although it’s bound to be complicated. For example, people had similar concerns with the spread of personal computing and tools like spreadsheets.

And yes, spreadsheets automated many bookkeeping tasks, which put many people out of work. Even though some types of jobs were destroyed, new accounting jobs were created that involved human-computer collaboration.

Technology took over more of the rote math calculations, and humans focused on the more nuanced and client-facing aspects of accounting work. Even though this idea can get overblown in the mass media, AI and automation has and will take people’s jobs. No question.

And we don’t want to downplay the impact that has on people’s lives. But by understanding how AI works, what it’s good at, and where it struggles, we can also find opportunities to work more effectively and to create new types of jobs that involve collaboration.

Machines can help us do things that we can’t do as well (or at all) by ourselves.

Human-AI collaboration can help us narrow down complex decision trees and make better choices. Human-Robot collaboration has the potential to give us super strength or resilience. Different kinds of AI will impact the world in powerful ways, but not without costs.

So it’s up to us to decide which costs are worth it, how to minimize harm, and create a future we want to live in.

Recommender systems are a type of AI that try to understand our brains and make useful recommendations to us.

This kind of AI can guide the things we watch by recommending YouTube videos or shows on Netflix for example. On Amazon, it’s recommending items to buy, when I search on Google, it’s recommending relevant and interesting links. And everywhere online, advertisement servers are trying to recommend products and services.

Recommender systems combine supervised learning and unsupervised learning techniques to learn about us. And because we’re so complicated, recommending stuff to us is a tough problem that can produce lots of unexpected results.

Maybe we get caught in an online bubble and only see tweets from our friends and people who think like us. Maybe we miss a new TV show because streaming sites don’t think we’d like it.

Or maybe that creepy thing happens where you’re talking to your friends about supercomputers and then every single ad you see for the next day is for supercomputers.

AI that makes recommendations can really change what version of the internet we all see. But to understand the benefits and drawbacks of these algorithms, we have to understand where they get their data and how they work.

As an example, let’s focus on an algorithm that could recommend YouTube videos. Because “The Algorithm” is a really big deal if YouTube is your job, and everyone’s talking about the mysterious changes behind the algorithm anyway.

Three Common Approaches by AI Recomenders

Three common approaches are content-based recommendation, social recommendation, and personalized recommendations. Content-based recommendations look at the content of the videos, not the audience. Like, for example, our algorithm may decide to recommend more recent videos, or videos that are made by someone on a list of “quality creators.”

But this means someone has to decide who “quality creators” are, or program an AI that tries to predict creator quality.

On the other hand, social recommendations pay attention to the audience. YouTube is on the internet so we can use social ratings such as “likes” or “views” or “watch time” to decide what people are watching and should be recommended. But not everybody likes the same stuff, so maybe pure popularity isn’t the way to go. Different people have different preferences, so our AI can incorporate that with personalized recommendations.

But the problem with personalized recommendations is that it might be difficult to stumble onto new interesting stuff. So, to get the best of all worlds, recommender systems generally use collaborative filtering, which combines all three of these recommenders.

When we see a recommendation on YouTube, it could be because that video is similar to other videos that we’ve watched and liked and other people who have similar tastes watched and liked that video. Or (especially if you’re new to YouTube) that video might be recommended because it’s popular and lots of people are watching and liking it.

Collaborative filtering combines several of the techniques we’ve already talked about in this AI course. It uses unsupervised learning to find similar people or content, and it tries to use data from those things to predict how we would feel about something we haven’t even seen yet.

Problems

Recommender systems that use collaborative filtering AI can take in lots of different data, not just a 1 or a 0, for whether a user subscribed to a YouTube channel or bought a product. A movie rating site might use a one-to-five star rating system. Or a social media AI could keep track of the number of milliseconds a user dwells on a post.

Regardless, the basic strategy is the same: use known information from users to predict preferences. And this can get complicated on big websites that gather lots of user information using a combination of different algorithms.

The real world is full of a lot of data and there are three key problems that can lead to recommender systems making small or big mistakes.

First, datasets that recommender system AIs are usually very sparse. Most people don’t watch most shows or videos — there just isn’t enough time! And even fewer people give social ratings such as “likes.”

Doing any kind of analysis with sparse datasets is very computationally intense, which gets expensive, which means some companies are willing to lose some accuracy to reduce costs.

Second, there’s the cold start problem. When we go on a website for the first time, for example, the AI doesn’t know enough about us to provide good personalized recommendations.

Third, even if an AI makes statistically likely predictions, that doesn’t mean those recommendations are actually useful to us. Online ads run into this failure a lot, where we’ll be shown ads for sites we recently visited, or something we just bought. Sure, that’s probably something I’m interested in, but I could’ve figured that out without a recommender system.

In a potentially more harmful way, recommender systems don’t understand important social context, so “statistically likely” recommendations can be worrying.

Recommendations may stereotype users in a socially uncomfortable way. Like, for example, an AI might assume that because John-Bot is a robot, he really wants to watch WALL-E and Robocop. Just because he’s a robot doesn’t mean he wants to watch robot stuff.

Or recommendations might be inappropriate for certain users, like recommending a video that a parent would consider too violent to their children after they had watched a bunch of NERF War videos. And, on social media, recommendations can trap us in ideological echo chambers, where we tend to only see the opinions from people that agree with us, which can skew our knowledge about the world.

This idea that we all see slightly different versions of the internet, and data is constantly being collected about us, can be a little concerning. But understanding how recommender systems work, can help us live more knowledgeable lives, and coexist with AI.

When we don’t want data added to a recommender system’s model of us, we can use a private or incognito browser window and not log into sites. If we open a news homepage this way, we might see what the average human (or robot!) is being recommended.

Of course, incognito browsers don’t mean total privacy, but this strategy prevents sites from connecting data — like, for example, my Twitter account with my searches for tiny polo shirts on Google (because I needed to get John-bot a birthday present).

Moreover, since we spend so much time online, we might want to make the most of it with really personalized recommendations. So… seriously… “like, comment, and subscribe” to your favorite creators because as we leave ratings, reviews, and other traces of online activities, recommender systems can learn better models.

Recommender systems are a part of the internet as we know it, whether we like it or not. And as AI becomes a bigger part of our lives, these kinds of recommendations will be too. So it’s on us to be aware of this technology, so that we know what kind of world we’re living in, and the ways AI might influence us every single day.

And if you’re here to learn how to build recommender systems, my advice would be to think explicitly about the trade-offs that are involved.

Deciding how to define the clusters of users or items, can create more or less personalized spaces.

There used to be a time when a group of friends at dinner could ask a question like “is a hot dog a sandwich?” and it would turn into a basic shouting match with lots of gesturing and hypothetical examples. But now, we have access to a LOT of human knowledge in the palm of our hands… so our friends can look up memes and dictionary definitions and pictures of sandwiches to prove that none of them have a connected bun like hot dogs (disappointed).

Search engines are a huge part of modern life. They help us access information, find directions to places, shop, and participate in sandwich arguments. But how does Google find answers to questions? How are Siri and Alexa so smart but also easily stumped? How did IBM’s Watson beat the best Jeopardy players in the world?

Well, search engines are just AI systems that are getting better and better at helping us find what we’re looking for.

What are Search Engines?

When we talk about search engines, we typically think about the AI systems online, like Google, Bing, Duck Duck Go and Ask Jeeves. But the basic ideas behind non-AI search engines have existed for centuries.

Essentially, search engines gather data, create organization systems to sort that data, and find results to a question.

How Search Engines Work?

For example, when you needed an answer to a question and couldn’t search online, you could go to the library! Libraries gather data in the form of books and newspapers that are stacked neatly on the shelves.

Librarians have organization systems to help you find what you’re looking for. Knowing that magazines are on shelves by the water fountain, while kids books are on the second floor is a kind of organization. Also, fiction books are sorted by the author’s last name, while nonfiction has the Dewey Decimal System, and so on.

Once you (or the librarian) have the resources you need, you’ll be able to find results to your question!

Web vs Internet

Now, rather than looking through books, web search engines look through all the data on the World Wide Web, aka “the Web”. And instead of asking a human librarian where to find information, we ask an AI like John-bot instead.

And just so we’re clear, we’re using “Web” throughout this video even though it might sound a little old-fashioned. That’s because the Internet and the Web are not the same thing.

The Internet is a collection of computers that send messages to each other. Video services like Netflix that play on your TV, for example, use the Internet, not the Web.

The Web, on the other hand, is part of the Internet and uses the Internet’s connections to send documents and other content in a format that can be displayed by a browser like Chrome or Safari.

Gather Data

As with most AI systems, the first step is to gather lots of data. To gather data on the Web, we can use a computer program called a Web crawler, which systematically finds and downloads Web pages. This is a HUGE task and happens before the search engine AI can take any questions.

It starts on some Web page that we pick, called a seed, and downloads that page and finds all its links. Then, the crawler downloads each of the linked Web pages and finds their links, and so on… until we’ve crawled the whole Web.

Inverted Index

After we have collected all the data, the AI’s next step is to organize it by building an index, which is a kind of lookup system. The kind that’s used for organizing Web pages is called an inverted index, which is like the index in the back of a textbook.

For each word, it lists all of the Web pages that contain that word. Usually, the Web pages are represented by I.D. numbers so we don’t have a long, messy list of URLs.

Let’s say 0 is the seed – which happens to be a page about Genghis Khan. It has a lot of words on it like “the, mongol, Khan, Genghis, who, and is”. In this inverted index, page 1 is about Marco Polo, but it mentions the word “Genghis” along with words like “the, Marco, Polo, who, are, and is.”

Page 2 is about the Mongols, page 3 is a different webpage about Marco Polo, and page 4 is about Water Polo.

So, let’s say we type “Who is Genghis Khan?” into a search engine. Our AI can use this inverted index to find results, which in this case, are links to Web pages. The AI will look at the words “who”, “is”, “Genghis”, and “Khan” and use the inverted index to find relevant pages.

Our AI might find that Web pages zero, one, two and five have at least one of the words from the question “who is Genghis Khan?” When Siri says “I found this for you,” the AI is just returning a list of Web pages that contain the same terms as the question.

Rank Web Pages

Except… most search engines include one more step. There are millions of pages online that contain the same terms. So it’s important for search engines to rank Web pages, so that the top result is more likely to be relevant than the tenth result or the hundredth.

Of course, Google and Bing don’t hire “supervisors” to grade each possible question and answer to help their AI systems learn from training data. That would take forever, and they wouldn’t be able to keep up with all the new content that gets created every day.

Really, regular users like us do this training for free all the time. Every time we use a search engine, our behavior tells the AI whether or not the results answered our question.

For example, if we type in “who is Genghis Khan” into a search engine, and click on a Web page about Star Trek II: The Wrath of Khan, we might be disappointed to find Genghis Khan isn’t ANYWHERE in that movie. So we’ll bounce back to the search results, and try again until we find a page that answers our question. A bounce indicates a bad result.

But if we click on a Wikipedia article about Genghis Khan and stay for a while reading, that’s a click through, which probably means that we found what we were looking for… so that indicates a good result.

Human behavior like bounces and click throughs give AI systems the training data they need to learn how to rank search results and better answer our questions. Data from the Web and data from how we use the Web helps make better and better search engines.

Knowledge Bases

Now, sometimes we ask our smart devices questions and we want actual answers… not links to Web pages. When I say “OK Google, what’s the weather like in Indianapolis?” I don’t want to scroll through the results.

For this kind of problem, instead of using an inverted index, AIs rely on knowledge bases. Which you might remember from our video about Symbolic AI. A knowledge base encodes information about the universe as relationships between objects like “chocolate donut” and “John-Bot wears polo”.

One of the main problems with knowledge bases is that it’s really hard to write down all of the facts in the universe, especially common sense things that humans take for granted but computers need to be told.

Enter AI researcher Tom Mitchell and his team of scientists from Carnegie Mellon University. In 2010, they created a huge knowledge base called the Never Ending Language Learner or NELL, which was able to extract hundreds of thousands of facts from random Web pages.

The way it works is really clever, so let’s go to the example to see how. NELL starts with some facts provided by a human, for example, the genre of music that Mozart plays is classical. Which was represented like this:

Mozart. musicGenre. Classical.

Similarly,

Jimi Hendrix. plays. Guitar.

And Darth Vader. has Child. Luke Skywalker.

Then, NELL gets to work and reads through each Web page one-by-one for words mentioned in those facts. Maybe it finds the text “Mozart plays the piano.” NELL doesn’t know much about these symbols, but this text matches the same pattern as one of the facts provided by a human, specifically, the “plays” relationship.

So NELL learns a new object: Piano. And a new fact: Mozart. plays. Piano. By searching over the entire Web, NELL can learn lots of facts based on just the three original ones that humans gave it!

Some facts might appear hundreds or thousands of times online, like Lenny Kravitz. hasChild. Zoë Kravitz. But NELL might also find facts that are mentioned SOMEWHERE online and extract them as potentially true. Like, for example, Darth Vader. plays. Kloo Horn. We just don’t know!

Just like how we look for multiple sources when writing a paper, NELL uses repetition and multiple sources to build confidence that the facts it’s finding are actually true.

To consider other relationships, NELL uses the highly confident facts it learned and searches through the Web again. Only this time, NELL is looking for new relationships.

Maybe it finds the text “Darth Vader cuts off Luke Skywalker’s hand,” and NELL learns a new (very specific) relationship: cutsOffHand. Over and over again, NELL will use known relationships to find new objects, and known objects to find new relationships — creating a huge knowledge base.

AI systems can use huge knowledge bases, like this one extracted by NELL, to answer our questions directly.

Instead of using the words from our questions to search through an inverted index, an AI like Siri can reformulate our questions into incomplete facts and then look for matches in a knowledge base.

Hey John Bot….

John Bot: Yes, Smith?

Smith: “Who wrote The Bluest Eye?”

His AI could then reformulate that question into an incomplete fact, replacing “who” with a question mark. If John-bot extracted that information earlier, he can find matches in his knowledge base and return the most confident result.

John-Green-bot: Toni Morrison wrote The Bluest Eye!

Smith: Hey. Thanks, John-bot!

Different words are categorized differently, so an AI like John-bot can tell the difference between questions asking “who” and “when” and “where.” But that gets more complicated, so we’re not going to dive into the details here. If you want to learn more, you can read about part of speech tagging systems.

Using all these strategies, search engines have become really good at answering common questions. But questions like “How many trees are in Ohio?” or “How many hotdogs are eaten in the South Sandwich Islands annually?” still stump most AI systems, because not enough people ask them and AI hasn’t learned how to answer them well yet.

It’s also important to watch out for search engine answers to questions like “Who invented the time machine?” because AI systems have a tough time with nuance and incomplete data.

And a big, sort of hidden, problem is that search engine AI systems are influenced by any biases in data online. For example, if I ask Google for images of “nurses,” it will mostly show pictures of female nurses.

So in the next topic, we’ll talk about how an algorithm can be biased, where bias comes from, and what we can do to address bias in AI.

Algorithms are just math and code, but algorithms are created by people and use our data, so biases that exist in the real world are mimicked or even exaggerated by AI systems.

This idea is called algorithmic bias. Bias isn’t inherently a terrible thing.

Our brains try to take shortcuts by finding patterns in data. So if you’ve only seen small, tiny dogs, you might see a Great Dane and be like “Whoa that dog is unnatural.”

This doesn’t become a problem unless we don’t acknowledge exceptions to patterns or unless we start treating certain groups of people unfairly. As a society, we have laws to prevent discrimination based on certain “protected classes” (like gender, race, or age) for things like employment or housing.

So it’s important to be aware of the difference between bias, which we all have, and discrimination, which we can prevent.

And knowing about algorithmic bias can help us steer clear of a future where AI is used in harmful, discriminatory ways.

Hidden Biases

There are at least 5 types of algorithmic bias we should pay attention to.

First, training data can reflect hidden biases in society. For example, if an AI was trained on recent news articles or books, the word “nurse” is more likely to refer to a “woman,” while the word “programmer” is more likely to refer to a “man.” And you can see this happening with a Google image search: “nurse” shows mostly women, while “programmer” mostly shows mostly men.

We can see how hidden biases in the data gets embedded in search engine AI. Of course, we know there are male nurses and female programmers and non-binary people doing both of these jobs!

For example, an image search for “programmer 1960” shows a LOT more women. But AI algorithms aren’t very good at recognizing cultural biases that might change over time, and they could even be spreading hidden biases to more human brains.

It’s also tempting to think that if we just don’t collect or use training data that categorizes protected classes like race or gender, then our algorithms can’t possibly discriminate. But, protected classes may emerge as correlated features, which are features that aren’t explicitly in data but may be unintentionally correlated to a specific prediction.

For example, because many places in the US are still extremely segregated, zip code can be strongly correlated to race. A record of purchases can be strongly correlated to gender. And a controversial 2017 paper showed that sexual orientation is strongly correlated with characteristics of a social media profile photo.

Second, the training data may not have enough examples of each class, which can affect the accuracy of predictions. For example, many facial recognition AI algorithms are trained on data that includes way more examples of white peoples’ faces than other races.

One story that made the news a few years ago is a passport photo checker with an AI system to warn if the person in the photo had blinked. But the system had a lot of trouble with photos of people of Asian descent.

Being asked to take a photo again and again would be really frustrating if you’re just trying to renew your passport, which is already sort of a pain! Or, let’s say, you got a cool gig programming a drone for IBM… but it has trouble recognizing your face because your skin’s too dark… for example.

Hard to Quantify

Third, it’s hard to quantify certain features in training data. There are lots of things that are tough to describe with numbers. Like can you really rate a sibling relationship with a number? It’s complicated!

You love them, but you hate how messy they are, but you like cooking together, but you hate how your parents compare you… It’s so hard to quantify all that!

In many cases, we try to build AI to evaluate complicated qualities of data, but sometimes we have to settle for easily measurable shortcuts. One recent example is trying to use AI to grade writing on standardized tests like SATs and GREs with the goal to save human graders time.

Good writing involves complex elements like clarity, structure, and creativity, but most of these qualities are hard to measure. So, instead, these AI focused on easier-to-measure elements like sentence length, vocabulary, and grammar, which don’t fully represent good writing… and made these AIs easier to fool.

Some students from MIT built a natural language program to create essays that made NO sense, but were rated highly by these grading algorithms.

These AIs could also potentially be fooled by memorizing portions of “template” essays to influence the score, rather than actually writing a response to the prompt, all because of the training data that was used for these scoring AIs.

Positive Feedback Loop

Fourth, the algorithm could influence the data that it gets, creating a positive feedback loop.

A positive feedback loop basically means “amplifying what happened in the past”… whether or not this amplification is good. An example is PredPol’s drug crime prediction algorithm, which has been in use since 2012 in many large cities including LA and Chicago.

Geolitica was trained on data that was heavily biased by past housing segregation and past cases of police bias. So, it would more frequently send police to certain neighborhoods where a lot of racial minority folks lived.

Arrests in those neighborhoods increased, that arrest data was fed back into the algorithm, and the AI would predict more future drug arrests in those neighborhoods and send the police there again.

Even though there might be crime in neighborhoods where police weren’t being sent by this AI, because there weren’t any arrests in those neighborhoods, data about them wasn’t fed back into the algorithm.

While algorithms like PredPol are still in use, to try and manage these feedback effects, there is currently more effort to monitor and adjust how they process data.

So basically, this would be like a new principal who was hired to improve the average grades of a school, but he doesn’t really care about the students who already have good grades. He creates a watchlist of students who have really bad grades and checks up on them every week, and he ignores the students who keep up with good grades.

If any of the students on his watchlist don’t do their homework that week, they get punished. But all of the students NOT on his watchlist can slack on their homework, and get away with it based on “what happened in the past.”

This is essentially what’s happening with PredPol, and you can be the judge if you believe it’s fair or not.

Finally, a group of people may mess with training data on purpose. For example, in 2014, Microsoft released a chatbot named Xiaoice in China. People could chat with Xiaoice so it would learn how to speak naturally on a variety of topics from these conversations. It worked great, and Xiaoice had over 40 million conversations with no incidents.

In 2016, Microsoft tried the same thing in the U.S. by releasing the Xbot Tay. Tay trained on direct conversation threads on Twitter, and by playing games with users where they could get it to repeat what they were saying.

In 12 hours after its release, after a “coordinated attack by a subset of people” who biased its data set, Tay started posting violent, sexist, anti-semitic, and racist Tweets.

This kind of manipulation is usually framed as “joking” or “trolling,” but the fact that AI can be manipulated means we should take algorithmic predictions with a grain of salt.

The common theme of algorithmic bias is that AI systems are trying to make good predictions, but they make mistakes. Some of these mistakes may be harmless or mildly inconvenient, but others may have significant consequences.

To understand the key limitations of AI in our current society, let’s go to the example.

Let’s say there’s an AI system called HireMe! that gives hiring recommendations to companies. HireMe is being used by Robots Weekly, a magazine where John-bot applied for an editorial job.

Just by chance, the last two people named “John” got fired from Robots Weekly and another three “Johns” didn’t make it through the hiring process. So, when John-Bot applies for the job, HireMe! predicts that he’s only 24% likely to be employed by the company in 3 years.

Seeing this prediction, the hiring manager at Robots Weekly rejects John-bot, and this data gets added to the HireMe! AI system. John-Bot is just another “John” that got rejected, even though he may have been the perfect robot for the job! Now, future “Johns” have an even lower chance to be hired. It’s a positive feedback loop, with some pretty negative consequences for John-Bot.

Of course, being named “John” isn’t a protected class, but this could apply to other groups of people. Also, even though algorithms like HireMe! Are great at establishing a link between two kinds of data, they can’t always clarify

why they’re making predictions. For example, HireMe! may find that higher age is associated with lower knowledge of digital technologies, so the AI suggests hiring younger applicants.

Not only is this illegally discriminating against the protected class of “age,” but the implied link also might not be true. John-bot may be almost 40, but he runs a robot blog and is active in online communities like Nerdfighteria!

So it’s up to humans interacting with AI systems like HireMe! to pay attention to recommendations and make sure they’re fair, or adjust the algorithms if not.

Monitoring AI for bias and discrimination sounds like a huge responsibility, so how can we do it? The first step is just understanding that algorithms will be biased.

It’s important to be critical about AI recommendations, instead of just accepting that “the computer said so.” This is why transparency in algorithms is so important, which is the ability to examine inputs and outputs to understand why an algorithm is giving certain recommendations.

But that’s easier said than done when it comes to certain algorithms, like deep learning methods. Hidden layers can be tricky to interpret.

Second, if we want to have less biased algorithms, we may need more training data on protected classes like race, gender, or age. Looking at an algorithm’s recommendations for protected classes may be a good way to check it for discrimination. This is kind of a double-edged sword, though.

People who are part of protected classes may (understandably) be worried about handing over personal information. It may feel like a violation of privacy, or they might worry that algorithms will be misused to target rather than protect them.

Even if you aren’t actively working on AI systems, knowing about these algorithms and staying informed about artificial intelligence are really important as we shape the future of this field.

Anyone, including you, can advocate for more careful, critical interpretation of algorithmic outputs to help protect human rights. Some people are even advocating that algorithms should be clinically tested and scrutinized in the same way that medicines are.

According to these opinions, we should know if there are “side effects” before integrating AI in our daily lives. There’s nothing like that in the works yet.

But it took over 2400 years for the Hippocratic Oath to transform into current medical ethics guidelines.

So it may take some time for us to come up with the right set of practices.

Hey everyone!

Welcome to the final topic of Free AI Course with Courses Buddy.

We’ve covered a lot of ground together, from the basics of neural networks to game playing, language modeling, and algorithmic bias. And as we’ve been learning about different parts of artificial intelligence as a field, there have been a couple themes that keep coming up..

First, AI is in more places than ever before.

The machine learning professor Andrew Ng says that “Artificial Intelligence is the New Electricity.”

This is a pretty bold claim, but lots of governments are taking it seriously and planning to grow education, research, and development in AI. China’s plan alone calls for over 100 billion U.S. dollars in funding over the next 10 years.

Second, AI is awesome.

It can help make our lives easier and sort of gives us superpowers. Who knows what we can accomplish with the help of machine learning and AI?

And third, AI doesn’t work that well yet.

I still can’t ask my phone or any “smart” device to do much, and we’re far away from personal robot butlers. So what’s next? What’s the future of AI?

The Future of AI

One way to think about the future of AI is to consider milestones AI hasn’t reached yet. Current soccer robots aren’t quite ready to take on human professionals, and Siri still has a lot of trouble understanding exactly what I’m saying.

For every AI system, we can try and list what abilities would take the current technology to the next level. In 2014, for example, the Society of Automotive Engineers attempted to do just that for self-driving cars. They defined five levels of automation.

For each additional level, they expected that the AI controlling the car can do more without human help.

At level 1, cruise control automatically accelerates and decelerates to keep the car at a constant speed, but everything else is on the human driver.

At level 3, the car is basically on its own. It’s driving, monitoring its surroundings, navigating, and so on… but a human driver will need to take over if something goes wrong, like really bad weather or a downed powerline.

And at level 5, the human driver can just sit back, have a smoothie, while the car takes them to work through rush-hour traffic. And obviously, we don’t have cars with the technology to do all this yet.

But these levels are a way to evaluate how far we’ve come, and how far our research still has to go. We can even think about other AIs using “levels of automation.”

Like, for example, maybe we have level 1 AI assistants right now that can set alarms for us, but we still need to double-check their work. But what are levels 2 through 5?

What milestones would need to be achieved for AI to be as good as a human assistant? What would be milestones for computer vision or recommender systems or any of the other topics in this course?

Sometimes it’s useful to think about different kinds of AI on their own as we make progress on each very difficult problem. But sometimes people try to imagine an ultimate AI for all applications: an Artificial General Intelligence or AGI.

To understand why there’s such an emphasis on being “general,” it can be helpful to remember where all this AI stuff first started. For that, let’s go to the example.

Alan Turing was a British mathematician who helped break the German enigma codes during World War II, and helped define the mathematical theory behind computers.

In his paper “Computing Machinery and Intelligence” from 1950, he introduced the now-famous “Turing Test”, or “The Imitation Game.” Turing proposed an adaptation of a guessing game.

In his version, there’s an “interrogator” in one room, and a human and a machine in the other. The interrogator talks to the hidden players and tries to figure out which is a human and which is a machine.

Turing even gave a series of talking points, like: Please write me a sonnet on the subject of the Forth Bridge. Add 34,957 and 70,764. Do you play chess? I have K at K1 and no other pieces. You have only K at K6 and R at R1. It’s your move. What do you play?

The goal of The Imitation Game was to test a machine’s intelligence about any human thing, from math to poetry. We wouldn’t just judge how “real” a robot’s fake human skin looks.

As Turing put it: “We do not wish to penalize the machine for its inability to shine in beauty competitions, nor to penalize a man for losing in a race against an airplane.” This idea suggests a unified goal for AI, an artificial general intelligence.

But over the last 70 years, AI researchers have focused on subfields like computer vision, knowledge representation, economic markets, planning, and so on.

Interdisciplinary Research

And even though we’re not sure if an Artificial General Intelligence is possible, many communities are doing interdisciplinary research, and many AI researchers are taking baby steps to combine specialized subfields.

This involves projects like teaching a robot to understand language, or teaching an AI system that models the stock market to read the news and better understand market fluctuations.

To be clear, most of AI is still science fiction… we’re nowhere near Blade Runner, Her, or any similar movies. Before we get too excited about combining everything we’ve built to achieve AGI, we should remember that we still don’t know how to make specialized AIs for most problems.

Some subfields are making progress more quickly than others and we’re seeing AI systems pop up in lots of places with awesome potential.

To understand how AI might be able to change our lives, AI Professors Yolanda Gil and Bart Selman put together the Computing Research Association’s AI Roadmap for the next 20 years.

They predict AI reducing healthcare costs, personalizing education, accelerating scientific discoveries, helping national defense, and more. Part of the reason they expect so much progress is that more people than ever (including us!) are learning how to build AI systems. And all of these problems have lots of data to train new algorithms.

It used to be hard to collect training data, going to libraries to copy facts and transcribe books. But now, a lot of data is already digital. If you want to know what’s happening on the other side of the planet, you can download newspapers or grab tweets from the Twitter API. Interested in hyperlocal weather prediction?

You can combine free data from the weather service with personal weather stations to help know when to water your plants. And if you feed that data into a robot gardener, you could build a fully-automated weather-knowing plant-growing food-making garden!

Maker communities around the globe are combining data, AI, and cheap hardware to create the future and personalize AI technologies. While imagining an AI/human utopia is exciting, we have to be realistic, too.

In many industries, automation doesn’t only enhance human activities, it can replace humans entirely. Truck, delivery, and tractor drivers are some of the most common jobs in the US as of 2014.

If self-driving vehicles revolutionize transportation in the near future, will all those people lose their jobs?

We can’t know for sure, but Gödel Prize winning Computer Science Professor Moshe Vardi points out that this is already the trend in some industries. For example, U.S. manufacturing output will likely keep rising, but manufacturing jobs have been decreasing a lot. Also, computers use energy, and that means we’re not getting any benefits from AI for free.

Massive amounts of machines running these algorithms can have a substantial carbon footprint. On top of that, as we’ve discussed, you have to be pretty careful when it comes to trusting AI systems because they often end up with all kinds of biases you may not want. So we have to consider the benefits of massive AI deployment with the costs.

In a now-famous story from a few years ago, Target figured out a woman was pregnant based on her shopping history, and they sent her maternity coupons. But she was still in high school, so her family saw the mail, even though she hadn’t told them.

Do we want our data being used like this, and potentially revealing personal details? Or what about the government. Should it be allowed to track people with facial recognition installed on cameras at intersections?

When we provide companies location data from our phones we could help them build better traffic models so we can get to places faster. Cities could improve bus routes, but it also means … someone … is … always … watching you.

AI could also track your friends and family, where you shopped, ate, and who you hung out with. If statistics have shown that people who leave home late at night are more likely to commit a crime… and an AI knows you left (even though it’s just for some late night cookie dough), should it call the police to watch you — just in case?

So, we can go down any number of scary thought experiments. And there’s a lot to consider when it comes to the future of AI. AI is a really new tool and it’s great that so many people have access to it, but that also means there are very few laws or protections about what they can and can’t do.

Innovations in AI have awesome potential to make positive changes, but there are also plenty of risks, especially if the technology advances faster than the average person’s understanding of it.

It’s probably the most accurate to say that the future is… complicated. And the most important thing we can do is be educated and involved in AI as the field changes. Which we’re doing right now!

So when we see a company or government rolling out a new technology, we know what questions to ask: Where did they get their data? Is this even a situation where we want AI to help humans? Is this the right tool to use? What privacy are we giving up for this cool new feature? Is anyone auditing this model? Is this AI really doing what the developers hoped it would?

We’re excited to see what future you decide to build.

If you want to learn more about AI we’ll include more free learning resources.

In the meantime, don’t forget to be awesome!

https://www.ibm.com/topics/artificial-intelligence

https://cloud.google.com/learn/what-is-artificial-intelligence

https://www.ibm.com/blog/category/artificial-intelligence/

https://builtin.com/artificial-intelligence

https://www.investopedia.com/terms/a/artificial-intelligence-ai.asp

https://www.cloudflare.com/learning/ai/what-is-artificial-intelligence/

https://blog.google/technology/ai/

https://www.coursera.org/articles/what-is-artificial-intelligence

https://www.britannica.com/technology/artificial-intelligence

https://openai.com/blog