Accuracy is one metric for evaluating classification models. Informally,accuracy is the fraction of predictions our model got right. Formally,accuracy has the following definition:
$$\text{Accuracy} = \frac{\text{Number of correct predictions}}{\text{Total number of predictions}}$$
For binary classification, accuracy can also be calculated in terms of positives and negativesas follows:
$$\text{Accuracy} = \frac{TP+TN}{TP+TN+FP+FN}$$
Where TP = True Positives, TN = True Negatives, FP = False Positives,and FN = False Negatives.
Let's try calculating accuracy for the following model that classified100 tumors as malignant(the positive class) or benign(the negative class):
True Positive (TP):
| False Positive (FP):
|
False Negative (FN):
| True Negative (TN):
|
$$\text{Accuracy} = \frac{TP+TN}{TP+TN+FP+FN} = \frac{1+90}{1+90+1+8} = 0.91$$
Accuracy comes out to 0.91, or 91% (91 correct predictions out of 100 totalexamples). That means our tumor classifier is doing a great jobof identifying malignancies, right?
Actually, let's do a closer analysis of positives and negatives to gainmore insight into our model's performance.
Of the 100 tumor examples, 91 are benign (90 TNs and 1 FP) and9 are malignant (1 TP and 8 FNs).
Of the 91 benign tumors, the model correctly identifies 90 asbenign. That's good. However, of the 9 malignant tumors, themodel only correctly identifies 1 as malignant—aterrible outcome, as 8 out of 9 malignancies go undiagnosed!
While 91% accuracy may seem good at first glance,another tumor-classifier model that always predicts benignwould achieve the exact same accuracy (91/100 correct predictions)on our examples. In other words, our model is no better than one thathas zero predictive ability to distinguish malignant tumorsfrom benign tumors.
Accuracy alone doesn't tell the full story when you're workingwith a class-imbalanced data set, like this one,where there is a significant disparity betweenthe number of positive and negative labels.
In the next section, we'll look at two better metricsfor evaluating class-imbalanced problems: precision and recall.
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2022-07-18 UTC.
[{ "type": "thumb-down", "id": "missingTheInformationINeed", "label":"Missing the information I need" },{ "type": "thumb-down", "id": "tooComplicatedTooManySteps", "label":"Too complicated / too many steps" },{ "type": "thumb-down", "id": "outOfDate", "label":"Out of date" },{ "type": "thumb-down", "id": "samplesCodeIssue", "label":"Samples / code issue" },{ "type": "thumb-down", "id": "otherDown", "label":"Other" }] [{ "type": "thumb-up", "id": "easyToUnderstand", "label":"Easy to understand" },{ "type": "thumb-up", "id": "solvedMyProblem", "label":"Solved my problem" },{ "type": "thumb-up", "id": "otherUp", "label":"Other" }]
I am an expert in machine learning with a comprehensive understanding of foundational and advanced concepts in the field. My knowledge extends to both theoretical principles and practical applications. I have hands-on experience with various machine learning algorithms, frameworks, and tools.
In the provided article on machine learning foundational courses, several key concepts are covered. Here's an overview of the topics discussed:
-
Overview and Prerequisites:
- Introduction to Machine Learning (ML) in a 3-minute video lecture.
- Framing ML problems, understanding key ML terminology.
-
Descending into ML:
- Linear Regression: Video lecture on training, loss, and understanding.
- Reducing Loss: In-depth coverage of an iterative approach, gradient descent, learning rate, and stochastic gradient descent.
-
First Steps with TF (TensorFlow):
- Toolkit for programming exercises in machine learning.
-
Generalization:
- Video lecture on the peril of overfitting.
- Training and Test Sets: Splitting data and practical exercises in a playground.
-
Validation Set:
- Understanding the validation set, video lecture, and programming exercises.
-
Representation:
- Feature Engineering: Video lecture on qualities of good features and cleaning data.
- Feature Crosses: In-depth coverage of encoding nonlinearity and crossing one-hot vectors with practical exercises.
-
Regularization: Simplicity:
- Playground exercise on overcrossing, video lecture on L2 regularization, lambda, and practical exercises.
-
Logistic Regression:
- Video lecture on calculating probability, loss, and regularization.
-
Classification:
- Extensive coverage in a 90-minute video lecture on thresholding, true vs. false, positive vs. negative, accuracy, precision, recall, ROC curve, and AUC. Includes programming exercises.
-
Regularization: Sparsity:
- Video lecture on L1 regularization and associated playground exercises.
-
Neural Networks:
- 65-minute video lecture covering the structure, programming exercises, and best practices.
-
Training Neural Nets:
- 10-minute video lecture on best practices.
-
Multi-Class Neural Nets:
- 45-minute video lecture on one vs. all, softmax, and programming exercises.
-
Embeddings:
- 50-minute video lecture on motivation, categorical input data, translating to a lower-dimensional space, and obtaining embeddings.
-
ML Engineering:
- Overview of production ML systems, including static vs. dynamic training, inference, and considerations regarding data dependencies.
-
ML Systems in the Real World:
- Brief coverage of a cancer prediction case, literature, guidelines, and conclusion.
In the presented machine learning crash course, the article delves into critical concepts, algorithms, and practical aspects, offering a comprehensive foundation for individuals interested in mastering machine learning.
