Applications of Existing Convolutional Neural Networks to Deepfake Detection

Abstract

As technology becomes increasingly advanced, AI-generated media known as Deepfakes have raised concerns in areas like cybersecurity, misinformation, and privacy. Since Convolutional Neural Networks (CNNs) have proved effective in other fields like image classification tasks, the objective of this research is to apply these CNNs to the growing problem of detecting Deepfake images. Even if they are not initially designed for this task, this study will determine which of these architectures perform best for detecting Deepfake images. A dataset of real and Deepfake images was constructed and several CNN architectures including pre-existing and pre-trained models like VGG 16, VGG 19, DenseNet121, and ResNet50, were tested. A previously developed architecture for age classification was also modified to fit this task and all were compared to a custom architecture designed. The custom model achieved the highest validation accuracy of 97%, though it lacks generalizability and had a validation accuracy of 81.1% when tested on another dataset. Pre-trained transfer learning models underperformed even with fine-tuning, with ResNet50 yielding a maximum accuracy of 94%. The highest performing modified age-classification model had a validation accuracy of 92.36%. Increasing the number of convolutions in the modified age-classification model roughly improved accuracy. Overall, this study found that though pre-trained image classifiers can be applied to Deepfake detection, they were not designed for this task. Evidence of overfitting and differences between saliency maps for different models show the importance of developing large, generalizable datasets specifically for Deepfake detection. Custom CNNs and adapted models can be developed and run quickly with high performance on one dataset, but the lack of representative training data and State Of The Art (SOTA) methods leave models at risk of overfitting and limited applications of their results.

Keywords: Machine Learning, Computer Science, Deepfakes, Artificial Intelligence,  Convolutional Neural Networks, Image Classification, AI-Generated Media, Model Performance Evaluation

Introduction

Motivation

Deepfakes, or digitally manipulated media, are becoming more common and sophisticated, creating challenges in areas like cybersecurity, misinformation, and privacy. Deepfakes have significant implications for cybersecurity, as they can be used to impersonate individuals in phishing attacks or bypass biometric authentication systems. In the realm of misinformation, Deepfakes can spread false narratives or discredit legitimate sources, causing widespread confusion and distrust. From a privacy standpoint, the ability to create hyper-realistic synthetic media poses a risk to personal reputations and can be used for blackmail, harassment, or identity theft. Although AI detection exists in current research, it is often inaccurate when faced with new, sophisticated deep fakes, limiting their usefulness¹. By enhancing detection systems, this research aims to fill this gap, mitigating the threats of AI generated media and protecting individuals, institutions, and the public.

This research focuses on answering the question: Which CNN architectures are best to detect deep fake media effectively? To achieve this, this study aims to train, test, and compare the performance of various CNN architectures with different features to determine which architectures are better suited to this task. The CNN architectures tested in this study are most, if not all, key performers in image recognition, but have yet to be applied to Deepfake detection. The scope of this research is limited to binary classification of images as either real or fake, since CNN architectures are trained on labeled data. Deepfake detection for images is focused on within this study, so conclusions or data drawn from this study may not be applicable to other forms of media such as videos, time lapses, etc.

Background

The industry standard tools for detecting Deepfake media consist of various deep learning techniques, most notably CNNs and Generative Adversarial Networks (GANs). CNNs are a type of artificial neural network primarily used for image recognition tasks due to their ability to detect minute details through kernel or filter optimization. One of the most used CNNs for identifying AI-generated images is XceptionNet, a CNN that excels in detecting pixel anomalies often indicative of Deepfakes.

Patel et. al. conducted a case study on Deepfake Generation and Detection in 2023², and they report three approaches to Image/ Video Detection: Physical/Physiological based approaches, Signal Level Feature based approaches, and Data Driven models. Data Driven models relate the most to this proposed study, and Patel et. al. describes common CNNs that have performed well for problems like this. Rather than focusing on specific features of images or faces (for face forgery problems), data driven models like CNNs and Recurrent Neural Networks (RNNs) are trained over a general dataset of AI and real images/videos. One of the models they discuss alongside other industry standard models like Xception and DenseNet is MesoNet, which uses Inception as the foundation of its architecture, allowing it to detect compressed images and Deepfakes on social media platforms, where not all images are high resolution.

Additionally, Ashok V and Dr. Preetha Theresa Joy applied XceptionNet to Deepfake detection in 2023³, where they used XceptionNet to categorize images and video frames into real or fake. They compared the results to other SOTA models like VGG-16, ResNet-152, and Inception V3 and found that XceptionNet performed better than the other models. However, using CNNs for this problem risks overfitting the data, essentially training the model to memorize rather than predict. The model becomes used to seeing the same data but is not actually learning anything new during the epochs. Approaches including CNNs often struggle to generalize their results to other datasets because of this, meaning that their model performs well only on their training datasets, leading to underperformance on new data and an overall not robust conclusion. Ashok V and Dr. Joy combated this by taking their data from FaceForensics++ Database, a forensics dataset containing 1000 videos that have been altered using four different face forging techniques: Face2Face, Deepfakes, FaceSwap, and NeuralTextures. This helps minimize confounding variables due to only one technique being used to encode the real videos into Deepfakes. They train their model using 5000 real images and 5000 Deepfakes and then test their data on 794 video frames taken from FaceForensics++.

This study uses a large dataset to combat overfitting and increase the generalizability of the custom architecture CNN. It uses 60,000 synthetically generated images and 60,000 real (Collected from Krizhevsky & Hinton’s CIFAR-10 dataset) images to build a dataset for the models; a large dataset like this helps expose the model to many different images and helps create a more complex model through a thorough training process. Furthermore, the personal CNN model was tested on data from Manjil Karki’s Deepfake and Real Images dataset to identify how it performs on unseen data and test its generalizability.

Furthermore, A GAN is another widely used tool that creates two neural networks: a generator network and a discriminator network. The two are pitted against each other such that the generator outputs progressively convincing fake images while the discriminator attempts to differentiate the AI-generated images from the real ones. This technique allows GANs to develop high accuracies and better identify subtle inconsistencies and patterns within data⁴. GANs and other deep learning methods like CNNs are staples within the field of AI-generated media detection as they provide a robust system for detecting and recognizing differences between real and fake images.     

Though GANs are commonly used for Deepfake generation, Galamo Monkam, Weifeng Xu, and Jie Yan introduced G-Job GAN in 2023⁵, a GAN-based machine learning model that outperformed other industry standard GAN models. Their proposed architecture consists of two CNNs, a discriminator and a generator that work together to progressively create and detect Deepfake images. They pulled images from CelebA, a large-scale dataset containing over 200,000 celebrity images with 40 different facial attributes per image, using 162,770 images for training; 19,867 images for validation; and 19,962 images for testing. They measured their results using F1 score and additionally noted that their accuracy positively improved as their dataset size increased.

Though GANs bring unprecedented accuracy and immense potential to the field of Deepfake detection, they are notoriously hard to train, and the generators can easily become biased if not trained with a large and diverse dataset. Finding datasets like this is a key limitation of GANs since current real world Deepfake detection datasets often lack variance between different demographics (ethnicity, gender, image quality, age, etc) and manipulation techniques (lip syncing, expression transfer, face swapping, etc). Deepfake technology evolves quickly, which means that datasets can quickly become outdated as new technology surfaces. Privacy concerns with people’s real facial data is also a concern when trying to compile real images for these datasets, which is also why some public datasets are unable to capture the full range of images.

This was combated by utilizing transfer learning for the industry standard models incorporated into the study. These models will be imported using pre-trained weights that have already been determined due to exposure to different datasets, which will help increase the generalizability of the results and reduce the likelihood of overfitting. Additionally, the personal CNN model is tested on Manjil Karki’s Deepfake and Real Images Dataset⁶ to test generalizability of the solution and expose the model to different kinds of images. Use of images from different sources helps expose the model to a diverse set of images and helps determine the actual strength of the model.

Methodology Overview

This problem falls under supervised learning and is a classification task. In supervised learning, the model is trained on labeled data, where we know whether each image is real or fake. Since the output is a prediction of whether an image is real or fake, this is a binary classification problem. The labels for the data are 0 for fake images and 1 for real images. A mix of real images and deep fake images was gathered to build the dataset. The data used is vision data—specifically, images that were either generated by artificial intelligence (fake) or captured in the real world (real). After gathering the images, they were converted into numerical values that can be used by the CNN model. The model’s output is a label, either 0 or 1, depending on whether it predicts the image is fake or real.

After data preprocessing, several accepted image-recognition models with differing architectural priorities and goals such as DenseNet121, ResNet50, VGG-16, and VGG-19 were tested. For comparison, an age classifier was modified for this problem and a personalized CNN architecture was developed. Post testing and fine-tuning, these models were evaluated based on validation accuracy, validation loss, validation precision, validation recall, and validation f1 score. The generalizability of the personal CNN architecture was further tested on the Manjil Karki Deepfake and Real Images dataset from Kaggle⁶. By experimenting with different CNN architectures, the goal is to identify which one produces the most accurate predictions in order to understand which types of architectures in general are best applied to this situation. After all the models have been evaluated, saliency maps were visualized for true real predictions, true fake predictions, false real predictions, and false fake predictions for the ResNet50 model, 4 Convolution [512, 512, 2] model from the Age Classifier, Personal CNN on CIFAKE Dataset, and Personal CNN on the Manjil Karki Dataset. This research is significant because finding the best CNN architecture can improve deep fake detection systems and help address the growing issue of media manipulation.

Results

The transfer learning models had the lowest validation accuracy rates overall, as shown in the table below. ResNet50 overall had the highest test validation accuracy of 94 percent but was lower than the experimental model. The modified Age classifier model with the highest accuracy was the 4 Convolution [512, 512, 2] model with an accuracy of 92.36 percent.

These models were evaluated by comparing their validation losses, accuracies, precision, recall, and f1 score. Loss is an evaluation metric that works by comparing the true labels to the predicted outputs, using a specific loss function such as cross-entropy loss or mean squared error. This is helpful to visualize and compare how inaccurate different models are compared to each other. This study uses cross-entropy loss due to its common use in classification problems, and it works by comparing the true probability distribution of the data to the predicted probability distribution. Cross-entropy loss also helps to identify areas where possible over or underfitting occurred, which is essential in evaluating the overall success of the image classification models. Equation 1 shows the mathematical formula for calculating Cross-entropy loss.

(1)  

Validation test accuracy, on the other hand, measures the percentage of correct labels over the percentage of total predictions. To make it a percentage, simply multiply by 100. Equation 2 shows the formula for accuracy.

(2)  

Including precision, recall, and F1 score helps identify the predicting habits of the model by assessing the likelihood of correctly identifying Deepfake images compared to the real images. This in addition to the accuracy and loss results creates a thorough group of evaluation metrics for the study. Below, Equation 3, Equation 4, and Equation 5 show the mathematical formulas used to calculate Precision⁷, Recall⁸, and F1 Score⁹, respectively.

(3)  

(4)  

(5)  

Below is a table comparing the successes of the Transfer Learning Models, personal experimental model, and the raw architectures of VGG-16 and VGG-19 (Note the following metrics are all after fine-tuning).

Model Type	Validation Loss	Validation Accuracy	Validation Precision	Validation Recall	Validation F1 Score
VGG-16 Architecture	0.6931	0.5025	0.5025	0.5025	0.6689
VGG-19 Architecture	0.6931	0.5025	0.5025	0.5025	0.6689
VGG Transfer Learning	0.576	0.8097	0.8097	0.8097	0.8221, 0.7955
DenseNet121 Transfer Learning	0.4374	0.8378	0.8378	0.8378	0.8474, 0.8268
ResNet50 Transfer Learning	0.17463	0.9355	0.9355	0.9355	0.9335, 0.9374
Personal CNN	0.1314	0.9696	0.9696	0.9696	0.9697, .9695
Personal CNN on Manjil Karki’s Dataset	0.9635	0.8112	0.8215	0.8112	0.8103

For the Modified Age Classifier, 12 different models were generated. For each number of convolutional layers, the number of nodes vary. This pattern was repeated for convolutional layers of 2,3,4 and for node amounts of [2], [512,2], [512,512,2], and [1024,256,2].

Model Name	Validation Accuracy	Validation Loss	Validation Precision	Validation Recall	Validation F1 Score
2 convolution [2]	0.8617500067	0.3390810788	0.8617500067	0.8617500067	0.863423 0.86003536
2 convolution [512, 2]	0.8851666451	0.2985413671	0.8851666451	0.8851666451	0.89142764 0.87813926
2 convolution [512, 512, 2]	0.9014166594	0.2953568101	0.9014166594	0.9014166594	0.9040628 0.8986202
2 convolution [1024, 512, 256, 2]	0.8991666436	0.3799754679	0.8991666436	0.8991666436	0.90309143 0.89491045
3 convolution [2]	0.9109166861	0.264406532	0.9109166861	0.9109166861	0.9118205 0.90999407
3 convolution [512, 2]	0.9051666856	0.2593968809	0.9051666856	0.9051666856	0.91045004 0.89922065
3 convolution [512, 512, 2]	0.9159166813	0.249979496	0.9159166813	0.9159166813	0.9135167 0.91818696
3 convolution [1024, 512, 256, 2]	0.917750001	0.4092084467	0.917750001	0.917750001	0.91954017 0.91587824
4 convolution [2]	0.9123333097	0.2316058725	0.9123333097	0.9123333097	0.9096375 0.9148729
4 convolution [512, 2]	0.9188333154	0.2288808525	0.9188333154	0.9188333154	0.9226492 0.91462123
4 convolution [512, 512, 2]	0.923666656	0.2263379991	0.923666656	0.923666656	0.92419726 0.9231285
4 convolution [1024, 512, 256, 2]	0.9219166636	0.2523550689	0.9219166636	0.9219166636	0.92404956 0.9196604

The model with the highest accuracy for the age classifier was the 4 convolution [512, 512, 2], while the model with the lowest accuracy for the age classifier was 2 convolution [2]. The following confusion matrices show highest and lowest performing models. A label of 0 means fake, and a label of 1 means real.

Methodology

Data Preprocessing

This project uses CIFAKE: Real and AI-Generated Synthetic Images¹⁰ taken from Kaggle, which contains 60,000 synthetically generated images (generated the equivalent of CIFAR-10 with Stable Diffusion version 1.4) and 60,000 real (Collected from Krizhevsky & Hinton’s CIFAR-10 dataset) images. At the beginning of the project, the images are converted to numerical values of 0 and 1, where 0 represents an AI generated image and 1 represents a real image. Data was split using Sci-kit Learn’s train test split using an 80:20 split. The data is preprocessed by looping through the image paths and separating them into four lists: real_test, real_train, fake_test, fake_train. The labels are converted into 0 and 1 and set equal to the y. The X became all four of the lists added together and the y is the labelled 0 and 1 numbers. The test set is later split in half to create a separate validation set.

To preprocesses specifically for CNN testing, the values are converted back from binary into an image of 32×32 such that the sizes of X_train, X_test, X_validation, y_train, y_test, y_validation, respectively, are as follows: (96000, 32, 32, 3), (24000, 32, 32, 3), (12000, 32, 32, 3), (96000, 2), (24000, 2), (12000,2). The goal of this is to better organize our data into different lists to be used later when experimenting with scikit-learn models and visualizing the data.

To better understand the data, the different RGB color distributions, as well as the average brightness for pixels, are visualized for the fake images and the real images separately.

To visualize the RGB color intensities, the red value in each of the images is isolated and used in a row major order to loop through each individual pixel in the image. For each pixel, the red, blue, and green values were summed individually, and the dimensions of the image was used to return the average color value of that specific image color. Then, for each color per fake and real images, a total of 6 lists was created and a histogram was visualized for each. Shown in Figure 5 are the average red, blue, and green color intensity values for the AI-Generated images, and in Figure 6 for the Human images (total dataset, not relevant to train or test). We observe that though the distributions of red and green color intensity are approximately normal for both human and ai images, the distribution of blue color intensity appears skewed right, which may indicate that most images do not include a lot of blue color in them. However, there are no significant differences in the distributions between human and AI images.

To visualize the brightness values, the images are converted into black and white values and then using the same row major approach to loop through each pixel in the image. To find the average brightness per image, each grayscale image was added to a total sum for the image and divided by the dimensions to find the average value. Then by looping through the list of real and fake images, this code was repeated for each image in the study and saved in two lists: one for average brightnesses of all real images, and one for all fake images.

Both are plotted on a histogram to make comparison easier and both are shown within Figure 7 below. Both graphs show an approximately normal distribution, with most images having a color intensity not far on either end of the brightness spectrum. This indicates that there is not a significant skew for color intensity between AI images or human images. However, we observe that the average color intensity for human images has a slightly greater spread than that of AI images, which may be due to colors and shades found in real life that AI images do not replicate.

To solve the research problem, this work began by implementing a few scikit- learn methods (KNN Model, Random Forest Model, Linear Regression Model, SVC Model) using both the brightness training data described in the Dataset Section, and the full image data. After experimenting with the data to better understand how in general it responds to model classification, the study progressed to experimentation with the neural network MLP classifier and then began testing various CNN architectures on the data.

Introduction to Models Used

The VGG-16 and 19 are two pre-existing CNN architectures commonly used for image recognition within this field. The architecture is known to be simple and consists of 16 and 19 layers respectively, with small 3×3 filters. They are known for their simplicity and are the benchmark in image classification, often also being implemented in image style transfer problems¹¹. These models are manually implemented by building their layers in Tensorflow and Keras without pre-trained weights.

The idea of transfer learning is also applied by importing the VGG model from Keras Applications that includes pre-determined weights based on past training. Transfer learning uses models that have already been trained on large datasets and adapts them to new tasks, meaning that this model has already been configured to produce its best accuracy for image recognition tasks, which is what it is commonly used for in this field¹². For the VGG-16, VGG-19, and transfer learning VGG models, the models are later fine-tuned by unfreezing the layers for all the models from layer 14 onward, which includes the last two convolutional blocks (each block includes one max pooling layer and three convolutional layers) and connected dense layers. This is so that the earlier layers that are useful for more general observations were kept frozen, while the later layers that are involved in leading abstraction and pattern identification are retrained to better distinguish deepfakes from real images.     

DenseNet121 is another architecture looked at, with a unique structure where each layer is connected to every other layer, instead of stacking layers one after another. DenseNet121 ensures each layer receives input from all previous layers which makes the flow of information more efficient¹³. This model is imported from Tensorflow Keras applications to obtain its predetermined weights from prior training on image recognition tasks. This model is commonly used for text generation, image generation, and image feature extractions. For DenseNet121’s finetuning, it was unfrozen from layer 299 out of 429, keeping the final Dense Block and classifier layers, which are responsible for the decision making and minor pattern identification, two crucial aspects of Deepfake detection. Retraining the model on the later dense blocks from the architecture helped reduce the risk of overfitting from unnecessarily rerunning earlier layers and helps improve performance by letting the model specialize its predictions.

Additionally, this study uses the ResNet architecture, which is unique in the fact that it prioritizes model efficiency by using residual blocks to skip unnecessary layers to reduce computation time and make it easier for the model to train itself. This clever approach allows ResNet to have extremely deep networks while avoiding performance degradation that often comes with an increase in convoluted layers. Due to its often deep, complicated nature, this model is commonly used for deep computer learning tasks like image segmentation and object detection¹⁴. Within the field it is most applied to image recognition tasks, and in this study is used transfer learning to import it from the Tensorflow Keras application library. For the ResNet50 transfer learning, the first 7 blocks are preserved, which primarily serve as general feature analysis, and unfroze the layers starting at Block 8, which includes the final convolutional layers, global average pooling, and fully connected output layer. The model is retrained with these upper layers so that it could adapt and detect subtle patterns within deepfake images that were not determined during original training.

Finally, to act as a comparison point, this study looks at a personalized CNN with strategically composed convolutional and pooling layers to see if it could outperform the standard models used for image classification. This methodology helps determine whether it is necessary to develop specific and targeted models or if other models perform well too.

This model takes images of size 32×32 with three RGB color channels with 128 filters. The first block contains an alternating pattern of a convolutional layer, a Leaky ReLU activation function, and then runs a Batch Normalization. A Leaky ReLU activation function is a variant of a ReLU that helps the network learn complex patterns by reducing the likelihood of the network becoming inactive while training. Batch Normalization helps to normalize the outputs of the previous layers to help stabilize the model¹⁵. This three-layer combination is repeated twice more. The first block ends with a pooling layer that reduces the dimensions by half, so it becomes 16×16 and a dropout layer that drops 20% of the neurons to prevent overtraining.

The second block has 256 filters of size (3,3) but follows the same repetition pattern of the first block. The third block follows the same pattern but with 512 filters and only two repetitions of the three-layer combination consisting of a convolutional layer, LeakyReLU layer, and batch normalization layer.

Finally, it finishes by flattening the output into a 1D vector in preparation for the following dense layers that ultimately output the probabilities for each class, used for binary classification. The Adam optimizer, an adaptive learning rate method commonly used for deep networks¹⁶, is utilized here. It was trained with a batch size of 64 over 10 epochs.  See the figure below for a visual representation of the architecture.

While performing transfer learning, the model is unfrozen starting at layer 23 of 40 so that the model layers of the final block can be retrained (one convolutional layer, one LeakyReLU layer, Batch Normalization, Convolutional, LeakyReLU, Batch Normalization, Max pooling, and a dropout layer) and the upper most dense layer block. Essentially, the low to mid-level features are kept the same while allowing the higher up layers to be re-trained based on the dataset, which creates better performance since the model is able to identify new patterns from the data.

Figure 8 below visualizes the architecture and labels the different layers of the model.

The section below gives an in depth explanation and walk through of M. Fatih Aydogdu and M. Fatih Demirci’s Age Classifier CNN¹⁷ which uses an extremely unique technique to determine a person’s age from an image. Their methodology is then applied to this research problem and replicated.

Age Classification CNN Architecture

This methodology was taken from research conducted by M. Fatih Aydogdu and M. Fatih Demirci, where they used a unique approach to classify age from images using a convolutional neural network. In their paper, they created multiple CNN stages that consist of increasing layers and varying node amounts. Then they combined each layer with each other layer to create an exponential number of models to find the most accurate one. This approach was successful because they ensured that the ending format of the images was common amongst each CNN stage.

The age classification model is included due to its unique architecture and methodology. The systematic combination of multiple CNN models is an interesting way to increase model performance since it allows for experimentation with different amounts of layers and nodes at once. Using a multi-stage approach, helps build a more robust model with varying levels of complexity which ultimately allows for flexible and progressive analysis of fine image details, which is crucial for technical problems like age classification and Deepfake detection.

A 2-layer, 3 layer, and four-layer approach is used to replicate their approach. Next, matrix math is performed to find a combination of layers and pooling that would result in the final image size for all three CNN stages and create the architecture to be the same. However, the approach needed to be adjusted to fit the image sizes for the CIFAKE dataset that differ from theirs.

After building and saving each individual CNN stage, the magnitude of nodes to be used for each structure was determined; Four different groups were selected. These four nodes will be used on each CNN stage, creating 12 different models. After creating a for loop and looping through the number of nodes, each model was trained and saved in a list of all 12 of them to make training easier later. A for loop was used to cycle through the list of models and their names. It trained them, saved their validation curves as a file, and saved their accuracies in a table as shown for clarity. Changing the number of node groups or CNN groups would have led to different amounts of models, but to not deviate from the original study, 4 node groups and 4 CNN groups are used. This created 12 models of varying convolutional layers and amounts of nodes.

Evaluation Metrics and Modes of Comparison

For each of these models, the preprocessed dataset of real and fake images is trained using each model for ten epochs. The 12 models generated in the Age Classifier Replica were individually trained and the validation accuracies, loss, precision, recall, and f1 score were saved in a table shown in the results section and confusion matrices were generated for the highest and lowest performing models. The confusion matrices assisted in evaluating model precision, and the accuracies helped generalize the overall performance of all the models.

Analyzing the accuracy of the models helped provide a general sense of model performance, but can be misleading with imbalanced datasets, which is why other metrics are included as well. Validation loss was useful because it quantified the error between the predicted and actual outputs, which helped identify potential areas of over or underfitting by the models.

Precision helps quantify the likelihood of the model correctly identifying a Deepfake image as fake, while Recall similarly indicates how many fake images the models correctly identified. F1 Score is the harmonic mean of precision and recall, which balances false positives and false negatives into a robust score that gives a broader idea of the exactness of the model. Though these three scores are similar, using all of them provides a nuanced understanding of how well the model predicted fake or real images, and provides a more complete and reliable evaluation of model performance in addition to the validation accuracy and loss.

Error Analysis

This section visualizes, analyzes, and compares the saliency maps for ResNet50 (Highest performing SOTA Image Classifier), 4 Convolution [512, 512, 2] (Highest Performing Model from Modified Age Classifier), Personal CNN on CIFAKE Dataset (Original Dataset), and Personal CNN on Manjil Karki’s Deepfake and Real Images Dataset (Test Dataset). These models are used for visualizing the saliency maps because they performed the best in each category: The ResNet50 model had the highest performance across all metrics for the pretrained image classifier, the 4 Convolution [512, 512, 2] model had the highest performance out of 12 of the models from the Modified Age classifier, and the Personal CNN had the highest performance across all models on the CIFAKE Dataset. To make the saliency maps, images are randomly selected based on whether they had a high confidence score or not. Confidence score refers to the level of confidence that the model had when making its predictions for a certain image.

            From the saliency map visualizations, a clear pattern emerges within the focuses of the four different models. The personal CNN, which exhibits a higher accuracy on the CIFAKE dataset compared to the ResNet50 and 4 Convolution [512, 512, 2] models, focuses on the central part of the image, typically around key features of the image such as human faces and animal faces. The size of the area of focus is also smaller and contains brighter colored pixels, indicating that the center of the images was extremely crucial to the model’s predictions. This is good because it means the model is efficient but can also lead to overfitting on limited patterns of images specifically in the CIFAKE dataset. In comparison, the ResNet50 and 4 Convolution [512, 512, 2] models, while also sometimes focusing on the center of the image, typically have a wider area that the model considers in its predictions with less bright pixels. This indicates a less precise and specific area influencing the model predictions, which could have contributed to their lower performance.

 ResNet50 in particular, shows saliency maps in Figure 13 that do not exhibit a circular area of focus; For all visualizations displayed, the area of focus by the model is scattered around the image compared to the other models that are clearly circular in the center of the image. The 4 Convolution [512, 512, 2] model also does not display a circular locus of focus but instead seems to consider the whole image. This is a clear difference from the personal CNN, which exhibits a very small area of focus in the form of a circle usually in the center of the image shown.

Also, it can be observed that the model had more difficulty with images where the center object or person was a similar color to the background. In Figure 13 with the personal CNN on the new dataset, the two images that were misclassified showed people in the dark so that there was a less distinct difference between the people and the background. In comparison, the two correct images have a clear difference between the people and the background since they are taken in better lighting with starker lines.

The visualizations for the ResNet50 Model, 4 Convolution [512, 512, 2] model, Personal CNN on CIFAKE dataset, and Personal CNN on Manjil Karki Dataset are shown below. There are four subplots per visualization. Each subplot contains the actual image on the left side by side with the saliency map overlay to the right. All images used for saliency map analysis have high confidence levels (close to or equal to 1.000), which indicates that the model was completely sure about their prediction. This is a helpful metric to consider while analyzing the process of the different models and the role that plays in its overall performance.

Overfitting Analysis

     Detailed analysis of the training and validation metrics for the pre-trained image classifiers as well as the custom CNNs show that alongside the potential for recognizing underlying patterns in images lies the risk of memorizing instead of analyzing the data.

Comparison of the training accuracy with the validation accuracy after fine tuning for the ResNet50 model show that though that training accuracy continued to increase over the 10 epochs, its validation accuracy fluctuates between 94 and 93 percent accuracy and doesn’t get stronger. This is a strong indicator of potential overfitting, where the model is memorizing the training data but is not increasing its performance on unseen validation data. This could occur since the image classifiers were imported with pre-existing weights from prior training on datasets used for image classification, not Deepfake detection. Findings where the model’s validation accuracy fluctuates between two values while the training accuracy continues to increase are also consistent with the ResNet50 model results for before fine tuning, and the Personal CNN post fine tuning on the original dataset (CIFAKE dataset). The values for each evaluation metric for both training and validation data before and after fine tuning are shown in Figure 17.

Conclusion

For the CIFAKE dataset, the pretrained image classifiers were outperformed by the personal CNN architecture. The best performing model out of them was the RestNet50 Model, with a validation accuracy of 94 percent. This was surprising considering the model had an accuracy of about 97 percent. The age classifier model had accuracies that approximately increased with the number of nodes and number of convolutional layers. The highest accuracy for the age classifier replication was for the 4 convolution [512, 512, 2] model with a validation accuracy of 92.36 percent. However, testing the personal CNN on Manjil Karki’s Deepfake and Real Images dataset resulted in a validation accuracy of about 81 percent, a significant drop from its performance on the CIFAKE dataset.

While transfer learning and fine tuning are expected to improve the performance of the models, the results showed that on certain datasets, they can be outperformed by raw CNN architectures without pre-existing weights. This can also be since the weights for the pre-trained image classifiers are determined for image classification, an entirely separate problem than Deepfake detection. 

These results contribute to the idea that although pre-trained image classifiers can be applied for Deepfake detection, the lack of specific datasets for this task and lack of SOTA methods for benchmarking creates models prone to overfitting and limited generalizability. Creating the personal CNN was easier and faster than the industry standard models and even performed better on the CIFAKE dataset but underperformed on Manjil Karki’s Deepfake and Real Images Dataset. This is due to potential overfitting with the training dataset, shown by the increase in training validation over 10 epochs while the validation accuracy did not change and often decreased. This calls into question the diversity and strength of current datasets used for Deepfake detection.

However, one limitation of this study is the use of one dataset to train the transfer learning models and the modified age classifier. Although the dataset contains many images, they have all been taken from the same AI generated platform, which calls into question if there is a potential bias skewing the data. All the fake images were generated the equivalent of CIFAR-10 with Stable Diffusion version 1.4, and all the real images were collected from Krizhevsky & Hinton’s CIFAR-10 dataset. This may have caused the data to not be representative of all images. Furthermore, the personal CNN was tested on Manjil Karki’s Deepfake and Real images to determine its generalizability and found a significant drop in its performance from a validation accuracy of about 97 percent to a validation accuracy of 81 percent, indicating that the model’s performance is not as strong as previously thought. Additionally, this study was limited to only a few different architectures for testing, and did not explore further beyond the transfer learning models (VGG 16, VGG 19, DenseNet121, ResNet50) and the Age Classifier Model. To stay accurate to the original age classifier model, the Age Classifier methodology did not stray too far from their original design, but did find that the accuracy kept increasing.

Although there are existing datasets standardly used for Deepfake detection tasks, this study highlights the importance of developing evolved datasets that account for the latest changes to generative AI. These newer datasets, alongside clearer SOTA models specifically for this task, will allow for clear benchmarking standards to improve both training and evaluation. Future research should focus on developing more specialized CNN architectures and even experiment with the modified age classifier to see if it will continue to follow the trend of increasing validation accuracy with increasing number of nodes and CNN groups. Further research should test the impacts of changing nodes, CNN groups, and CNN layers.

As deep fake technology continues to evolve and become more undetectable, it becomes clearer that specific models designed to detect fake images are becoming more and more necessary. Though this study works to search for applications of existing models to this new problem, it also encourages the development of more specified models and more substantial datasets to mitigate this problem. Detecting deepfakes is not just a technical challenge but a necessary step in safeguarding digital content and safety.

Appendix

Epoch Number:	Validation Loss	Validation Accuracy	Validation Precision	Validation Recall	Validation F1 Score
1	.8218	.7968	.8173	.7968	.7944
2	.6918	.8140	.8140	.8140	.8140
3	.7282	0.8088	.8144	.8088	.8084
4	.7506	.8108	.8155	.8108	.8105
5	.7647	.8044	.8074	.8044	.8043
6	.7409	.8164	.8165	.8164	.8163
7	.7355	.8092	.8098	.8092	.8092
8	.7369	.8232	.8240	.8232	.8232
9	.7180	.8112	.8113	.8112	.8112
10	.9635	.8112	.8215	.8112	.8103

Epoch	Model	Training Accuracy	Training F1	Training Loss	Training Precision	Training Recall	Validation Accuracy	Validation F1	Validation Loss	Validation Precision	Validation Recall
1	2 convolution [2]	0.7889896035	0.78474957 0.79306567	0.5307711363	0.7889896035	0.7889896035	0.7846666574	0.7479515 0.81204534	0.4688479304	0.7846666574	0.7846666574
2	2 convolution [2]	0.8362187743	0.8331547 0.8391723	0.3739428222	0.8362187743	0.8362187743	0.8519999981	0.85810155 0.84535	0.3565165699	0.8519999981	0.8519999981
3	2 convolution [2]	0.847874999	0.84482974 0.8508029	0.3544433117	0.847874999	0.847874999	0.843083322	0.8356463 0.84987634	0.3576130271	0.843083322	0.843083322
4	2 convolution [2]	0.8554062247	0.8528417 0.85788286	0.3362457752	0.8554062247	0.8554062247	0.8676666617	0.8647359 0.8704731	0.3108260036	0.8676666617	0.8676666617
5	2 convolution [2]	0.8598020673	0.8575299 0.8620028	0.3276377618	0.8598020673	0.8598020673	0.8667500019	0.861041 0.87200826	0.3160246313	0.8667500019	0.8667500019
6	2 convolution [2]	0.8661041856	0.8638174 0.8683153	0.3167644739	0.8661041856	0.8661041856	0.8571666479	0.8485597 0.8648478	0.3294017017	0.8571666479	0.8571666479
7	2 convolution [2]	0.8686979413	0.8665841 0.8707458	0.3119476438	0.8686979413	0.8686979413	0.8561666608	0.8573317 0.85498226	0.3379768729	0.8561666608	0.8561666608
8	2 convolution [2]	0.8740833402	0.8719789 0.8761196	0.3027942777	0.8740833402	0.8740833402	0.8629166484	0.8594617 0.86620575	0.3272535503	0.8629166484	0.8629166484
9	2 convolution [2]	0.8739166856	0.87192607 0.8758462	0.299333185	0.8739166856	0.8739166856	0.8733333349	0.86952776 0.876923	0.3083691895	0.8733333349	0.8733333349
10	2 convolution [2]	0.8761146069	0.8741973 0.8779742	0.2962694764	0.8761146069	0.8761146069	0.8617500067	0.863423 0.86003536	0.3390810788	0.8617500067	0.8617500067
1	2 convolution [512, 2]	0.8774999976	0.8767941 0.8781978	0.2949949503	0.8774999976	0.8774999976	0.874333322	0.87466747 0.8739972	0.299644351	0.874333322	0.874333322
2	2 convolution [512, 2]	0.8908125162	0.8901396 0.89147717	0.2657128572	0.8908125162	0.8908125162	0.8879166842	0.8847965 0.8908722	0.2731055617	0.8879166842	0.8879166842
3	2 convolution [512, 2]	0.8983437419	0.89782536 0.89885676	0.2496741712	0.8983437419	0.8983437419	0.8896666765	0.8896114 0.8897217	0.2631596923	0.8896666765	0.8896666765
4	2 convolution [512, 2]	0.9018645883	0.90132076 0.90240234	0.242579326	0.9018645883	0.9018645883	0.8974166512	0.89723676 0.8975959	0.2541601062	0.8974166512	0.8974166512
5	2 convolution [512, 2]	0.9052916765	0.90473187 0.90584475	0.2333247215	0.9052916765	0.9052916765	0.8986666799	0.8995539 0.89776355	0.2574464381	0.8986666799	0.8986666799
6	2 convolution [512, 2]	0.9085416794	0.90805125 0.9090268	0.2256880701	0.9085416794	0.9085416794	0.8974999785	0.9010298 0.8937088	0.2693291306	0.8974999785	0.8974999785
7	2 convolution [512, 2]	0.9110521078	0.91054606 0.9115523	0.2213362902	0.9110521078	0.9110521078	0.8960833549	0.8967629 0.8953946	0.2587312758	0.8960833549	0.8960833549
8	2 convolution [512, 2]	0.9135521054	0.91306394 0.91403466	0.2155703753	0.9135521054	0.9135521054	0.8982499838	0.8972308 0.899249	0.254516691	0.8982499838	0.8982499838
9	2 convolution [512, 2]	0.9165312648	0.9159578 0.91709685	0.208176285	0.9165312648	0.9165312648	0.8995833397	0.89789 0.90122133	0.2556301355	0.8995833397	0.8995833397
10	2 convolution [512, 2]	0.9198020697	0.91935086 0.92024815	0.2008465976	0.9198020697	0.9198020697	0.8851666451	0.89142764 0.87813926	0.2985413671	0.8851666451	0.8851666451
1	2 convolution [512, 512, 2]	0.9036041498	0.90339077 0.90381646	0.2605671883	0.9036041498	0.9036041498	0.8922500014	0.8912073 0.8932727	0.2925795019	0.8922500014	0.8922500014
2	2 convolution [512, 512, 2]	0.9158020616	0.9156254 0.9159779	0.2089969516	0.9158020616	0.9158020616	0.8938333392	0.89258003 0.8950576	0.2703920007	0.8938333392	0.8938333392
3	2 convolution [512, 512, 2]	0.9208229184	0.9205805 0.92106384	0.1985894591	0.9208229184	0.9208229184	0.8933333158	0.8909338 0.8956294	0.2732286453	0.8933333158	0.8933333158
4	2 convolution [512, 512, 2]	0.9234895706	0.92315245 0.9238236	0.1899559796	0.9234895706	0.9234895706	0.8956666589	0.89757854 0.89368206	0.2745866179	0.8956666589	0.8956666589
5	2 convolution [512, 512, 2]	0.9282187223	0.92794174 0.92849356	0.1795120686	0.9282187223	0.9282187223	0.8967499733	0.899814 0.8934926	0.2683616877	0.8967499733	0.8967499733
6	2 convolution [512, 512, 2]	0.9316979051	0.93130076 0.9320904	0.1705223471	0.9316979051	0.9316979051	0.909250021	0.90990317 0.9085872	0.2396608293	0.909250021	0.909250021
7	2 convolution [512, 512, 2]	0.9355208278	0.9351831 0.93585485	0.1620711833	0.9355208278	0.9355208278	0.9008333087	0.9017341 0.89991593	0.2565122545	0.9008333087	0.9008333087
8	2 convolution [512, 512, 2]	0.9388853908	0.93852097 0.9392454	0.1535343081	0.9388853908	0.9388853908	0.9020833373	0.90024614 0.90385395	0.2851825953	0.9020833373	0.9020833373
9	2 convolution [512, 512, 2]	0.9409895539	0.94064516 0.94132996	0.1487781405	0.9409895539	0.9409895539	0.9051666856	0.90446603 0.905857	0.3012621403	0.9051666856	0.9051666856
10	2 convolution [512, 512, 2]	0.9447291493	0.9444281 0.9450269	0.1392964721	0.9447291493	0.9447291493	0.9014166594	0.9040628 0.8986202	0.2953568101	0.9014166594	0.9014166594
1	2 convolution [1024, 512, 256, 2]	0.9146770835	0.91436934 0.91498256	0.2635029852	0.9146770835	0.9146770835	0.8962500095	0.89877224 0.8935988	0.2685905397	0.8962500095	0.8962500095
2	2 convolution [1024, 512, 256, 2]	0.9296146035	0.9293961 0.9298316	0.1778874695	0.9296146035	0.9296146035	0.9027500153	0.9034339 0.90205616	0.2542472482	0.9027500153	0.9027500153
3	2 convolution [1024, 512, 256, 2]	0.9363333583	0.93612164 0.93654346	0.161615923	0.9363333583	0.9363333583	0.9006666541	0.8977174 0.9034504	0.3373866677	0.9006666541	0.9006666541
4	2 convolution [1024, 512, 256, 2]	0.9395624995	0.9391772 0.9399428	0.1522066742	0.9395624995	0.9395624995	0.898999989	0.89616174 0.9016872	0.2756465375	0.898999989	0.898999989
5	2 convolution [1024, 512, 256, 2]	0.9435625076	0.9432087 0.94391185	0.1433329284	0.9435625076	0.9435625076	0.906916678	0.9075407 0.9062841	0.2685157955	0.906916678	0.906916678
6	2 convolution [1024, 512, 256, 2]	0.9475833178	0.9473286 0.9478354	0.1342261285	0.9475833178	0.9475833178	0.9024999738	0.9031135 0.9018785	0.30235219	0.9024999738	0.9024999738
7	2 convolution [1024, 512, 256, 2]	0.9511562586	0.9508752 0.95143396	0.125568375	0.9511562586	0.9511562586	0.9024166465	0.9008718 0.9039139	0.3042684793	0.9024166465	0.9024166465
8	2 convolution [1024, 512, 256, 2]	0.9544062614	0.9541113 0.9546974	0.1166738272	0.9544062614	0.9544062614	0.9054999948	0.9052631 0.9057356	0.364079833	0.9054999948	0.9054999948
9	2 convolution [1024, 512, 256, 2]	0.9565312266	0.9561627 0.9568935	0.1105271801	0.9565312266	0.9565312266	0.9034166932	0.90396875 0.9028581	0.3179295361	0.9034166932	0.9034166932
10	2 convolution [1024, 512, 256, 2]	0.9609062672	0.9606747 0.96113497	0.1010118499	0.9609062672	0.9609062672	0.8991666436	0.90309143 0.89491045	0.3799754679	0.8991666436	0.8991666436
1	3 convolution [2]	0.8249478936	0.82221633 0.8275968	0.4699504077	0.8249478936	0.8249478936	0.8759999871	0.8743879 0.8775711	0.2986406982	0.8759999871	0.8759999871
2	3 convolution [2]	0.8733749986	0.87207705 0.8746468	0.3031539619	0.8733749986	0.8733749986	0.8786666393	0.88704413 0.86894685	0.2963351905	0.8786666393	0.8786666393
3	3 convolution [2]	0.8854062557	0.8843351 0.8864576	0.2781650126	0.8854062557	0.8854062557	0.8917499781	0.89231527 0.8911786	0.2681454718	0.8917499781	0.8917499781
4	3 convolution [2]	0.8940937519	0.8930633 0.8951044	0.2597511113	0.8940937519	0.8940937519	0.8809999824	0.8732693 0.88784164	0.2764364779	0.8809999824	0.8809999824
5	3 convolution [2]	0.9002916813	0.89940506 0.90116256	0.2467767	0.9002916813	0.9002916813	0.8884999752	0.88310325 0.8934204	0.267567575	0.8884999752	0.8884999752
6	3 convolution [2]	0.9041770697	0.9033281 0.9050112	0.2377151549	0.9041770697	0.9041770697	0.897333324	0.89728194 0.8973846	0.2506014705	0.897333324	0.897333324
7	3 convolution [2]	0.9085624814	0.9077281 0.90938175	0.228764683	0.9085624814	0.9085624814	0.8805000186	0.87226075 0.8877406	0.2803449631	0.8805000186	0.8805000186
8	3 convolution [2]	0.9103854299	0.90951157 0.91124237	0.223387599	0.9103854299	0.9103854299	0.9079166651	0.9062208 0.9095522	0.2434569299	0.9079166651	0.9079166651
9	3 convolution [2]	0.9122291803	0.9113705 0.9130712	0.2199628949	0.9122291803	0.9122291803	0.903249979	0.90572464 0.9006418	0.2525610328	0.903249979	0.903249979
10	3 convolution [2]	0.9136354327	0.91276383 0.91448957	0.2137070894	0.9136354327	0.9136354327	0.9109166861	0.9118205 0.90999407	0.264406532	0.9109166861	0.9109166861
1	3 convolution [512, 2]	0.9151041508	0.91490203 0.9153052	0.2186580151	0.9151041508	0.9151041508	0.9068333507	0.909503 0.9040013	0.2358119041	0.9068333507	0.9068333507
2	3 convolution [512, 2]	0.922979176	0.9228908 0.9230673	0.193210721	0.922979176	0.922979176	0.9083333611	0.9088649 0.9077954	0.2326092869	0.9083333611	0.9083333611
3	3 convolution [512, 2]	0.9268020988	0.9267158 0.9268881	0.1844657212	0.9268020988	0.9268020988	0.9138333201	0.9154399 0.9121644	0.2267202139	0.9138333201	0.9138333201
4	3 convolution [512, 2]	0.9305416942	0.93049526 0.9305879	0.176573202	0.9305416942	0.9305416942	0.9103333354	0.90995806 0.9107054	0.2470216453	0.9103333354	0.9103333354
5	3 convolution [512, 2]	0.9321874976	0.9322002 0.93217474	0.1714785546	0.9321874976	0.9321874976	0.9049999714	0.9010588 0.9086392	0.2420443594	0.9049999714	0.9049999714
6	3 convolution [512, 2]	0.9358958602	0.93586504 0.93592644	0.1635885388	0.9358958602	0.9358958602	0.8969166875	0.9021128 0.8911379	0.2677628696	0.8969166875	0.8969166875
7	3 convolution [512, 2]	0.9349062443	0.9347859 0.93502593	0.1667564362	0.9349062443	0.9349062443	0.9139166474	0.91192764 0.9158177	0.216148302	0.9139166474	0.9139166474
8	3 convolution [512, 2]	0.9394999743	0.9393863 0.93961316	0.1549074948	0.9394999743	0.9394999743	0.9125833511	0.91073096 0.9143602	0.2607713342	0.9125833511	0.9125833511
9	3 convolution [512, 2]	0.9411249757	0.94101304 0.9412363	0.1498103589	0.9411249757	0.9411249757	0.9167500138	0.9161842 0.9173081	0.2181386352	0.9167500138	0.9167500138
10	3 convolution [512, 2]	0.9433854222	0.94323933 0.9435307	0.1457278579	0.9433854222	0.9433854222	0.9051666856	0.91045004 0.89922065	0.2593968809	0.9051666856	0.9051666856
1	3 convolution [512, 512, 2]	0.9406874776	0.9405872 0.9407874	0.2158383578	0.9406874776	0.9406874776	0.9125833511	0.910594 0.91448593	0.2328374088	0.9125833511	0.9125833511
2	3 convolution [512, 512, 2]	0.9453958273	0.9453023 0.94548887	0.1410887092	0.9453958273	0.9453958273	0.9182500243	0.9178598 0.91863644	0.2421475053	0.9182500243	0.9182500243
3	3 convolution [512, 512, 2]	0.9459479451	0.945813 0.9460821	0.1388552338	0.9459479451	0.9459479451	0.9127500057	0.9116529 0.91382	0.2217516452	0.9127500057	0.9127500057
4	3 convolution [512, 512, 2]	0.9477187395	0.94756293 0.9478734	0.1333048195	0.9477187395	0.9477187395	0.9129999876	0.9153284 0.9105398	0.2240469307	0.9129999876	0.9129999876
5	3 convolution [512, 512, 2]	0.9506145716	0.9504063 0.950821	0.127213344	0.9506145716	0.9506145716	0.9186666608	0.9196046 0.9177065	0.2416230887	0.9186666608	0.9186666608
6	3 convolution [512, 512, 2]	0.9511666894	0.9508739 0.95145583	0.1243553236	0.9511666894	0.9511666894	0.9153333306	0.91555846 0.9151069	0.2250097096	0.9153333306	0.9153333306
7	3 convolution [512, 512, 2]	0.9539999962	0.95381814 0.95418036	0.1192808971	0.9539999962	0.9539999962	0.917750001	0.9175783 0.917921	0.2761963606	0.917750001	0.917750001
8	3 convolution [512, 512, 2]	0.9415937662	0.94090354 0.94226784	0.152856797	0.9415937662	0.9415937662	0.9118333459	0.9102629 0.9133496	0.2387892604	0.9118333459	0.9118333459
9	3 convolution [512, 512, 2]	0.9573125243	0.9570764 0.95754594	0.1107505709	0.9573125243	0.9573125243	0.9154166579	0.91467 0.9161502	0.2767106593	0.9154166579	0.9154166579
10	3 convolution [512, 512, 2]	0.9577812552	0.95750993 0.9580491	0.1115583256	0.9577812552	0.9577812552	0.9159166813	0.9135167 0.91818696	0.249979496	0.9159166813	0.9159166813
1	3 convolution [1024, 512, 256, 2]	0.9550208449	0.9549353 0.95510584	0.1605819762	0.9550208449	0.9550208449	0.9178333282	0.91717064 0.91848534	0.3067594171	0.9178333282	0.9178333282
2	3 convolution [1024, 512, 256, 2]	0.9594270587	0.9594089 0.95944524	0.1065118015	0.9594270587	0.9594270587	0.9164999723	0.9182841 0.9146362	0.2321927547	0.9164999723	0.9164999723
3	3 convolution [1024, 512, 256, 2]	0.9599062204	0.9597574 0.960054	0.1055173203	0.9599062204	0.9599062204	0.9167500138	0.9193899 0.9139312	0.2464149743	0.9167500138	0.9167500138
4	3 convolution [1024, 512, 256, 2]	0.9593541622	0.95924795 0.9594597	0.105272837	0.9593541622	0.9593541622	0.9139999747	0.9126903 0.91527086	0.3068576753	0.9139999747	0.9139999747
5	3 convolution [1024, 512, 256, 2]	0.9589062333	0.95874596 0.95906526	0.1103717983	0.9589062333	0.9589062333	0.9046666622	0.90064263 0.90837735	0.3168414831	0.9046666622	0.9046666622
6	3 convolution [1024, 512, 256, 2]	0.9609895945	0.9607479 0.9612282	0.1034229025	0.9609895945	0.9609895945	0.9139166474	0.9144513 0.9133752	0.2662386596	0.9139166474	0.9139166474
7	3 convolution [1024, 512, 256, 2]	0.9625833631	0.9623866 0.9627779	0.09819595516	0.9625833631	0.9625833631	0.9185833335	0.9190354 0.9181262	0.3584215641	0.9185833335	0.9185833335
8	3 convolution [1024, 512, 256, 2]	0.9647395611	0.96453524 0.96494156	0.09278436005	0.9647395611	0.9647395611	0.9157500267	0.91424197 0.91720575	0.6382867694	0.9157500267	0.9157500267
9	3 convolution [1024, 512, 256, 2]	0.9517604113	0.95137995 0.9521348	0.1289575547	0.9517604113	0.9517604113	0.9056666493	0.9087096 0.9024138	0.2861087918	0.9056666493	0.9056666493
10	3 convolution [1024, 512, 256, 2]	0.9636458158	0.9634179 0.9638708	0.09659095109	0.9636458158	0.9636458158	0.917750001	0.91954017 0.91587824	0.4092084467	0.917750001	0.917750001
1	4 convolution [2]	0.8253853917	0.82519776 0.82557255	0.4069786072	0.8253853917	0.8253853917	0.8786666393	0.87836254 0.87896913	0.2918222249	0.8786666393	0.8786666393
2	4 convolution [2]	0.8786979318	0.8785397 0.8788556	0.2918336093	0.8786979318	0.8786979318	0.8926666379	0.89300543 0.8923257	0.2733984292	0.8926666379	0.8926666379
3	4 convolution [2]	0.8934687376	0.89320725 0.89372885	0.2612748146	0.8934687376	0.8934687376	0.9027500153	0.90410054 0.90136075	0.2455496639	0.9027500153	0.9027500153
4	4 convolution [2]	0.902125001	0.9018448 0.90240353	0.244403407	0.902125001	0.902125001	0.8960833549	0.89981514 0.8920626	0.2572347522	0.8960833549	0.8960833549
5	4 convolution [2]	0.9086041451	0.90820444 0.9090003	0.2281206846	0.9086041451	0.9086041451	0.8957499862	0.8909613 0.90013564	0.2481725067	0.8957499862	0.8957499862
6	4 convolution [2]	0.9123749733	0.91196597 0.9127801	0.2203645408	0.9123749733	0.9123749733	0.9049999714	0.9031106 0.9068171	0.2390948683	0.9049999714	0.9049999714
7	4 convolution [2]	0.915635407	0.9153488 0.91592	0.2122298926	0.915635407	0.915635407	0.9058333039	0.9018074 0.9095421	0.2395686209	0.9058333039	0.9058333039
8	4 convolution [2]	0.920104146	0.919857 0.9203498	0.2013357282	0.920104146	0.920104146	0.9155833125	0.9156605 0.9155058	0.2152620703	0.9155833125	0.9155833125
9	4 convolution [2]	0.9220312238	0.92166644 0.92239255	0.197153911	0.9220312238	0.9220312238	0.9229999781	0.923712 0.92227453	0.2007297128	0.9229999781	0.9229999781
10	4 convolution [2]	0.9248229265	0.92449015 0.92515266	0.1889170706	0.9248229265	0.9248229265	0.9123333097	0.9096375 0.9148729	0.2316058725	0.9123333097	0.9123333097
1	4 convolution [512, 2]	0.9219375253	0.92195374 0.9219212	0.2024347186	0.9219375253	0.9219375253	0.9156666398	0.9171849 0.91409165	0.2168573141	0.9156666398	0.9156666398
2	4 convolution [512, 2]	0.9277083278	0.92784804 0.92756796	0.1824731827	0.9277083278	0.9277083278	0.9132500291	0.9140308 0.9124547	0.2153961957	0.9132500291	0.9132500291
3	4 convolution [512, 2]	0.9320312738	0.9321308 0.9319312	0.1727632433	0.9320312738	0.9320312738	0.890083313	0.89893484 0.87953234	0.250546515	0.890083313	0.890083313
4	4 convolution [512, 2]	0.9329791665	0.9330767 0.93288124	0.1701387912	0.9329791665	0.9329791665	0.9234166741	0.923525 0.92330796	0.1996691674	0.9234166741	0.9234166741
5	4 convolution [512, 2]	0.9343541861	0.93445516 0.93425274	0.1676291525	0.9343541861	0.9343541861	0.9228333235	0.9233063 0.92235446	0.2175154835	0.9228333235	0.9228333235
6	4 convolution [512, 2]	0.9373437762	0.937377 0.9373104	0.1584379822	0.9373437762	0.9373437762	0.9217500091	0.92200345 0.9214948	0.2129809111	0.9217500091	0.9217500091
7	4 convolution [512, 2]	0.9381458163	0.9381045 0.93818694	0.1570590883	0.9381458163	0.9381458163	0.9265833497	0.92703927 0.9261215	0.1925024837	0.9265833497	0.9265833497
8	4 convolution [512, 2]	0.9418854117	0.94176704 0.94200313	0.1504942477	0.9418854117	0.9418854117	0.9125000238	0.91780174 0.9064671	0.2330851108	0.9125000238	0.9125000238
9	4 convolution [512, 2]	0.9422187209	0.9421772 0.9422602	0.1490700245	0.9422187209	0.9422187209	0.926666677	0.9279868 0.9252971	0.2045307606	0.926666677	0.926666677
10	4 convolution [512, 2]	0.9437500238	0.9436372 0.9438622	0.1424072832	0.9437500238	0.9437500238	0.9188333154	0.9226492 0.91462123	0.2288808525	0.9188333154	0.9188333154
1	4 convolution [512, 512, 2]	0.946416676	0.946335 0.946498	0.1591994464	0.946416676	0.946416676	0.9250000119	0.925187 0.92481196	0.200510636	0.9250000119	0.9250000119
2	4 convolution [512, 512, 2]	0.9478437304	0.94783884 0.9478486	0.1340953708	0.9478437304	0.9478437304	0.9251666665	0.9245124 0.92580956	0.2024490982	0.9251666665	0.9251666665
3	4 convolution [512, 512, 2]	0.9456666708	0.94552934 0.9458032	0.1389929652	0.9456666708	0.9456666708	0.9258333445	0.9266644 0.9249831	0.2080940753	0.9258333445	0.9258333445
4	4 convolution [512, 512, 2]	0.9483437538	0.9482245 0.94846237	0.1324654967	0.9483437538	0.9483437538	0.9253333211	0.92754316 0.9229843	0.2081094086	0.9253333211	0.9253333211
5	4 convolution [512, 512, 2]	0.9511458278	0.9510459 0.9512453	0.1265737563	0.9511458278	0.9511458278	0.9184166789	0.9158428 0.92083764	0.2443780303	0.9184166789	0.9184166789
6	4 convolution [512, 512, 2]	0.9486145973	0.94847333 0.94875497	0.1358262599	0.9486145973	0.9486145973	0.9235000014	0.923994 0.9229994	0.2235287875	0.9235000014	0.9235000014
7	4 convolution [512, 512, 2]	0.9535833597	0.9534641 0.95370185	0.1194250584	0.9535833597	0.9535833597	0.9242500067	0.92454547 0.92395204	0.2058784217	0.9242500067	0.9242500067
8	4 convolution [512, 512, 2]	0.9541875124	0.95410717 0.95426744	0.1198573336	0.9541875124	0.9541875124	0.9260833263	0.92690563 0.9252422	0.2383998632	0.9260833263	0.9260833263
9	4 convolution [512, 512, 2]	0.9545833468	0.95443803 0.95472753	0.1171445921	0.9545833468	0.9545833468	0.9189166427	0.9187609 0.91907173	0.2367716134	0.9189166427	0.9189166427
10	4 convolution [512, 512, 2]	0.9553645849	0.95523065 0.9554976	0.1150081977	0.9553645849	0.9553645849	0.923666656	0.92419726 0.9231285	0.2263379991	0.923666656	0.923666656
1	4 convolution [1024, 512, 256, 2]	0.9604791403	0.96037346 0.9605842	0.1350988448	0.9604791403	0.9604791403	0.9178333282	0.91562545 0.9199285	0.2529542744	0.9178333282	0.9178333282
2	4 convolution [1024, 512, 256, 2]	0.9602187276	0.9602125 0.96022487	0.1046332568	0.9602187276	0.9602187276	0.9246666431	0.9251035 0.92422456	0.2158418	0.9246666431	0.9246666431
3	4 convolution [1024, 512, 256, 2]	0.9564166665	0.9564003 0.95643294	0.1141224504	0.9564166665	0.9564166665	0.921333313	0.92248315 0.9201488	0.2902549505	0.921333313	0.921333313
4	4 convolution [1024, 512, 256, 2]	0.9604895711	0.9603735 0.96060485	0.1049497053	0.9604895711	0.9604895711	0.9136666656	0.91044253 0.9166666	0.2400039434	0.9136666656	0.9136666656
5	4 convolution [1024, 512, 256, 2]	0.9571041465	0.95709336 0.9571148	0.1137457043	0.9571041465	0.9571041465	0.9214166403	0.9218788 0.92094886	0.2131452858	0.9214166403	0.9214166403
6	4 convolution [1024, 512, 256, 2]	0.9585520625	0.95834154 0.9587603	0.1075858474	0.9585520625	0.9585520625	0.924333334	0.92524284 0.9234014	0.2382297963	0.924333334	0.924333334
7	4 convolution [1024, 512, 256, 2]	0.9616875052	0.96152073 0.9618527	0.09956307709	0.9616875052	0.9616875052	0.9199166894	0.9189097 0.92089874	0.2468322366	0.9199166894	0.9199166894
8	4 convolution [1024, 512, 256, 2]	0.9586353898	0.9584166 0.9588518	0.1074528694	0.9586353898	0.9586353898	0.9210000038	0.9190434 0.9228641	0.2674310207	0.9210000038	0.9210000038
9	4 convolution [1024, 512, 256, 2]	0.9623437524	0.96216434 0.9625213	0.09895321727	0.9623437524	0.9623437524	0.923666656	0.92550415 0.92173606	0.2657699585	0.923666656	0.923666656
10	4 convolution [1024, 512, 256, 2]	0.9621458054	0.9619803 0.9623099	0.09975361079	0.9621458054	0.9621458054	0.9219166636	0.92404956 0.9196604	0.2523550689	0.9219166636	0.9219166636

References

1. N. Jacobson, Deepfakes and Their Impact on Society, CPI OpenFox, 2024 [↩]
2. Y. Patel et al., Deepfake Generation and Detection: Case Study and Challenges, IEEE Access 11, 143296–143323, 2023 [↩]
3. Deepfake Detection Using XceptionNet, IEEE, https://ieeexplore.ieee.org/document/10363477 [↩]
4. What is a GAN? – Generative Adversarial Networks Explained, Amazon Web Services, https://aws.amazon.com/what-is/gan/ [↩]
5. G. Monkam, W. Xu, J. Yan, A GAN-based Approach to Detect AI-Generated Images, in 2023 26th ACIS International Winter Conference on SNPD-Winter, IEEE, 229–232, 2023 [↩]
6. deepfake and real images, Kaggle, https://www.kaggle.com/datasets/manjilkarki/deepfake-and-real-images [↩] [↩]
7. precision_score, scikit-learn, https://scikit-learn.org/ stable/modules/generated/sklearn.metrics.precision_score.html [↩]
8. recall_score, scikit-le arn, https://scikit-learn.org/stable/modules/generated/sklearn.metrics.recall_score.html [↩]
9. f1_score, scikit-learn, https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f 1_score.html [↩]
10. CIFAKE: Real and AI-Generated Synthetic Images, Kaggle, https://www.kaggle.com/datasets/birdy654/cifake-rea l-and-ai-generated-synthetic-images [↩]
11. G. Boesch, Very Deep Convolutional Networks (VGG) Essential Guide, viso.ai, 2021 [↩]
12. What Is Transfer Learning? A Guide for Deep Learning, Built In, https://builtin.com/data-science/transfer-learning [↩]
13. S. Das, Implementing DenseNet-121 in PyTorch: A Step-by-Step Guide, deepkapha notes, 2023 [↩]
14. G. Boesch, Deep Residual Networks (ResNet, ResNet-50) – 2024 Guide, viso.ai, 2023 [↩]
15. L. Gupta, Batch Normalization and ReLU for solving Vanishing Gradients, Analytics Vidhya, 2021 [↩]
16. Adam Optimizer – an overview, ScienceDirect Topics, https://www.scienced irect.com/topics/computer-science/Adam-optimizer [↩]
17. M. F. Aydogdu, M. F. Demirci, Age Classification Using an Optimized CNN Architecture, in Proceedings of the International Conference on Compute and Data Analysis, ACM, 233–239, Lakeland FL USA, 2017 [↩]