How Do Sponsorship Labels and Item Position Influence Consumer Click-Through Rates on Amazon?

Abstract

This study analyzes the impact of Amazon’s sponsorship mechanism on consumer clickthrough rates on Amazon. Based on consumer search theory I conducted an experiment with 221 participants who interacted with simulated Amazon product pages (pencils, toothpaste, and paper towels). Participants were randomly assigned to one of three experimental treatments: standard Amazon layout with “Sponsored” labels, standard Amazon layout without “Sponsored” labels, and randomized product positions with “Sponsored” labels. I investigated whether being in a top-position was advantageous and how the sponsorship label influenced participants’ item selection. I used logistic regression to analyze and determine how position affects clickthrough rates and found that items in the top row had a higher clickthrough rate. To determine the effect of the sponsorship label I used a binomial test which allowed us to determine that the “Sponsored” label had no statistically significant effect on clickthrough rate. These results emphasize the importance of product positioning rather than a sponsorship label, this provides marketers with insights on how to improve product clickthrough rate. However, limitations like the study’s sample size and reliance on a simulated page could restrict the implications of the study.

Keywords: Amazon, sponsorship, click-through rates, consumer behavior, e-commerce, labeling effects, consumer search.

Introduction

With over 12 million items sold daily, Amazon is instrumental in helping consumers find and select products to purchase. Consumers are constantly faced with decisions influenced by numerous factors, such as product placement, price, brand recognition, ratings, and promotional labels. For sellers, understanding this behavior is key to optimizing sales. One standard tool advertisers use to gain exposure is paid advertising. In 2023, sellers spent over $40 billion on Amazon advertising, competing for consumer attention through higher product positions. These positions place products prominently on the results page, increasing their visibility. However, the top positions do not guarantee customer engagement, making it necessary for sellers to determine whether paying for top positions is worth the potential increase in sales.

A common tool sellers use to increase their products’ visibility is Amazon’s “Sponsored” label. The “Sponsored” label not only signals to consumers that the placement of the product was due to a payment from the seller but also raises the product to the top row of the product result page, thereby supposedly giving the product a positional advantage.

However, whether this mechanism of being sponsored results in a significant increase in clicks is undetermined. While it is generally assumed that products at the top of search results have an advantage, the impact on consumer decisions is not fully understood. Additionally, it is unknown whether other factors, such as price or brand recognition, play a more substantial role in influencing whether a product is clicked. Without evidence to isolate how position and labeling affect an item’s clickthrough rates, marketers could face difficulties evaluating the value of sponsored listings.

Additionally, Amazon is required to disclose when sellers pay for these noticeable positions. Specifically, Section 5 of the Federal Trade Commission Act prohibits ‘‘unfair or deceptive acts or practices in or affecting commerce.” This Act prompted Amazon to include a “Sponsored” label on each of these products to ensure transparency and not deceptively give an unfair advantage to these products. However, it is unclear how sponsorship labels affect user behavior or if consumers ignore them. By analyzing how consumers respond to sponsorship labels, this study provides insights for policymakers evaluating the effectiveness of disclosure regulations and contributes to questions about the impact of transparency mandates in digital marketplaces.

Research about consumer search and position bias has shown that product prevalence significantly influences online decisions. They found that users are more likely to click on products that are higher on a page simply because they are more visible and accessible. Although Amazon uses a Generalized Second-Price auction to determine top positions on a results page, I focus on how product placement and the “Sponsored” label affect consumer decisions.

Based on these goals, our study explores the following research questions: Does a product’s position on the page affect how likely it is to be clicked?

Does the presence of a “Sponsored” label significantly influence a consumer’s decision to click?

I hypothesize that products in higher positions or with a sponsorship label would be clicked more frequently than products that lacked these characteristics.

This study aims to analyze the effect of Amazon’s sponsorship mechanism on consumer clickthrough rates. Theoretical models suggest that online consumers engage in a sequential search, where evaluating each additional product incurs a cognitive or temporal cost.

Sponsorship helps mitigate this cost by elevating products to prominent positions, increasing the likelihood of consideration. However, the effectiveness of this mechanism also depends on how consumers interpret item ranking. High-ranking products that lack a “Sponsored” label may be viewed as more trustworthy or valuable, as their position is perceived to reflect algorithmic relevance rather than paid promotion. Thus, the presence or absence of the “Sponsored” label influences visibility and acts as a signal that shapes perceptions of quality and credibility.

To do this, I created an experiment involving simulated Amazon pages for three products: pencils, toothpaste, and paper towels. Each participant was shown one of three result pages that varied in product position, presence of a “Sponsored” label, or both. This allowed us to isolate and measure the effects of position and the sponsorship label on consumer behavior.

Literature Review

Previous research shows that the order in which items appear significantly influences which ones consumers choose to click. Due to this, firms shape product positioning through auctions, optimization strategies, and other tools to steer consumers toward preferred items. Product placement is a commercial strategy and behavioral tool on platforms like Amazon. This study investigates how these positions influence clickthrough rates.

Consumer Search

When a user searches for a product on Amazon, the results they first see are likely sponsored items rather than the most relevant items. This is known as search diversion, and it has been theorized to help platforms increase consumer engagement¹. While this can benefit sellers and the selling platform, it could distort consumer search and affect which products are clicked. Our study tests how these diversions impact clickthrough rates.

Another strategy platforms use to influence consumer behavior is search distortion. Search distortion is when results are ordered to prioritize profitability over relevance². On Amazon, this can occur when sponsored or Amazon-branded products are given top positions regardless of search-match quality. A recent study found that Amazon systematically favors its private-label products, such as AmazonBasics, in search rankings³. Such distortion may redirect user attention toward products in top positions, affecting click likelihood. Our study examines how position impacts clickthrough rates, clarifying how search design influences user behavior. While I observe some top organic (non-sponsored) products receiving high clickthrough rates, our current data do not allow us to test whether these products are inherently more desirable. Estimating product fixed effects in future research will help assess whether Amazon’s organic rankings reflect product quality or if they are themselves a source of search distortion.

Previous research indicates that the order in which products appear may influence which items consumers choose to click. Athey and Ellison (2011) developed a model of position auctions that incorporates user search patterns, finding that consumers often concentrate on the top set of results⁴. Their research suggests that product visibility is crucial in shaping user decisions. While their study was theoretical, its findings align with our experiment: the order in which products appear may influence which items consumers choose to click. Supporting this, Chen et al. (2023), in “Advancing Ad Auction Realism: Practical Insights & Modeling Implications,” modeled a real-world ad auction environment that found that users favored higher-placed results regardless of content quality⁵. Their findings reinforce that position is one of the driving factors in consumer engagement.

Consumer Clickthrough Rates

One of the main metrics search engines use to determine the effectiveness of their advertisements is clickthrough rates (CTR). Clickthrough rates measure the percentage of users who click on an advertisement after viewing it. Many researchers are interested in the factors influencing users to click on specific ads and how to predict CTR. One paper specifically focused on investigating the factors related to click-sponsored links. The authors found that users overwhelmingly prefer organic links, clicking them first 82% of the time. Participants of this study also rated organic links as more relevant than sponsored links despite identical content being rotated between the two types of links⁶. A significant preference for organic recommendations was also observed on a different platform, Docear, with organic results having 8.86% CTR compared to 5.86% of commercial results⁷. Another study highlighted the complexity of how the position of items affects consumer clickthrough rates. Many users do not click sequentially; in fact, 46% of users click out of order, and 57% of multi-click users do not follow cascading behavior. In addition, the higher relevance of neighboring ads can negatively impact the CTR of a given ad⁸. A different study supported these findings, extending them by developing a structural model to better capture the observed behavior from consumers. They found that eliminating competition among ads could increase CTR by 51%⁹. Overall, these findings suggest that consumer behavior in response to positional advertisements is influenced by a range of factors. In our study, I can compare our findings about consumer behavior on Amazon to those of previous studies of CTR on other search engines.

There has also been research regarding the factors influencing organic traffic to retail sites, specifically through search engines¹⁰. A specific paper confirmed that organic links are clicked more often than sponsored links for retail traffic. The paper also found that a retailer’s brand reputability helped with directly receiving clicks from consumers and gaining visibility through search engine preference for recognizable brands.

Additionally, there has been research regarding the prediction of CTR using statistical models. One paper discussed how CTR data is often skewed because users are more likely to click on ads in higher positions, regardless of their actual relevance⁵. To address this issue, the authors of this paper proposed a probabilistic factor model to address this position bias, and tested it on real-world data to ensure accuracy. I can compare the results of our study with this model, which would give us insight on how CTR on Amazon compares with other search engines.  While clickthrough rate provides valuable information about consumer engagement, it is not necessarily indicative of purchasing behavior. In marketing literature, purchase intent is defined as a consumer’s conscious inclination or likelihood to buy a product. This is considered a more direct predictor of actual transactions. Studies have linked purchase intent with factors like trust, perceived value, and presentation¹¹. In this study, I interpret click behavior as a behavioral signal that may reflect early-stage purchase intent, allowing us to connect interface design elements like product position and sponsorship labeling to the broader process of consumer decision-making.

Having already discussed the theory behind consumer search, it is important to consider the empirical results from previous studies regarding consumer search and clickthrough rates. Although there are no preexisting papers regarding the sponsorship mechanism on Amazon, there are papers examining the effect of top positions on Google.

In a paper titled “Search engine advertisements: The impact of advertising statements on click-through and conversion rates”¹², the researchers used Google AdWords campaigns to conduct an experiment with real ads from two companies: one selling electronic supplies and the other selling travel insurance. The researchers randomly assigned consumers who searched for certain keywords one of four ad variants. The four types of ads were ones that included 1) expert evidence, 2) statistical evidence, 3) causal evidence, and 4) baseline ads with no specific evidence. Expert evidence ads contained endorsements, statistical evidence ads contained data supporting the product’s effectiveness, and causal evidence ads contained explanations of how the product works. From measuring and analyzing clickthrough rates, the study found that ads with expert evidence and statistical evidence had higher clickthrough rates than other ad types. In our study, I analyzed the effect of the “Sponsored” label, which could be considered a type of “expert evidence” ad. This indicates that sponsored products on Amazon might have a positive effect on clickthrough rates.

There also have been some studies focusing purely on clickthrough rate of sponsored positions compared to non-sponsored positions on Google. Top organic search results received on average 19 times more clicks than the top sponsored result¹². This suggests that consumers often skipped past sponsored content to the organic search results on Google. The study found that the first organic search result had a clickthrough rate of roughly 40%. Another study found that the top organic result had a clickthrough rate of roughly 28%¹³. Although these numbers vary quite drastically, it is clear that the top organic result receives significantly more clicks than other positions. However, it is important to note that the top organic result displays a snippet of a website, which is a large display of what the website contains. So, this could be a reason why the top organic result receives such a large number of clicks compared to top sponsored positions and lower organic positions on Google. This empirical information contradicts theoretical knowledge regarding search diversion, as people who search on Google regularly skip over the sponsored content. In our experiment, I would be able to see whether the behavior of consumers on Amazon aligns more with the theoretical findings or the empirical findings of consumers on Google.

The layout of the Amazon search engine is much different than the Google search engine as products are displayed in a 3 by 5 grid rather than a list of items. Therefore, the distribution of clicks for top results will likely vary drastically between Amazon and Google. However, I can still compare the results of our study to the empirical data regarding clickthrough rates on the Google search engine.

Methods

Experiment Design

The objective of this study is to examine the factors influencing consumer preferences on Amazon.com and evaluate the extent to which they impact purchasing decisions. Participants were recruited through general online survey forums that were not targeted to any specific demographic group, helping to reduce sampling bias and reflect a broader range of consumer behavior. At the beginning of the experiment, the participants were instructed to select which item they are most likely to purchase on the following simulated Amazon webpages. These pages are shown sequentially, displaying a simulated Amazon page for the product categories: pencils, toothpaste, and paper towels.

These simulated pages look nearly identical to that of Amazon.com, ensuring that consumers feel as if they are actually purchasing from the site. Each page contains the top three rows of items for that respective product category, and sponsorship item banners, filter options, and a search bar—just like a real product page on Amazon.com. Each row contains five products (excluding row three of paper towels which has only four products) displaying 15 products per page (excluding paper towels which display 14 products). The product listings were taken directly from live Amazon search results with no content modified. An example of our simulated webpage can be seen in Figure 1.

After the participant selects the item they are most likely to purchase from the “number 2 pencils” webpage, they are automatically directed to the “toothpaste” web page followed by the “paper towels” webpage. The decision to choose a certain item over another could depend on a variety of factors including position, price, ratings, sponsorship label, aesthetic appeal, brand name, etc. Although many factors contribute to consumer clickthrough rates, I specifically wanted to analyze the effect of the sponsorship label, which gives products a “Sponsored” tag and a top position on the page.

To analyze the effects of the sponsorship label, I developed 3 treatments of this experiment to control for sponsorship visibility and position, which are described as follows:

• Treatment 1: This treatment mirrors a standard Amazon search results page. Sponsored products are positioned at the top, followed by other popular products ranked by relevance or popularity. This arrangement is reflective of the normal Amazon webpage. Sponsored products are clearly labeled as “Sponsored.” An image of this treatment can be seen in Figure 1. 
• Treatment 2: In this treatment, the product positions are identical to Treatment 1, but the “Sponsored” labels are removed, presenting the items as though they were not sponsored. This treatment allows us to observe how removing the sponsorship label affects consumer choice when product positioning remains constant. An image of this treatment can be seen in Figure 2.
• Treatment 3: In this treatment, product positions were randomized for each participant. However, the “Sponsored” labels are still in their standard locations at the top of the page. This treatment allows us to control for product positioning. An image of this treatment can be seen in Figure 3.

Each participant is randomly given one of the three treatments. The treatment they were given is consistent throughout all product types (pencils, toothpaste, paper towels).

In Figure 2, the red arrows show how the 2 of the products in the top row changed position. The other 3 products in the Treatment 3 webpage were originally in the second and third rows of the page, but moved up to the top row since the positions are randomized for Treatment 3.

A preliminary test of the experiment was conducted with a group of 20 individuals to evaluate the clarity of instructions, functionality of the simulated Amazon pages, and the overall flow of the survey. This pilot phase helped identify areas of confusion, such as unclear instructions and button visibility, which were addressed before launching the full study. Feedback from this test ensured that participants in the main experiment would have a smoother and more intuitive experience.

Data Collection

In total, 221 people took the survey. 45 responses for treatment 1, 50 responses for treatment 2, and 126 responses for treatment 3 were collected. I intentionally oversampled Treatment 3, more than twice the size of the other groups, because the analysis of product position required greater statistical power. Because treatment types were randomly assigned to participants, the uneven group sizes are not expected to introduce systematic bias.

Throughout the experiment, data is collected on several key variables for each product displayed on the webpages. Because the same products were used across all treatments, variation in characteristics like brand, price, and rating were held constant which eliminates these as confounding variables. For every item presented, the following information is recorded:

• Position: The row and column of the clicked product
  • Sponsored Label: A binary indicator noting whether a product is labeled as “Sponsored” or not
  • Price: The cost of the product
  • Brand: The brand of the product

After selecting which products they would be most likely to purchase, participants are asked to respond to a series of survey questions designed to provide further context for their choices and shopping habits. An image of this survey can be seen in Appendix A1. The questions include:

• Why did you choose the pencils you chose?
  • Why did you choose the toothpaste?
  • Why did you choose the paper towels you chose?  
  • Did the fact that there were some items labeled “Sponsored” influence your decision?
  • How often do you shop online?
  • What is your age?
  • What gender do you identify as?

From these questions, I am able to observe several descriptive statistics (online shopping frequency, age, gender, and “Sponsored” label awareness) of the population that participated in our experiment. A table displaying the raw data of these descriptive statistics can be seen in Figure 3.The distribution of these descriptive statistics amongst the groups, separated by treatment, can be seen in Appendix Figures A2 through A13.

The demographic information distribution varied between treatments for age, but not as much for online shopping frequency and gender. For age, treatment 1 was fairly spread out, while treatments 2 and 3 had significantly more younger participants between the ages 0-34. For online shopping frequency, all three treatments had a similar trend. Most people shop weekly, and slightly less people shop monthly. Significantly less people shop rarely/never, with even less people shopping daily. For gender, there was a roughly even split between males and females across all three treatments, with some variation. For sponsorship label effect, most people did not care or did not realize any items were sponsored across all treatments. In treatment 1, some  people noticed the “Sponsored” label, but most people who noticed the label avoided it. In treatment 2, very few people responded that the “Sponsored” label affected them, which is reasonable as there was no label present in this treatment. For treatment 3, ~45% of people did not care about the “Sponsored” label, ~25% did not realize it was there, ~15% noticed and avoided it, and ~5% had a preference towards sponsored items.

Data Analysis and Results

To analyze the collected data, various statistical methods were applied. This section focuses on three questions related to the role of product position, sponsorship labeling, and product-level popularity.

Question 1: Is Being in the Top Position Advantageous?

An important question of this study was whether being placed in the top position on the page is advantageous for a product, independent of other factors like brand recognition or having a “Sponsored” label. I found that being in the top row, which is given to sponsored products, is advantageous. However, being in the leftmost position, the “top” position in terms of the auction, was not observed to be advantageous for all types of products.

To answer this question, logistic regression was applied to the data collected from Treatment 3, where product positions were randomized. Because Treatment 3 randomized the positions of all the products for each participant, it allows us to interpret the effects of row and column position on clickthrough rates as casual. The random product locations remove the confounding effects from product quality or brand recognition. This isolates position as the primary variable. As a result, logistic regression can be applied to this treatment to determine the effect of placement independent of product differences. On the other hand, Treatments 1 and 2 have fixed

product-label-position combinations and are interpreted as correlational instead of causal.

Logistic regression is a statistical method used for situations with binary dependent variables¹⁴. I chose to run a logistic regression for our data because this type of method is used for modeling the relationship between independent variables (position) and a binary dependent variable (clicked or not clicked), which was exactly what I wanted to do.

The logistic model provides useful information about the correlation between the row/column position and clicks (our binary dependent variable). Whether or not a certain product was clicked, the dependent variable, was represented by 0 = not clicked and 1 = clicked. The independent variable was the product’s position—row and column—on the webpage. The rows and columns were dummy variables as I cannot assume that preference does not scale linearly as the row/column number increases. Therefore, I represented the row and column position as dummy variables to allow us to measure the individual effect of each row and column.  Additionally, I tested the effect of the “Sponsored” label, although I was more concerned with the effect of position.

In a logistic regression, a sigmoid function is used to output coefficients of the model. Figure 4 shows the specific sigmoid equation for our experiment. These coefficients allow us to determine the strength of correlation between clicks and position, where a negative number indicates a negative correlation, and positive number indicates a positive correlation. Additionally, each coefficient has a p-value, which signifies if the result is significant. A p-value below 0.05 or 5% means that the result is statistically significant.

I validated the regression model I used by running a log-likelihood test. The results from this test can be seen in Figure 5. The likelihood ratio test results indicate that the logistic regression models significantly improve fit compared to a null model with no predictors for all three product types. The p-values are all well below the 0.05 threshold, suggesting that the variables included in the models (Sponsored, Row, and Column) meaningfully explain variation in clickthrough behavior across all product types. Figure 6 shows the logistic regression results on the data from the pencil products.

The data from Figure 6 revealed several statistically significant effects of product position on clickthrough likelihood. The regression coefficient for Row 2 showed that these items were significantly less likely to be clicked compared to those in Row 1. More specifically, products in Row 2 had a log-odds of being clicked that was 1.076 lower than products in Row 1. This means that the odds of a product being clicked in Row 2 were about 34.1% of the odds for products in Row 1, calculated by converting the log-odds to an odds ratio value ( 𝑐𝑜𝑒𝑓𝑓𝑖𝑐𝑖𝑒𝑛𝑡). As indicated by the coefficient for Row 3, this effect was even more pronounced for items in Row 3. Items in this row had an odds of being clicked that were approximately 27% that of items in Row 1 (odds ratio ≈ 0.27), indicating that items further down the page are much less likely to be selected. The 95% confidence interval for the row variables confirms the reliability of their negative effects.

In terms of column effects, Columns 3 and 4 had negative coefficients, meaning being in these columns had significant negative impact on click likelihood compared to Column 1. The odds ratio for Column 3 was 0.54, meaning items in this column were 46% less likely to be clicked than those in Column 1. Column 4 had an odds ratio of 0.46, so these items were 54% less likely to be clicked than those in Column 1. The coefficient for Column 5 showed an even stronger negative effect for items in this column. Resulting in an odds ratio of roughly 0.25, Column 5 items had 75% lower odds of being clicked compared to Column 1. The 95% confidence interval for the column variables confirms the reliability of their negative effects. Figure 7 shows the logistic regression results on the data from the toothpaste products.

The logistic regression on the data from the toothpaste Amazon page, as shown in Figure 7, showed limited statistical significance. The only statistically significant result was for Column 2, where there was a positive coefficient, indicating better chances of being clicked compared to Column 1. Specifically, the coefficient for Column 2 corresponds to an odds ratio of approximately 2.06, meaning items in this column were about twice as likely to be clicked compared to items in Column 1. This result is contradictory to sellers’ intuition as the top position in Column 1 is treated as the top position in the auction.

All other factors had p-values exceeding the typical threshold for significance (0.05). An explanation for this result could be the amount of repeated products on the toothpaste page compared to the other pages. That is, there were multiples of certain products within the page, making product position have less of an effect. Specifically, items in position 12 and 32 were the exact same product. The items in 11, 22, and 31 were also the exact same. The other pages—pencils and paper towels—had no repeat products within the Amazon page. This could be a possible explanation for why position had nearly no effect for toothpaste, while the other products observed some effect. An image of each product page can be seen in Appendix Figure A18 through A20.

In addition, as discussed in the section for Question 3 later in this paper, items 12 and 22 were more frequently clicked than other products on the page. Both of these products had multiple copies on the search result page, so consumers could have clicked on these products regardless of position, as they were deemed some of the most popular. This repetition could have reduced variance across positions, which could have potentially contributed to the lack of significant findings. The results from this regression could also indicate that consumer preferences affected clickthrough rates more than item position. Figure 8 shows the logistic regression results on the data from the paper towel products.

The results as seen in Figure 8 revealed statistically significant effects related to the row position of items. Items in row 2 and 3 had negative coefficients, meaning were significantly less likely to be clicked compared to those in Row 1. Items in Row 2 and Row 3 had odds of being clicked that were approximately 68% and 69% lower, respectively, than items in Row 1 (odds ratios ≈ 0.32 and 0.31), indicating a strong negative effect of lower row positions on clickthrough rates. The 95% confidence interval for the row variables confirms the reliability of their negative effects. For column positions, none of the variables had statistically significant effects.

The results from the logistic regressions show that being in the top row is generally advantageous. It should be noted that our regression results for toothpaste show that the top row does not always receive the most clicks, which could have various explanations discussed further in the Limitations section. Similar to the results from the empirical data about top positions on Google, the products in the top row had a significant advantage over lower positions. Although, the products in the first row did not receive significantly more clicks than other rows, like what was observed for Google.

Question 2: How does the “Sponsored” label on items affect consumer choices?

I found that the “Sponsored” label had essentially no effect on consumer choices.

To answer this question, I applied a binomial test. The objective of a binomial is to test a hypothesis about the proportion of the successes in a population¹⁵. To use this test, I must set a null hypothesis, which is a baseline assumption that is assumed to be true from the start. For example, if a binomial test was used to test if a coin is fair, the null hypothesis would be that the coin lands on heads 50% of the time. The test uses the binomial distribution to determine the probability of observing a certain number of successes in a given sample. The

formula for a binomial distribution is shown in Figure 9, and the specific formula for our experiment is shown in Figure 10.

In our study, I observed whether the number of clicks for sponsored items differed from a standard random probability. Our null hypothesis was that the “Sponsored” label has no effect on consumer choices, meaning that the probability of a consumer clicking a sponsored item is the same as clicking an unsponsored item. The binomial test was performed for each product, with a combined dataset of Treatments 1 and 2. I combined the data from these two treatments since the products in Treatment 1 had a “Sponsored” label while the products in Treatment 2 did not.

Because I combined the data from the two treatments, I was able to control for whether or not a product had a “Sponsored” label. That is, I directly compared the clickthrough rates for the exact same products between the treatments, and analyzed the presence of the “Sponsored” label independent of other factors.

The dataset included total clicks and total number of participants between the two treatments. An image of the pencil dataset can be seen in Appendix Figure A14. The results from the binomial test are as follows:

The test statistic, a Z-score, measures how far the observed proportion of successes deviates from the expected proportion under the null hypothesis, in units of standard deviation. This is useful in determining whether the observed outcomes are unusual enough to reject the null hypothesis. As previously mentioned, our null hypothesis was that the “Sponsored” label has no effect on consumer choices. Generally for negative z-scores, if the value is less than −1.645 (which is a 5% significance level), then it suggests a statistically significant difference from the null hypothesis. The equation for a z-score is shown in Figure 12.

While the z-score related to the “Sponsored” label is not statistically significant at the 5% level, this does not demonstrate that the label has no effect. The confidence interval for all three products lie outside a ±2% range, indicating that the effect may not be negligible. This indicates that the true effect of the Sponsored label could be slightly negative or negligible, but it is not estimated with high precision. Due to this, I can interpret these results as inconclusive. The data does not provide enough information to denote the impact of advertising in this context.

The z-scores for all three products—pencils, toothpaste, and paper towels—were negative, suggesting that the “Sponsored” label could have a negative effect on clickthrough rates. As shown in z-score for pencils is significant as it is more negative than the typical significance

level of -1.645. However, this test statistic is only marginally significant, which suggests that the “Sponsored” label may influence sales negatively, but this is not conclusive.

Overall, I cannot definitively say that the “Sponsored” label has a negative effect as all of the results were not statistically significant enough to reject the null hypothesis. Therefore, more data is needed to conclusively make a claim about whether or not the null hypothesis is true.

Question 3: What are the best products for each item category and are they being advantaged — either by sponsorship or the search engine?

For the observed product categories, certain popular items seemed to be advantaged by the search engine, but not as much by the sponsorship mechanism.

In answering this question, I compared the click likeness for each item between Treatment 3 and a combined dataset of Treatments 1 and 2. First, I totaled the amount of clicks for each item for Treatment 3, and compared these numbers against the dataset of Treatment 1 + 2. I decided to analyze the data from Treatment 3 separate from Treatment 1 + 2 because Treatment 3 had randomized positions while Treatments 1 + 2 did not. Therefore, I can observe which products were most popular when position was controlled for, compared to when it was not. I then computed the clickthrough rate for each item by dividing the totals for each item by the total number of observations. Clickthrough rates were multiplied by 100 to be shown as percentage.

The tables displaying this data are shown in Figures 13 and 43. In the tables, the position code is given by the row number followed by the column number. For example, the position “12” represents the item in row 1 and column 2. For treatment 3 dataset, I am interested in observing the original position, or the position an item would be in if the products were not randomized , which tells us exactly which item the user clicked on. The clicked position in Treatments 1 and 2 tell us the exact item that was clicked. I can compare these positions with the original positions from Treatment 3. To clarify, these position codes represent the exact product through the row and column number, however I am not interested in observing the effect of position in this experiment. The codes simply allow us to compare the popularity of specific items between the two datasets. Popularity is established by identifying products whose clickthrough rates rank within the top 10-15% of items on that page.

For pencils, it was clear that item 14, 15, and 32 were the most popular in Treatment 3. An image of the products themselves can be seen in Appendix Figure A15. These items remained the most popular products when compared to data from Treatments 1+2. The click likeliness was a very similar value for items 15 and 32, but item 14 received 4% more clicks in Treatment 1+2. It is also important to note that the click rates for items 11, 12, 13 (the Sponsored products) increased significantly between Treatment 3 and Treatment 1+2. This indicates that being in the top positions is beneficial, but is not beneficial enough to increase click likeliness to above the most popular products. To answer the question at hand, the top products are not being advantaged by sponsorship. However, item 14—the most popular pencils in all treatments, was given an “Overall Pick” label. This label is given to products by Amazon’s choice, so certain top products are indeed advantaged by the search engine.

For toothpaste, items 23 and 24 were clearly the most popular products in Treatment 3. In Treatment 1+2, their click likeliness decreased significantly, and other products’ rates were boosted. Some less popular items, but still chosen a fair percentage of the time, such as 12, 21, and 22 received a greater percentage of clicks in Treatment 1+2. Item 12 was advantaged by

sponsorship and item 24 received the “Overall Pick” label, indicating that some popular items are advantaged.

For paper towels, the most popular product was item 15 in Treatment 3. Items 11, 12, 21, 22, and 25 had similar clickthrough rates, and were significantly more popular than the other items. In Treatments 1 + 2, these items remained the most popular. However, items 12 and 15 had significantly increased clickthrough rates as compared to the data in Treatment 3. Item 12 was advantaged by sponsorship and item 15 was given the “Overall Pick” label. Therefore, I also observed sponsorship and search engine advantage given to popular products for paper towels.

However, I can observe that being in the first row has a positive effect on organic search results. This finding of organic products being more popular than sponsored products aligns with information from previous literature. This also expands on our results from Question 1, which was that being in the top row has a positive effect on position. These observations will be further examined in the Discussion section.

Discussion

Implication of Results

These results of this study suggest that both row and column position significantly influence consumer behavior, with positions beyond the first row and first 2 columns having a negative effect on clickthrough rate. Logistic regression was applied to randomized layouts to confirm that products in Row 1 had a higher click likelihood compared to those in Rows 2 and 3.

Similarly, items in Columns 3 through 5 performed worse relative to Columns 1 and 2. These findings show that having a more prominent position on a page plays a key role in shaping consumer attention.

Our findings on the effects of the “Sponsored” label on consumer behavior were inconclusive. I used binomial testing to compare clickthrough rates of identical products with and without sponsorship labels and the results failed to reach statistical significance. Although the z-score for pencils was nearly marginally significant, sample size limitations and variability could have obscured subtle effects. Future studies with greater statistical power could be needed to more precisely determine how the “Sponsored” label affects consumer decision-making.

Our analysis of product-level clickthrough rates across treatments suggests that a certain subset of products for each category was inherently more popular than others. Certain items consistently received higher consumer engagement in Treatment 3, which randomized product positions.

These items were among the top performing items in Treatment 1 and 2, despite changes in position. Interestingly, many of these items were labeled “Overall Pick.” The “Overall Pick”

label is chosen based on consumer preferences, marked by strong ratings, frequent purchases, and infrequent returns¹⁶, which may have increased consumer trust. In contrast, the “Sponsored” label did not yield the same result, suggesting that users responded more positively to algorithm-based endorsements than paid promotions.

While position and label effects were generally consistent across product categories, the toothpaste products deviated from this pattern. In contrast to pencil and paper towel logistic regression results, the toothpaste data showed that most position variables were not statistically significant. A likely explanation for this is that toothpaste products appeared more than once on the results page. For example, identical items were shown in multiple rows or columns which allowed the participants to encounter the same product regardless of position. This repetition may have weakened the influence of position by increasing familiarity or perceived popularity. As a result, click behavior in the toothpaste category may have been affected more by product recognition rather than position or labeling.

The limited impact of the “Sponsored” label observed in this study raises broader questions about trust and transparency in platform architecture. Although platforms like Amazon follow FTC disclosure requirements, our findings suggest that these labels may lack the visual or cognitive salience needed to influence consumer behavior. In contrast, internal signals like the “Overall Pick” designation appeared to resonate more with users. As algorithmic trust signals become more influential than traditional advertising cues, regulators and designers alike may need to revisit how disclosure mechanisms are implemented in the evolving digital landscape.

Note:

These results suggest that both row and column position significantly influence consumer behavior, with positions beyond the first row and first 2 columns having a negative effect on clickthrough rate. As position was controlled for in this treatment, this effect could have been due to consumer preferences towards the properties of the products in Column 2. However, I am not able to observe the effect of the Column 2 products since the products on the webpage were completely randomized. I can conclude from the logistic regression results that being in the top row on the webpage is advantageous for an item. In general, products in Row 2 and Row 3 were less likely to be clicked compared to those in Row 1, with the effect being more pronounced for Row 3. However, being in the top left position is not advantageous across all products, although it appeared to be advantageous for the pencils webpage. These findings can be related to federal policy regarding the sponsorship label. As previously mentioned, the Federal Trade Commission mandates that sponsored content be labeled to inform consumers about paid advertisements, such as the “Sponsored” labels on Amazon. The purpose of this policy is to prevent search engines from misleading consumers. As I observed, these “Sponsored” labels are not influencing consumer behavior. This raises questions about the labels’ visibility and effectiveness, as they are not influencing consumer behavior as intended. Policymakers might use this data to reconsider

the current disclosure standards, potentially developing more prominent labeling requirements to ensure that consumers can clearly identify which products are sponsored. Overall, I can conclude that certain popular products are being advantaged by the search engine through the “Overall Pick” label, but most of the popular products are not being advantaged by the sponsorship label. The “Overall Pick” label is chosen based on consumer preferences, marked by good ratings, often purchases, and infrequent returns¹⁶. Amazon boosts the visibility of this item by marking it with a brightly colored label, counterbalancing the prominence given to potentially less relevant sponsored items by highlighting this highly relevant “Overall Pick” product. These results show that the sponsored products were often not the products with the highest CTR. The weaker position effects for toothpaste may have been due to repetition of popular items, which diluted the impact of position. Although higher positions are beneficial, the positional advantage has a more pronounced effect on organic results compared to sponsored ones.

Limitations

While this study provides evidence about how sponsorship influences consumer click behavior on Amazon, there are several limitations that may affect how broadly these results can be applied. First, the experiment was conducted using simulated Amazon pages rather than the live platform. Although I replicated the interface and user experience in our experiment, there were some differences our experiment had from the actual platform. A main difference was that users did not actually purchase the products they selected, so the lack of purchasing intent could have affected our results.

Moreover, like previously mentioned, the results of the logistic regression for toothpaste in answering Question 1 could have been skewed. Since there were multiple of the same products in different positions, participants may have been more likely to select a familiar item regardless of where it appeared on the page. This repetition reduced the variation in responses tied to specific locations, making it harder to isolate the effect of position alone. As a result, the product placement impact may have been understated for this category compared to pencils and paper towels.

Another limitation of our study is due to our limited sample size. With only 221 participants spread across three treatments the statistical power of our tests is constrained. This makes it harder to detect smaller effects in the data. In addition, the age distribution in particular was uneven, which could have skewed results to align with behaviors of younger people (ages 0-24). A larger sample would allow for more precise estimates and ability to reasonably weight responses based on demographic groups, which cannot be accurately done currently due to the small sample size.

Conclusion

This paper analyzes the impact of Amazon’s sponsorship mechanism on consumer clickthrough rates, explicitly focusing on the effect of top positions and the “Sponsored” label. Our study was focused on three central questions: (1) Is being in the top position advantageous? (2) Does the “Sponsored” label influence consumer behavior? (3) Are popular products advantaged by Amazon’s search algorithm or the sponsorship mechanism?

Our results found that a product’s position was a significant factor in clickthrough rate. This was especially true for pencils and paper towels, where products in Row 3 had a substantially lower clickthrough rate than those in Row 1. On the other hand, the toothpaste data showed a weaker correlation, potentially due to duplicate items on the page. While position seemed to affect clickthrough rate, the “Sponsored” label had no statistically significant effect on consumer behavior, with a binomial test for pencils showing marginal results (z = −1.803, p ≈ 0.07). Users appeared to be more affected by product popularity, particularly items labeled “Overall Pick.”

These results have broader implications in consumer behavior, where trust, prior knowledge, and perceived quality often overshadow advertising. The lack of a statistically significant effect for the “Sponsored” label suggests that users may subconsciously discount paid content, especially when other labels like “Overall Pick” are present. This reinforces prior literature showing that labeling and position influence consumer expectations. Our findings suggest that shoppers are influenced more by perceived credibility and trust-based indicators rather than by paid promotion.

Despite the insights gained, this study had many limitations. The sample size (n = 221) was relatively small and uneven across treatments, which can limit generalizability. I recruited participants through online platforms and social media, which could have been a factor in skewing the data to be younger, especially in Treatments 2 and 3. In addition, participants viewed product categories in the same order, which could have introduced sequence effects. Due to time and resource constraints, I was unable to incorporate mixed-effect models or conduct a power analysis. These constraints should be considered when interpreting the results.

Future research could explore additional factors, such as brand preferences and pricing dynamics, to build a more comprehensive understanding of consumer behavior on Amazon. Looking into other variables like customer ratings, shipping speed, and product imagery could provide a deeper insight into what factors affect clickthrough rates. Researchers could also narrow the research to specific demographic groups such as age or gender to find if different sets of groups interpret platform design elements like position and labeling differently. By incorporating these variables, future work can offer a deeper understanding of how users navigate digital marketplaces.

Appendix

Appendix A: Additional Experiment Design Specifications

References

1. Hagiu, A., & Jullien, B. (2011). Why do intermediaries divert search? Harvard Business School. https://doi.org/10.1111/j.1756-2171.2011.00136.x [↩]
2. Eliaz, K., & Spiegler, R. (2011). A SIMPLE MODEL OF SEARCH ENGINE PRICING. The Economic Journal, 121(556), F329–F339. http://www.jstor.org/stable/41301346 [↩]
3. Dash, A., Chakraborty, A., Ghosh, S., Mukherjee, A., Gummadi, K. (2021) . When the Umpire is also a Player: Bias in Private Label Product Recommendations on E-commerce Marketplaces. ACM Conference on Fairness, Accountability, and Transparency.

   https://doi.org/10.48550/arXiv.2102.00141 [↩]

4. Athey, S., & Ellison, G. (2011). Position auctions with consumer search. The Quarterly Journal of Economics, 126(3), 1213-1270. https://doi.org/10.1093/qje/qjr031 [↩]
5. Chen, M., Nabi, S., & Siniscalchi, M. (2023). Advancing ad auction realism: Practical insights & modeling implications. KDD 2023 Workshop on Artificial Intelligence for Computational Advertising (AdKDD). https://doi.org/10.48550/arXiv.2307.11732 [↩] [↩]
6. Jansen, B. J., Brown, A., Resnick, M. (2007). Factors relating to the decision to click on a sponsored link Decision Support Systems, 44(1), 46–59. https://doi.org/10.1016/j.dss.2007.02.009 [↩]
7. Beel, J., Langer, S., Genzmehr, M. (2013). Sponsored vs. Organic (Research Paper) Recommendations and the Impact of Labeling. Research and Advanced Technology for Digital Libraries (TPDL 2013).https://doi.org/10.1007/978-3-642-40501-3_44 [↩]
8. Jeziorski, P., Segal, I. (2015). What makes them click: empirical analysis of consumer demand for search advertising. American Economic Journal: Microeconomics, August 2015, Vol. 7, No. 3 (August 2015), pp. 24-53. https://www.jstor.org/stable/24467001 [↩]
9. Craswell, N., Zoeter, O., Taylor, M., Ramsey, B. (2008). An experimental comparison of click position-bias models. Proceedings of the 2008 International Conference on Web Search and Data Mining. https://doi.org/10.1145/1341531.1341545 [↩]
10. Baye, M. R., De los Santos, B., & Wildenbeest, M. R. (2015). Search engine optimization: What drives organic traffic to retail sites? Journal of Economics & Management Strategy, 25(1), 6–31. https://doi.org/10.1111/jems.12141 [↩]
11. Shastry, S. Venkatesh. Consumer Attitude and Their Purchase Intention: A Review of Consumer Attitude and Their Purchase Intention: A Review of Literature Literature. International Review of Business and Economics, November 2021, Vol. 5, Issue 2 (November 2021), p. 3, digitalcommons.du.edu/cgi/viewcontent.cgi?article=1148&context=irbe. [↩]
12. Bailyn, E. (2024). Google Click-Through Rates (CTRs) by Ranking Position in 2024. First Page Sage. https://firstpagesage.com/reports/google-click-through-rates-ctrs-by-ranking-position/ [↩] [↩]
13. Dean, B. (2024). Google CTR Stats. Backlinko. https://backlinko.com/google-ctr-stats Edelman, B., & Schwartz, M. (2007). Internet advertising and the generalized second-price auction: Selling billions of dollars worth of keywords. American Economic Review, ISSN 1944-7981. https://www.aeaweb.org/articles?id=10.1257/aer.97.1.242 [↩]
14. Jansen, B. J., Brown, A., Resnick, M. (2007). Factors relating to the decision to click on a sponsored link Decision Support Systems, 44(1), 46–59. https://doi.org/10.1016/j.dss.2007.02.009; [↩]
15. Jansen, B. J., Brown, A., Resnick, M. (2007). Factors relating to the decision to click on a sponsored link Decision Support Systems, 44(1), 46–59. https://doi.org/10.1016/j.dss.2007.02.009; [↩]
16. Amazon. (2024). Amazon’s Choice FAQs. https://www.amazon.com/b?ie=UTF8&node=21449952011 [↩] [↩]