How Hacker News ranking works: scoring, controversy, and penalties (2013)
Recorded: Sept. 19, 2026, 11 p.m.
| Original | Summarized |
How Hacker News ranking really works: scoring, controversy, and penalties Ken Shirriff's blog Computer history, restoring vintage computers, IC reverse engineering, and whatever How Hacker News ranking really works: scoring, controversy, and penalties The basic formula for Hacker News ranking has been known for years, but questions remained. By carefully analyzing the top 60 HN stories for several days, I can answer those questions and more. How ranking works Because the time has a larger exponent than the votes, an article's score will eventually drop to zero, so nothing stays on the front page too long. This exponent is known as gravity. You might expect that every time you visit Hacker News, the stories are scored by the above formula and sorted to determine their rankings. But for efficiency, stories are individually reranked only occasionally. When a story is upvoted, it is reranked and moved up or down the list to its appropriate spot, leaving the other stories unchanged. Thus, the amount of reranking is significantly reduced. There is, however, the possibility that a story stops getting votes and ends up stuck in a high position. To avoid this, every 30 seconds one of the top 50 stories is randomly selected and reranked. The consequence is that a story may be "wrongly" ranked for many minutes if it isn't getting votes. Raw scores and the #1 spot on a typical day This chart shows a few interesting things. The score for an article shoots up rapidly and then slowly drops over many hours. The scoring formula accounts for much of this: an article getting a constant rate of votes will peak quickly and then gradually descend. But the observed peak is even faster - this is because articles tend to get a lot of votes in the first hour or two, and then the voting rate drops off. Combining these two factors yields the steep curves shown. There are a few articles each day that score much above the rest, along with a lot of articles in the middle. Some articles score very well but are unlucky and get stuck behind a more popular article. Other articles hit #1 briefly, between the fall of one and the climb of another. Looking at the difference between the article with the top raw score (top of the graph) and the top-ranked article (red line), you can see when penalties have been applied. The green triangles and text show where "controversy" penalties were applied. The blue triangles and text show where articles were penalized into oblivion, dropping off the top 60. Milder penalties are not shown here. It's clear that the content of the #1 spot on HN isn't "natural", but results from the constant application of penalties to many articles. It's unclear if these penalties result from HN administrators or from flagged articles. Submissions that get automatically penalized One interesting theory by eterm is that news from popular sources gets submitted in parallel by multiple people resulting in more upvotes than the article "merits". Automatically penalizing popular websites would help counteract this effect. The impact of penalties Controversy My methodology 2.802 Pyret: A new programming language from the creators of Racket Note that three of the top 10 articles are ranked lower than expected from their score: The NY Times, Marelle and ISO 1. Since The NY Times is ranked between articles with 1.407 and 0.785, its penalty factor can be computed as between .47 and .85. Likewise, the other penalties must be .87 to .93, and .60 to .82. This technique shows the existence of a penalty and gives a range for the penalty, but determining the exact penalty is difficult. You can look at the range over time and hope that it converges to a single value. However, several sources of error mess this up. First, the neighboring articles may also have penalties applied, or be scored differently (e.g. job postings). Second, because articles are not constantly reranked, an article may be out of place temporarily. Third, the penalty on an article may change over time. Fourth, the reported vote count may differ from the actual vote count because "bad" votes get suppressed. The result is that I've been able to determine approximate penalties, but there is a fair bit of numerical instability. Penalties over a day On average, about 20% of the articles on the front page have been penalized, while 38% of the articles on the second page have been penalized. (The front page rate is lower since penalized articles are less likely to be on the front page, kind of by definition.) There is a lot more penalization going on than you might expect. Here's a list of the articles on the front page on 11/11 that were penalized. (This excludes articles that would have been there if they weren't penalized.) This list is much longer than I expected; scroll for the full list. Why the Climate Corporation Sold Itself to Monsanto, The code for the scoring formula (def frontpage-rank (s (o scorefn realscore) (o gravity gravity*)) In case you don't read Arc code, the above snippet defines several constants: gravity* = 1.8, timebase* = 120 (minutes), etc. It then defines a method frontpage-rank that ranks a story s based on its upvotes (realscore) and age in minutes (item-age). The next factor hits an article flagged as a gag (joke) with a heavy value of .1, and a "lightweight" article with a factor of .17. The actual penalty system appears to be much more complex than what appears in the published code. Conclusion Update (11/18): article on penalties is penalized Email ThisBlogThis!Share to XShare to FacebookShare to Pinterest Labels: 15 comments: Anonymous Nice analysis! You should check for stories that involve YC companies and/or their competitors. I've often thought that HN gives unfair advantage to stories about YC companies, beyond just the normal echo-chamber effect. November 18, 2013 at 2:56 PM Fabien Would you mind posting a link to the corresponding HN discussion, as it's burried and searching in HN is impractical at best? November 19, 2013 at 4:18 AM Cd-MaN Thank you! Very nice analysis! November 19, 2013 at 6:40 AM Ken Shirriff Fabien: the discussion on HN is here and Reddit has some discussion here. The Reddit discussion has some interesting links. November 20, 2013 at 7:41 AM Unknown I'll doing a postmortem of my article and I would be really amazed to see the graph my article i posted yesterday made on HN (What if successful startups are just lucky?).Does your crawler still running?I wonder if I had some penalties and what the graph looks like. November 26, 2013 at 2:11 AM HN Reader and Poster Awesome post! I'd love to see more about the voting ring detection penalty. At this point, every one of my posts that makes the front page gets penalized. According to PG this is due to voting ring detection. I'm certainly not organizing any voting rings. I believe this may be another inadvertent type of penalty for popular domains -- having too many friends that upvote you and set off "voting ring detection". It's a bummer because I put a lot of time in the content and truly think it is good content. The end result is that I "set it and forget it" on Hacker News. Trying to engage there just leads to frustration when the comment thread suddenly drops from the front page. Another observation - my posts on gender (which I no longer write about due to the personal risk) got the "flamewar" penalty, even though they were honest, noncontroversial pieces generating some really good discussion. Apparently it was too much discussion. The algorithm probably protects us from a lot of junk but it also hurts sometimes too. November 26, 2013 at 10:24 AM Ken Shirriff Vianney Lecroart: unfortunately I'm note running my crawler any more, so I don't have data for your article.HN Reader: I'd like to know more about the voting ring detection too. Apparently that's what nailed my article. Like you, I'm definitely don't have any voting ring, so I don't know why I got hit by the detector. November 26, 2013 at 11:38 AM Unknown I was wondering why we* dropped off the front page so quickly... Hm.Thanks for putting this together.* we = Prime. We had the "Get your health records from any doctor" post. November 26, 2013 at 12:32 PM Rohit This article has got 705 point on HackerNews, quite an amazing feat. How Hacker News ranking really works: scoring, controversy, and penalties (righto.com)705 points by jseip 1 day ago | flag | 156 commentsAny guess, How much traffic it would be? I guess, over 100K page views?? November 27, 2013 at 12:40 AM Ken Shirriff Hi Rohit! I received about 25K page views. November 27, 2013 at 10:46 AM Anonymous Very interesting! Two questions..1) Any way to tell if different accounts' votes are valued differently based on karma points, age etc.?2) What about if accounts' can be penalized rather then just certain sites? Not just being hellbanned. January 22, 2014 at 5:36 PM JavaUser @Anonymous, I think they should be, more trusted votes are counted more, If I am not wrong. Simple example, when a new account publish or vote its not reflected immediately. October 3, 2015 at 6:11 PM Robert Welain This comment has been removed by the author. June 22, 2017 at 8:18 AM Suresh Dasari Thanks for the article. Nice analysis. May 24, 2020 at 11:11 PM Shahzad Aslam very good explanation and nice analysis. Thanks for the article. February 23, 2024 at 9:23 AM Post a Comment Newer Post Older Post Home Subscribe Contact info and site index Popular Posts Cores in space: The core memory module from a 1980 Spacelab computer A Multi-Protocol Infrared Remote Library for the Arduino Microcode in Intel's 8087 floating-point chip: the scale instruction Apple iPhone charger teardown: quality in a tiny expensive package A dozen USB chargers in the lab: Apple is very good, but not quite the best Teardown and exploration of Apple's Magsafe connector Mining Bitcoin with pencil and paper: 0.67 hashes per day An Arduino universal remote: record and playback IR signals Search This Blog Labels 386 6502 8008 8085 8086 8087 8088 aerospace alto analog Apollo apple arc arduino arm beaglebone bitcoin c# cadc calculator chips css datapoint dx7 electronics f# fairchild fpga fractals genome globus haskell HP html5 ibm ibm1401 ibm360 intel ipv6 ir java javascript math microcode oscilloscope Pentium photo power supply random reverse-engineering sheevaplug snark space spanish synth teardown theory unicode Z-80 Blog Archive ► 2026 (12) ► September (1) ► August (1) ► July (1) ► June (3) ► May (2) ► April (1) ► March (1) ► February (1) ► January (1) ► 2025 (22) ► December (2) ► November (1) ► October (1) ► September (1) ► August (4) ► July (1) ► June (1) ► May (2) ► April (1) ► March (3) ► February (1) ► January (4) ► 2024 (21) ► December (1) ► November (1) ► October (1) ► September (3) ► August (2) ► July (2) ► June (1) ► May (1) ► April (1) ► March (2) ► February (3) ► January (3) ► 2023 (35) ► December (4) ► November (2) ► October (3) ► September (1) ► August (2) ► July (3) ► May (1) ► April (2) ► March (4) ► February (5) ► January (8) ► 2022 (18) ► November (3) ► August (1) ► July (1) ► June (1) ► May (1) ► April (4) ► March (2) ► February (3) ► January (2) ► 2021 (26) ► December (4) ► November (2) ► September (1) ► August (1) ► July (2) ► June (2) ► May (1) ► April (2) ► March (4) ► February (4) ► January (3) ► 2020 (33) ► December (2) ► November (3) ► October (2) ► September (4) ► August (5) ► July (2) ► June (3) ► May (4) ► April (2) ► March (5) ► January (1) ► 2019 (18) ► November (3) ► October (2) ► September (3) ► August (1) ► July (4) ► April (2) ► February (1) ► January (2) ► 2018 (17) ► December (1) ► September (4) ► August (1) ► June (1) ► May (1) ► April (1) ► March (3) ► February (1) ► January (4) ► 2017 (21) ► December (5) ► November (2) ► October (3) ► August (1) ► July (2) ► June (2) ► April (2) ► March (2) ► February (1) ► January (1) ► 2016 (34) ► December (2) ► October (5) ► September (8) ► August (2) ► July (3) ► June (4) ► May (1) ► April (1) ► March (1) ► February (4) ► January (3) ► 2015 (12) ► December (2) ► November (1) ► October (3) ► August (1) ► May (2) ► March (2) ► February (1) ► 2014 (13) ► December (1) ► October (1) ► September (3) ► May (2) ► March (1) ► February (5) ▼ 2013 (24) ▼ November (2) How Hacker News ranking really works: scoring, con... ► September (4) ► August (4) ► July (4) ► June (2) ► April (1) ► March (2) ► February (2) ► January (3) ► 2012 (10) ► December (1) ► November (5) ► October (1) ► May (1) ► March (1) ► February (1) ► 2011 (11) ► December (2) ► July (2) ► May (2) ► April (1) ► March (1) ► February (3) ► 2010 (22) ► December (2) ► November (4) ► October (3) ► August (1) ► June (1) ► May (2) ► April (3) ► March (4) ► January (2) ► 2009 (22) ► December (2) ► November (5) ► September (1) ► August (3) ► July (1) ► June (3) ► April (1) ► March (3) ► February (2) ► January (1) ► 2008 (27) ► July (3) ► June (1) ► May (3) ► April (4) ► March (10) ► February (6) Don't miss a post!Subscribe to get updates by email.SubscribeBuilt with Kit Powered by Blogger. |
The fundamental structure of the Hacker News ranking system involves a complex scoring mechanism that extends beyond simple upvotes, incorporating temporal factors and various penalty structures. The author investigated whether the published algorithm accurately reflects the observed rankings, revealing that external factors and penalties significantly shape article placement. The core scoring is based on an upvote score and the time elapsed since submission. This system incorporates a concept termed gravity, where the time component is raised to a larger exponent than the vote count, ensuring that scores naturally descend over time, preventing any single article from remaining on the front page indefinitely. Although stories are generally scored by this formula upon page load, the system employs occasional, randomized reranking—selecting one of the top fifty stories every thirty seconds—to maintain dynamic sorting. Furthermore, pages are cached for ninety seconds to manage efficiency. The observed scoring dynamics demonstrate that articles tend to accumulate votes rapidly in the initial hours, followed by a gradual decline. This pattern is accounted for by the scoring formula, which balances the rate of voting accumulation against the passage of time. The analysis of raw scores versus actual rankings indicated that numerous articles were consistently ranked lower than their calculated scores suggested, strongly implying the presence of a substantial penalty system affecting visibility. Penalties are applied based on several criteria, including content and domain. Automatic penalties are levied on articles if their titles contain specific terms, such as "NSA," and various popular websites, such as those associated with technology or news publications, are subjected to penalties. The author theorizes that automatically penalizing popular sources helps counteract the effect of parallel submissions resulting from multiple upvotes. A significant mechanism for reducing scores is the controversy factor, which is designed to mitigate flamewars by penalizing articles that accumulate a large number of comments relative to their upvotes. While the published code suggests a formula based on the ratio of votes to comments squared, the author suspects the actual exponent might be three, which allows the controversy penalty to cause sudden and severe drops in ranking when an article reaches a certain threshold of comments. This mechanism is responsible for dramatic shifts where a popular story can vanish from the front page upon hitting a specific comment count. The impact of these penalties is computationally significant; a penalty factor of 0.4, for instance, effectively reduces each vote to 0.3 votes or accelerates the ranking drop by 66 percent. The analysis suggests that most observed deviations from expected ranking are attributable to these penalties, rather than solely the raw vote count. The process of determining the exact penalty factor is complicated by potential numerical instability, as neighboring articles may also be affected, and a lack of constant reranking means temporary misplacement can occur. In summary, the ranking on Hacker News is not a pure meritocracy determined only by upvotes. While upvotes are the primary driver of scoring, the visibility and prolonged presence of an article are heavily modulated by a complex, often unseen penalty system related to content controversy and domain popularity. This system functions to manage community behavior, resulting in a dynamic and often counterintuitive distribution of content on the platform. |