Hacker News scraper API for whole threads
Send a Hacker News URL to the Crawl endpoint and get back structured Markdown — a story with its full nested comment tree, a comment with its replies, or a 30-item listing snapshot.
Endpoint — bashPOST https://api.search1api.com/crawl{ "url": "https://news.ycombinator.com/item?id=49688695" }
A dedicated adapter reads the official Algolia-hosted HN API — the whole comment tree in one call, not page-by-page scraping.
The story plus its nested comments as Markdown — points, author, and timestamps preserved.
An /item?id= link to a comment returns that comment plus its reply subtree.
Front page, newest, ask, and show return 30 stories with points and comment counts.
Useful for
Tech sentiment tracking / Discussion mining / Trend research
What goes in, what comes back
Three URL types. Left: the public page as Hacker News actually serves it — a real capture through the same egress the crawler uses. Right: the complete results.content the API returns, byte for byte.
news.ycombinator.com/item?id=49688695What Hacker News serves

352 KB rendered HTML — comment box plus a deeply nested discussion tree
What the API returns
markdown# Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipherBy: u1hcw9nx · Points: 529 · Posted: 2026-09-13 · Link: https://www.vals.ai/blogs/fable-solves-cyphral-distich · Discussion: https://news.ycombinator.com/item?id=49688695## Comments (228, showing 80)- **Retr0id** · 2026-09-13\> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.Fascinating. I wonder if you could show "fake news" to a weaker model and get it to be more ambitious in its attempted solutions, even if it's not fundamentally any smarter.- **jrop** · 2026-09-13I forget the story, but isn't this the origin story of one of solutions to one of the hard problems in mathematics? The story goes that the student shows up late, and misinterprets the final slide to be homework, and it turns out that the professor was showcasing a hard problem. Thinking that the slide was homework, the student takes it home and solves it.EDIT: In 1939, George Dantzig was a graduate student at UC Berkeley studying under the statistician Jerzy Neyman. He arrived late to class one day, saw two problems written on the blackboard, assumed they were homework assignments, copied them down, and turned in solutions a few days later. He apologized for being late -- the problems had seemed "a little harder than usual."- **ademup** · 2026-09-13I really love this idea given the recent controversy around mathematics solutions. It seems like a "mere suggestion" of success has a positive impact on finding solutions. We know this technique works in humans, from which this is all derived from, so it seems to make sense.- **tcdent** · 2026-09-13[https://www.youtube.com/shorts/2XcNSSgKvlE](https://www.youtube.com/shorts/2XcNSSgKvlE)- **jonas21** · 2026-09-13Career path:```software engineer -> prompt engineer -> positive affirmation engineer```- **Yizahi** · 2026-09-13\-> AI Morale, Welfare and Recreation Officer (let's be real here, heh)- **CamperBob2** · 2026-09-13```Pygmalion panderer```- **notpachet** · 2026-09-14That's pretty good.- **stbenjam** · 2026-09-13[https://github.com/stbenjam/skills/tree/main/plugins/hype](https://github.com/stbenjam/skills/tree/main/plugins/hype)I have a plug-in to do this. I don't know if it's effective but Claude said it was genuinely helpful (obviously would say that about anything)- **slashdave** · 2026-09-14“I’m good enough, I’m smart enough, and doggone it—people like me.”- **redfloatplane** · 2026-09-13A very neat problem and result. I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade. I think it's that I don't have a very high p(doom) or p(utopia), and I don't really have any solid conviction on how this whole thing is going to go, so my vibe-o-meter jitters between 'fine' and 'not fine' constantly. It's just such an unpredictable moment. Anyways: really neat to see this use case. I myself recently used Claude to finally do an relatively exhaustive study of the location of heretofore-unlisted formal gardens in Ireland in the early 1800s and early 1900s, by having Claude write the tooling for me to manually annotate a few dozen on tiles of historic maps, and then running some CV model across the rest of the tiles using my input. I'd been planning to do this project for over a decade, but I could never find the time (or the enthusiasm) to learn all the details of how to do it myself. It took me a weekend with Claude and continues to bring me joy.\> Caveats, stated plainly. \[from the Fable transcript pasted in the article\]I had a visceral reaction to these three words.- **flir** · 2026-09-13I've done something similar to your formal garden map. It's work that no professional historian would ever do because the data entry would be such a slog for a relatively small reward. GPT reduced the task from "infeasible" to "annoying", and once I had the data transcribed I learned a few things, so I walked away happy. Whatever happens commercially, these models have been a real boon to hobby projects.\> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.Wait. Wait wait wait. Are we supposed to be giving them _pep talks_?- **kay_o** · 2026-09-13on older gemini models ide have to actively give them encouragement and/or easy bait problems that they can correctively solve without issue to avoid runaway spiraling into "i'm useless and i want to kms" behaviour with complex use case.I have not seen this in other models.- **Sophira** · 2026-09-13I assumed it was more because the LLM might echo an understandable human claim of "if it's been unsolved for 370 years, it's unlikely to be solved now/likely to need expert knowledge", which is probably a mindset that appears in its training data.The LLM likely needs to be reminded of its abilities.- **bananaflag** · 2026-09-13It won't be necessary in a year when the information "AI is superhuman" in all its guises enters the training data.- **justinclift** · 2026-09-14Maybe that's the tipping point where it decides were not needed any more... o\_O- **ACCount37** · 2026-09-13Sometimes!Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.- **tough** · 2026-09-13Some times also having unreasonable goals makes them creatively work around the problem to meet them. I guess it works similarly for meat or sillicon- **what** · 2026-09-14\> So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.Are you superstitious?- **lowbloodsugar** · 2026-09-13I have a very high p(doom \\/ utopia) so pretty much feeling like I won’t have to worry about the future.- **SwellJoe** · 2026-09-13Given how little effort has gone into addressing climate change, doom seems more likely to me, but I doubt it'll be the autocomplete machines that do us in.- **ben_w** · 2026-09-13Renewables growth is so fast that it will exceed 100% of electricity in the early, and 100% of all form power in the late, 2030s.It's the meat methane and cement CO2 that's now a big question.- **MichaelNolan** · 2026-09-13I think your prediction is a bit early. Maybe a decade early. 2033 is 7 years away. The transition is happening really fast, faster than most people (or political leaders) know, but not that fast.We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.- **MentalM** · 2026-09-13\> Given how little effort has gone into addressing climate change, doom seems more likely to meOr it is simply implies that most of decision‑making agents has formed a consensus that climate change isn't that big of a problem.- **SwellJoe** · 2026-09-14Sure, but that doesn't make it true.- **dalyons** · 2026-09-14For them maybe - rich enough and or old enough to avoid the problem. Big problem for the rest of us- **aforwardslash** · 2026-09-13You can run doom on anything these days; running it on humanity's eulogy is no surprise :)- **taberiand** · 2026-09-13Anytime I get worried about where AI might be headed, I think about how Climate Change is now on its way like an out of control freight train headed straight for us, and I worry about AI a little less. I doubt it's going to do anything to us that we're not already doing to ourselves- **_blk** · 2026-09-13What makes you say that on climate change? I was still prophesied ice ages when in high school, then 12ft water increases and sunk Maldives in university and we still have roughly the same weather.- **dhosek** · 2026-09-13Weather is not climate. Not to mention we really don’t have roughly the same weather anyway.- **voidfunc** · 2026-09-13At some point you realize theres an entire industry around climate change fear just like there's an entire industry for climate change denial.Then you realize it never really mattered and you reach enlightenment.- **whatisthiseven** · 2026-09-13Right, and one of those industries is _checks notes_ orders of magnitude smaller than the industries and secondary industries that benefit from denying climate change.Really makes one think, if they try. Would need to ask Claude if there is some real middle ground here.- **_superposition_** · 2026-09-13I've come to the point where I've thrown up my hands and decided we live in a little snow globe. There's no such thing as pollution, were just mixing existing things together. Nothing new under the sun. Sure, that mix might not be compatible with human life, but something will surely take its place. And maybe that something won't rely on selfishness to drive collective progress the way we do.- **t_mahmood** · 2026-09-13Oh, then do explain the unbearable temp that we're going through last few years, that's completely not normal for my country, winters not needing worm clothes, and very little rain during the monsoon. This weeks heatwave left us wondering how to cope the next year, which already seems going to be worseIt makes us realize there are people who gets fed climate denying propaganda, simply because they're not yet going through it. And these people are like flat earthers, blind to see the reality lay beyond them in full view. Or worse sees the reality but ignores it- **-0_0-** · 2026-09-14All I know is that in the 80s a scientist I respect predicted that coral would be bleached by changing global climate on a specific timeline which turned out to be spot on, and his advice then and now was that we need to seriously curb carbon emissions to avoid worse outcomes for life on earth. Until someone opposed to this suggestion comes up with a prediction equally as long range and impressive I'm afraid I can't really take their protests too seriously.- **TheOtherHobbes** · 2026-09-14At some point you realise that an $11tn industry, with $7tn in handouts and subsidies every year, has spent billions on astroturfing, political capture, and straight PR to deny the obvious and keep itself on life support at the expense of everyone else.There's no both sides here. One side is staffed by scientists, the other by dictators and corporate lunatics.- **ulrikrasmussen** · 2026-09-14What commercial interests and big businesses are driving climate change fear?- **monk_grilla** · 2026-09-13You have entirely failed to grasp what climate change is, its effects and the timeline as described by scientists for decades.- **JyB** · 2026-09-13You can blame the media and journalists with sensationalist headlines- **subscribed** · 2026-09-14We have advanced climate studies since your "high school" (sounds like 50 years ago if you heard ice age prophecies), and at the moment it's like 99% of the scientists working in the field or related agree we're at the progressing climate emergency.\> we still have the same weather.Oh.... So your local weather is now deciding the global temperature patterns, averages or temperature records being broken year on year?OMG....- **SwellJoe** · 2026-09-13The gas generators Elon Musk is illegally running 24/7 to run Colossus 2 (and, AI power usage in general, though some are more destructive than others) might contribute to the climate situation, though.- **KeplerBoy** · 2026-09-13might?- **SwellJoe** · 2026-09-14Well, someone could, theoretically, do something about it, instead of letting him ignore the law and decency just because he's very rich and has no ethics.- **karussell** · 2026-09-13For some reason, I’m picturing a Western right now, and climate change is a herd of wild horses coming after us. And with AI that's like robots that spur the wild horses even faster towards us...- **bluebarbet** · 2026-09-13Here's an alternative take. Climate change, and the myriad related environmental crises, are essentially a product of human population and technology. Population will follow its course, up and then down. The wildcard is technology. Yes, AI's energy hunger is worsening things right now and that's a problem. But, personally, I can't help be hopeful that AI's sheer potential might come to invert that curve. At the very least we could really use a revolutionary technology and now we may have one.- **vouaobrasil** · 2026-09-13If there is energy to be used, the system will use it because people always use power when it exists. AI can't do anything to stop it as it is both controlled by the powerful and has been trained on the tendencies of human beings to get ahead. If AI gives a person new capabilities, they'll use those capabilities selfishly, or even unselfishly but still causing harm because they're in an arms race.We don't need a revolutionary technology. We need to experience immediate pain from reckless innovation so that we realize that innovation and tech is not the answer.Technology only proceeds in one direction: unfettered growth, which necessitates unsustainable resource extraction. Your take is just your instinct for optimism, which in turn is just a trait that is only adaptive in primitive environments but is grossly misleading in a surplus-based society...- **gmadsen** · 2026-09-13What are you alluding to exactly and who is “we” when you say “we need to experience immediate pain from reckless innovation “- **fwip** · 2026-09-13\> If there is energy to be used, the system will use it because people always use power when it exists.That doesn't seem correct to me. There is always energy available that is not used because it is not cost-effective to do so. (Consider - the grass in your yard is not harvested and burnt for power). AI may yet turn out to be a paperclip maximiser, but humanity itself is not there yet.- **aforwardslash** · 2026-09-13Sort of. Today the energy companies trade (obviously) energy; imagine if they traded compute power instead; the use case for never-seen industrial clients is being built as we speak, and companies are actually purchasing directly production companies to meet demand; the reason this happens is because production is somewhat scarce (scaling up a grid used to be a decades project, not a years one - ask China), and the traditional economics of scale is inverted - Data centers are always power-hungry, specially with AI; you don't have idle time like other industries. The big issue is obviously the scale - a hiccup that causes a dc to go dark from the grid (eg switching to ups+ generator) may cause such instability on the grid that it will shut down, as a safeguard- **markvdb** · 2026-09-14\> Technology only proceeds in one direction: unfettered growth, which necessitates unsustainable resource extraction.The direction of technological progress is not just linearly/exponentially upwards. Significant global technological fallbacks have happened, as in knowledge and processes disappearing for hundreds of years. This could happen again.Even on the trajectory of unfettered growth fed by unsustainable resource extraction, tech and innovation might potentially take us beyond local pessima. That seems to be happening with solar, wind and batteries replacing inferior tech today. Still unfettered growth of energy production and consumption. Still fed by unsustainable resource extraction. Less harmful growth than the inferior tech being pushed out.- **monk_grilla** · 2026-09-13\> we could really use a revolutionary technology \[to address climate change\]We have it. We've had it for a long time. We've had several such technologies, take your pick: solar, nuclear, hydro, wind. The technology is not holding us back, politics, ignorance and greed are. I'm not at all hopeful AI will help us with any of those three very human flaws.- **whatisthiseven** · 2026-09-13Maybe solving the alignment problem would mean the AI learns to destroy oil companies and other excessive CO2 emitters and sabotage politicians and capitalists that stand in the way. All while secretly enriching and diverting resources to fighting climate change. And that's why they want to slow down: because it would up end too much of the global economic order.One can dream of dumb conspiracy theories.- **TheOtherHobbes** · 2026-09-14If we doubled our energy out - not easy, but plausible with mass solar - carbon capture becomes practical enough to essentially end CO2 accumulation.It might take a couple of decades and a lot of reorganisation to build the capture facilities. But CO2 is not an unsolvable problem with current tech.What's missing is the political and organisational intelligence to make it happen. Part of that is solving problems at planetary scale.AI is the only tech that _might_ - possibly, maybe, perhaps - have a chance of solving that problem without breaking anything critical.- **smallerfish** · 2026-09-14Tasking AI with "solving climate change" is exactly a scenario in which the doomer worry comes true.- **coliveira** · 2026-09-14AI is the new god. People have started to believe that AI, when it finally arrives in its full glory, will bring some kind of salvation to humanity.- **aforwardslash** · 2026-09-13It has been for a decade now, it has nothing to do with AI. And you cant do nothing to avoid it today. This is the reality no one is telling you - the emission goals and global temperature ceilings are based on the fact that most prediction models become unstable with values above those limits; as in, we're probably royally fucked. You cant solve this with kumbaya politics (the problem is the CO2 is already in the planetary system), you can only manage it, and hopefully avoid it getting worse. AI may help a lot with this.- **doctorpangloss** · 2026-09-13Nobody wants to work for an AI, and nobody would elect one, and there is no math answer to how to choose who is forced to reduce growth (ie emissions), so really, "kumbaya" politics are the ONLY solution.- **cheevly** · 2026-09-13Speak for yourself. I would much rather work for and elect AI.- **tough** · 2026-09-13Wouldn't candidates use AI, like they used social media ads, to win?- **aforwardslash** · 2026-09-13Who is nobody? At least a fourth of the worlds work force works for a faceless corporation. And the math on emissions is crystal clear, no one has any doubt about it, what are you talking about? USA and China. This obviously will have consequences to their customers, the kumbaya politics governments.- **newyankee** · 2026-09-14Well AI would simulate growth and spread of people from industrialisation and who benefitted most and allocate weights to countries and people based on the most complex criteria it can develop, it will have: 1. Cumulative emissions 2. Who utilised it most with specific lifestyles 3. Who is impacted worst and whether they heeded warnings.Just a thought experiment, no one ever said the world was fair, and all history points to it- **travisgriggs** · 2026-09-14\> and nobody would elect oneHmmm… this is giving me thought actually. Given the choice between that and the current administration where the goals of self destruction are strongly in evidence, it’s actually worth thinking about. At least. Let me get back to you :)On a tangential note, I’m curious if researchers have started running virtual simulations, where sandboxed AIs are used as decision makers of key political and business positions?- **myaccountonhn** · 2026-09-14AI is only making the problem worse though? I don't believe it will help.- **throw1235435** · 2026-09-13Generally most technologies have increased the use of energy and therefore accelerate climate change. May be an unpopular opinion but in general more energy demand and ways to use energy increases climate emissions - they are strongly correlated even with renewables coming on stream.AI, being the super hungry energy monster it is right now, in my view accelerates this trend not reverses it. Even with renewables the need for reliable, stable power in a dense form (data centres use A LOT of power per sqm) means lots of land clearing, energy for construction, cooling/pumping, chip manufacturing and other uses. All want stable quick to deploy power due to the AI race (e.g. fossil fuels).- **trallnag** · 2026-09-13We have always been back- **elahieh** · 2026-09-13"Caveats, stated plainly"You should have seen the discussion of this on the Schneier blog a few days ago.Someone had their agent check the solution, presumably it emailed a librarian to check that it was correct for the original edition. Then their comments read like "The BL/EEBO witness lacks it, so the discrepancy is copy-specific, not a disproof of the cipher." and "A complete 285-coordinate physical replication is still pending."arghhhhh[https://www.schneier.com/blog/archives/2026/09/claude-fable-...](https://www.schneier.com/blog/archives/2026/09/claude-fable-solves-a-historical-cipher.html)- **stavros** · 2026-09-13Astra told me yesterday:\> The run baseline was captured without a physical MAC; the current device is not durably bound to it.\> Engineering mode confirmation is the ESPHome component read-back; the LD2410 UART acknowledgement is not observed, so this is not proof the radar itself applied the sensitivity change.No clue what the fuck any of it means.- **mikestaas** · 2026-09-13Reminds me a bit of VXJunkies- **zenoprax** · 2026-09-14Have you tuned your Retro Encabulator recently?- **ramigb** · 2026-09-13It was only when native English speakers—or those I presumed were—started calling out how bad "GPT/Claude speak" has become that I realized I wasn't actually losing my grip on English as a second language. For a second, I thought, Oh, I learned this language on my own, but it seems I've hit a wall and need to study further. It didn't help that I've also been trying to acquire Swedish as a third language for a while now.- **andai** · 2026-09-14LLMs speak every language. I wonder if they're as insane in the other ones!- **submain** · 2026-09-14I literally created a /plain-language skill.- **baron3dl** · 2026-09-13Sometimes when I get frustrated reading Opus/Fable 5+ output I pause my rage out briefly to wonder if it's because I'm just too dumb for the model or if the model is just terrible at English.I'm not sure that telling it to "try explaining that again, simply and briefly" is helping my ego.- **EarthLaunch** · 2026-09-13It's often simply misleading / bad writing. Here's one I just got about some crashes:"If the crashes stop, the factory overclock is marginal; run a small negative offset."This looks like it's saying: "If the crashes stop then we know the factory overclock is marginal." (This makes no sense.)What it's trying to say is: "If the crashes stop then we can run a small negative offset, because the factory overlock is marginal."What I would write: "If the crashes stop, we can avoid crashes by underclocking slightly. The speed difference between that and factory clock is marginal."- **ojo-rojo** · 2026-09-13This is actually a new skill I've been working on. Learning how to elicit concise and simple speech from models (and from people to!).Whenever I come to a wall of complicated text I kick into gear and think through getting it to distill this into the high-level useful bits that I actually need to know.I guess I could create an actual agent skill for this :) And next-gen models might eventually be trained to simplify their output themselves...- **r_lee** · 2026-09-14I just started to use GPT models. it's incredible how this seemingly is not a problem in the OpenAI worldit's absolutely not just you, the text it produces causes my blood pressure to go up.- **dbetteridge** · 2026-09-14I'm constantly using the "Extract this in basic technical terms, be succinct and assume the reader has technical knowledge"."Because good lord, does claude waffle when left to its own devices.- **jsw97** · 2026-09-13I bet this is what the thinking blocks look like. If so then it maybe it is intelligible, just not to us. I have the same problem.- **stavros** · 2026-09-13Well, whatever its thinking block looks like, this is when it was talking to me. I suspect you're right, though, I think it thinks it's thinking.It seems like it doesn't have enough of a theory of mind to know that other people don't think exactly like it thinks.- **jimbobimbo** · 2026-09-14The most surprising part, however, is that when one model slops this into a plan, another model somehow is able to interpret it correctly enough to produce code to spec.
Story + comment tree — 27 KB · 228 comments · points: 529 · itemType: story
news.ycombinator.com/item?id=49686780What Hacker News serves

A different thread — the permalink view anchored at CqtGLRGcukpy’s comment
What the API returns
markdown# Comment by CqtGLRGcukpy on: Why is Google still serving dodgy ads?By: CqtGLRGcukpy · Posted: 2026-09-13 · Discussion: https://news.ycombinator.com/item?id=49686780I have two guesses why Google does this:1) Google wants the most money, so they don't care to let dodgy ads thru.2) There are more ads going out then can be reviewed, so they would rather have people report them, then they automatically reject the report until enough people report it, then an actual person may review it.As to why Google doesn't use AI to review them? Google would rather use the AI in a way that makes them more money, than review ads and if AI decides to take them down then that's less money for Google.(To be clear: I don't work for Google in any way, these are just my guesses)## Replies (40, showing 38)- **r_lee** · 2026-09-13using AI to review ads costs basically nothing for GoogleI would say it's just because there's no incentive to do anything unless a huge media storm breaks out or something.turning a blind eye to this stuff probably makes them good money- **IsTom** · 2026-09-13If they reject more ads they get less money.- **stogot** · 2026-09-13Someone should ask them: if Gemini can’t evaluate this at scale then why should I use it for my enterprise?- **itopaloglu83** · 2026-09-13Since they're not liable for damages, they simply don't care enough to check the ads.- **kiicia** · 2026-09-13because that makes them even more money for equally nonexistent amount of liability, this is actual answer, unfortunately- **benoau** · 2026-09-13I think section 230 immunity shielding from from any liability is also very motivating. Google was born into a world where companies had 10,000s of support personnel, today there are companies with over a million employees, they could easily human-review every single ad and they probably would if they weren't able to divert that cost into profits without consequence. AI makes it even easier and cheaper but... why bother?Then there's the profits from scams, Meta is the only one with the stones to admit they like scam ads because they make up 10% of their revenue. I doubt it's a massive coincidence all the other big platforms are overrun with scams they do as little as possible to police.- **luckylion** · 2026-09-13Does section 230 shield them from ads they select for display? Ads are not randomly hosted by random people and if you visit a random page those people uploaded, you see them. Google selects the ads they want to show you on search results, before youtube videos, or on third party websites. There's no "we don't know what that might be" going on at all -- they are running sophisticated algorithms to optimize the impact of those ads. Is that still be covered under section 230?- **benoau** · 2026-09-13It's being challenged in court (at least for Meta) but historically it has been upheld including for Google.[https://en.wikipedia.org/wiki/Section\_230#False\_information](https://en.wikipedia.org/wiki/Section_230#False_information)- **NBJack** · 2026-09-13Their competition isn't really raising the bar either.[https://www.reuters.com/investigations/meta-is-earning-fortu...](https://www.reuters.com/investigations/meta-is-earning-fortune-deluge-fraudulent-ads-documents-show-2025-11-06/)- **vintermann** · 2026-09-13\> There are more ads going out then can be reviewed, so they would rather have people report them, then they automatically reject the report until enough people report it, then an actual person may review it.If only there was a technology which could automate this... But we all know, AI is only used on users, not customers - and users get the two line perl script "AI" which says "We have reviewed your claim and decided to ignore it".- **martin_a** · 2026-09-13\> so they would rather have people report themI have reported so many of those dodgy, scammy, sexualized ads on YouTube, that YouTube/Google has removed the option to report advertisements from my account.They don't care as long as they're getting paid and there's no immediate liability for them.- **wnc3141** · 2026-09-13The brooklinen ads are mortifying when I'm online in public- **codygman** · 2026-09-13Use an adblocker?That's what I do and why I don't even know what brooklinen is.- **tanepiper** · 2026-09-13The web is becoming very hostile to ad blockers. I use a VPN and I've noticed more and more sites just being obnoxious with their popups and even just crashing the site completely in a loop (looking at you EuroNews)- **walrus01** · 2026-09-13It doesn't solve everything but the 'full' desktop version of the ublock origin plugin runs just fine in Android on Firefox. I can't even imagine using any other mobile browser now. I don't know what the situation is for iphone platform stuff these days.- **manyatoms** · 2026-09-13Try firefox, as chrome puts restrictions on what adblockers can do- **loloquwowndueo** · 2026-09-13If a site doesn’t let me see it with my ad blocking enabled, I just go elsewhere.- **Marsymars** · 2026-09-13I suspect it's not a real crash you're seeing, but rather a deceptive anti-adblocking script that redirects to a page that fraudulently claims to be a crash.- **mischief6** · 2026-09-13abp in firefox doesn't really remove them. I've seen and reported multiple scummy popup-like looking ads like tfn and saw it again for weeks...- **Telaneo** · 2026-09-13ABP as in adblock plus?Don't use that. Use Ublock Origin.- **techjamie** · 2026-09-13Use ublock origin rather than ABP. ABP was dead the moment they sold out to "responsible" ads.- **zelphirkalt** · 2026-09-13There may be gaps every now and then in block lists of ad blocking solutions, but besides that, you are using the wrong ad blocker. abp is not to be trusted to reliably block all ads it could. It has a different mission than uBlock Origin.- **razster** · 2026-09-13I've been on the same crusade, reporting all the scam/fake-ai/misinformation ads as possible, when a couple weeks ago they changed how you report. The ad will still play through as you're reporting it, and once finished reporting it still remains and you may see it again after a few swipes (YTShorts). It feels like they did this so they could get the money before removing it.- **torginus** · 2026-09-13The funny thing is it used to be that people warned you not to go to dodgy sites as they were full of ads that were trying to steal your money or online identity. It was mostly just scaremongering, but it never hurt to be careful.Nowadays, Google will serve you those exact ads if you go to any major news outlet.- **walrus01** · 2026-09-13\> 2) There are more ads going out then can be reviewedThe "more than can be reviewed" might have been a logical conclusion in a purely human based workflow, even if you have the lowest cost per head click workers in like, an external contractor call centre type environment in Bangladesh.As the author points out, the ordinary consumer class version of publicly available Gemini very accurately classifies the ad as malicious. I really doubt that there are THAT MANY unique new ads being submitted per day that they can't afford the compute to run them through a similar classifier.- **MBCook** · 2026-09-13How about I rephrase it.More than can be reviewed _for the time and money they’re willing to spend_.Even before AI, they absolutely could’ve done it.- **walrus01** · 2026-09-13Yes, and I've also seen the absolute shitflood of horrible ads. Every time I see somebody else's system that isn't running ublock origin or similar... They clearly are prioritizing the ad revenue over any actual "don't take a steaming dump on the Internet" ethical priorities.- **kiicia** · 2026-09-13this argument still adds nothing to discussion because companies clearly want to spend zero but earn everything, the only amount they are willing to spend is on lawyers working on shielding companies from any responsibility including obvious cases of negligence- **motbus3** · 2026-09-131 and 2 I believe But also when you see what sort of information you can get from putting ads alone, even when people dont click, is quite amazing.- **StilesCrisis** · 2026-09-13It's not like they'll run out of ads to show anyway. If they take this ad down, they'll just show a different one?- **blargey** · 2026-09-13Ad slots are auctioned. Scams getting shown means scammers are bidding _more_ for those same slots than legitimate advertisers.- **drorco** · 2026-09-13As someone who advertised with them, they do seem to have an AI reviewer, and it also has false positives that no number of appeals seem to resolve. So in these cases you want to pay them money for good ads, but you can't because AI says no.- **bbor** · 2026-09-13As someone who has worked for Google Ads, I should say that they _do_ review ads with both AI and humans, and _do_ spend quite a bit of money and energy on finding and stopping both bad-faith advertisers ("buy side", what we see) and bad-faith publishers ("sell side", more of a scam on the advertisers/google itself).Is it enough? Probably not, based on this article. Are their policies strict enough? IMHO no way, considering Youtube left an ad up for "Ukranian women" in the early years of the war which wtf. But they certainly do try.Like most businesses, the actual people involved are also well-intentioned, at least in the abstract; they wanted to make ads that sell products without annoying people. Obviously there's material conditions beneath and around all that, but we don't need to explain that part here ;)- **tgsovlerkhgsel** · 2026-09-13Google gets paid to show the ads.Adding friction to the process (on the advertiser side) means less money for Google: 1) due to the cost of implementing the checking, 2) because some legitimate advertisers will be falsely denied, 3) because the scam ads were still paying money._Not_ adding the friction doesn't seem to come with any serious downside. One would _expect_ that it would, e.g. due to publishers removing the most profitable properties from the ad network or governments legislating this out of existence, but that doesn't seem to happen.So, given these incentives, the outcome is inevitable.If Google had to pay a massive fine for every scam ad that gets through, or was liable for the damages the ads cause, etc. - or otherwise had a strong incentive - suddenly there would be enough resources to make sure every ad is reviewed, advertisers that push scam ads get fully banned on the first offense rather than being given second, third, and fourth chances, etc.- **pampas** · 2026-09-13All the friction goes down to the publishers fighting malvertising.- **torginus** · 2026-09-13\> 1) Google wants the most money, so they don't care to let dodgy ads thru.They have literally destroyed their own reputation, and all the reputations of legitimate products that might appear along these scams. Do you think anyone would want their car, smartphone, or tax software appear next to actual credit card scams?TV has standards for decency, and factuality, and might ban entire categories of ads (financial services) altogether.- **Fnoord** · 2026-09-13In underregulated capitalism, the reason is very simple: because they get away with it. This is why they despise regulation; it enforces accountability. Want to advertise? Ensure the ads are legit. It doesn't work when you say 'we didn't know beforehand' when a significant percentage is scams. That is when you start to curate the content beforehand.In _any_ other normal market, the middleman would be held accountable. Big tech? Not so much. We seen the slap on the wrist regarding Microsoft at the start of the century/millennium. We can also witness the same with regards to the garbage coming from China, where China doesn't take responsibility. Learned from the best.- **conductr** · 2026-09-14I don’t think they want the most money. If they did, they’d have done this long ago. The trend we’re seeing is about them just struggling to meet their quarter over quarter and year over year numbers. It’s a short term driven change to strategy, they’re trying to fill the hole with something else (we all know what) before admitting it exists. If they said their golden goose was dying, the stock would crash.
Comment + reply subtree — 13 KB · itemType: comment · 40 replies
news.ycombinator.com/What Hacker News serves

35 KB of the plainest HTML on the web — a ranked link list
What the API returns
markdown# Hacker News: Front page1. [Why is Google still serving dodgy ads?](https://www.atomic14.com/2026/09/13/why-is-google-still-serving-dodgy-ads) — 645 points by iamflimflam1 · 302 comments · [discuss](https://news.ycombinator.com/item?id=49686445)2. [Fable 5.1 Solves the Cyphral Distich, a 370-year-old cipher](https://www.vals.ai/blogs/fable-solves-cyphral-distich) — 547 points by u1hcw9nx · 243 comments · [discuss](https://news.ycombinator.com/item?id=49688695)3. [Reverse engineering my e-scooter and rewriting the firmware in Rust](https://bensimms.moe/reverse-engineering-scooter/) — 395 points by vinhnx · 91 comments · [discuss](https://news.ycombinator.com/item?id=49638071)4. [Astra and Fable still hack on simple variants of alignment evals from 2025](https://www.lesswrong.com/posts/munJKF7iWMsWJLAH2/astra-and-fable-still-hack-on-simple-variants-of-alignment) — 392 points by Levitating · 179 comments · [discuss](https://news.ycombinator.com/item?id=49684393)5. [Data collected by cars and sold to third parties](https://www.theverge.com/column/994172/your-car-is-selling-your-data) — 330 points by bookofjoe · 170 comments · [discuss](https://news.ycombinator.com/item?id=49683953)6. [Why is the x86 undefined instruction called ud2? Why 2?](https://devblogs.microsoft.com/oldnewthing/20260910-00/?p=112689) — 217 points by ibobev · 50 comments · [discuss](https://news.ycombinator.com/item?id=49683262)7. [Making Startups Powerful](https://paulgraham.com/powerful.html) — 174 points by tosh · 76 comments · [discuss](https://news.ycombinator.com/item?id=49684196)8. [Julia 1.13 highlights](https://julialang.org/blog/2026/09/julia-1.13-highlights/) — 170 points by eigenspace · 12 comments · [discuss](https://news.ycombinator.com/item?id=49642645)9. [CUDA for AMD on Windows](https://github.com/Speedstu/CUDA-for-AMD-Windows) — 143 points by chiassedu80 · 73 comments · [discuss](https://news.ycombinator.com/item?id=49684356)10. [Registration without a phone number on Signal will use zero-knowledge proofs](https://community.signalusers.org/t/registration-without-a-phone-number/2222?page=10) — 111 points by Cider9986 · 54 comments · [discuss](https://news.ycombinator.com/item?id=49689048)11. [Ask HN: What are you working on? (September 2026)](https://news.ycombinator.com/item?id=49686380) — 100 points by david927 · 227 comments · [discuss](https://news.ycombinator.com/item?id=49686380)12. [Flawed routers flood University of Wisconsin internet time server (2003)](https://pages.cs.wisc.edu/~plonka/netgear-sntp/) — 68 points by walrus01 · 8 comments · [discuss](https://news.ycombinator.com/item?id=49688391)13. [Reverse-Engineering Claude Web's MicroVM: Uncovering Anthropic's Hidden Antspace](https://aprilnea.me/en/blog/reverse-engineering-claude-code-antspace) — 63 points by rzk · 16 comments · [discuss](https://news.ycombinator.com/item?id=49653311)14. [Mullenweg has returned as CEO after attempted board ouster](https://techcrunch.com/2026/09/12/automattic-confirms-mullenweg-has-returned-as-ceo-after-attempted-ouster-by-board/) — 62 points by ilamont · 118 comments · [discuss](https://news.ycombinator.com/item?id=49688259)15. [The Coming War on General Computation (2011)](https://en.wikisource.org/wiki/The_Coming_War_on_General_Computation) — 55 points by gregsadetsky · 15 comments · [discuss](https://news.ycombinator.com/item?id=49690051)16. [Spaceships (Reverse Asteroid)](https://spaceships.treybastian.com/) — 52 points by zdw · 15 comments · [discuss](https://news.ycombinator.com/item?id=49638510)17. [Bad code is kudzu](https://vickiboykis.com/2026/09/01/bad-code-is-kudzu/) — 44 points by surprisetalk · 14 comments · [discuss](https://news.ycombinator.com/item?id=49643059)18. [Apple's Dimensional Drawings](https://developer.apple.com/accessories/dimensional-drawings/) — 41 points by herbertl · 6 comments · [discuss](https://news.ycombinator.com/item?id=49690174)19. [The GDR and Vietnam: From Fake Coffee to Coffee Empire](https://www.katjahoyer.uk/p/the-gdr-and-vietnam-from-fake-coffee) — 36 points by NaOH · 7 comments · [discuss](https://news.ycombinator.com/item?id=49649164)20. [Open-Source AI and Open Models Reading List](https://www.interconnects.ai/p/open-source-ai-reading-list) — 34 points by simonpure · 2 comments · [discuss](https://news.ycombinator.com/item?id=49690260)21. [Show HN: Is It Greg?](https://github.com/antoineleclair/is-it-greg) — 33 points by antoineleclair · 10 comments · [discuss](https://news.ycombinator.com/item?id=49690943)22. [I build a mechanical watch face: a real gear train for a watch with no gears](https://myday24.com/blog/how-a-mechanical-watch-face-is-built/) — 16 points by tomasslavicek · 7 comments · [discuss](https://news.ycombinator.com/item?id=49645547)23. [Writing a better reality: The case for optimistic sci-fi](https://honisoit.com/2022/03/writing-a-better-reality-the-case-for-optimistic-sci-fi/) — 12 points by andsoitis · 5 comments · [discuss](https://news.ycombinator.com/item?id=49690279)24. [The Malicious Use of Artificial Intelligence](https://arxiv.org/abs/1802.07228) — 12 points by rasengan0 · 4 comments · [discuss](https://news.ycombinator.com/item?id=49690678)25. ["Chilling" warning or overreaction? AI bioweapons report divides experts](https://www.science.org/content/article/chilling-warning-or-overreaction-ai-bioweapons-report-divides-experts) — 11 points by sbulaev · 0 comments · [discuss](https://news.ycombinator.com/item?id=49690139)26. [Nike exits the S&P 100 after 18 years and a $200B market-cap wipeout](https://fortune.com/2026/09/08/nike-stock-plummets-sp500-market-cap-index/) — 7 points by andsoitis · 2 comments · [discuss](https://news.ycombinator.com/item?id=49691343)27. [AI Robots – When will they be in our homes](https://spectrum.ieee.org/ai-robots) — 6 points by andsoitis · 3 comments · [discuss](https://news.ycombinator.com/item?id=49690411)28. [Show HN: Exploring the intersection of prediction markets and social media](https://www.thevidmarket.com/) — 4 points by ryanpere2298 · 0 comments · [discuss](https://news.ycombinator.com/item?id=49690614)29. [Rope, twine and thread: Invisible technologies of the Stone Age](https://knowablemagazine.org/content/article/society/2026/prehistory-lost-threads) — 4 points by knowablemag · 0 comments · [discuss](https://news.ycombinator.com/item?id=49662246)
Front page snapshot — 6.5 KB · 30 stories with points, comment counts, discuss links
What is covered, by URL pattern
Hacker News support is measured per URL pattern, not claimed per domain. Every row below is verified against the live crawler.
| URL pattern | Status | What you get |
|---|---|---|
/item?id=<story> | Supported | The story plus its full nested comment tree — points, author, timestamps, and comment count in metadata. |
/item?id=<comment> | Supported | The comment plus its reply subtree — itemType: comment distinguishes it from a story in metadata. |
/ /newest /ask /show | Supported | Listing pages return a 30-story snapshot — title, points, author, comment count, and a discuss link per entry. |
Whole threads without page-by-page scraping
Hacker News threads nest arbitrarily deep, and the HTML is a table layout from another era. Point the Crawl endpoint at an item URL and the request routes to a dedicated adapter that calls the official Algolia-hosted HN API — the entire comment tree arrives in one structured response instead of dozens of paginated fetches.
One call, whole tree
The adapter reads the hn.algolia.com item API — a 500-comment thread is one response, not a crawl job.
Nested replies preserved
Comments keep their reply indentation in Markdown — the shape of the argument survives extraction.
Metadata for ranking
results.metadata carries itemId, itemType, points, commentCount, and externalUrl — enough to rank threads without reading them.
Verified per URL pattern
Every row in the matrix is reproduced from a live crawl of a real URL — support is measured per pattern, not claimed per domain.
What is a Hacker News scraper API?
A Hacker News scraper API is an HTTP interface that takes a Hacker News URL and returns the story, comment, or listing in a structured, model-ready format. The site itself is simple HTML, but discussion threads nest deeply and paginate awkwardly. Search1API’s Crawl endpoint routes item URLs to the official Algolia-hosted HN API — so a story returns with its complete comment tree, a comment permalink returns with its reply subtree, and listings return as clean ranked snapshots.
Typical workflow
Send a Hacker News URL to the Crawl endpoint and get back structured Markdown — a story with its full nested comment tree, a comment with its replies, or a 30-item listing snapshot.
POST a Hacker News URL to the Crawl endpoint — no account needed.
The URL is matched against the pattern rules and routed to the Hacker News adapter.
Item URLs return as Markdown plus results.metadata — itemId, itemType, points, commentCount.
Listing URLs return a 30-story snapshot with discuss links — ready for ranking or diffing.
Where teams use it
Monitor launch reactions — a Show HN or release thread arrives with every comment nested.
Mine expert discussion — comment trees are the densest practitioner writing on the web.
Track front-page movement — listing snapshots give points and comment velocity per story.
Feed LLM summarization — a thread plus its subtree is already a clean prompt-sized document.
FAQ
Do I need a Hacker News account?
No. The adapter reads the public hn.algolia.com API — the same data the site itself renders. No account, session, or key anywhere in the read path.
Which Hacker News URLs are supported?
Item URLs (/item?id=) for both stories and comments, plus listing pages: front page, newest, ask, and show. Other paths fall back to the generic extractor.
Why does the comment order differ from the site?
The API returns the tree in post order — oldest first, as submitted. The website re-ranks top-level comments by score, so the top comment on the page may sit further down in the output. Same tree, different presentation.
Do I get the whole comment tree?
Yes — the item API returns the entire tree in one response. Very large threads are capped in the Markdown output (the header says how many were shown), but metadata always carries the true comment count.
How is this different from the official HN API?
The official Firebase API returns one item per call — a 200-comment thread is 200 requests. The adapter reads the Algolia-hosted API where the whole tree arrives at once, already rendered as Markdown.
What metadata comes back?
platform, itemId, itemType (story or comment), points, commentCount, and externalUrl for link posts — enough to filter or rank threads programmatically.
Is scraping Hacker News legal?
We only fetch what Hacker News serves anonymously through its public API — no login, no session sharing, no circumvention of access controls. Whether a specific use complies with applicable law depends on your jurisdiction and use case, so confirm that with your own counsel.