Astral Codex Ten Podcast

I hate the term "hallucinations" for when AIs say false things. It's perfectly calculated to mislead the reader - to make them think AIs are crazy, or maybe just have incomprehensible failure modes.
AIs say false things for the same reason you do.
At least, I did. In school, I would take multiple choice tests. When I didn't know the answer to a question, I would guess. Schoolchild urban legend said that "C" was the best bet, so I would fill in bubble C. It was fine. Probably got a couple extra points that way, maybe raised my GPA by 0.1 over the counterfactual.
Some kids never guessed. They thought it was dishonest. I had trouble understanding them, but when I think back on it, I had limits too. I would guess on multiple choice questions, but never the short answer section. "Who invented the cotton gin?" For any "who invented" question in US History, there's a 10% chance it's Thomas Edison. Still, I never put down his name. "Who negotiated the purchase of southern Arizona from Mexico?" The most common name in the United States has long been "John Smith", applying to 1/10,000 individuals. An 0.01% chance of getting a question right is better than zero, right? If I'd guessed "John Smith" for every short answer question I didn't know, I might have gotten ~1 extra point in my school career, with no downside.
You can go further.
https://www.astralcodexten.com/p/shameless-guesses-not-hallucinations

Guest post by David Speiser
 
The Problem
Everyone hates Congress. That poll showing that cockroaches are more popular than Congress is now thirteen years old, and things haven't improved in those thirteen years. Congressional approval dipped below 20% during the Great Recession and hasn't recovered since.
A republic where a supermajority of citizens neither like nor trust their representatives is not the most stable of foundations, so it should not be shocking that the legislative branch is being subsumed by the executive.
What's the solution? Many have been proposed, some with very snazzy websites. FairVote thinks that ranked choice voting and proportional representation will solve it. The Congressional Reform Project has another snazzy website with such bold proposals as "Increase the opportunity for Members to form relationships across party lines, including by bipartisan issues conferences." There are more think tanks. They want to enlarge the House by a few hundred members, switch to a biennial budget system, spend more on Congressional staffers, and introduce term limits, among many other suggestions.
There are op-eds too. Here's how the Atlantic wants to fix Congress. The New York Times of course has a solution. Here on Substack, Matt Yglesias thinks proportional representation is the solution, and Nicholas Decker has an especially interesting solution.
These proposals, no matter which direction they're coming from, have two things in common. The first is that they largely agree on the problem: members of Congress are disconnected from their constituents. Thanks to a combination of huge gerrymandered districts, national partisan polarization, and the influence of large donors, a representative has little incentive to care about the experience of individual people in their district.
The second thing that all these proposed solutions have in common is that none of them will ever be implemented.
https://www.astralcodexten.com/p/last-rights

California lets interest groups propose measures for the state ballot. Anyone who gathers enough signatures (currently 874,641) can put their hare-brained plans before voters during the next election year.
This year, the big story is the 2026 Billionaire Tax Act, a 5% wealth tax on California's billionaires. Your views on this will mostly be shaped by whether or not you like taxing the rich, but opponents have argued that it's an especially poorly written proposal:
It includes a tax on "unrealized gains", like a founder's share of a private company which hasn't been sold yet. This could be an existential threat to the Silicon Valley model of building startups that are worth billions on paper before their founders see any cash. Since most billionaires keep most of their wealth in stocks, any wealth tax will need some way to reach these (cf. complaints about the "buy, borrow, die" strategy for avoiding taxation). But there are better ways to do this (for example, taxing at liquidation and treating death as a virtual liquidation event), other wealth tax proposals have included these, and the California proposal doesn't.
It appears to value company stakes by voting rights rather than ownership, so a typical founder who maintains control of their company despite dilution might see themselves taxed for more than they have. Garry Tan explains the math here with reference to Google. However, Current Affairs has a good article (?!) that pushes back, saying the proposal exempts public companies like Google. Although private companies would still be affected, this would be so obviously unfair that founders would easily win an exemption based on a provision allowing them to appeal nonsensical results. Still, some might counterobject that proposed legislation is generally supposed to be good, rather than so bad that its victims will easily win on appeal.
It's retroactive, applying to billionaires who lived in California in January, even though it won't come to a vote until November. Proponents argue that this is necessary to prevent billionaire flight; opponents point out that alternatively, billionaires could flee before the tax even passes (as some have already done). One plausible result is that the tax fails (either at the ballot box or the courts), but only after spurring California's richest taxpayers to flee, leading to a net decrease in revenue.
Some people propose that it could decrease state revenues overall even if it passed, if it drove out enough billionaires, though others disagree.
Pro-tech-industry newsletter Pirate Wires finds that 20 out of 21 California tech billionaires interviewed were "developing an exit plan" and quotes an insider saying that "if this tax actually passes, I think the technology industry kind of has to leave the state". Even Gavin Newsom, hardly known for being an anti-tax conservative, has argued that it "makes no sense" and "would be really damaging".
The ACX legal and economic analysis team (Claude, GPT, and Gemini) doubt the direst warnings, but agree that the tax is of dubious value and its provisions poorly suited to Silicon Valley.
https://www.astralcodexten.com/p/seiu-delenda-est

Having Your Own Government Try To Destroy You Is (At Least Temporarily) Good For Business
On Friday, the Pentagon declared AI company Anthropic a "supply chain risk", a designation never before given to an American company. This unprecedented move was seen as an attempt to punish, maybe destroy the company. How effective was it?
Anthropic isn't publicly traded, so we turn to the prediction markets. Ventuals.com has a "perpetual future" on Anthropic stock, a complicated instrument attempting to track the company's valuation, to be resolved at the IPO. Here's what they've got:
https://www.astralcodexten.com/p/mantic-monday-groundhog-day

Last Friday, Secretary of War Pete Hegseth declared AI company Anthropic a "supply chain risk", the first time this designation has ever been applied to a US company. The trigger for the move was Anthropic's refusal to allow the Department of War to use their AIs for mass surveillance and autonomous weapons.
A few hours later, Hegseth and Sam Altman declared an agreement-in-principle for OpenAI's models to be used in the niche vacated by Anthropic. Altman stated that he had received guarantees that OpenAI's models wouldn't be used for mass surveillance or autonomous weapons either, but given Hegseth's unwillingness to concede these points with Anthropic, observers speculated that the safeguards in Altman's contract must be weaker or, in a worst-case scenario, completely toothless.
The debate centers on the Department of War's demand that AIs be permitted for "all lawful use". Anthropic worried that mass surveillance and autonomous weaponry would de facto fall in this category; Hegseth and Altman have tried to reassure the public that they won't, and the parts of their agreement that have leaked to the public cite the statutes that Altman expects to constrain this category. Altman's initial statement seemed to suggest additional prohibitions, but on a closer read, provide little tangible evidence of meaningful further restrictions.
Some alert ACX readers1 have done a deep dive into national security law to try to untangle the situation. Their conclusion mirrors that of Anthropic and the majority of Twitter commenters: this is not enough. Current laws against domestic mass surveillance and autonomous weapons have wide loopholes in practice. Further, many of the rules which do exist can be changed by the Department of War at any time. Although OpenAI's national security lead said that "we intended [the phrase 'all lawful use'] to mean [according to the law] at the time the contract is signed', this is not how contract law usually works, and not how the provision is likely to be enforced2. Therefore, these guarantees are not helpful.
To learn more about the details, let's look at the law:
https://www.astralcodexten.com/p/all-lawful-use-much-more-than-you