Archives AI News

When it comes to driving screws, this quarter-inch ratchet gets the job done for me.

The bar for AI-powered handsets has been set a little higher thanks to Google's flagship Pixels.

Modern workloads are driving the need for more RAM - but how much is enough? Here's what you should know before upgrading.

Published on August 27, 2025 5:31 PM GMTThis past month I’ve been experimenting with video posts, as you’ve probably noticed (unless you read all these posts on Less Wrong, in which case, now you know). It started because I didn’t think I had time to make a post, but then on a whim decided to record a video. People responded well, so I’ve tried it two more times.I like that videos do a better job of netting me new subscribers, which is important because this blog ultimately exists for the purposes of promoting the book. Otherwise I’d blog only on Less Wrong, write marginally fewer posts, and not worry about having a subscriber list.I don’t like that the written transcripts of the videos don’t read as well as written posts would. Or at least that’s what I think. They contain a lot more fluff, which is more tolerable when speaking, but less so in writing. Or at least it is to me.I’m not yet sure what to do about it, but I’m going to continue to experiment a little longer. I’ll likely settle somewhere that includes a mix of written posts and video posts once I figure out what kinds of posts work best which way.Speaking of writing, I spent August wrestling with Chapter 4. Turns out this one had a lot of stuff in it I left behind for myself to fix later in editing, I just didn’t remember that. So I spent most of the month trying to get the opening section right. I’m finally there, so now I’m on to the other big task of editing this chapter, which is fixing the last section by addressing the Löbian Obstacle (thanks to Abram Demski for many useful comments two years ago that I’m just now finally addressing!)All this editing work has made a few things clear to me about how to fix writing faster. My current methods:If some text doesn’t work, the problem is usually not with the text itself, but earlier text that forced this text to take the shape it’s taking. The solution is usually to go up the page and change whatever is making the problem text exists so that some different text can take its place.Figuring out that I need to change up-page text usually takes a few passes, since it’s not always immediately obvious that what’s off about a sentence or paragraph is that it’s been written into a corner by some previous statement rather than normal draft writing that needs to be cleaned up.When I get stuck on how to move forward, LLMs are a good source of creativity. What I usually do is ask Claude to generate several possible completions for a snippet of text. It never gives me back anything I end up using directly, but it gets me thinking about ideas I hadn’t considered talking about, which usually leads to figuring out what to do.If I’m really stuck, Claude and I will have some back and forth. I turn on my annoyed, short-on-time manager mode to press it by simply rejecting the results and saying what I don’t like. After a couple rounds, this is usually enough feedback to get it to give me something useful.In other writing outside the blog, I did a dialog with fellow Less Wrong user SpectrumDT about ’s hypertext book Meaningness. In the end, SpectrumDT says I didn’t end up explaining anything better, but he enjoyed the exchange. You can read it here.Discuss

Published on August 27, 2025 5:04 PM GMTHere's a relatively important question regarding transparency requirements for AI companies: At which points in time should AI companies be required to disclose information? (While I focus on transparency, this question is also applicable to other safety-relevant requirements, and is applicable to norms around voluntary actions rather than requirements.) A natural option would be to attach transparency requirements to the existing processes of pre-deployment testing and releasing a model card when a new model is released. As in, companies would be required to include the relevant information whenever they release a new model (likely in the model card). This is convenient because pre-deployment testing and model cards are already established norms in the AI industry, which makes it easier to attach something new to these existing processes rather than creating a new process. However, I think attaching requirements to model releases (such that the requirement must happen before release proceeds) has large downsides: Right now, AI companies are often in a massive rush to deploy models, and deployment is already a very complex process. If something is a blocker for model release, then there will be massive pressure to get that thing done on time (regardless of the cost in terms of quality). Thus, adding something to the requirements for deployment is likely to make that new thing get done poorly in a heavily rushed way (and also makes it more likely that time of safety researchers is used inefficiently to get it done in a short period of time). Probably most of the risk from AI comes from internal rather than external deployment[1], so requiring transparency prior to external deployment isn't particularly important anyway (as the model was very likely already internally deployed and if nothing else already could have been stolen). Correspondingly, another cost of attaching requirements to external releases of models is that it implies that external deployment is where relevant risks exist and that other activities are safe so long as a model isn't externally deployed. This could be a pretty costly misconception, and even if relevant people agree that internal deployment risks are important (and understand that this is a misconception), I still think that tying relevant processes to external deployment might make these people end up acting as though this is where the risk lives. Building on the prior bullet, transparency about models which aren't externally deployed is very important in some cases and attaching transparency to external model releases makes it awkward to require transparency about models which aren't released externally (in the cases where this is important). In particular, this is important if internal models are way more capable than externally deployed models (much more capable on key metrics, much more capable qualitatively, or they are the equivalent of more than a year ahead at the 2025 rate of AI progress), especially if the absolute level of capability is high (e.g., human engineers in AI companies are now mostly managing teams of AI agents). Internal models being much more capable than externally deployed models could be caused by increased secrecy or rapid progress from AI automation of AI R&D. Rather than associating requirements with model release, you could instead require quarterly releases or require information to be released within some period (e.g. a month) of a model release. I'd prefer something like quarterly, but either of these options seem substantially better to me. (The cadence would need to be increased if AI progress is made substantially more rapid due to AI automation of AI R&D.) Ideally, you'd also include a mechanism for requiring transparency about models which aren't externally deployed in some situations. E.g., you could require transparency about models which are non-trivially more capable than externally deployed models if they've been internally deployed for longer than some period (this could be required in the next quarterly report).[2] Whatever this requirement is, you could make this requirement only trigger above some absolute capability threshold to reduce costs (if you sufficiently trust the process for evaluating this capability threshold) and you could allow companies to delay for some period before having to release the information so that it's less costly[3]. By default, I predict that more requirements will be pushed into the release cycle of models and that AI companies will pressure (safety-relevant) employees to inefficiently rush to meet these requirements in a low-quality way. It's not obvious this is a mistake: perhaps it's too hard to establish some other way of integrating requirements. But, at the very least, we should be aware of the costs. Why I don't like making (transparency) requirements trigger at the point of internal deployment (prior to rapid acceleration in AI progress) All of these approaches (including attaching requirements to external model release) have the downside that requirements don't necessarily trigger for a given model prior to risks from that specific model. In principle, you could resolve this by having requirements trigger prior to internal deployment (or prior to training that model for security-relevant requirements). However, this would make the rush issue even worse, the details of this get increasingly complex as processes for training and internally deploying models get more complex, and companies would resist requiring (unilateral) transparency around models which they haven't yet released.[4] Having things operate at a regular cadence or within some period of model releases (rather than having requirements trigger before the situation is risky) is equivalent to using the results from an earlier model as evidence about the safety of a later model. As long as AI development is sufficiently slow and continuous, and there is some safety buffer in the mitigations, this is fine. This is one reason why I think it would be good if the expectation was that safety measures should have some buffer: they should suffice for handling AIs which are as capable as the company has a reasonable chance of training within the next several months (ideally within the next 6 months).[5] If the rate of progress greatly increased (perhaps due to AI automating AI R&D or a new paradigm), this buffer wouldn't be feasible, so more regular reporting would be important. At least this is true prior to large economic effects or obvious AI-enabled military buildup. At this point we can possibly adjust our approach. Also, transparency requirements (especially transparency requirements done in advance) are less important in worlds with massive obvious external effects of AI prior to potential catastrophe. ↩︎ Other options include: You could also require transparency within some period of the point when risk-relevant thresholds have been crossed (regardless of whether this model is externally deployed), but currently AI companies don't have granular enough thresholds for this to be sufficient and it's hard to figure out what these thresholds should be in advance. You could also try to operationalize "way more capable than externally deployed models", but this might be tricky to do and it might be easier for companies to stretch the relevant threshold without this being brazen. ↩︎ Allowing for a (substantial) delay reduces costs in multiple ways. It makes it so the company has more time to figure out what to disclose and it also means the company is less likely to be forced to leak information about a future release. A delay of a few months is probably acceptable unless AI progress has sped up massively. ↩︎ As discussed above, transparency around models which aren't deployed externally is important in some cases, but companies would probably be less resistant if this was scoped more narrowly to these particular cases rather than always requiring immediate transparency when more capable models are deployed internally. ↩︎ Interestingly, using the results from an earlier model as evidence for a later model is effectively how the Anthropic RSP v2 works. Security requirements are only triggered by evaluations which are only required prior to model release, even though security is in principle important since the model was trained. Precisely, security is only required once a model is determined to reach some ASL level and this determination is done as part of the release process. This seems fine so long as there is effectively some buffer (such that we'd expect security requirements to trigger early enough that not having secured this model for some period still keeps risk sufficiently low). ↩︎ Discuss

Published on August 27, 2025 5:04 PM GMTHere's a relatively important question regarding transparency requirements for AI companies: At which points in time should AI companies be required to disclose information? (While I focus on transparency, this question is also applicable to other safety-relevant requirements, and is applicable to norms around voluntary actions rather than requirements.) A natural option would be to attach transparency requirements to the existing processes of pre-deployment testing and releasing a model card when a new model is released. As in, companies would be required to include the relevant information whenever they release a new model (likely in the model card). This is convenient because pre-deployment testing and model cards are already established norms in the AI industry, which makes it easier to attach something new to these existing processes rather than creating a new process. However, I think attaching requirements to model releases (such that the requirement must happen before release proceeds) has large downsides: Right now, AI companies are often in a massive rush to deploy models, and deployment is already a very complex process. If something is a blocker for model release, then there will be massive pressure to get that thing done on time (regardless of the cost in terms of quality). Thus, adding something to the requirements for deployment is likely to make that new thing get done poorly in a heavily rushed way (and also makes it more likely that time of safety researchers is used inefficiently to get it done in a short period of time). Probably most of the risk from AI comes from internal rather than external deployment[1], so requiring transparency prior to external deployment isn't particularly important anyway (as the model was very likely already internally deployed and if nothing else already could have been stolen). Correspondingly, another cost of attaching requirements to external releases of models is that it implies that external deployment is where relevant risks exist and that other activities are safe so long as a model isn't externally deployed. This could be a pretty costly misconception, and even if relevant people agree that internal deployment risks are important (and understand that this is a misconception), I still think that tying relevant processes to external deployment might make these people end up acting as though this is where the risk lives. Building on the prior bullet, transparency about models which aren't externally deployed is very important in some cases and attaching transparency to external model releases makes it awkward to require transparency about models which aren't released externally (in the cases where this is important). In particular, this is important if internal models are way more capable than externally deployed models (much more capable on key metrics, much more capable qualitatively, or they are the equivalent of more than a year ahead at the 2025 rate of AI progress), especially if the absolute level of capability is high (e.g., human engineers in AI companies are now mostly managing teams of AI agents). Internal models being much more capable than externally deployed models could be caused by increased secrecy or rapid progress from AI automation of AI R&D. Rather than associating requirements with model release, you could instead require quarterly releases or require information to be released within some period (e.g. a month) of a model release. I'd prefer something like quarterly, but either of these options seem substantially better to me. (The cadence would need to be increased if AI progress is made substantially more rapid due to AI automation of AI R&D.) Ideally, you'd also include a mechanism for requiring transparency about models which aren't externally deployed in some situations. E.g., you could require transparency about models which are non-trivially more capable than externally deployed models if they've been internally deployed for longer than some period (this could be required in the next quarterly report).[2] Whatever this requirement is, you could make this requirement only trigger above some absolute capability threshold to reduce costs (if you sufficiently trust the process for evaluating this capability threshold) and you could allow companies to delay for some period before having to release the information so that it's less costly[3]. By default, I predict that more requirements will be pushed into the release cycle of models and that AI companies will pressure (safety-relevant) employees to inefficiently rush to meet these requirements in a low-quality way. It's not obvious this is a mistake: perhaps it's too hard to establish some other way of integrating requirements. But, at the very least, we should be aware of the costs. Why I don't like making (transparency) requirements trigger at the point of internal deployment (prior to rapid acceleration in AI progress) All of these approaches (including attaching requirements to external model release) have the downside that requirements don't necessarily trigger for a given model prior to risks from that specific model. In principle, you could resolve this by having requirements trigger prior to internal deployment (or prior to training that model for security-relevant requirements). However, this would make the rush issue even worse, the details of this get increasingly complex as processes for training and internally deploying models get more complex, and companies would resist requiring (unilateral) transparency around models which they haven't yet released.[4] Having things operate at a regular cadence or within some period of model releases (rather than having requirements trigger before the situation is risky) is equivalent to using the results from an earlier model as evidence about the safety of a later model. As long as AI development is sufficiently slow and continuous, and there is some safety buffer in the mitigations, this is fine. This is one reason why I think it would be good if the expectation was that safety measures should have some buffer: they should suffice for handling AIs which are as capable as the company has a reasonable chance of training within the next several months (ideally within the next 6 months).[5] If the rate of progress greatly increased (perhaps due to AI automating AI R&D or a new paradigm), this buffer wouldn't be feasible, so more regular reporting would be important. At least this is true prior to large economic effects or obvious AI-enabled military buildup. At this point we can possibly adjust our approach. Also, transparency requirements (especially transparency requirements done in advance) are less important in worlds with massive obvious external effects of AI prior to potential catastrophe. ↩︎ Other options include: You could also require transparency within some period of the point when risk-relevant thresholds have been crossed (regardless of whether this model is externally deployed), but currently AI companies don't have granular enough thresholds for this to be sufficient and it's hard to figure out what these thresholds should be in advance. You could also try to operationalize "way more capable than externally deployed models", but this might be tricky to do and it might be easier for companies to stretch the relevant threshold without this being brazen. ↩︎ Allowing for a (substantial) delay reduces costs in multiple ways. It makes it so the company has more time to figure out what to disclose and it also means the company is less likely to be forced to leak information about a future release. A delay of a few months is probably acceptable unless AI progress has sped up massively. ↩︎ As discussed above, transparency around models which aren't deployed externally is important in some cases, but companies would probably be less resistant if this was scoped more narrowly to these particular cases rather than always requiring immediate transparency when more capable models are deployed internally. ↩︎ Interestingly, using the results from an earlier model as evidence for a later model is effectively how the Anthropic RSP v2 works. Security requirements are only triggered by evaluations which are only required prior to model release, even though security is in principle important since the model was trained. Precisely, security is only required once a model is determined to reach some ASL level and this determination is done as part of the release process. This seems fine so long as there is effectively some buffer (such that we'd expect security requirements to trigger early enough that not having secured this model for some period still keeps risk sufficiently low). ↩︎ Discuss

At $799, the entry-level flagship doesn't fix what isn't broken, but includes a handful of new AI features.

The bar for AI-powered handsets has been set a little higher thanks to Google's flagship Pixels.

The Suunto Race 2 is a compelling sports watch with customized coaching plans and a brilliant display.