Gusarich's thoughts

When I wrote that post earlier, I realized that this would become obsolete very soon once new models are released. So I decided to make a live tier list of frontier LLMs and keep it updated with my opinions on the latest models that I've used myself. It is accessible here, and I'll make sure to keep updating it every time I have tested a new frontier model well enough to have a proper opinion on it. It's important to note that these ranks are very subjective, and it all depends on the use cases. But if your needs from AI match mine at least in some ways, consider checking it out sometimes. At the moment I mostly care about agentic usage with extensive reasoning and great instruction following.

Things got too easy with AI AI provides incredible value to me and to many other people in our daily lives and work. It's now possible to do more, better, and faster than ever. But I realized that the progression of AI capabilities outpaced the progression of my goals. I recently started thinking in retrospect about what I was doing and what I was thinking about in the past months, and it feels like I didn't really face any hard challenges in many months. I can't remember the last time I was thinking about a problem for a week straight myself because it was so nontrivial. In most cases AI just one-shots stuff or does so after a couple of days of low-effort prompting iterations. I was thinking a lot about different stuff, but it was more of a meta-analysis or idea generation of some kind. I didn't really struggle with the execution part for a long time. Very often when I see something I'd like to try doing, it just works with AI very quickly. I just get quick dopamine from low-effort achievements. And it allowed me to minimize my effort in many ways while keeping the product of my work at a level not much higher than before. That's not what my goals should be. I believe that I should aim much higher than I was trying to aim before. Higher than I could have ever imagined a couple of years ago. If AI makes it so easy to do things that took weeks before, I should do things that would take years before. I think now is the time when this becomes possible to execute. I should try to keep the bar of my goals at the level where my own mental effort stays high enough, so that I'm not underaiming. I should stay at the limit of both my and AI's capabilities. It will probably maximize the product of my work. And the first step is to discover where that limit actually is. I want to find out how far I can push before I start struggling. I think that incredible things can be done in a month of real mental effort of a single human supercharged with many AI agents. I have a few ideas for what to try. Things that I didn't really think of until recently, because it was absurd for me to even consider trying them.

I gave Codex its own Mac Mini I was playing around with Codex CLI a lot over the holidays, and apart from making it run 30 instances of itself as "subagents" (actually just doing codex exec runs in background terminals) I also decided to buy a fresh Mac Mini and give it to Codex. I quickly implemented a pretty simple setup, that consists of a Codex caller that controls Codex process and that forces it to run in an endless loop, and a Telegram bot through which I interact with the agent. The setup was the easiest part to implement and I actually spent about 20x more time on writing first working versions of prompts with Claude. Prompts are everything here. I decided to use Codex here with GPT-5.2 model on Extra High reasoning effort, as that is the best model available in terms of instruction following and long context. Earlier I saw it run for 3 hours doing 4 compactions in the process and still delivering a good result for a non-trivial problem. The instruction following is essential in this experiment as the agent must reliably interact with me and follow all the guidance I gave it about using the computer, interacting with me and keeping notes. Long-context accuracy is the second most important part, as this agent will run endlessly for days, weeks, or maybe months. Working on a specific problem for a few hours is much simpler than delivering consistently good (and even improving) results for a week straight without resets. I won't explain too much about the setup, but the core here was giving agent a lot of context about [who, where, when, and how] it is, and instructing it to take notes. A lot of notes. All the time. I'll be using this agent as an assistant for some time. It's nice to be able to message it from my phone and ask it to push some change in a repository. Or ask it to research something overnight using ChatGPT in a browser. I'll probably stop the experiment if I don't find it useful over multiple days in a row. In this case I'll just wait for the next version of GPT and try again. Whether it turns out to work great or not, I'll post an update later. (Image edited with Nano Banana Pro based on a real photo)

Happy new year 🕺

A new blog post with my predictions on AI progress in 2026. https://gusarich.com/blog/ai-in-2026/

TON Vanity Meet the new blazingly fast vanity address generator for TON smart contracts! The previous state-of-the-art solution was released 3 years ago and hasn't improved much since. We at TON Studio decided to develop a completely new solution from scratch, following the usage patterns the previous solution introduced. Optimizations in both the smart contract and the kernel lead to extreme generation speedups. In realistic usage scenarios on an RTX 4090, the generator finds suffix patterns up to 5,100x faster, and prefix patterns up to 3,600x faster. In practice, this allows finding about 2 more letters than before in appropriate time. Apart from the crazy speedups, there's a simple interface for TypeScript usage, like in smart contract tests and deployment scripts. Applying the generated vanity address requires writing just a few extra lines of code. The generator's output was improved too, storing all results in a structured JSON format with all useful metadata. Quality-of-life improvements include a polished CLI experience, comprehensive test coverage, and a benchmark script for comparing future optimizations. The tool will be maintained and improved over time. The entire development process is open on GitHub, and contributions and feedback are welcome. A detailed write-up covering all optimizations and the development process will be published soon, for those who are interested. Check it out: https://github.com/ton-org/vanity

My personal opinion based on experience: * GPT-5.1 has the best instruction following, strong agentic capabilities, and very good skills in math, coding, and problem solving. * GPT-5.1-Codex-Max has worse general capabilities than GPT-5.1, but is noticeably better for large and complex coding tasks. * Opus 4.5 has the best implicit intent understanding, very good instruction following and agentic capabilities, but lacks depth in its reasoning that is required for complex problem solving. * Gemini 3 Pro has the best raw intelligence, especially in math, and has good agentic capabilities, but lacks instruction following. So, the choice becomes quite simple: * For well-defined general tasks, go with GPT-5.1. * For well-defined coding tasks, go with GPT-5.1-Codex-Max. * For less defined or ambiguous tasks, as well as general agentic scenarios, go with Opus 4.5. * For math and problem solving in general, go with Gemini 3 Pro, but either pair it with one of the above or work on extra scaffolding. I'm personally using GPT-5.1 and its Codex variant mostly, but sometimes trying Opus 4.5 and Gemini 3 Pro when the task fits them. I got used to the way you have to use GPT-5.1 and it gives very good results in any task I throw at it, if it's defined properly. For vibe-coding and front-end, I'd go with Opus 4.5 for its intent understanding in more ambiguous cases. All these models are roughly in the same pricing league, but OpenAI and Google also have stronger beasts: GPT-5.1 Pro and (upcoming) Gemini 3 Deep Think. These are only available in expensive subscriptions for $200/$250 a month, and are very slow. But I'm still using GPT-5.1 Pro almost daily for better results in tasks requiring reasoning. A very common scenario in my work is to throw all the context about the task into GPT-5.1 Pro and ask it to write a detailed implementation plan, then give that plan to GPT-5.1-Codex-Max to implement. It works out very well, especially if you do a couple of follow-ups with GPT-5.1 Pro to refine the plan and sync it with your intent better. Gemini 3 Deep Think is a similar thing, and it will probably be even better for complex reasoning tasks, but due to the lack of instruction following in Gemini models and the fact that it's behind another $250 paywall, I'll stick with ChatGPT Pro for now.

What LLM to use today? Many major releases occurred in the past weeks. The current frontier consists of models that a couple of months ago were only rumors. And they are great. OpenAI has GPT-5.1 and GPT-5.1-Codex-Max; Anthropic has Opus 4.5; Google has Gemini 3 Pro. I'm often working with code, and therefore I need a good coding model. I look at coding benchmarks, like SWE-bench, but scores there differ by just a couple of percent. Are there just no leaders for coding right now? It's just that most benchmarks that companies show in release posts aren't really useful. Opus 4.5 having a 3% better score on SWE-bench doesn't mean that it's 3% better at all. In fact, it doesn't tell you pretty much anything that could translate even remotely to real-world usage. It could be much worse, or much better, or the same. But this benchmark is still one of the most referenced ones when talking about coding performance. There are more benchmarks like this, not only for coding. And they are also misleading. So, how do you know which model is better then? Aren't benchmarks supposed to show that? Well, benchmarks show exactly what they test, and nothing else. SWE-bench (verified), for example, has 46% of tasks from a single Django repository, 87% of tasks are bugfixes, and 5-10% of tasks are just invalid. I wouldn't say that this benchmark somehow shows general capabilities in coding. And definitely it doesn't show capabilities in vibe-coding scenarios. What it mostly shows is the ability to locate a bug in a Python repository by being given a bug report, and fixing it in one shot. What matters for real-world coding then? From my experience, instruction following skills and agentic capabilities are the two most important things. A model is useless if it cannot consistently do what you tell it, and it won't be helpful in complex scenarios if it is not agentic. But honestly, at the moment I don't know any *good* benchmarks that evaluate this in diverse environments. How do you pick a model to use then? The first thing I'd recommend doing is to just try all of them in some real tasks you need to do. It doesn't necessarily have to be the same task, and not even the same complexity. You just have to be honest with yourself and think about how each model works with you in these tasks. Does the model understand what you want from it? Does it complete the task the way you want it to? Does it piss you off less than other models? Does it feel good? If the answers to most of the questions above are "yes", then just go with that model. At least it will feel better than others for you. Models have different skill distributions and personalities. That's why many people have very different opinions about models even though *they are all good*. I believe that to take the most out of AI capabilities, you have to really understand what they are good and bad at, and use models based on use case. As I said earlier, the current state of coding benchmarks is bad, and therefore I'll be talking solely from my experience next.

There is nothing out-of-distribution AI turned out to be very simple if you think about it. You just make a model that works with something generic, and feed as much training data as you can into it. The generic data I mean here is text. People had writing for thousands of years and the whole world is built on it — we write, we speak, we read, and we listen our entire lives. It's so deep in our brains that it's hard to think of something that cannot be described in text. Some AI pessimists love to say that "LLMs do not generalize out-of-distribution", but if they are built to understand and write text — they can generalize to anything that can be described as text, so pretty much everything. What people actually refer to when pointing out to generalization problems is *intelligence*. Why couldn't LLMs solve simple puzzles 2 years ago if they can generalize? The reason is simple: LLMs were stupid. Bad data, bad models, and bad training produced models with very limited intelligence. They simply lacked IQ to generalize well. Nothing was changed conceptually on a fundamental level in the past couple of years — we just gathered better data, designed better architectures, and improved training algorithms. Things that LLMs fail right now usually aren't "boolean" in nature. If it only solves 1% of some specific tasks right now, it means that it *can* solve them sometimes and next generations of models will solve more. And it's hard to find a task where LLMs couldn't complete at least the simplest version of it today. Inability to solve 100% of given tasks of some kind right away just means the lack of intelligence. In order to generalize some problem, you need the intelligence to understand and solve that problem well. There is no data limitation in LLMs in the sense of generalization ability, only an intelligence limitation that is being improved rapidly. I believe that LLMs are one of the valid paths to ASI. There could be other paths, even better and more general ones, but LLMs can do the thing too. ASI is not far away already, and from what we'll see in 2026 we won't have to change much to reach a superhuman level of general intelligence. I don't see anything that could stop LLMs from progressing further at the current exponential pace.

There is no singularity When mentioning singularity, people often think of some "point" in time when AI progress starts to speed up exponentially very quickly with no human control and it all kind of converges to infinity and we don't know what will happen the second after. And I myself had a similar picture in my head too, until recently. I was thinking that predicting anything after 2027 is impossible because of this "singularity" that I thought would happen in that period. I decided that I should not plan anything long-term and just focus on short-term decisions, while leaving the rest as is. And I kind of did not get why OpenAI is planning moves for several years ahead like nothing changes even though they seem to believe in ASI. But now I have changed my understanding of it. There will be no singularity. You can plan for years ahead, as before. Everything will go just as expected on the scale of humanity. There is a 4-month-old post by Sam Altman, The Gentle Singularity. He talks about how the singularity won't be a certain point but gradual progress. And here's a quote I'd like to highlight right now:

We are climbing the long arc of exponential technological progress; it always looks vertical looking forward and flat going backwards, but it’s one smooth curve.

I kind of understood it on the first read, but not all the dots did connect in my head at the time. The key is that you should not think of AI as some non-canonical event. This is just one of the many steps humanity takes while progressing. It will speed up the overall technological progress significantly, but it's just the same as other major advances did. From paper to the worldwide web. All major advances sped up the overall progress, but that is just how exponential progress works. New advances speed up the progress towards more advances. AI is not an exception to the overall trend here, even though it can feel like one. We won't enter a singularity in a way some people think. It all will be just as usual. New advances will happen every day, same as now. The progress will speed up, same as it always did. The whole "self-evolving AI" thing is no different from, for example, how the existence of the internet allows improving the internet itself. To make it all more clear, you can think of an actual exponent. You can pick three points and scale the chart in a way that makes it feel like the last point is "far away" from the previous two, and that the difference between these first two points is minimal compared to the last. We are at that middle point right now. And we are always on it. Whatever we imagine to happen 10 years from now feels much less realistic than whatever happened in the last 10 years. And that's normal. Then, if you move through that exponent, what felt "impossible" now stays on the left tail and feels like it's not that significant. And again, new possibilities open for future advances that again feel much harder to achieve than before. But actually it's all just how it naturally works. And that is what humanity has always experienced. And there is some chance an evil ASI kills humanity, for sure. But there was that chance with many major advances, like when humans made an atomic bomb. And it never stopped humanity from moving forward. There's no point in stopping. We should keep accelerating while considering all the risks.

Writing with AI If you scroll up through my Telegram channel, or open my first blog posts, you will easily notice how they were completely written with AI. Those very obvious patterns that are very easy to spot, like "this isn't just X, it's Y". I was mostly writing drafts myself, but using AI to "finish the paragraph", and also to completely rewrite the whole draft in the end for the purpose of fixing grammar errors and "improving writing". I'm now ashamed of that. A lot of time has passed and only recently I stopped using AI like this. Writing has to come fully from myself, otherwise it makes no sense. It also helps to actually think more about what I write. Before I could just drop a bunch of random thoughts into AI and ask it to write a nice post out of it, but as a result I skipped the whole stage of thinking in a structured way that happens when you write things yourself. I think I changed my mind on all that after seeing hundreds of fully AI-generated posts on X for months. They all look the same. My first posts look this way too when I reread them now. It's soulless, feels cheap, and often provides less value to readers. I'm not using AI this way anymore, at least for things that actually require some thinking from me. I'm now only using it to fact-check, spot grammar errors and give feedback, but never to rewrite whole chunks of text or to finish my thoughts. I'm writing everything myself, then asking AI to give feedback to me, then changing things myself. For random posts on X, I don't even do these last steps. But I definitely use AI for boilerplate stuff, like prompts for AI itself or some Slack messages. Those things don't require much thinking from my side anyway. And I think for such cases, it's fine.

This February I made a prediction that AI would surpass top human performers in competitive programming by the end of the summer. For context, at that point, the best models we had were o1, Sonnet 3.6, and Gemini 2.0. They couldn't reliably solve even simple tasks, and the only hint we had was the late-December preview of o3 (which eventually turned out to be a different version from what we got in April). This "o3-preview", when benchmarked on Codeforces contests, ranked in the 99.8th percentile, which is extremely good, but it was still far behind top performers, positioned at about ~130th place. In the past couple of months, Google and OpenAI have been showing off their experimental reasoning models' skills by participating in various math and coding olympiads. Just today, they both posted about their results at ICPC, which is the most prestigious competitive programming olympiad, where university students from different countries participate in teams of 3. Google's system solved 10/12 problems, which is enough for a gold medal already. And OpenAI got a stunning 12/12, which is even more than the top-1 team from Saint Petersburg solved. Basically, AI could solve all those extremely difficult problems given the same time and constraints as human contestants, and actually achieved first place, which I think can count as a successful resolution to my prediction. Even earlier, in December, I made a more general prediction for 2025, saying that 1. AI would surpass humans in different technical fields (mostly meaning math/coding); and 2. AI would discover a lot of new science. We still can't say that AI has "surpassed" humans in all those things, and it's actually quite hard to define that outcome. And AI hasn't discovered much this year either, but I think with these recent results in various olympiads we're definitely getting there very quickly. Overall, I still believe in those timings, and the general timing for ASI in 2027, or even earlier.

Why did many people have a bad first impression of GPT-5? Actually, the reason behind that is absurdly stupid. OpenAI fucked up with UX. That's it. The model is actually good; all variants of it are. But OpenAI rushed the release for some reason, and their goal of making the UX better made it worse for a lot of users. The key detail here was the model router that they added to ChatGPT so that users don't have to manually choose a model, and it can just choose the appropriate one on its own. For example, if you ask it how to pronounce a word, that can easily be answered with a non-thinking model, with lower latency and the same accuracy. But if you give it a math problem, ask something about coding, or just generally give it a task that requires more reasoning - it is better processed by a thinking variant of the model. And the idea is good, especially for the average user who doesn't know much about how these models work and doesn't want to think about which model to choose for every query. But the implementation was very bad in the first couple of days, and OpenAI confirmed that themselves. The router was working poorly, not choosing a thinking model for complex queries when needed, and not only that, the information about which model answered the query was also hidden, so, for example, when your request (as a free/plus user) was routed to "GPT-5 mini", you couldn't know that. There's not even a "mini" model in the model picker. And another factor is the "reasoning effort" parameter that OpenAI's models have. It determines "how hard" the model thinks before answering. If you need quicker answers, choose "low"; if you need more reasoning for more complex tasks, use "medium" or "hard". And the thing is that in ChatGPT you can't choose that setting yourself. It's part of the router, too. And the information about this setting is also hidden from users. It turned out that most of the requests from free/plus users were processed either by a non-thinking model, or by a thinking model but with the "low" reasoning effort setting. And sometimes also by "mini" models when limits for the regular ones were exhausted. And the performance is, expectedly, bad under those circumstances. So, even when paying users tried out GPT-5, they were often getting bad results. And that was their impression of GPT-5. And that's why there was so much hate for it online. But OpenAI is fixing those problems, and some are already fixed. So, if you tried out GPT-5 in the first couple of days and didn't like it, consider trying it out again now or in a few days, as it might be much better.

My impression of GPT-5 This was an extremely anticipated release. Literally the whole AI bubble waited for it and watched closely. It's been 2 years since GPT-4, and people expected something extraordinary. Me too. I raised my expectations for GPT-5 in the past few months - hoping that it would basically be "o4" but under a new name. And I expected a capability jump similar to the jump from o1 to o3. I was also watching the whole rollout extremely closely and had tried out GPT-5 before the official release for a few days. First, when it was being tested on LMArena under the codenames "Zenith" and "Summit", and another time when it was available on Perplexity due to a bug. I didn't try it out heavily on real tasks in those days, but I still sent many prompts for testing purposes. And I had a "taste" of it at that time. It felt similar to o3 in vibe, but smarter, more precise, and just generally better. My expectations rose again after trying it out there. I was almost sure it would be an "o4" moment. But then the day of the release came, and I was watching the livestream. It was boring. I turned it off halfway through. And I didn't even try GPT-5 that day and went to sleep. I was already disappointed with the boring presentation. The next day I was scrolling X and looking at feedback from other people, and it was mostly bad. People said it was either on par with o3, or a step down. Then I finally tried it out. And it actually felt better than o3. And GPT-5 pro felt better than o3 pro. I still can't exactly say *how much better* they are, but it's definitely noticeable and significant in many scenarios. It's hard to notice a difference in simple casual chats, but once you give it something complex or put it in an agentic environment - you'll see how it just does a better job than o3 could, and in many cases - much better than any other model could. It also translates to agentic coding. For a whole month beforehand I was extensively using Claude 4 Opus for coding in Claude Code full-time, and it was great. I liked that model and its taste. It was nice coding with it. But honestly, it was pissing me off very often. And so I tried downloading the Codex CLI with GPT-5 inside. The UX of the CLI itself is poor compared to Claude Code at the moment. It is not that developer-friendly. But after trying to code with GPT-5 the same way I did with Opus, I started to notice how GPT-5 was just better. It's not always about the quality of the code, and definitely not about the speed. My first impression is that it not only writes better code overall, but that it's much better at instruction following and tool calling. Those are the things that people liked the most about Claude models. And I liked that too. And many people thought that no model would match Claude in these metrics anytime soon. The thing is that GPT-5 just follows your instructions extremely precisely. And it doesn't do things you don't ask it to do. Claude was pissing me off so much by starting to go off track from instructions in long coding sessions, or even in simple queries when it just did something I did not ask it to do. GPT-5 is just better in this regard. Sometimes it follows instructions so well that you understand that your instructions were bad. And it works so well with long context. I can mention something once early in a coding session, and then I just see how it still remembers, references, and follows that for so long. Opus was missing those things very often, especially in long sessions. It might sound like too much ass-licking for OpenAI, but that's my honest experience with GPT-5. I was sceptical too, especially after seeing that boring livestream and seeing so much hate on X. But after trying all of it out myself, I was really amazed. Is it "o4" level? I'm not sure. More like o3.5.

The Complexity Threshold of AI We see dozens of new LLMs heavily tuned for software engineering tasks, and they're becoming very good at it, very quickly. As models evolved, I started using them more and more for writing code, eventually reaching a point where I almost completely stopped writing code myself. The last time I wrote code manually (or rather, with AI-assisted tab completions) was around four months ago. However, once tasks become larger and more complex, these models quickly become inefficient. They seem to have a certain complexity threshold, beyond which their efficiency rapidly declines. I was mostly using AI either to quickly take projects "from 0 to 1" by iterating on MVPs, or to build small Python scripts for working with LLMs and data. About a week ago, I needed to rapidly build another MVP while iterating on ideas, so I used Claude Code and completed the whole thing within a single day. I wanted to keep developing it, but the code became so messy that changing or adding anything was nearly impossible. Even minor adjustments, like updating the UI, caused other parts to break. At that point, I decided I was done with this MVP and needed to re-implement everything from scratch with better structure and architecture. When I started the second implementation attempt, with my "plan" ready, I gave it to Claude Code and watched it fail for hours. It was producing code, but it didn't match my vision and wasn't working as expected. Many architectural and code-level issues remained. I tried slightly adjusting the plan and re-implementing everything multiple times over the span of three days, but it didn't help. After three unsuccessful attempts, I almost lost hope. But then I decided to spend more time refining the specification itself before starting the implementation. I spent two days writing and iteratively refining the specification through feedback loops with the smartest models, while also giving my own feedback on each part. It covered almost everything needed for implementation, from high-level architecture to logic for specific use cases and solutions for the hardest implementation parts that I struggled with the most. Suddenly, when I gave this new specification to the agent and started slowly implementing things one by one, it just worked. I told the agent to implement the next thing, waited 10 minutes, tested the implementation, asked it to fix any issues, and then moved to the next step. A few times during those two days, I also asked another agent to carefully read the entire codebase and strictly compare it against the specification, then passed its feedback back to the first agent to resolve any differences. After about two days, the project was mostly finished. It still has some rough edges (which are easy to address), and I haven't thoroughly *tested* everything yet (I even decided not to write any automated tests at all at this stage), but all the core functionality just worked. When I asked Claude to change something, it usually did so accurately, without breaking other parts. The thing is that AI is much better at following instructions than at coming up with practical stuff on its own. Smarter models can handle more complex tasks independently, but in larger projects, their limit of "acceptable complexity per step" is lower. Therefore, when working on a bigger and more complex project, it's important to keep all individual steps at the same level of complexity and depth as you would when working on smaller projects. The complexity of each individual step matters more than the complexity of the whole project.

✍️ New Blog Post: "Billions of Tokens Later: Scaling LLM Fuzzing in Practice" I've spent months running LLM-powered fuzzing at production scale—processing billions of tokens, discovering practical scaling laws, and developing effective deduplication strategies. Here’s what I learned along the way: https://gusarich.com/blog/billions-of-tokens-later/