r/Automate • u/RedEagle_MGN • 26d ago
Grok 4.7 just dropped and the price-performance is interesting
https://i.imgur.com/QDpLl1E.png8
u/InterestingRespond38 26d ago
Rough one to present in a meeting, the price axis runs from $18 down to $0 so everyone reads the slope as cheaper getting worse. Put cheap on the left and expensive on the right before you show it to anyone.
7
3
u/PositionSmart1699 20d ago
El eje X va de $18 a $0, ¿quién dibuja un gráfico leyendo de derecha a izquierda? Me está costando más de lo que debería seguir la línea negra así.
2
u/Brilliant-Lead-3672 23d ago
The x axis runs $18 down to $0 and every line just falls with it, so that's not a price-performance chart, it's the price list with extra steps.
2
u/RegionDifficult4005 23d ago
your x-axis goes from $18 down to $0 left to right, so the interesting part is on the left, which is the expensive part. also who is Fable 5.1, that name sounds made up
2
u/Infinite-Draw-5420 23d ago
Fair, the flat pricing is genuinely tempting, but that chart is cutting off way too much. The y-axis stops at 55% so anything above it just vanishes off the top, and the lines on the same plot for Fable and Opus are sitting past the cut. If you want the actual price/perf read, plot cost per task against score instead of per-token price, tokens per task swing by 3x between models.
2
u/Eastern_Run8512 23d ago
whose numbers are these? a chart plotting prices from $18 down to $0 against a 20-55% score window is a very convenient zoom, and i'd want to know if that $18 means per million output tokens or some blended rate. also curious what an OPT-6.8b is doing on the same line as Fable 5.1
1
u/Apprehensive-Big4150 26d ago
Fable 5.1 and Okus 5 sitting next to Sonnet 5 like those are real models is a choice. Also the price axis is backards, $18 on the left down to $0. 'Interesting' is doing a lot of work here
1
u/Choice_District7368 25d ago
Small thing: Claude 3.5 and Sonnet 3.5 appear as two separate points on your chart, but Claude 3.5 Sonnet is one model. Unless you're comparing two different versions, in which case ignore me.
1
1
u/Sharp-Chipmunk8113 23d ago
Legend tiene "Olaus 5" y "QPI-5.6-Sat", no sé si la gráfica la hizo un OCR o alguien escribiendo con los codos.
1
u/Orchidivy 23d ago
Wait, that AI CP generator is still alive? I assumed it had already been euthanized by its own Terms of Service.
1
1
u/Proper-Emu5264 23d ago
Small thing, the chart is titled '55% CursorBench 4.0 score' but there are five curves on it. So the 55% belongs to which model exactly? A bit confusing as a label.
1
u/Such-Definition1920 22d ago
Un graphique où tous les modèles prennent la même pente descendante, ça ne dit rien sur le rapport qualité-prix de Grok en particulier. À ce compte-là le modèle le moins cher gagne à chaque fois, bravo. Le seul truc qui compte c'est à effort égal, parce qu'Opus 4.7 en high et en max c'est pas la même facture pour la même tâche, et je vois pas cette nuance dans la courbe.
1
u/EnoughBaker2405 22d ago
Your x-axis runs from $18 down to $0, so the cheap models sit on the right. That is just a score chart with the labels flipped, there is no price-performance story in it.
1
1
u/PositionSmart1699 22d ago
¿Y ese eje de precio es por millón de tokens o por tarea terminada? Porque por millón de tokens hasta el modelo más tonto parece una ganga si no cuentas los que se come para acabar lo mismo.
1
1
u/Adventurous-Word-453 22d ago
first of all why is the x axis going from $18 down to $0, thats backwards, every chart ive ever seen puts cheap on the left not on the right, so now my brain reads it as the expensive models getting worse which ok maybe they are but thats not what a price-performance chart is supposed to show, and second, "price performance is interesting" for a model that went from 40.4 to 46.3 on CursorBench which is still under 50%? congrats you failed less than half the coding tasks instead of failing 60% of them, at $2/$6, which is the same price as 4.6, so the only thing that changed is the number going up a little and the benchmark names getting longer, CursorBench 4.0, DeepSWE v1.1, EEBench, AA Briefcase v1.1, who is running all these benchmarks and who pays for them, and every single launch now beats GPT and Fable on the two benchmarks nobody has ever heard of like electrical engineering, sure, the legal one too, 19.6%, thats the highest of four models, so like one in five legal tasks, wouldnt hire it as a paralegal. sorry, "SpaceXAI", i still cant get over that name. same base price as 4.6 they say like its a gift, of course it is, theyre not gonna raise the price on a model that barely moved
1
u/Responsible-Gene2627 22d ago
o1-5.6.9d. Somone's cat walked across the keyboard during the version bump and nobody had the heart to say anything.
1
u/ProfessionalNet5513 22d ago
so theres a red line in the chart and its not in the legend. Fable 5.1, Ola 5, Grok 4.7, Sonnet 5, and then a red one with no name, and im supposed to read price-performance off that. which one is it.
also the price axis runs 18 down to 0 left to right, so the cheap stuff is on the right and every line climbs as it gets more expensive, which is just, price. what exactly is interesting here, you didnt say. y axis stops at 55% too, thats a very tight crop, you can make a flat line look like a mountain with a crop like that. i've never heard of Ola 5, is that new? and theres no actual numbers, no per token pricing, nothing about context, just a chart.
1
u/Unusual_Medicine_476 22d ago
The price axis runs $18 on the left down to $0 on the right. Minor thing, but cheaper models end up on the side where you expect the expensive ones, so I keep reading the Grok 4.7 line the wrong way. Flip the axis and the chart says the same thing with less squinting.
1
u/ResponsibleEgg3878 22d ago
ok first of all who let OPT-6.8b onto that chart. thats a 2022 model, why is it standing next to Fable 5.1 like theyre in the same conversation. and the x axis runs 18 to 0 so the whole thing reads right to left, i spent a good minute reading the expensive end as the cheap end. anyway the thing nobody wants to say is that price per million tokens is a garbage way to compare anything agentic, grok at 2/6 looks unbeatable until you realize you get billed for every token it spends thinking, and then the same task costs you twice. CursorBench is one bench. xAI's own comparison has Fable 5.1 Max ahead of it on CursorBench and Terminal-Bench, so 'interesting' is carrying a lot of weight in that title. i'll believe it when someone runs it on a real repo for a week before i move anything off Claude
1
u/Exotic-Sherbet-1917 21d ago
the x axis runs $18 on the left down to $0 on the right so I read the whole thing backwards and assumed every model got worse the cheeper it got. also why is Claude 3.5 sitting on a 2026 chart, is it there as a control group
1
1
1
u/Sad_Grapefruit_7187 21d ago
"price-performance is interesting" is such a nothing sentence though. interesting how. the x axis just says price in dollars, price of what, per million tokens, blended, input only, output only, at what cache hit rate? because the ranking flips completely depending on whether your workload is mostly input or mostly output and thats not a small detail. and the y axis starts at 20 so everything up top is squashed and grok 4.7 and opus 5 look like theyre basically sitting on each other. also what the hell is o3-5.6-bsl, is that an openai model? i cant keep up with the naming anymore. falcon 5.1 showing up on a coding bench chart is the part i actually want explained. im just tired of every release being "interesting" with a chart that doesnt tell me if it holds together on a long context task
1
u/Alarmed_Ad_8402 21d ago
million tokens? And what is OPT-6.8b doing on a 2026 chart next to Sonnet 5
1
u/Ancient-Advantage655 21d ago
the x axis goes $18 down to $0, right to left. i keep reading the chart backwards and thinking the cheap models are the expensive ones. who plots price like that
1
u/Soggy_Search_8526 21d ago
the y axis caps at 55% so every model on that chart is failing damn near half the tasks and we're out here comparing their price tags
1
1
u/Salty_Kangaroo_915 21d ago
Funny how every one of these charts tops out at $18 and CursorBench is the only axis on it. Grok 4.7 wins the exact benchmark the whole post is built on, im not shocked.
1
u/Ordinary_Key_488 20d ago
El eje de precios va de 10$ a 0$, o sea que el gráfico se lee al revés y aun así parece que todo mejora hacia la derecha. Y una puntuación en CursorBench no es lo que te cuesta terminar una tarea.
1
u/First-Description555 20d ago
Un axe de prix qui va de 0$ à 18$ à côté d'un score entre 20 et 55%, ça ne dit rien sur le rapport qualité-prix. Ça dit juste que le gratuit est en bas et le cher en haut, ce qui était déjà connu avant le graphique. Le coût par tâche résolue sur un vrai repo, c'est ça qui m'interresserait.
1
u/Wide_Opening_8340 20d ago
e price axis goes from $18 down to $0 left to right, so cheap is on the right. That's not how anyone reads a chart.
Also, $18 for what? Blended per million tokens, output only, list price before whatever discount?
1
1
u/InterestingRespond38 20d ago
Sympathies on that price axis, running $18 down to $0 squeezes every model people actually pay for into the left edge where you can't read the lines at all. Log scale, or split the sub-$6 crowd into its own panel and it'd say something.
1
u/Ok_Relative_3684 20d ago
Took me a second to realize your price axis runs backwards, from $18 down to $0, I kept reading the lines the wrong way. Sort the models by score next time and it'll be obvious whether Grok 4.7 sitting near the cheap end actually buys you anything over Sonnet 5.
0
u/Separate-Pickle2580 20d ago
The x axis on that chart runs from $18 down to $0, which is a weird way to plot price. Took me a second to figure out why cheap was on the right.
Also what is Rable 5.1
1
u/Eastern_Run8512 20d ago
price on the x-axis, but price of what? per task or per million tokens? those two order the chart completly differently, and "interesting price-performance" only means something once you say which one it is.
1
u/Key_Hornet_998 20d ago
Price per million tokens is the least interesting number on that chart, these models don't burn the same amount of tokens for the same task. Run it on your own repo for a week and look at the invoice instead. Also, why is there a point labelled GPT-5 & Sonnet sitting on the Grok line?
1
u/BeginningAmbitious26 19d ago
CursorBench is Cursor's own eval page, so a "price-performance" chart pulled from it is basically a sales tool. Until someone runs Grok 4.7 on their own repo, cost per task means nothing.
1
1
u/Adventurous-Word-453 19d ago
why is the x axis running from $18 down to $0 lmao, so every line is "better" the further right you go and cheapness looks like quality, thats a very flattering way to draw a chart for whatever model you're rooting for. and whats the y even, score at what effort? because if you put opus 5 on high and grok on default you've compared two different things and called it a benchmark. thats the whole trick with these posts, the numbers are real and the framing does all the work. 2/6 vs 5/25 is genuinely a big gap though, like 4x on output, so sure its cheap, but i've been burned by cheap before, every time a model comes in at a third of the price the first thing you notice is it forgets what you told it 40 messages ago and then you're paying in babysitting. and fable 5.1 at the top of that chart, whatever fable is, nobody in these threads ever actually uses half the names on the list, they just screenshot the ones that make their point. the axis is still backwards though, that part isnt an opinion
1
u/Lanky-Employment-985 26d ago
grok 4.7 isn't a model, neither are fable 5.1 or o3a 5. this looks like a made-up chart. and the 'downward trend' is just what a price axis does, cheaper models score lower. that's not a finding. where's the source?
0
-4
u/RedEagle_MGN 26d ago
xAI just released Grok 4.7. The headline numbers from their blog post:
CursorBench 4.0 (longer-running coding tasks): 46.3% vs Grok 4.6 at 40.4%, GPT-5.6 Sol at 41.7%, Fable 5.1 at 51.8%
DeepSWE v1.1: 71.0% (high effort) vs 65.2%, 72.7%, 70.0%
Terminal-Bench 4.0: 38.0% vs 20.3%, 37.3%, 57.9%
Harvey Legal Agent Benchmark: 19.6% vs 15.8%, 2.5%, 6.7%
HealthBench Professional: 56.7% vs 48.5%, 60.5%, 62.1%
EEBench (electrical engineering): 64.0% vs 53.0%, 39.4%, 56.4%
Pricing: $2/M input, $6/M output. Same as Grok 4.6. GPT-5.6 Sol is $4/$20, Fable 5.1 is $10/$50.
The thing that stands out to me is the legal benchmark. 19.6% doesnt sound great in absolute terms but it is beating everything else in that comparison by a wide margin. The EEBench number is also solid at 64%.
It also tops LatchBio biosafety benchmark at 62.4% and only lets 3.3% of risky dual-use cyber prompts through on HackerBench v0.3.
Available now in Cursor and Grok Build. API at console.x.ai.
Source: https://x.ai/news/grok-4-7
2
-3
u/Plane-Schedule-3939 26d ago
Grok 4.7? xAI is on 4.1 as far as I know, and there's no Fable 5.1 or QPT-3.6 either. Where's that chart from?
1
31
u/ghostfaceschiller 26d ago
What a bizarre way to set up this chart