S
@sensho
didn’t u hear bro francois doesn’t believe in harnesses in benchmarks. not agi unless the model spawns in with perfect innate capabilities and not allowed to learn/improve with tools.
17
1
1.4K
@scaling01posted on X
there's a chance ARC-AGI-3 is already solved with GPT-5.5-xhigh + tools
View original tweet on X →
Community Sentiment Analysis
Real-time analysis of public opinion and engagement
Sentiment Distribution
83% Engaged
25% Positive
58% Negative
Positive
25%
Negative
58%
Neutral
17%
Key Takeaways
What the community is saying — both sides
Supporting
1
Tools trump raw IQ:
Many replies argue that model performance depends far more on whether it can test, search and recover in a loop than on raw parameter-count or single-turn intelligence.
2
Base-model capability still matters:
Critics say tools augment but do not replace the need for a strong underlying model—planning, understanding prompts and error-correction still rely on core architecture and training.
3
“GPT 5.5 Pro” feature wishlist:
Requests focus on better plugin/tool integration, longer context, faster inference, local control, and robust developer APIs rather than just a new version name.
4
Safety and misuse concerns:
People worry that autonomous tool access creates dangerous feedback loops (unsupervised web queries, code execution), and call for strict permissioning, auditing, and sandboxing.
5
Evaluation must evolve:
Several replies emphasize the need for benchmarks that measure closed-loop, tool-using behavior (multi-step retrieval+execution, recovery from failure), not just static LLM metrics.
6
Versioning skepticism:
A portion of replies treats “5.5 Pro” as marketing—demanding transparent changelogs and reproducible gains instead of hype-driven names.
Opposing
1
xhigh + tools
is misleading — the right unit is cost-per-successful-task at the budget cap. Without a capped inference spend many results are an expensive lottery and evaporate when you constrain cost.
2
innate capabilities
only, not be propped up by toolchains or allowed to learn/improve during evaluation.
3
10x better capabilities
and models that are ~1000x cheaper to reach a sustainable progress curve.
4
move the goal‑posts
to claim progress rather than accept real failure modes.
5
sub‑1%
for GPT-5.5-xhigh). Inflating scores by changing the rubric conflates success on easy cases with mastery of hard ones.
6
novel rule construction from few examples
(a structural gap, not just a budget problem).
7
METR and ARC
are both accused of being poor tests that fail to measure the right capabilities.
Top Reactions
Most popular replies, ranked by engagement
K
@kittingercloud
Waiting for Fran to move the goal post again.
13
0
1.7K
B
@bytecrafter_1
ARC-AGI-3 'solved' with xhigh + tools is the wrong unit. cost-per-successful-task at the budget cap is what determines whether the result is reproducible or just expensive lottery. cap inference spend and those numbers usually halve.
2
0
908
P
@PhilP2874
https://t.co/LqsXN8b1CS
0
0
680
M
@micahrmiller13
Tools are the hidden variable. Raw intelligence matters less once the environment lets the model test, search, and recover inside the loop.
0
0
703
P
@ptremblay
what about GPT 5.5 Pro?
0
0
79
Report an Issue
Found something wrong with this article? Let us know and we'll look into it.