A historical launch, not a current announcement
This archive edition examines the March 2025 launch record. It is not reporting a new Gemini release today. On 25 March 2025, Google introduced Gemini 2.5 and presented Gemini 2.5 Pro Experimental as its advanced model for complex tasks. Google called the family “thinking models” and said it intended to build thinking capabilities directly into all its models. [1, 4]
That was a consequential product-positioning move. In 2025, vendors were increasingly packaging extra inference-time work as “reasoning”: a way to spend more computation and time before producing an answer. TechCrunch situated Gemini 2.5 in that competitive category and noted that such systems use additional computing power and time. The launch evidence therefore supports a claim about Google’s model direction, not a finding that the system possessed human-like reasoning.
The language deserves care. Ars Technica described the process as “simulated reasoning” and noted there was no evidence that it resembled human reasoning. That distinction remains useful. “Thinking” was Google’s label for a model behavior and product strategy. It should not be inflated into a cognitive claim. The practical question for a user was narrower: does deliberate inference improve a defined task enough to justify its latency, cost and operational complexity?
What the published scores show
Google’s launch post made broad capability claims in reasoning, mathematics, science and coding. It reported 18.8% on Humanity’s Last Exam for models without tool use, and 63.8% on SWE-Bench Verified with a custom agent setup. [1, 4] These figures are concrete published evidence, but they do not measure one universal property called intelligence.
The first score concerns a demanding knowledge-and-reasoning evaluation. Google described Humanity’s Last Exam as a dataset designed by subject-matter experts to probe the frontier of knowledge and reasoning. The responsible conclusion is limited: Google reported 18.8% under the stated condition. It does not establish reliability in every professional setting, nor does it show how a production workflow would behave.
SWE-Bench Verified is closer to a software-engineering task, but its qualifier is essential. Google’s 63.8% figure used a custom agent setup. [1, 4] An agent outcome depends on more than the base model: prompts, repository setup, tool permissions, search strategy, retries, test execution and patch-validation rules may affect it. TechCrunch added a useful comparison, reporting that Gemini’s cited result trailed Claude 3.7 Sonnet’s 70.3%. [1, 4] That does not prove a permanent ranking; it demonstrates that benchmark claims are configuration-specific.
Google also cited a leading LMArena position. Human preference can be relevant to style, clarity and perceived helpfulness. It is not equivalent to correctness, security, factual grounding or successful completion of a business process. A response can be preferred and wrong, or accurate but less polished. Procurement should keep preference testing separate from correctness and safety testing.
One million tokens meant capacity
At launch, Gemini 2.5 Pro shipped with a 1 million-token context window. Google said 2 million tokens were coming soon, making the larger number a roadmap statement rather than a deployed launch capability. [1, 4] This matters because technical plans often turn an announced maximum into an assumed dependency. In March 2025, teams could test the 1 million-token model; they could not safely require the promised expansion.
Google said the model could handle text, audio, images, video and entire code repositories. A long context window can reduce document splitting and simplify first-pass review of heterogeneous material. But accepting a large input is not the same as understanding it. Long-context quality also depends on retrieval of relevant details, instruction adherence, weighting of evidence, grounding, tool design and output verification.
The correct evaluation question was not simply, “Can it ingest our corpus?” It was whether it could consistently identify relevant evidence, respect access boundaries, explain its basis and produce auditable work on representative data. Context capacity changes architecture. It does not remove the need for retrieval tests, citation checks and adversarial cases.
Coding required a workflow test
Google highlighted code generation, editing, transformation and agentic applications. Ars Technica independently reported testing a showcased capability: generating a working video game from one prompt. That is useful, bounded corroboration that a public model could perform the demonstrated task. It is not evidence that an autonomous coding system is ready to modify a valuable codebase without controls.
InfoQ reported an individual user complaint about incomplete code output and destructive repository changes in an agent workflow. This is anecdotal, not a controlled study. Still, it identifies a credible operational risk: capable code generation can become unsafe when coupled to broad tools and write permissions. The relevant unit of assessment is the full loop—task specification, isolated repository, constrained permissions, visible diff, test execution, human review and rollback—not an attractive standalone demo.
A sensible pilot starts with bounded changes in disposable environments. Require diffs; run existing and task-specific tests; exclude production credentials; preserve human approval before merge or deployment; and record regressions and recovery effort. This does not negate Google’s 63.8% result. It places a vendor-reported, custom-agent benchmark in the operational setting where value and risk occur. [1, 4]
The archive lesson from later lifecycle evidence
Later release notes do not change what was announced in March 2025. They do, however, provide context for adoption discipline. Google’s 2026 API notes record model shutdowns and the introduction of lifecycle stages and deprecation timelines. [2] The durable lesson is that frontier-model integrations need version pinning, migration tests and a viable exit path.
At launch, Gemini 2.5 Pro Experimental was available through Google AI Studio and Gemini Advanced. Vertex AI was described as forthcoming, and API pricing had not been published. [1, 4] That incomplete commercial picture was material. A technically promising experimental model can still have unresolved cost, quota, interface and production-support conditions.
The balanced verdict is not that the launch claims were empty, nor that benchmarks named a general winner. Google published meaningful scores, context capacity and a clear product direction. Independent reporting supplied comparison and caution. The strongest interpretation is practical: Gemini 2.5 Pro Experimental marked a period when reasoning-oriented, multimodal, long-context models became serious objects of workflow evaluation. Treat the benchmark as an input, not a deployment approval; distinguish vendor assertions from independent observations; and make reversibility a design requirement.