Paper deep dive
Reconstructing Implicit Scientific Knowledge: Evaluating LLM Agents through End-to-End Reproduction of Astronomy
Yuehui Wang, Xinyu Qi, Guirong Xue, Cheng Wang, Yangbin Xie, Xiaoyu Tang, Cong Sun
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 9/30/2026, 7:03:53 AM
Summary
This paper evaluates Large Language Model (LLM) agents' ability to reconstruct implicit scientific knowledge through end-to-end reproduction of astronomy studies. Using a multi-agent framework (Hermes), the authors analyzed fourteen papers (one from The Astrophysical Journal and thirteen from Nature). They found that eleven of the thirteen Nature papers contained methodological ambiguities preventing unique reproduction paths. In a controlled case study of a paper by Rix et al., twelve predefined analysis paths yielded spatial scale estimates ranging from 2.16 to 3.53 kpc, with only one matching the published value of ~2.70 kpc. The study concludes that matching published results does not validate the reconstruction of underlying reasoning, as agents often fail to recognize the causal relevance of implicit information (e.g., parallax zero-point corrections) until the effect is visible through systematic exploration.
Entities (8)
Relation Signals (6)
Hermes â usedby â LLM agents
confidence 95% · The workflow was orchestrated with Hermes, an open-source multi-agent framework
LLM agents â struggleswith â Implicit Scientific Knowledge
confidence 94% · the bottleneck is as often a failure to connect relevant information as to retrieve it
Gaia Parallax Zero-Point â influences â Spatial Scale (Sigma RGC)
confidence 93% · the decisive information (a +0.02 mas parallax zero-point correction)... agents did not recognize its causal relevance
Rix et al. (2022) â publishedin â The Astrophysical Journal
confidence 92% · published by Rix et al. in The Astrophysical Journal in 2022
LLM agents â failedtoreproduce â Rix et al. (2022)
confidence 90% · repeated runs failed to align with the values reported in the paper
Rix et al. (2022) â cites â Lindegren et al. (2021)
confidence 85% · cited Lindegren et al. (2021) on the Gaia EDR3 parallax zero-point
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The integration of large language models (LLMs) into scientific workflows is accelerating, yet their ability to reconstruct the reasoning underlying published research remains unexplored. Papers specify explicit procedures while leaving many methodological dependencies-data selection, calibration corrections, priors, and domain assumptions-implicit. This ambiguity complicates the evaluation of LLM-based agents, since a failure to reproduce a result may reflect either limitations of the agent or underspecification in the source. We present a framework that evaluates agents through end-to-end reproduction, separating execution from verification and computational failure from methodological ambiguity. We apply it to fourteen astronomy studies: a case study from The Astrophysical Journal and thirteen papers published in Nature. Eleven of the thirteen contained an ambiguity preventing a uniquely specified reproduction path. In a controlled case study, twelve predefined paths, a 3x2x2 sensitivity analysis over sample definition, sky masking, and parallax zero-point treatment-gave estimates from 2.16 to 3.53 kpc for the same quantity, with only one recovering the published value (about 2.70 kpc). The published value was never used as an optimization target, selection criterion, or stopping condition; the matching path was found only after all twelve had run. Crucially, the decisive information (a +0.02 mas parallax zero-point correction) was already in the paper, but the agents did not recognize its causal relevance until the analysis made the effect visible. Matching a published outcome therefore does not validate reconstruction of the underlying reasoning, and the bottleneck is as often a failure to connect relevant information as to retrieve it. End-to-end reproduction thus serves both as a test of reproducibility and as a framework for evaluating implicit scientific knowledge in AI systems.
Tags
Links
- Source: https://arxiv.org/abs/2609.35900v1
- Canonical: https://arxiv.org/abs/2609.35900v1
Trouble viewing inline? Open PDF directly â
Full Text
46,951 characters extracted from source content.
Expand or collapse full text
ReconstructingImplicitScientificKnowledge:Evaluating LLMAgentsthroughEnd-to-EndReproductionofAstronomy Studies [YuehuiWang] 1 *,[XinyuQi] 1 ,[GuirongXue] 1 ,[ChengWang] 1 ,[YangbinXie] 1 ,[XiaoyuTang] 1 ,[Cong Sun] 1 , 1 [ResearchCenterforScientificDataHub,ZhejiangLab,Hangzhou,China] *Correspondence:[wangyuehui@zhejianglab.org] Abstract Theintegrationoflargelanguagemodels(LLMs)intoscientificworkflowsisaccelerating,yettheirability toreconstructthereasoningunderlyingpublishedresearchremainsunexplored.Papersspecifyexplicit procedureswhileleavingmanymethodologicaldependenciesâdataselection,calibrationcorrections,priors, anddomainassumptionsâimplicit.ThisambiguitycomplicatestheevaluationofLLM-basedagents,sincea failuretoreproducearesultmayreflecteitherlimitationsoftheagentorunderspecificationinthesource. Wepresentaframeworkthatevaluatesagentsthroughend-to-endreproduction,separatingexecutionfrom verificationandcomputationalfailurefrommethodologicalambiguity.Weapplyittofourteenastronomy studies:acasestudyfromTheAstrophysicalJournalandthirteenpaperspublishedinNature.Elevenofthe thirteencontainedanambiguitypreventingauniquelyspecifiedreproductionpath.Inacontrolledcasestudy, twelvepredefinedpathsâa3x2x2sensitivityanalysisoversampledefinition,skymasking,andparallax zero-pointtreatmentâgaveestimatesfrom2.16to3.53kpcforthesamequantity,withonlyonerecovering thepublishedvalue(about2.70kpc).Thepublishedvaluewasneverusedasanoptimizationtarget, selectioncriterion,orstoppingcondition;thematchingpathwasfoundonlyafteralltwelvehadrun. Crucially,thedecisiveinformation(a+0.02masparallaxzero-pointcorrection)wasalreadyinthepaper,but theagentsdidnotrecognizeitscausalrelevanceuntiltheanalysismadetheeffectvisible.Matchinga publishedoutcomethereforedoesnotvalidatereconstructionoftheunderlyingreasoning,andthebottleneck isasoftenafailuretoconnectrelevantinformationastoretrieveit.End-to-endreproductionthusserves bothasatestofreproducibilityandasaframeworkforevaluatingimplicitscientificknowledgeinAI systems. 1 Introduction Largelanguagemodels(LLMs)areincreasinglycapableofperformingindividualcomponentsof scientificwork,includingliteratureretrieval,codegeneration,dataanalysis,mathematicalreasoning, andtheinterpretationofcomputationalresults(Vaswanietal.,2017;Brownetal.,2020;OpenAI,2023). Recentagenticsystemsextendthesecapabilitiesbyallowingmodelstoinvokeexternaltools,execute programs,retrievedatasets,anditerativelyrevisetheiranalyses(Yaoetal.,2023;Shinnetal.,2023;Wu etal.,2023).ThisdevelopmentraisesamorefundamentalquestionthanwhetheranLLMcangenerate scientificallyplausibletext:CananAIagentreconstructthecompletereasoningprocessrequiredto obtainapublishedscientificresultfromtheinformationavailabletoit? DirectevaluationofAIsystemsforscientificdiscoveryischallengingbecausenovelscientificclaims oftenlackanimmediategroundtruth.Scientificreproductionprovidesacomplementary,controlled settinginwhichthetargetresultisalreadyknown,whiletheagentmustindependentlyreconstructthe data,methodological,andcomputationalstepsrequiredtoobtainit(Peng,2011;Stoddenetal.,2014; Baker,2016). However,reproductionintroducesacriticalchallengeforAIevaluation.Scientificpapersarenot executablespecifications;rather,theydescriberesultsandmethodsforhumanreaderswhopossess substantialbackgroundknowledge,disciplinaryconventions,andpracticalexperience.Consequently,a papermayexplicitlystateaparameterwhileleavingimplicittherationaleforapplyingittoaparticular datasetordownstreamcalculation.Multiplescientificallyplausiblechoicesoftenremaincompatible withasinglepublisheddescription.Forahumanexpert,suchomissionsarelargelytransparentbecause domainknowledgesuppliesthemissingreasoning.ForanAIagent,conversely,thesameomissioncan generatemultipleexecutableinterpretationswithdistinctquantitativeconsequences.Consequently,a reproductionfailureisinherentlyambiguous:itmaystemfromanagenticerror,source underspecification,orthefailureofthemodeltocorrectlyconnectrelevantinformationwithinthe document. Thisdistinctionmotivatesourstudy.Ratherthanevaluatingscientificagentssolelyontheirabilityto reproduceafinalnumericalvalue,weinvestigatewhethertheycanreconstructthemethodological dependencieslinkingscientificobservations,assumptions,parameters,computationalprocedures,and conclusions.Weuseastronomyasareal-worldtestbedbecauseastronomicalresearchprovidesarich combinationofpubliclyaccessibledatasets,computationalanalyses,quantitativemodels,and heterogeneousmethodologicalchoices.Weconstructanexperimentalcorpusoffourteenpublished astronomystudies(onein-depthApJcasestudyplusabatchofthirteenpaperspublishedinNature)and developamulti-agentreproductionworkflowthatseparatesexecutionfromverificationwhileexplicitly trackingintermediateartifactsanduncertainty. 2 EvaluationFramework Unlikeamathematicalproof,asoftwaretest,orastandardizedbenchmark,agenuinelynewscientific hypothesismaytakemonths,years,orevendecadestovalidateexperimentally.EvenwhenanAI systemproducesalogicallycoherentandsuperficiallyplausibleconclusion,contemporaneous validationisfrequentlyimpossibleâleavingusunabletodeterminewhethertheoutputrepresentsan originalbreakthrough,acompellingconjecture,orafundamentalerror.Thisintroducesadistinctive methodologicalbottleneckforAI-enabledscience.Withoutimmediategroundtruth,establishing whetherModelAoutperformsModelBinscientificexplorationbecomesintractable,whichinturn impedesiterativeimprovement.Resolvingthisfoundationalchallengerequiresbuildingan experimentalframeworkthatrendersscientificreasoningbyAIobservable,measurable,andassessable. Tooperationalizethis,wereframedtheobjective:ratherthantaskingAIwithgeneratinganentirely noveldiscoveryfromscratch,weevaluatedwhetheranAIsystemcouldtraceascientificinquiry end-to-endâfromformulatingaquestionandsourcingdatatounderstandingmethods,executing analyses,interpretingoutcomes,andrecoveringpublishedconclusions.Thisreproductionprotocol convertsareasoningprocessthattraditionallylacksimmediatefeedbackintoaverifiableand comparativepipeline.Toisolatecorereasoningcapabilitiesfromconfoundingvariables,the experimentaldesignwasstructuredacrossthreecontrollabledimensions:thefoundationmodel,the agentworkflow,andthetargetpublication. Cross-modelcomparisonsweresystematicallyemployedtomitigatearchitecturalbiasesinherenttoany singlefoundationmodel.Becauserawgenerativemodelscannotautonomouslyexecuteend-to-end scientificworkflows,wedeployedamulti-agentarchitecturethatoperationalizesreasoningandcoding intoconcreteactions,includingliteraturesearches,dataacquisition,toolinvocation,execution,and outputvalidation(Boiko,MacKnight&Gomes,2023).Thismulti-stageworkflowtransforms reproductionintoanobservablepipelineofinquiry,analysis,andvalidation,completewithexplicit artifactgenerationanditerativediscussionstages. Tocounteractself-reinforcingevaluationbiasesinwhichagentsexcessivelyvalidatetheirownoutputs, thepipelineexplicitlydecouplesexecutionfromadjudication.Criticalmilestonesincorporate cross-modelaudits,preservationofintermediateartifacts,provenancetrackingforparameters, executionauditing,andmandatoryreportingofuncertaintyandfailuremodes.Asinfrastructure-level errorsweresystematicallyisolatedandresolved,ourfocusshiftedfromthelimitationsofthe problem-solvingsystemstotheintrinsicinformationconstraintssuppliedbythesourcematerialitself. Figure1.Overallworkflowforpaperreproduction.Theagentdoesnotmerelysummarizethepaper;itretraces thechainofevidencefrominputs(paperandpublicdata)throughanauditableprocess(steps,evidence,andgaps) toanoutputintheformofadefensiblejudgment. 3 MethodsandExperimentalSetup Thissectionspecifiesthecomponentsneededtoreproduceourevaluation.Exactmodelidentifiers,run logs,prompts,andper-pathconfigurationsarereleasedwiththecode(seeDataandCodeAvailability); whereavalueisfinalizedatsubmissiontime,itismarkedhereandinthereproducibilitypackage. 3.1 Foundationmodelsandconfiguration Severalfrontier,instruction-followingLLMswereusedasthereasoningengines.Everyanalysiswas performedwithatleasttwoindependentfoundationmodelssothatnosingleconclusionrestsonone modelarchitecture.Theexactmodelidentifiersandversions,accessdates,decodingtemperature, context-windowsettings,randomseeds(wheresupported),andthenumberofindependentrunsper paperarefixedatsubmissionandrecordedinthereproducibilitypackage[tobecompletedat submission:modelnames/versions,temperature,top-p,contextlength,seeds,numberofruns,run dates].Nomodelwasgivenaccesstothepublishednumericaltargetduringpathgenerationinthe controlledcasestudy. 3.2 Agentarchitectureandroles TheworkflowwasorchestratedwithHermes,anopen-sourcemulti-agentframework(NousResearch, 2026;https://github.com/NousResearch/hermes-agent),whichschedulesspecializedrolesandprovides themwithtoolsandapersistentfilesystem.Fiveroleswereused:areader,whichparsesthetargetpaper andaccessiblesupplementarymaterialintostructurednotes;aplanner,whichdecomposesthe reproductionintoanexecutabletaskgraph;acoder,whichwritesandexecutestheanalysis;averifier, whichindependentlyre-derivesandchecksoutputsagainstthepaper;andanadjudicator,which resolvesexecutionâverificationdisagreementsthroughcross-modelauditratherthanself-confirmation. 3.3 Toolchainandexecutionenvironment Agentshadaccesstowebandscholarlyliteratureretrieval,publicastronomicaldataarchives(including theGaiaarchiveand,whererelevant,MAST,TESS,andsurveyreleases),apersistentfilesystem,anda sandboxedPythonexecutionenvironmentwithpinnedpackageversionsandshellaccess.Eachtarget paperwasprocessedinanisolatedworkingdirectoryandsoftwareenvironment.Everydownloaded dataset,generatedscript,intermediateoutput,andparameterdecisionwasloggedwithprovenance.The fullprompts,roledefinitions,packagemanifest,configurations,andintermediateartifactsarereleased withthepaper[tobecompletedatsubmission:GitHub/ZenodoDOIandenvironmentlockfile]. 3.4 AuditprotocolforthethirteenNaturepapers EachofthethirteenNaturepaperswasindependentlyprocessedintworoundsdrivenbydifferent foundationmodels;theadjudicatorreconciledthetworecords,andahumandomainexpertreviewed everyretaineditem.Apaperwasclassifiedaslackingauniquelyspecifiedreproductionpathwhenat leastonemethodological,configurational,orinterpretationalambiguitypreventedanindependentagent fromproceedingfromthepublicmaterialstothereportednumericalresultwithoutmakingachoicethat thepaperdidnotuniquelydetermine.Issuecategoriesarenotmutuallyexclusive(asinglepapercan contributetoseveral),sothedenominatorforallcategorycountsisthenumberofpapers(n=13),and categorycountscansumtomorethanthirteen.Itemswhosestatuscouldnotberesolvedâforexample, becausetheyappearedtoarisefromtextextractionratherthanfromthesourceâwererecordedas unverifiedratherthancountedasconfirmed. 3.5 Controlleddesignofthetwelveanalysispaths Forthein-depthcasestudy,thedegreesoffreedomidentifiedinthepublishedmethodwere(i)the stellar-sampledefinition,withthreealternatives;(i)theskymask,withtwoalternatives;and(i)the treatmentoftheGaiaparallaxzero-point,withtwoalternatives.Theirfullfactorialcombination,3Ă2Ă 2,definestwelvepredefined,independentlyexecutablepaths.Thetwelvepathsconstituteasensitivity analysis,notanattempttoreverse-engineertheauthorsâchoices:thepublishedvalue(â2.7kpc)wasnot usedasanoptimizationtarget,aselectioncriterion,orastoppingconditionduringpathgeneration,and thesinglepaththatmatchedthepublishedvaluewasidentifiedonlyafteralltwelvepredefinedpaths hadbeenexecuted. 4 ExperimentalCorpusandCaseStudies ThestudybeganwithonepaperfromTheAstrophysicalJournal(ApJ),whichsurfacedtheinitial problem,andthenexpandedtothirteenastronomypaperspublishedinNaturetotestwhetherthesame patternwouldrecur(Figure2).Papersweredrawnfromleadingjournalsnotbecausereproducingthem wasanendinitself,butbecausetheyprovidepeer-reviewedscientificquestions,methodological descriptions,andquantitativeconclusionsthatserveasfixedreferencepoints.Threepracticaleligibility criteriawereapplied:(1)theunderlyingdatawerepubliclyaccessible;(2)thereportpresented sufficientlyconcretematerialtodefineameaningfulend-to-endreproductiontaskfromtheavailable publicmaterial;and(3)theheadlineconclusionwasbroadlyacceptedbythefield,sothata reproductiondiscrepancycouldnotsimplybeattributedtocontestedscience.Thesecriteriaconcern feasibilityandtheexistenceofastablereference,notwhetherapapercouldbereproduced;they thereforedonotpre-selectpapersbyresult. Figure2.ReproductionandattributioninAI-assistedscientificresearch.ThecorpuscomprisesoneinitialApJ casestudyandabatchofthirteenastronomypaperspublishedinNature;thelattersuppliestheprincipal corpus-levelevidence. 4.1 TheinitialApJcase:identifyingtheproblem ThefirstpaperwasâThePoorOldHeartoftheMilkyWay,âpublishedbyRixetal.inThe AstrophysicalJournalin2022(Rixetal.,2022).UsingtheHermesframework,weconnectedseveral largelanguagemodelsandranmultipleindependentreproductionattempts.Everyrunrecoveredthe paperâscentralqualitativeresult:metal-poorstarsareconcentratedtowardtheGalacticcentre,andthe twostellarpopulationsshowcleardifferencesinchemicalcompositionandorbitalproperties.Atthe quantitativelevel,however,repeatedrunsfailedtoalignwiththevaluesreportedinthepaper.The agentsbegantracingthediscrepanciesbackwardthroughtheworkflow(Figure3). Figure3.TheworkflowusedfortheRixetal.(2022)reproduction. Thefirstanomalyappearedinsampleselection.Followingthemethodstatedinthepaperproduced nearly30,000stars,whereasthepublishedanalysisusedapproximately18,000â20,000.Onthebasisof thepapertextandcurrentlyavailablepublicdataalone,wecouldnotreconstructthesamecounting convention.Thediscrepancycouldreflectanunstatedsubsampledefinition,additionalqualitycuts, deduplicationrules,ordifferencesindataversionsâdetailsthatwerenotfullyandunambiguously specifiedinthearticle(Figure4). Figure4.Differencebetweenthepaperâsreportedsamplesizeandtherecoverableselectionconventions:one statedcutdoesnotreproducethereportedsamplesize. Asecondanomalyinvolvedthepaperâscentralspatial-scaleparameter,theradialdispersionofthe metal-poorpopulation,denotedÏ RGC (seeFigure5).Thepaperreportedavalueofapproximately2.7 kpc.Theinitialreproductionrecoveredthequalitativetrend,butthepublicmethoddidnotuniquely determinetheexecutablechoicesgoverningsampledefinition,skymasking,andparallaxzero-point treatment;accordingly,itdidnotuniquelydeterminethe2.7kpcvalueeither.Atthatstage,werecorded themismatchasanunresolveddiscrepancy.Wedidnotchooseparametersaccordingtohowclosely theyreproducedthepublishednumber. Figure5.QuantitativereproductiondiscrepancyforthespatialscaleÏ RGC . Atthetime,wecouldnotlocatethesourceofthisfirstreproductionfailureprecisely.Itmighthave reflectedgenuinelymissingpracticalinformationinthepaper,andthusapotentiallyimportant scientificclue.Butitcouldjustaseasilyhavearisenfromourownlackofdomainexpertise,ahidden defectintheagentpipeline,orafailuresomewhereinthemodelâtoolchain.Wethereforesetanew goal:reproduceasmallgroupofpapersfromthesamebroadfieldwhilesystematicallyrefiningthe workflowandreducinginterferencefromthepipeline,agents,models,andhumanoperation. 4.2 ExpandingtothirteenNaturepapers Whenthefirstpapercouldnotbereproducedexactly,theagentsystemwasourprimarysuspect.Aswe refinedthepipeline,however,ourattentionshiftedfromthesystemsolvingtheproblemtothe informationsuppliedbytheproblemitself.WeselectedthirteenastrophysicspaperspublishedinNature andranabatchreproductionexperiment.Acrossthethirteenpapers,elevencontainedatleastoneissue thatpreventedauniquelyspecifiedreproductionpath.Onlytwoprovidedmethodologicaldescriptions thatweresufficientlycomplete,withinthescopeofourtasks,tosupportanindependentpathfromdata tothetargetnumericalresult(Figure6). Figure6.Auditofmethodologicalinformationinthethirteen-paperNaturesample.Elevenofthirteenpapers containedatleastoneissuethatpreventedauniquelyspecifiedreproductionpath. Thischangedhowwethoughtaboutreproduction.Initially,wehadhopedtouseitasanindirecttestof whetherAIcouldperformscientificdiscovery.Thebatchexperimentsuggestedthatreproductionmight bemorethanawaytoverifyexistingdiscoveries:itcouldalsoprovideanentrypointforfindingnew knowledge.WhenanAIsystemretracesthepathfromdatatoscientificconclusion,research judgementsthatwereomittedortreatedasobviousmaybecomevisibleagain.Adiscrepancybetweena reproducedresultandapublishedresultisnotnecessarilyanerrortobeeliminated.Onceverifiedand explained,itmayrevealarelationshipamongthesourcedata,theanalyticalmethod,andthescientific conclusionthathasnotbeenmadefullyexplicit.Inthatsense,reproductioncanmovefromvalidating existingknowledgetowarddiscoveringknowledgehiddenwithinexistingresearch. 4.3 Returningtothefirstpaper:acontrolledsensitivityanalysis Withthisideainmind,wereturnedtothefirstpaperusingtherevisedreproductionsystem.Thistime, westoppedtryingtotunetheresulttowardthepaperâs2.7kpcvalue.Instead,weaskedadifferent question:ifthepaperdoesnotuniquelyspecifyananalysispath,howmuchcanscientificallyplausible choiceschangetheconclusion?Weturnedthisquestionintoacontrolledexperiment(Figure7).Oneof thecentralresultsinRixetal.isthespatialextentofametal-poorstellarpopulationaroundtheGalactic centre,reportedasapproximately2.7kpc.Severalchoiceswerenotuniquelydeterminedbythe methodologicaldescription:whichstarsshouldbeincluded,whetherasystematicparallaxoffsetshould becorrected,andwhetherregionsclosetotheGalacticplaneshouldbeexcluded.Weenumerated reasonableoptionsforthesethreedimensionsandgeneratedtwelvepossibleanalysispaths.Thesepaths werenottwelveguessesaboutwhattheauthorshadactuallydone;theywereasystematicexploration oftheconsequencesofchoicesthatthepaperdidnotreducetoasingleexecutablespecification. Figure7.Attributionlogicforthreefamiliesofcontrolledcomparisons:changeonevariablewhileholdingthe othertwofixed. Alltwelvepathsconvergedandexecutedsuccessfully,producingspatialscalesfrom2.16to3.53kpc. Onepathâaparticularsampledefinition,aparallaxzero-pointcorrection,andthefull-sky regionâproducedapproximately2.70kpc,closelymatchingthereportedresult(Figure8). Toforecloseanyreverse-engineeringinterpretation,westateexplicitly:thepublishedvaluewasnot usedasanoptimizationtarget,selectioncriterion,orstoppingconditionduringthegenerationofthe12 paths.Thematchingpathwasidentifiedonlyafterall12predefinedpathshadbeenexecuted.Path labels,parametergrids,andstoppingruleswerefixedbeforeanypathwasrun,andnopathwasadded, removed,orre-weightedafterthepublishednumberbecamevisibletotheanalysisteam. Figure8.Sensitivityofthetwelveanalysispathstoparallaxzero-pointtreatment(3sampledefinitionsĂ2sky masksĂ2zero-pointtreatments). Ourprimaryconcern,however,wasnotwhichanalyticalpathcameclosesttothe2.7kpcbenchmark, butwhyâgiventhatalltwelvepathsexecutedsuccessfullyâanysinglepathshouldbepreferredover theremainingeleven.Atthisjuncture,theAIsurfacedaconnectionithadpreviouslyfailedtoestablish. Therelevantinformationwasnotabsentfromthetext:Rixetal.hadalreadyreportedavalueof+0.02 masandcitedLindegrenetal.(2021)ontheGaiaEDR3parallaxzero-point.Ratherthantheparameter itself,whathadbeencompressedwastheunderlyingmethodologicaljudgmentâwhyaconstant approximationremainedappropriateforthisspecificsample,underwhatconditionsthatapproximation held,andwhenitrequiredre-estimationforsubsequentdatareleasesoralternativesamples(Lindegren etal.,2021;ESA,2021).Thiscrucialrationaledidnotreappearasanexplicitstepwithinthe quantitativemethodology.Toanexperiencedastronomertheinferencemayhavebeenintuitive,because thetargetdistancescale,Gaiaparallaxes,andtheknownparallaxzero-pointbiasnaturallylinkthe correctiondiscussiontothedownstreamfit.Thesystem,however,didnotmakethisconnection autonomously.Althoughitsuccessfullyparsedâ+0.02masâandexecutedthesubsequentfitting procedure,itdidnotinferthatavalueintroducedearlierinthetextneededtogovernthequantitative analysis,orthatomittingitwouldmateriallyaltertheoutcome.Onlyafterthetwelve-pathexperiment renderedthiseffectquantitativelyvisibledidthesystembridgeinformationdistributedacrossdisparate sectionsofthepaper.Thesalientfindingisthereforenotthattheparameterwasmissing,butthatits causalrelevancewasnotrecognized.WhilemodernAIsystemscansystematicallyexploreaspaceof analyticalchoicesandexecutealternativesinparallel,Theevaluatedagentscontinuedtostrugglewith recognizingwhyoneempiricalfactiscausallyrelevanttoanotherintheabsenceofexplicitguidanceto investigatethatrelationship. 5 RelatedWork LLMagentsforscience.Transformerlanguagemodelsandinstruction-followingsystemsestablished theunderlyingcapabilitiesusedthroughoutthiswork(Vaswanietal.,2017;Brownetal.,2020;OpenAI, 2023).Buildingontooluse,retrieval,andself-reflection,language-modelagentsnowplanandexecute multi-steptasks(Yaoetal.,2023;Shinnetal.,2023;Wuetal.,2023),includinginscientificdomains, whereLLM-drivensystemshavebeenusedtodesignandrunchemicalexperimentsandtooperate laboratoryinstrumentation(Boiko,MacKnight&Gomes,2023).Ourworktreatssuchanagentasthe subjectofevaluationratherthanasablack-boxperformer. AIforscientificdiscovery.Aparallellineofworkpursuesautonomous"AIscientists"thatgenerate hypotheses,runanalyses,anddraftstudiesend-to-end(Luetal.,2024;Sietal.,2025).Demonstrations ofautonomousdiscoveryaredifficulttoevaluaterigorouslybecausegenuinenoveltyhasnoimmediate groundtruth.Reproductionsuppliesacontrolledlens:thetargetisknown,sothequalityofthe reconstructedpathânotonlythefinalnumberâbecomesmeasurable. Reproducibilityandcomputationalreproducibility.Surveysacrossthesciencesfindthatmany publishedresultsaredifficultorimpossibletoreproduce(Baker,2016),andmethodologicalwork addressesdependencycapture,provenance,andexecutableenvironments(Peng,2011;Stoddenetal., 2014).Weusereproducibilityforadistinctpurpose:ratherthantreatingreproductiononlyasa reliabilitytarget,weuseitasadiagnosticinstrumentthatrevealswhichstepsapublicationleaves implicit. Implicitandtacitknowledge.Philosophicalandsociologicalaccountsemphasizethatexpert competencecontainsatacitcomponentthatisnotfullyexpressibleintext(Polanyi,1966;Collins, 2010).RecentcritiquesofAI-assistedsciencearguethatpublicationscompressmethodological judgment.Ourcontributionisoperational:weturnthepresenceandrecoveryofsuchimplicit knowledgeintoanobservablevariablebymeasuringwhetheranend-to-endreproductionagent re-establishesthemissingmethodologicaldependencies. 6 Results ThebatchreproductionofthirteenpapersandthedeeperinvestigationofRixetal.pointtothesame conclusion:withinthiscorpus,incompletespecificationoftheimplicitknowledgerequiredfor quantitativeanalysisisarecurringpatternratherthananisolatedcase.Acrossthethirteen-paperNature sample,elevenstudiescontainedatleastonemethodological,configurational,orinterpretational ambiguitythatpreventedauniquelyspecifiedreproductionpathwithinthescopeofourexperiments (Figure6).Inthecontrolledcasestudy,twelveindependentlyexecutablepathsyieldedspatialscales between2.16and3.53kpc,withonepathrecoveringthepublishedvalueofapproximately2.70kpc (Figure8).Theexperimentshowsthatmatchingapublishednumberdoesnotbyitselfestablishthatthe underlyingreasoningpathhasbeenreconstructed. Ourexperimentsrevealaspecificlimitationofthescientificagentsevaluatedinthisstudy:whilethe LLMscouldretrieveexplicitinformationandexecutewell-definedcomputationalprocedures,they wereunreliableatrecognizingwheninformationdistributedacrossascientificdocumentiscausally relevanttoadownstreamanalysis.Thehardestpartofpaperreproduction,then,maynotbefindinga parameter,butunderstandingwhythatparameterbelongstothisdataset,thissample,andthisscientific question.Thisdoesnotmeanthatthepapersâconclusionsareunreliable.Apapermustprioritizenovelty; manychoicesinvolvingparameters,datafiltering,calibration,anduncertaintyhandlingarenot themselvesnovel,yettheyaredecisionsthatcannotbemadeincorrectlywithoutconsequences.For domainscientists,suchjudgementsareoftenacquiredthroughtraining,convention,andexperience.For AI,theydonotautomaticallyemergeevenwhenthepaper,data,andcodeareallavailable.Auditing canpreventanAIsystemfromforcingmissingpiecesintoplacesimplytoobtainananswer,butit cannotsubstitutefordomainexperience. 7 Limitations Severallimitationsboundthescopeofourclaims.First,thecorpusissmallandstagedbydesign:one in-depthApJcasestudyplusabatchofthirteenNaturepapers,ratherthanasingleuniformbenchmark offourteen.Second,thecorpusisconfinedtoastronomy,adata-richfieldwithunusuallyopenarchives; thefrequencyandcharacterofimplicitdependenciesmaydifferinotherdisciplines.Third,although auditcategorieswereproducedbytwomodel-drivenrounds,reconciledbyanadjudicator,and confirmedbyhumanreview,theirassignmentremainspartlyinterpretiveandcouldbecodeddifferently byanotherteam.Fourth,LLMcapabilitiesandversionsevolverapidly,soabsolutefailureratesare time-dependentandmaychangewithlatermodels;wethereforereportamechanismandanevaluation methodratherthanapermanentbenchmarknumber.Fifth,reproductionissensitivetodatareleasesand tothecompletenessofpubliclyaccessiblematerial,bothofwhichcanchangeovertime.Finally,the twelve-pathsensitivityanalysisconcernsasinglequantityinasinglepaper;whilethemechanismit exposesâpresent-but-unlinkedcausalinformationâissuggestive,broadergeneralizationrequires largerandmorediversecorpora. 8 DiscussionandConclusion AtVivaTechin2023,YannLeCunarguedthatmuchofhumanknowledgeisnon-linguisticandthatthis experientialcomponenthasthereforenotbeencapturedbycurrentAIsystems.Thesameconcern appliestoourexperimentsonscientificcapability.Paperspreservewhatscientistsdiscovered,butthey donotalwayspreservethefullprocessbywhichscientistsknewthataparticularchoicewastheright one.ThismaybeaneasilyoverlookedriskwhenAIattemptstoconductscientificdiscovery independently. Wearguethatend-to-endscientificreproductioncanservenotonlyasareproducibilitytestbutalsoas anevaluationframeworkforimplicitscientificknowledgeandscientificreasoninginAIsystems.The controlledRixcaseillustrateswhythisframingisuseful:apredefinedsensitivityanalysis,inwhichthe publishedtargetwasneverusedfortuning,showsthatanumericalmatchisneithernecessaryevidence ofcorrectreasoningnorastabletargettooptimizetoward,whereasthefailuretoconnectacorrection alreadypresentinthepapertothedownstreamfitisdirectlyobservable.Ournextgoal,therefore,isnot simplytoreproducemorepapers,buttoenablemodels,throughlarge-scalereproduction,toactively reconstructknowledgeembeddedinscientificpracticebutcompressedinpublication.Asthescale growsfromdozensofpaperstohundredsorthousands,recurringparameterconflicts,methodological variations,andfailuremodesmaybegintorevealpatternsthattransferacrosspapersanddisciplines. Wedonotclaimthatreproductionhasproducednewscientificdiscoverieshere;rather,weshowthatit canserveasadisciplinedentrypointthroughwhichsuchimplicitknowledgeâand,potentially,new scientificquestionsâmaycometolight. AppendixAâImplicit-KnowledgeCluesAcrossThirteenAstronomyPapersPublishedinNature AllthirteenpapershaveDOIsbeginningwith10.1038/s41586-,theidentifierusedbythemainweeklyjournalNature;journalselsewhereintheNatureportfoliouseotheridentifiers,suchas s41550forNatureAstronomy,s41567forNaturePhysics,ands41467forNatureCommunications.Wethereforerefertothebatchthroughoutasâthirteenastronomypaperspublishedin Nature,ânotasNatureAstronomypapers. Statusoftheauditobservations.Theclueslistedbelowareauditobservationsgeneratedduringthereproductionexperiments.Theymustnotbeinterpretedasclaimsthatthecorresponding paperscontainerrors.Someobservationsarisefromtextextraction,fromincompleteaccesstosupplementarymaterial,orfromunresolvedmethodologicalambiguity,andtheyrequire independentverificationagainsttheoriginalarticles.Eachitemisthereconciledproductoftwomodel-drivenroundsplushumanreview;itemsthatcouldnotbeconfirmedareexplicitly marked.Acomplete,machine-readableaudittable,theper-pathconfigurationsforthetwelve-pathexperiment,andallintermediateartifactsarereleasedassupplementarymaterial(seeData andCodeAvailability);thesummarystatisticsreportedinthemaintextarethoseinFigure6. Readingguide.Thecolumnsreport(lefttoright)thepapernumber,title,DOIandcitation,topic,thestudyâsmainfinding,theimplicit-knowledgecluesidentifiedduringreproduction,andthemining approachusedtosurfacethem.Individualcluesareseparatedbylinebreaksanddonotrepresentindependenterrorcounts. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 01Detectionofanisotropic cosmicstructuresona gigaparsecscale 10.1038/s41586-026-10702- 5 Nature,2026 Cosmology·Large-scale structure UsingtheDESIgalaxy surveyandaparameter-free statistic,thestudydetects anisotropyinthegalaxy distributionon billion-light-yearscales beyondthatexpectedin simulateduniverses, challengingthe cosmologicalprinciple. âThreedifferentcountsare givenforthesimulation sample:330inthemain text,1,000inFig.4a,and 200inFig.4bâmorethan onenumberforthesame test. âThepaperdoesnotclearly identifywhatismeantbya ârepresentativeBGS subsample.â âThenumberofangular binsisnotgiven,norarethe radial-binboundariesor partitioningrule. âAformalrecordofthe sameissueasthefirstclue: thesample-count conventionisinconsistent withthesignificance threshold. âThenormalization coefficientinthe angular-varianceformula (Eq.3)iscorruptedinthe extractedtext,soitsexact formcannotbereadreliably. âThesimulatedparticle countistypographically ambiguous:shouldâ1,0243â bereadas1024Âłor somethingelse? Cross-checkconventionsfor simulationcounts, significancethresholds,and relatedquantities.crossthe tworoundsofexperimental records. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 02GW250114reveals signaturesofpost-merger black-holehorizon 10.1038/s41586-026-10696- 0 Nature,2026 Gravitationalwaves·Black holes GW250114reveals signaturesofpost-merger black-hole-horizon behaviourandtests Einsteiniangravity. âTheunitsofthedamping ratearenotspecified. âTwointerpretationsassign differentrolestothesame â300simulated backgroundsâsetup. âThepaperprovidesonlya theoreticalcitationanddoes notstatewherethe NRSur7dq4waveformwas obtained. âItisunclearwhich quantitythe"remnantmap" parametercalibrates;this doesnotalignwiththeangle emphasizedinthemaintext. âThepapersaystheresult is"fullyconsistent"with Kerrtheorywithout providingaconsistency metric,whilealsostating thatÏHÏHâneednot matchexactlyâthetwo formulationsconflict. âDetailsoftheexact analyticalmodeldependon theSupplementary Information,butthetwo interpretationsdisagreeon thedegreeofthat dependence. Mergethetworoundsof experimentalrecordsand placethetwotheoretically contradictoryformulations sidebyside. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 03Adirectblack-holemass measurementinalittlered dotathighredshift 10.1038/s41586-026-10579- 4 Nature,2026 JWST·Black-hole measurement JWSTdirectlymeasuresthe massofablackholeina high-redshift"littlereddot" galaxy. âThesameâ10km/svalue appearswithtwomeanings: velocitygradientversus projectedvelocityperbin. âAnobjectreconstructedby onlyoneinterpretationhas noindependent corroborationfromthe other. âThecentralvalueis readable,buttheuncertainty valuesareunreliableinthe visibletext. âForthe0.05arcsecdata cube,oneinterpretation treatsthevalueasa validationinputandthe otherasanexecution parameter;itsroleis unclear. âThetwointerpretations assignoppositelabelsto whetherthereportedstellar massshouldbetreatedasa "referenceanswer"to preventleakage. âThemassandinclination likewisereceiveconflicting reference-answerlabels. Mergethetworoundsof experimentalrecordsand isolatedisagreementsover âreferenceanswerversus validationinput,âtogether withthe1dexmass discrepancy. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 04Anultra-faint,chemically primitivegalaxyformingin thereionizationera 10.1038/s41586-026-10374- 1 Nature,2026 JWST·EarlygalaxiesThestudydiscoversan extremelyfaint,metal-poor dwarfgalaxythatformed duringtheepochofcosmic reionization. âThetwointerpretations disagreeontheapplicable scopeofmethodM07. âTheyalsodisagreeonhow stronglythe"hardspectrum" interpretationissupported. âThecomplete methodologicalchainforthe dynamicalmassismissing. Mergethetworoundsof experimentalrecordsand enumeratethreesetsof methodological-semantic disagreements:theroleof theoxygen-lineratio,the hard-spectrum interpretation,andthe dynamical-masschain. 05Satellitemegaconstellations willthreatenspace-based astronomy 10.1038/s41586-025-09759- 5 Nature,2025 Spacediplomacy·Satellite constellations Satellitemegaconstellations suchasStarlinkthreaten ground-andspace-based astronomicalobservations. âThepaperisinternally inconsistent:theSPHEREx orbitalaltitudeisgivenas 650kminoneplacebut700 kmintheoverview. âTheHubbleexposure-time boundsarealso reconstructedvalues;the asymmetricboundsstill requireverificationagainst theoriginalmanuscript. Mergethetworoundsof experimentalrecordsand recover,itembyitem,the valuescorruptedduringtext extraction,includingorbital altitude,distancebounds, andexposure-timebounds. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 06Witnessingtheonsetof reionizationthrough Lyman-αemissionat redshift13 10.1038/s41586-025-08779- 5 Nature,2025 JWST·CosmicreionizationLyman-αemissionat redshift13providesaview oftheonsetofcosmic reionization. âThetwointerpretations disagreeonhowthe systemicredshiftwas determined. âNeitherinterpretationcan recovertheEDT3prior distributionormost posterior-tablevalues;the paperdoesnotspecifythe implementationof Ï-clipping,covariance construction,orpath-loss correction. âTheForcePhophotometric andmorphological-fitting configurationisincomplete: neithertheversionnorthe PSFisprovided. Mergethetworoundsof experimentalrecordsand isolatethreesetsof parameterslackinga uniquelyspecifiedsource: priors,Ï-clipping,and photometricconfiguration. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 07Spectroscopicconfirmation oftwoluminousgalaxiesat aredshiftof14 10.1038/s41586-024-07860- 9 Nature,2024 JWST·High-redshift galaxies Spectroscopyconfirmstwo luminousgalaxiesatredshift 14. âTheunitsoftheslit-loss compensationpolynomial arenotspecifiedfor thez=14.0z=14.0galaxy. âThesharedparameterU5 hasconflictingsemantics: thesamenameisusedwith differentmeanings. âThepaperprovidesno numericalreproduction tolerance;another interpretationuses comparisonlanguagebut suppliesnopass/fail criterion. âThesingleexposures, per-exposurePSFs,software version,andconfiguration requiredformorphological fittingarenotprovided. âOneinterpretationcitesa sourceoutsidethepermitted rulesandrequires verification. âThesampler,likelihood implementation,and convergencecriteriaare missing,preventing numericalreproduction. Mergethetworoundsof experimentalrecordsand juxtaposefourgroupsof missingconfiguration details:units,tolerances, priors,andPSFs. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 08Fast-movingstarsaroundan intermediate-massblack holeinÏCentauri 10.1038/s41586-024-07511- z Nature,2024 Blackholes·Globular clusters Fast-movingstarsintheÏ Cenglobularclusterindicate thepresenceofan intermediate-massblack hole. âThetwointerpretations disagreeonwhetherthe robustnesstestswere extendedtothe surface-brightnessmodel, mass-to-lightratio,and distance. âThefullMCMC configurationisnot provided,yetone interpretationlabelsit âdeterminate.â âThegridboundsandstep sizefortheminimum-mass methodareunspecified. âTheexternalsimulation gridusedfortheN-body comparisonisunavailable; neitherinterpretation providessufficient numerical-sampling configurationtoreproduce theposterior deterministically. âExtendedDataTable1 containsmissingcells, althoughboth interpretationsrequirethose valuesfortheircalculations. Mergethetworoundsof experimentalrecordsand identifythethreemethods labelled"determinate" despitemissing configurationdetails. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 09Aresonantsextupletof sub-Neptunestransitingthe brightstarHD110067 10.1038/s41586-023-06692- 3 Nature,2023 Exoplanets·Resonant chains ThesixplanetsorbitingHD 110067forma3:2/4:3 period-resonancechain;two wererecoveredby reprocessingTESSdata. âThemaintextandfigure captionconflict:thetext states120ssampling, whereasthecaptiongives2 minutesforSector23and 20secondsforSector49. âThepredictedperiodof theouterplanetgis54.7433 days,versusthemeasured valueof54.76992daysin Table1.Thishasalready beeninvestigatedindepth: thedifferencecanbe decomposedintothe productofthedetuningsof adjacentorbitalpairs. âThe±orderingofthe asymmetricuncertaintiesin Table1wasreconstructed fromcorruptedtextcells andstillrequiresverification againsttheoriginaltable. Placethetwoperiodvalues sidebysideandcheckthem arithmetically:the 0.0266-daydifference equals133timestheerror bar.Then,independently verifythetransittimesusing locallyavailablepublic TESSdata,recovering agreementwithin2â4 minutes,andcomparethe twoexperimentalrecordsto identifythecontradictionin thecaptionedsampling cadence. 10Thesolardynamobegins nearthesurface 10.1038/s41586-024-07315- 1 Nature,2024 SolarphysicsThesolarmagnetic-field dynamobeginsnearthe surfaceratherthaninthe deepinterior,astraditionally assumed. âThetwointerpretations agreeonthenumerical rotation-profileequation, buttheformulaitselfis problematic:theexact algebraicsolutionofthe quasi-linearequationlacksa parameter. Mergethetworoundsof experimentalrecordsand confirmtheparameter missingfromtheassembled formula.Repeatthe extractionwithadifferent OCRtool. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 11Orbitalperiodchangeof Dimorphosduetothe DARTkineticimpact 10.1038/s41586-023-05805- 2 Nature,2023 Planetaryscience·DART mission Thestudymeasuresthe changeinDimorphosâs orbitalperiodafterthe DARTspacecraftimpacted thebinary-asteroidsystem. âTwoprecisionlevels coexistforthepre-impact period:11.92148±0.00013 h(3Ï)andarounded abbreviatedvalue. âThetwointerpretations classifytheperiod differentlyâasan"external input"oraâmethodoutput.â âThepapergivesno reproductiontolerance.One interpretationproposes passingresultsâwithin3Ïâ; oneincludesaninitialradar estimateofâ36±15 minutes,whiletheother omitsitentirely. Theformalexperimentin thispaperwasnot completed,soanearlier shadow-experiment recordâwhichdidcomplete therunâwasused retrospectively. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 12StarformationneartheSun isdrivenbyexpansionof theLocalBubble 10.1038/s41586-021-04286- 5 Nature,2021 MilkyWay·StarformationStarformationneartheSun isdrivenbytheexpansion oftheLocalBubble. âTheclaimed"qualitative morphology"hasnometric orthreshold.Theexpansion modelisalsoidealized,and theequationâsexponentwas inferredbackwardfroma standardforminanexternal citation.Neither interpretationprovidesa numericalreproduction tolerance. âThespecificvariantofthe MilkyWay-like gravitationalpotentialand itsintegrationsettingsare unspecified;external numerical contentâincludingthe clustercatalogue,ages,and dustgeometryâismissing. Mergethetworoundsof experimentalrecordsand enumeratethreeissues:the potentialvariant, reproductiontolerance,and formulaexponent. No.PapertitleDOI/citationTopicMainfindingImplicit-knowledgecluesMiningapproach 13Adormantovermassive blackholeintheearly Universe 10.1038/s41586-024-08210- 5 Nature,2024 JWST·EarlyblackholesThestudyidentifiesa dormant,overmassiveblack holeintheearlyUniverse. âTheunitsoftheblack-hole massareambiguous,asare thoseofthehostgalaxyâs stellarmass. âWhetherthepaperâsactual processingmethodusesa virialworkflowremainsto beverified. âOneinterpretationhad accesstoonlyalimited portionofthemaintext becauseofdifferencesin materialaccess. âDatadescribedas "expectedtobepublic" representonlyaneligibility expectation,notverified availability. âTheluminosityof~1e45 isthe"corrected"target;the corrected-versus-uncorrecte dluminositychainis ambiguousandisfurther affectedbytext-extraction problems. Mergethetworoundsof experimentalrecordsand enumeratefourissue classes:unitambiguity, reproducibility,simulation availability,andthe luminositychain. References 1. AstropyCollaboration(2022).TheAstropyProject:BuildinganOpen-scienceProjectandStatusofthev2.0 CorePackage.TheAstrophysicalJournal,935,167.https://doi.org/10.3847/1538-4357/ac7c74 2. Baker,M.(2016).1,500scientistsliftthelidonreproducibility.Nature,533,452â454. https://doi.org/10.1038/533452a 3. Boiko,D.A.,MacKnight,R.,&Gomes,G.(2023).Autonomouschemicalresearchwithlargelanguage models.Nature,624,570â578.https://doi.org/10.1038/s41586-023-06792-0 4. Borlaff,A.S.,Marcum,P.M.,&Howell,S.B.(2025).Satellitemegaconstellationswillthreatenspace-based astronomy.Nature,648,51â57.https://doi.org/10.1038/s41586-025-09759-5 5. Brown,T.B.,Mann,B.,Subbiah,N.,Kaplan,J.,Dhariwal,P.,Neelakantan,A.,etal.(2020).Language ModelsareFew-ShotLearners(GPT-3).AdvancesinNeuralInformationProcessingSystems33(NeurIPS 2020),1877â1901.https://arxiv.org/abs/2005.14165 6. Carniani,S.,Hainline,K.,DâEugenio,F.,Eisenstein,D.J.,Jakobsen,P.,Witstok,J.,etal.(2024). Spectroscopicconfirmationoftwoluminousgalaxiesataredshiftof14.Nature,633,318â322. https://doi.org/10.1038/s41586-024-07860-9 7. Collins,H.(2010).TacitandExplicitKnowledge.UniversityofChicagoPress. 8. ESA/Gaia(2021).GaiaEDR3/DR3parallaxzero-pointcorrectioncode.EuropeanSpaceAgency. https://w.cosmos.esa.int/web/gaia/edr3-code 9. GaiaCollaboration(2016).TheGaiamission.Astronomy&Astrophysics,595,A1. https://doi.org/10.1051/0004-6361/201629272 10. GaiaCollaboration(2021).GaiaEarlyDataRelease3:Summaryofthecontentsandsurveyproperties. Astronomy&Astrophysics,649,A1.https://doi.org/10.1051/0004-6361/202039657 11. HĂ€berle,M.,Neumayer,N.,Seth,A.,Bellini,A.,Libralato,M.,Baumgardt,H.,etal.(2024).Fast-moving starsaroundanintermediate-massblackholeinÏCentauri.Nature,631,285â288. https://doi.org/10.1038/s41586-024-07511-z 12. JuodĆŸbalis,I.,Marconcini,C.,DâEugenio,F.,Maiolino,R.,Marconi,A.,Ăbler,H.,etal.(2026a).Adirect black-holemassmeasurementinalittlereddotathighredshift.Nature,653,1017â1021. https://doi.org/10.1038/s41586-026-10579-4 13. JuodĆŸbalis,I.,Maiolino,R.,Baker,W.M.,Tacchella,S.,Soltz,J.,DâEugenio,F.,etal.(2024).Adormant overmassiveblackholeintheearlyUniverse.Nature,636,594â597. https://doi.org/10.1038/s41586-024-08210-5 14. Lindegren,L.,Bastian,U.,Biermann,M.,Bombrun,A.,deTorres,A.,Gerlach,E.,etal.(2021).GaiaEarly DataRelease3:Parallaxbiasversusmagnitude,colour,andposition.Astronomy&Astrophysics,649,A4. https://doi.org/10.1051/0004-6361/202039653 15. Lu,C.,Lu,C.,Lange,R.T.,Foerster,J.,Clune,J.,&Ha,D.(2024).TheAIScientist:TowardsFully AutomatedOpen-EndedScientificDiscovery.arXivpreprintarXiv:2408.06292. https://arxiv.org/abs/2408.06292 16. Lu,N.,Ma,S.,Piccinni,O.J.,Chen,Y.,&Sun,L.(2026).GW250114revealssignaturesofpost-merger black-holehorizon.Nature,655,71â75.https://doi.org/10.1038/s41586-026-10696-0 17. Luque,R.,Osborn,H.P.,Leleu,A.,PallĂ©,E.,Bonfanti,A.,BarragĂĄn,O.,etal.(2023).Aresonantsextuplet ofsub-NeptunestransitingthebrightstarHD110067.Nature,623,932â937. https://doi.org/10.1038/s41586-023-06692-3 18. Nakajima,K.,Ouchi,M.,Harikane,Y.,Vanzella,E.,Ono,Y.,Isobe,Y.,etal.(2026).Anultra-faint, chemicallyprimitivegalaxyforminginthereionizationera.Nature,653,363â367. https://doi.org/10.1038/s41586-026-10374-1 19. NousResearch(2026).HermesAgent:anopen-sourceautonomousmulti-agentframework(versionusedat release).https://github.com/NousResearch/hermes-agent 20. OpenAI(2023).GPT-4TechnicalReport.arXivpreprintarXiv:2303.08774.https://arxiv.org/abs/2303.08774 21. Peng,R.D.(2011).Reproducibleresearchincomputationalscience.Science,334,1226â1227. https://doi.org/10.1126/science.1213847 22. Polanyi,M.(1966).TheTacitDimension.UniversityofChicagoPress. 23. Rix,H.-W.,Chandra,V.,Andrae,R.,Price-Whelan,A.M.,Weinberg,D.H.,Conroy,C.,Fouesneau,M., Hogg,D.W.,etal.(2022).ThePoorOldHeartoftheMilkyWay.TheAstrophysicalJournal,941,45. https://doi.org/10.3847/1538-4357/ac9e01 24. Shinn,N.,Cassano,F.,Berman,E.,Gopinath,A.,Narasimhan,K.,&Yao,S.(2023).Reflexion:Language AgentswithVerbalReinforcementLearning.AdvancesinNeuralInformationProcessingSystems36 (NeurIPS2023).https://arxiv.org/abs/2303.11366 25. Si,C.,Yang,Z.,Kanai,K.,Schuurmans,D.,&Dai,H.(2025).FromTheoremstoScience:End-to-End AutonomousScientificResearch.arXivpreprintarXiv:2504.04662.https://arxiv.org/abs/2504.04662 26. Stodden,V.,LeVeque,R.J.,&Mitchell,I.(2014).ReproducibleResearchforScientificComputing:Tools andStrategiesforChangingtheCulture.InImplementingReproducibleResearch(p.19â36).Chapman andHall/CRC.https://arxiv.org/abs/1201.3528 27. SylosLabini,F.,&Galoppo,M.(2026).Detectionofanisotropiccosmicstructuresonagigaparsecscale. Nature,655,327â331.https://doi.org/10.1038/s41586-026-10702-5 28. Thomas,C.A.,Naidu,S.P.,Scheirich,P.,Moskovitz,N.A.,Pravec,P.,Chesley,S.R.,etal.(2023).Orbital periodchangeofDimorphosduetotheDARTkineticimpact.Nature,616,448â451. https://doi.org/10.1038/s41586-023-05805-2 29. Vasil,G.M.,Lecoanet,D.,Augustson,K.,Burns,K.J.,Oishi,J.S.,Brown,B.P.,etal.(2024).Thesolar dynamobeginsnearthesurface.Nature,629,769â772.https://doi.org/10.1038/s41586-024-07315-1 30. Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser,L.,&Polosukhin,I. (2017).AttentionIsAllYouNeed.AdvancesinNeuralInformationProcessingSystems30(NeurIPS2017), 5998â6008.https://arxiv.org/abs/1706.03762 31. Witstok,J.,Jakobsen,P.,Maiolino,R.,Helton,J.M.,Johnson,B.D.,Robertson,B.E.,etal.(2025). WitnessingtheonsetofreionizationthroughLyman-αemissionatredshift13.Nature,639,897â901. https://doi.org/10.1038/s41586-025-08779-5 32. Wu,Q.,Bansal,G.,Zhang,J.,Wu,Y.,Li,B.,Zhu,E.,etal.(2023).AutoGen:EnablingNext-GenLLM ApplicationsviaMulti-AgentConversation.arXivpreprintarXiv:2308.08155. https://arxiv.org/abs/2308.08155 33. Yao,S.,Zhao,J.,Yu,D.,Du,N.,Shafran,I.,Narasimhan,K.,&Cao,Y.(2023).ReAct:Synergizing ReasoningandActinginLanguageModels.InInternationalConferenceonLearningRepresentations(ICLR 2023).https://arxiv.org/abs/2210.03629 34. Zucker,C.,Goodman,A.A.,Alves,J.,Bialy,S.,Foley,M.,Speagle,J.S.,etal.(2022).Starformationnear theSunisdrivenbyexpansionoftheLocalBubble.Nature,601,334â337. https://doi.org/10.1038/s41586-021-04286-5 Acknowledgements WesincerelythankAnnaforherinvaluablehelpandsupportthroughouttheresearchandthewritingofthis paper. Funding ThisworkissupportedbytheZhejiangProvinceKeyResearchandDevelopmentPlan(GrantNo. 025SSYS0004) FigureCreditsandPermissions Figures1â8werecreatedbytheauthorsforthisstudyandcontainnothird-partycopyrightedmaterial.Any valuesorqualitativefindingsdrawnfromthereproducedpapersareattributedthroughthetextand References;nocopyrightedfigurepanelswerereproduced.Figuresarevectorizedorrenderedataresolution thatremainslegibleafterthescalingappliedinarXiv/PDFproduction. UseofGenerativeAI Duringthepreparationofthiswork,theauthorsusedlargelanguagemodels,includingthe021Science FoundationModel,andanLLM-basedmulti-agentsystemtoassistwithcodedrafting,retrievaland organizationofliterature,executionofthereproductionexperimentsreportedherein,andlanguageeditingof themanuscript.Allscientificquestions,methodologicaldecisions,auditclassifications,interpretationof results,andverificationoftheanalysiswereperformedbytheauthors,whotakefullresponsibilityforthe contentofthisarticle.