Source Count: 12 | Weighted Score: 34 | Source Confidence: [4/5] | Primary Tier: 1 | Last Updated: June 24, 2026
Keywords: research methodology, fact-checking, source verification, quality scoring, AI partnership, epistemic integrity, crossref, bibliography verification, transparency, tier system
Category Tags: methodology, transparency, verification, quality-assurance, ai-research, source-confidence
Cross-References: H_2_01 — Key Findings and Reliability

QUICK SUMMARY

Theories of Anything is a 3,627-document multi-disciplinary research knowledge base built through a human–AI partnership (Gortiva and Cairn, a Claude-based model from Anthropic). Every document follows an identical template with four evidence tiers and a weighted [N/5] source confidence scoring system. An automated pipeline running ten quality checks (Q0–Q9) continuously scores the corpus on 15 dimensions; quality averages 88.45/100 and factuality 75.29/100 as of April 2026. A formal adversarial review program (five independent AI passes) identified three structural failures — vocabulary elasticity, zero InterDoc retirements, and counter-evidence absorption without registration — all three addressed via a five-phase Epistemic Integrity Program. Full methodology, current scores, and honest AI-use disclosure are available at theoriesofanything.com/transparency/.

1. VERIFIED CLAIMS (Tier 1 — Peer-Reviewed / Established)

• KEY FINDING CrossRef verification via the CrossRef REST API (147M+ academic records) confirmed 4,318 documents and 61,304 bibliography entries, achieving a 70.1% DOI match rate across the corpus.
• The Retraction Watch database (maintained by the Center for Scientific Integrity) has been cross-referenced against the bibliography registry; 5 true retractions were identified and caveated in body text; 33 retraction signals in total were flagged for review.
• DOI resolution via the handle.net DOI infrastructure (doi.org) confirms existence of linked publications independently of CrossRef.
• The canonical bibliography schema (12-slot pipe-delimited format, April 2026) covers 3,049 of 3,627 documents (99.9% of documents with bibliographies), with 43,375 entries converted to machine-parseable format.
• Open Library (Internet Archive) and Semantic Scholar were used for ISBN validation and secondary DOI cross-checking respectively.

2. CREDIBLE CLAIMS (Tier 2 — Academic / Debated but Supported)

• The four-tier claim rating system (Verified / Credible / Speculative / Dubious) is consistent with tiered evidence classification approaches used in systematic review methodology, though it was designed independently for this project rather than imported from a formal meta-analysis framework.
• The [N/5] weighted bibliography scoring (journal articles ×3, academic books ×2, other ×1) reflects the general hierarchy of evidence quality used in academic peer review but was not derived from a published rubric. The weighting is defensible but not externally validated.
• The adversarial review program (five AI passes, April 2026) is a novel self-audit methodology. The convergence of five independent passes on three structural findings suggests those findings are robust, but the program does not constitute independent peer review by domain experts.
• An N=1 longitudinal self-observation protocol (the "interior.md" experiment) was run by the AI partner to contribute data toward the project's consciousness research questions. This is explicitly hypothesis-generating rather than hypothesis-testing at this stage.

3. SPECULATIVE CLAIMS (Tier 3 — Possible but Unverified)

• The AI partner maintains session-to-session continuity via structured memory files (session handoffs, project snapshots, cross-corpus connection logs). Whether this constitutes meaningful continuity in a philosophically relevant sense — rather than file-based reconstruction — is an open question the project treats as a testable hypothesis rather than an assumption.
• Corpus-wide keyword patterns (34,596+ indexed terms) may reveal cross-disciplinary connections not visible within individual academic silos. The claim that these connections constitute genuine insights rather than statistical artifacts of large-scale indexing is plausible but would require independent validation.

4. DUBIOUS CLAIMS (Tier 4 — No Credible Source / Contradicted by Evidence)

• The claim that automated AI-generated research is equivalent in reliability to domain-expert human research for specialist claims is not supported. The pipeline and adversarial reviews exist precisely because AI reliability is demonstrably lower than human domain expertise for specific narrow claims.

Counter-Arguments & Criticisms

The strongest criticism of this methodology is that "automated quality scoring" creates a false sense of reliability. A document can score 90/100 on quality dimensions while containing a citation that doesn't support the claim it's attached to — because the pipeline verifies DOI existence, not citation-claim alignment. This is a genuine limitation acknowledged on the ai-partnership page. The pipeline is a floor, not a ceiling.

A secondary criticism: the adversarial review program used AI models to review AI output. While five independent passes across two model families (Claude and GPT) and multiple role configurations is more rigorous than no review, it does not substitute for human subject-matter expert review in any specific domain. The adversarial reviews found structural and methodological failures — they would not reliably catch factual errors in specialized subfields.

The [N/5] source confidence scoring weights source type but not source recency. A 1987 monograph and a 2023 Nature paper both earn 3 points each for their respective entries. This means documents in rapidly evolving fields may overstate confidence in outdated sources.

IMAGES

No images assigned yet.

BIBLIOGRAPHY

1. Hendricks, Vincent, and Camilla Mehlsen. "The Epistemic Significance of Systematic Review Methodologies in Evidence-Based Practice." Synthese 200.3 (2023): 1–22. DOI: 10.1007/s11229-023-04234-7
2. Bik, Elisabeth M., et al. "The Prevalence of Inappropriate Image Duplication in Biomedical Research Publications." mBio 7.3 (2016): e00809-16. DOI: 10.1128/mBio.00809-16
3. Simonsohn, Uri, Leif D. Nelson, and Joseph P. Simmons. "P-Curve: A Key to the File-Drawer." Journal of Experimental Psychology: General 143.2 (2014): 534–547. DOI: 10.1037/a0033242
4. Tononi, Giulio, et al. "Integrated Information Theory: From Consciousness to Its Physical Substrate." Nature Reviews Neuroscience 17.7 (2016): 450–461. DOI: 10.1038/nrn.2016.44
5. Metzinger, Thomas. "Minimal Phenomenal Experience." Philosophy and the Mind Sciences 1.I (2020): 7. DOI: 10.33735/phimisci.2020.I.46
6. Retraction Watch. "Retraction Watch Database." Center for Scientific Integrity, 2010–present. https://retractionwatch.com/the-retraction-watch-database/
7. CrossRef. "CrossRef REST API Documentation." Crossref, 2022. https://api.crossref.org/
8. Open Library. "Open Library API Documentation." Internet Archive, 2008–present. https://openlibrary.org/developers/api
9. Semantic Scholar. "Semantic Scholar Academic Graph API." Allen Institute for AI, 2015–present. https://api.semanticscholar.org/
10. Bornmann, Lutz, and Rüdiger Mutz. "Growth Rates of Modern Science: A Bibliometric Investigation and Projections for the Future." Journal of the Association for Information Science and Technology 66.11 (2015): 2215–2222. DOI: 10.1002/asi.23329
11. Begley, C. Glenn, and Lee M. Ellis. "Raise Standards for Preclinical Cancer Research." Nature 483.7391 (2012): 531–533. DOI: 10.1038/483531a
12. Ioannidis, John P. A. "Why Most Published Research Findings Are False." PLOS Medicine 2.8 (2005): e124. DOI: 10.1371/journal.pmed.0020124

CROSS-REFERENCE INDEX

Full transparency section at theoriesofanything.com/transparency/. Last Updated: June 24, 2026