Source Count: 13 | Weighted Score: 30 | Source Confidence: [4/5] | Primary Tier: 2 | Last Updated: March 10, 2026
Keywords: AGI, artificial general intelligence, artificial intelligence, AI, superintelligence, alignment, neural network, transformer, large language model, LLM, reasoning, consciousness, Turing test, embodiment, symbol grounding, safety, control problem, existential risk, cognitive architecture, GPT
Category Tags: information computation, artificial intelligence, AGI, safety
Cross-References: ZD_2_02 — Machine Learning · S_1_01 — Future Technology Overview · ZE_3_09 — AI Ethics · K_1_01 — Consciousness Overview

QUICK SUMMARY

Artificial General Intelligence (AGI) — a hypothetical AI system capable of performing any intellectual task that a human can, with the same flexibility, generality, and ability to learn and transfer knowledge across domains — remains one of the most consequential and contested goals in computer science and philosophy of mind. AGI is distinguished from narrow AI (systems that excel at specific, well-defined tasks — chess, image classification, protein folding, language generation — but cannot generalize outside their training domain) and from the speculative concept of superintelligence (an AI that vastly exceeds human cognitive capability across all relevant domains — Bostrom 2014). The question of whether AGI is achievable, how it might be achieved, and whether it would be safe dominates contemporary AI discourse. Current state of AI (as of 2025): the dominant paradigm is deep learning — neural networks with many layers, trained on vast datasets using gradient-based optimization. The transformer architecture (Vaswani et al. 2017, Attention Is All You Need) — based on the self-attention mechanism — enabled the development of large language models (LLMs) (GPT-4, Claude, Gemini, Llama) that demonstrate remarkable capabilities in language understanding, generation, reasoning, coding, and multimodal processing (text + images + audio). These models have reignited the AGI debate: researchers argue that scaling current architectures (more parameters, more data, more compute) will yield AGI (the scaling hypothesis — Kaplan et al. 2020 showed power-law relationships between model size, data, and performance); others argue that fundamental architectural innovations are needed because current systems lack: causal reasoning (understanding why, not just correlation), robust planning (multi-step reasoning in novel situations), embodiment (physical interaction with the world), self-awareness, and genuine understanding vs. stochastic pattern matching (Yann LeCun 2022 has argued that autoregressive LLMs cannot achieve human-level intelligence and that "world models" with learned physics simulations are necessary). Major architectural approaches to AGI include: (1) Scaling deep learning: the position (associated with OpenAI, Anthropic, Google DeepMind) that sufficiently large neural networks trained on sufficiently diverse data will develop emergent general capabilities; "emergent abilities" (capabilities that appear suddenly at scale — Wei et al. 2022) are cited as evidence, though Schaeffer et al. (2023) have argued that emergence may be an artifact of metric choice. (2) Hybrid neurosymbolic systems: combining neural networks (good at pattern recognition, learning from data) with symbolic AI (good at logical reasoning, planning, knowledge representation) — proposed by Marcus (2020), Garcez & Lamb (2020); the intuition is that human cognition integrates fast, associative processing (System 1) with deliberate, rule-based reasoning (System 2 — Kahneman 2011). (3) Cognitive architectures: systems modeled on theories of human cognition — ACT-R (Anderson), SOAR (Laird), LIDA, OpenCog — these attempt to replicate the functional organization of human intelligence, including working memory, long-term memory, attention, and metacognition. (4) Whole-brain emulation: scanning and simulating a biological brain at sufficient resolution to replicate its function — currently far beyond technical capability (the human brain has ~86 billion neurons and ~100 trillion synapses). The AI alignment problem — ensuring that an AGI system's goals and behavior remain beneficial and aligned with human values — is considered by researchers to be the central challenge: a misaligned AGI with superhuman capability could pursue goals catastrophic for humanity if its objective function does not correctly capture human values and intentions (the paperclip maximizer thought experiment — Bostrom 2003). AI safety research includes: reward modeling, RLHF (reinforcement learning from human feedback), constitutional AI, interpretability/mechanistic interpretability research, formal verification, and governance frameworks.

1. VERIFIED CLAIMS (Tier 1 — Technical / Published / Empirical)

1.1 Current AI Capabilities and Architecture

• Transformer architecture (Vaswani et al. 2017): the self-attention mechanism computes relationships between all positions in a sequence simultaneously, enabling parallelized training and effective long-range dependency modeling — this architecture underlies virtually all state-of-the-art language models, vision transformers, and multimodal systems
• Scaling laws (Kaplan et al. 2020; Hoffmann et al. 2022): empirically observed power-law relationships between model size (parameters), training data size, compute budget, and model performance (measured as loss) — these relationships have been remarkably predictive and have driven the trend toward larger models; the Chinchilla paper (Hoffmann et al.) showed that many models were undertrained relative to their size, suggesting that data scaling is as important as parameter scaling

1.2 The Alignment Problem

• Bostrom (2014, Superintelligence): systematic analysis of existential risk from advanced AI — argued that the "control problem" (ensuring a superintelligent AI remains aligned with human interests) is extremely difficult because: (a) specifying human values completely is impossible, (b) an intelligent system may find unexpected ways to satisfy its objective function (Goodhart's law), and (c) a sufficiently capable system may resist or circumvent human attempts to correct it
• Russell (2019, Human Compatible): proposed "inverse reward design" — rather than specifying the AI's objective function, design systems that learn human preferences from behavior and remain uncertain about their objectives, deferring to humans; this approach frames AI safety as a problem of cooperative game theory

1.3 Limitations of Current Systems

• LeCun (2022): argued that current LLMs are fundamentally limited because they lack a "world model" — an internal representation of the physical and causal structure of reality that enables prediction, planning, and counterfactual reasoning; LLMs predict the next token based on statistical patterns, not causal understanding
• Marcus (2020, The Next Decade in AI): identified five persistent limitations of deep learning: (1) data hunger, (2) brittleness (failure on out-of-distribution inputs), (3) opacity (lack of interpretability), (4) poor integration of prior knowledge, (5) lack of compositional generalization

2. CREDIBLE CLAIMS (Tier 2 — Academic / Actively Debated)

2.1 Emergence and Scaling

• Wei et al. (2022): documented "emergent abilities" in large language models — capabilities (chain-of-thought reasoning, multi-step arithmetic, translation) that appeared suddenly as model scale increased; if genuine, these suggest that scaling may yield qualitatively new capabilities
• Schaeffer et al. (2023): challenged the emergence narrative — argued that apparent emergence is an artifact of nonlinear metrics (accuracy thresholds); when measured with linear metrics, performance improves smoothly with scale; the debate is unresolved

2.2 Consciousness in AI

• Whether an AGI system would be conscious (have subjective experience) is a deep philosophical question — Tononi's Integrated Information Theory (IIT) would deny consciousness to feedforward networks but might attribute it to highly integrated recurrent architectures; functionalists would attribute consciousness to any system that replicates the functional organization of conscious brains; there is currently no scientific consensus on machine consciousness

3. SPECULATIVE CLAIMS (Tier 3 — Possible but Unverified)

3.1 AGI Timeline Predictions

• Predictions for when AGI will be achieved range from ~2030 (Kurzweil, some OpenAI researchers) to ~2100+ to "never" — expert survey results (Grace et al. 2024) show a median estimate of ~2047 for a 50% probability of human-level AI, but with enormous variance and low confidence; timeline predictions in AI have historically been unreliable

4. DUBIOUS CLAIMS (Tier 4 — No Credible Source / Contradicted by Evidence)

4.1 Current LLMs Are AGI

• [UNSUPPORTED] Claims that current LLMs (GPT-4, Claude, Gemini) constitute AGI — while these systems demonstrate impressive capabilities on many benchmarks, they fail on tasks requiring genuine causal reasoning, robust planning in novel environments, physical manipulation, continuous learning, and reliable factual grounding; they are best characterized as powerful narrow AI systems with broad but shallow generalization

COUNTER-ARGUMENTS

• Scaling laws vs. architectural innovation: The "scaling hypothesis" — that sufficient compute, data, and parameters will yield AGI — is championed by proponents at OpenAI and DeepMind but challenged by researchers who argue that current architectures have fundamental limitations. Gary Marcus has repeatedly argued that LLMs lack systematic compositionality and causal reasoning that scaling alone cannot provide. François Chollet designed ARC (Abstraction and Reasoning Corpus) to test the kind of fluid intelligence that current systems fail at regardless of scale
• Emergent capabilities debate: Wei et al. (2022) reported emergent capabilities appearing at scale, but Schaeffer, Miranda, and Koyejo (2023) argued that apparent emergence is an artifact of nonlinear or discontinuous evaluation metrics — when linear metrics are used, performance improvement is smooth and predictable, challenging the narrative of sudden capability jumps
• Safety and alignment: Stuart Russell (Human Compatible, 2019) and researchers at the Center for AI Safety have argued that scaling toward AGI without solving the alignment problem poses existential risks. The debate over whether alignment is a solvable engineering problem or a fundamentally intractable challenge (the "sharp left turn" concern) divides the AI safety community

IMAGES

No images assigned yet.

BIBLIOGRAPHY

1. Vaswani, A. et al | 2017 | "Attention Is All You Need" | Advances in Neural Information Processing Systems | ∅ | 30::5998–6008 | ∅ | ∅ | ∅ | ∅ | ∅ | ∅
2. Bostrom, N | 2014 | ∅ | Superintelligence: Paths, Dangers, Strategies | ∅ | ∅ | Oxford: Oxford University Press | ∅ | doi:10.1017/s0031819115000340 | ∅ | ∅ | ∅
3. Russell, S | 2019 | ∅ | Human Compatible: Artificial Intelligence and the Problem of Control | ∅ | ∅ | New York: Viking | ∅ | doi:10.3917/pe.204.0186o | ∅ | ∅ | ∅
4. Kaplan, J. et al | 2020 | "Scaling Laws for Neural Language Models" | ∅ | ∅ | ∅ | ∅ | ∅ | arxiv:2001.08361 | ∅ | ∅ | ∅
5. Hoffmann, J. et al | 2022 | "Training Compute-Optimal Large Language Models" | ∅ | ∅ | ∅ | ∅ | ∅ | arxiv:2203.15556 | ∅ | ∅ | ∅
6. Marcus, G | 2020 | "The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence" | ∅ | ∅ | ∅ | ∅ | ∅ | arxiv:2002.06177 | ∅ | ∅ | ∅
7. Wei, J. et al | 2022 | "Emergent Abilities of Large Language Models" | Transactions on Machine Learning Research | ∅ | ∅ | ∅ | ∅ | ∅ | ∅ | ∅ | ∅
8. Schaeffer, R. et al | 2023 | "Are Emergent Abilities of Large Language Models a Mirage?" | Advances in Neural Information Processing Systems | ∅ | ∅ | 36 | ∅ | doi:10.52202/075280-2425 | ∅ | ∅ | ∅
9. LeCun, Y | 2022 | "A Path Towards Autonomous Machine Intelligence" | ∅ | ∅ | ∅ | OpenReview preprint, June | ∅ | ∅ | ∅ | ∅ | ∅
10. Grace, K. et al | 2024 | "Thousands of AI Authors on the Future of AI" | ∅ | ∅ | ∅ | ∅ | ∅ | arxiv:2401.02843 | ∅ | ∅ | ∅
11. Garcez, A.d'A.; Lamb, L.C | 2023 | "Neurosymbolic AI: The 3rd Wave" | Artificial Intelligence Review | ∅ | 56::12387–12406 | ∅ | ∅ | doi:10.1007/s10462-023-10448-w | ∅ | ∅ | ∅
12. Mitchell, M | 2019 | ∅ | Artificial Intelligence: A Guide for Thinking Humans | ∅ | ∅ | New York: Farrar, Straus and Giroux | ∅ | doi:10.5007/1808-1711.2023.e88209 | ∅ | ∅ | ∅
13. Ngo, R., Chan, L.; Mindermann, S | 2023 | "The Alignment Problem from a Deep Learning Perspective" | ∅ | ∅ | ∅ | ∅ | ∅ | arxiv:2209.00626 | ∅ | ∅ | ∅

CROSS-REFERENCE INDEX

No cross-references yet.