Pathways to Machine Reasoning: From Logic-Based Learning to Language Models

Abstract

Machine reasoning, a cornerstone of artificial intelligence, seeks to replicate human cognitive abilities such as abstraction, reasoning, and decision-making. Despite significant advances in machine learning, reasoning remains a critical challenge, particularly in tasks requiring logical structure, interpretability, and adaptability to novel scenarios. While symbolic methods excel in transparency and structured reasoning, they are limited by scalability and combinatorial complexity. Neural models, on the other hand, offer remarkable pattern recognition and interpolation capabilities but often lack explicit reasoning mechanisms, constraining their ability to perform multi-step logical inference or generalise effectively beyond training data.

This thesis introduces novel approaches to advance machine reasoning, addressing the limitations of existing methodologies. Three key contributions are presented. First, an adaptive sampling diversity strategy is proposed for large language models (LLMs), improving their reasoning performance by dynamically aligning sampling parameters with task complexity. This method enhances logical coherence without requiring model retraining. Second, an empirical study of optimality in logic-based learning, particularly within Inductive Logic Programming (ILP), challenges the traditional assumption that optimal hypotheses always generalise better, demonstrating that sub-optimal solutions can outperform in noisy or complex domains. Third, a neuro-symbolic framework is developed, integrating the structured precision of symbolic methods with the flexibility of neural networks via contrastive learning. This approach enables robust cross-domain reasoning while maintaining interpretability.

These contributions are validated through rigorous experimentation on benchmarks such as the Natural Language Inference (NLI) and the Massive Multitask Language Understanding (MMLU), showcasing significant improvements in reasoning capabilities. The thesis concludes by discussing the implications of these findings, potential future directions, and the role of machine reasoning in advancing artificial intelligence.