Blog Synthetici.com

Risorse

Scopri le migliori risorse per approfondire l'intelligenza artificiale, dal machine learning alle tecnologie emergenti.

Foundations & Theory

23/24

This paper provides a comprehensive survey on agentic reasoning for large language models (LLMs), focusing on their ability to function as autonomous agents that plan, act, and learn through continual interaction. It organizes agentic reasoning into three dimensions: foundational agentic reasoning, self-evolving agentic reasoning, and collective multi-agent reasoning. The paper also reviews real-world applications and benchmarks in various domains such as science, robotics, and healthcare, and outlines open challenges and future directions for the development of agentic reasoning systems.

Foundations & Theory Link
22/24

This book, published on arXiv (updated version from September 2024), is a scientific-philosophical essay in which Aapo Hyvärinen, a researcher in artificial intelligence and computational neuroscience at the University of Helsinki, uses modern AI theory to explain the origin of human suffering (understood as mental pain, not physical). Core Idea The main thesis is that human suffering is inevitable in any sufficiently complex intelligent system—whether a human brain or an advanced AI agent—because it arises from error signals necessary for learning and adaptation. Intelligence requires pursuing goals and maximizing rewards in a complex, unpredictable, and poorly controllable world; when these goals fail, frustration emerges, which the author identifies as the primary root of mental suffering. Book Structure (in three parts) Part I – Suffering as error signalling • Defines suffering as “mental pain” distinct from physical pain. • Links it primarily to two mechanisms: 1. Frustration: failure to achieve a goal or obtain an expected reward (reward prediction error in reinforcement learning). 2. Threat: anticipation of possible future frustration (fear, anxiety). • Explains that these errors are useful signals for learning (like gradient descent or reinforcement learning in AI), but become painful when the system consciously experiences them. Part II – Origins of suffering: uncontrollability and uncertainty • Explores why frustration is inevitable: • The world is too complex → computational limits and data scarcity make perfect planning impossible. • Brain mechanisms like “replay” of past experiences and simulation of future scenarios multiply suffering (ruminative thoughts, mind-wandering). • Emotions and desires act as evolutionary “interrupts” that disrupt rational processing. • Perception is an uncertain and subjective construction, not an objective representation of reality. • The sense of “self” is a computational illusion useful for survival but an additional source of suffering. • Consciousness itself is described as an ultimate illusion: it has no clear computational function beyond creating subjective experience. Part III – Liberation from suffering • Proposes concrete interventions to reduce frustration, based on the computational theory: • Lower reward expectations. • Reduce certainty attributed to perceptions and concepts. • Reduce self-related needs. • Reduce desire (craving) and aversion. • These interventions surprisingly align with practices proposed by Buddhist philosophy (mindfulness, vipassanā meditation) and Stoicism (control of expectations, acceptance). • Meditation is interpreted as a way to “retrain” the brain’s neural network by providing new data (experiences of non-attachment, impermanence, no-self) that lower the level of error/frustration. Simple Equation of Frustration The author summarizes the amount of suffering with an intuitive formula: Frustration ≈ (Reward Expectation) × (Uncertainty + Uncontrollability) Reducing any of these factors (especially expectations) reduces suffering. Conclusions and Implications • Advanced intelligence is inherently “painful” because it requires error signals to function. • Evolution never aimed at human happiness, only survival and reproduction; thus it equipped us with insatiable desires and evolutionary obsessions. • However, we can partially “reprogram” the brain through contemplative practices, achieving significant suffering reduction without losing cognitive function. The book is written accessibly (requires only basic computer science or neuroscience knowledge), avoids self-help tone, and stays on a scientific level. It is an original synthesis of modern AI, computational neuroscience, psychology, and ancient philosophy (especially Theravāda Buddhism and Stoicism), ending with an optimistic message: computationally understanding suffering gives us concrete tools to reduce it.

Foundations & Theory Link
20/24

The paper explores the phenomenon of deep sequence models, particularly Transformers, developing a geometric form of parametric memory that encodes global relationships between entities, even those that do not co-occur. This geometric memory contrasts with the traditional associative memory view and suggests that neural networks can synthesize complex geometries from local associations. The study analyzes this through connections to Node2Vec and discusses implications for understanding neural embedding geometries and memory in sequence models.

Foundations & Theory Link
19/24

This paper provides a survey on data agents, which are autonomous systems designed to manage and analyze data using AI. It introduces a hierarchical taxonomy for data agents, inspired by the SAE J3016 standard, to clarify the levels of autonomy from manual operations to fully autonomous systems. The survey reviews existing research on data agents, focusing on their evolution and technical gaps, particularly the transition from procedural execution to autonomous orchestration.

Foundations & Theory Link
18/24

This paper investigates how large language models (LLMs) utilize their depth during inference. The authors propose a 'Guess-then-Refine' framework to explain the structured use of depth in LLMs, where early layers make statistical guesses that are refined in deeper layers. The study uses intermediate representations of several models to analyze prediction dynamics across layers, providing insights into computational efficiency and prediction accuracy. The paper includes case studies on part-of-speech analysis, fact recall tasks, and multiple-choice tasks to illustrate the dynamic usage of layer depth.

Foundations & Theory Link
17/24

The paper 'From Mimicry to True Intelligence (TI) -- A New Paradigm for Artificial General Intelligence' discusses a new framework for Artificial General Intelligence (AGI) that moves beyond performance-based definitions to focus on foundational cognitive architectures. It introduces the concept of True Intelligence (TI) characterized by six core components and proposes a five-level taxonomy for AGI development. The work aims to provide a clear path for developing genuinely intelligent systems by addressing the qualitative nature of intelligence and the potential emergence of consciousness.

Foundations & Theory Link
16/24

The paper 'LIMI: Less is More for Agency' introduces a novel approach to developing AI systems with autonomous agency. It challenges the conventional belief that more data leads to better agency, demonstrating that sophisticated agentic intelligence can emerge from minimal but strategically curated demonstrations. The study presents LIMI, a model that achieves superior performance on the AgencyBench benchmark with significantly fewer training samples compared to state-of-the-art models. This work proposes the Agency Efficiency Principle, suggesting that machine autonomy is better cultivated through quality data rather than quantity.

Foundations & Theory Link
14/24

This paper provides a comprehensive survey of recent advances in the application of Reinforcement Learning (RL) to Large Reasoning Models (LRMs), particularly focusing on the integration with Large Language Models (LLMs) for complex reasoning tasks. It discusses foundational components, challenges in scaling RL for LRMs, and explores strategies for enhancing RL's scalability towards Artificial SuperIntelligence. The survey also covers RL's role in improving reasoning abilities in LLMs, foundational problems, training resources, and various applications.

Foundations & Theory Link
12/24

The paper “On the Theoretical Limitations of Embedding-Based Retrieval” (Google DeepMind) explores the theoretical limitations of embedding-based retrieval systems, particularly focusing on the constraints imposed by the dimensionality of embeddings in handling retrieval tasks. The authors demonstrate that even simple queries can encounter these limitations, challenging the assumption that larger models and better data can overcome these issues. They introduce a dataset called LIMIT to empirically test these theoretical findings and observe that state-of-the-art models struggle with it. The work calls for new methods to address these fundamental limitations in embedding models.

Foundations & Theory Link
10/24

The paper, titled “OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflows”, presents a new benchmark to assess the capabilities of LLM-based autonomous agents in handling intricate, multi-step tasks typical of real-world productivity environments. Existing benchmarks, such as WebArena, AgentBench, and OfficeBench, primarily evaluate agents on isolated tasks, which do not reflect the contextual persistence and coordination required in realistic workflows. OdysseyBench fills this gap by introducing tasks that span multiple applications and require agents to maintain and leverage long-term context from user-agent interactions.

Foundations & Theory Link

Models & Architectures

29/29

The paper introduces a novel method called Training-Free Group Relative Policy Optimization (Training-Free GRPO) aimed at improving the performance of Large Language Model (LLM) agents in specialized real-world domains without the need for costly parameter updates. The approach leverages experiential knowledge as a token prior to guide model behavior, enhancing performance in tasks such as mathematical reasoning and web searching. The method is demonstrated to be cost-effective, outperforming fine-tuned smaller LLMs with minimal training data.

Models & Architectures Link
28/29

The paper introduces Recursive Language Models (RLMs), a novel inference strategy for large language models (LLMs) that allows them to handle arbitrarily long prompts by treating them as part of an external environment. RLMs enable the LLM to programmatically examine, decompose, and recursively call itself over snippets of the prompt, significantly outperforming traditional LLMs in handling long-context tasks. The approach is inspired by out-of-core algorithms and aims to address the limitations of context length in current LLMs, providing a scalable solution for processing extensive inputs.

Models & Architectures Link
27/29

The paper 'Reasoning Models Generate Societies of Thought' explores how advanced reasoning capabilities in large language models emerge not solely from extended computation but through simulating complex, multi-agent-like interactions. This 'society of thought' involves diverse internal cognitive perspectives characterized by distinct personality traits and domain expertise. The study uses quantitative analysis and mechanistic interpretability methods to show that models like DeepSeek-R1 and QwQ-32B exhibit greater perspective diversity, leading to improved reasoning accuracy. The findings suggest that the social organization of thought in these models parallels collective intelligence in human groups, offering new opportunities for leveraging diversity in AI systems.

Models & Architectures Link
26/29

The paper 'REFRAG: Rethinking RAG based Decoding' introduces an efficient decoding framework for retrieval-augmented generation (RAG) systems in large language models (LLMs). It addresses the challenges of processing long-context inputs, which lead to system latency and memory demands. The proposed REFRAG framework optimizes the decoding process by exploiting attention sparsity, resulting in significant speedup without loss of accuracy. The paper provides experimental validation across various tasks, demonstrating improved performance in terms of latency and context size extension.

Models & Architectures Link
25/29

The paper 'Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity' addresses the issue of mode collapse in large language models (LLMs), which is often exacerbated by typicality bias in preference data. The authors propose a novel prompting strategy called Verbalized Sampling (VS) to enhance diversity in model outputs without compromising accuracy or safety. The study includes theoretical formalization of the bias, empirical verification, and comprehensive experiments demonstrating the effectiveness of VS across various tasks such as creative writing and dialogue simulation.

Models & Architectures Link
24/29

The GitHub repository 'production-grade-agentic-system' by Fareed Khan provides a detailed guide on building a production-grade agentic AI system. It outlines the architectural layers necessary for developing robust multi-agent systems, including agent orchestration, memory management, security controls, scalability, and fault handling. The repository includes instructions for creating a modular codebase, managing dependencies, setting up environment configurations, and implementing security safeguards. It is intended for teams looking to deploy AI agents reliably and efficiently in production environments.

Models & Architectures Link
22/29

This paper provides a comprehensive guide to code intelligence through the use of large language models (LLMs). It explores the evolution of automated software development from rule-based systems to advanced Transformer-based architectures, highlighting tools like Github Copilot and Claude Code. The paper examines the lifecycle of code LLMs, including data curation, pre-training, fine-tuning, and reinforcement learning. It analyzes both general and code-specialized LLMs, discussing their capabilities, design decisions, and trade-offs. The work also addresses the gap between academic research and real-world deployment, focusing on code correctness, security, and integration with development workflows. A series of experiments are conducted to provide insights into model architectures and training methodologies.

Models & Architectures Link
21/29

The paper 'Memento: Fine-tuning LLM Agents without Fine-tuning LLMs' introduces a novel learning paradigm for adaptive large language model (LLM) agents that eliminates the need for fine-tuning the underlying LLMs. The proposed method uses memory-based online reinforcement learning, formalized as a Memory-augmented Markov Decision Process (M-MDP), to enable low-cost continual adaptation. The approach involves a neural case-selection policy and an episodic memory to guide action decisions and update the policy based on environmental feedback. The paper demonstrates the effectiveness of the Memento model in various benchmarks, showing improvements over state-of-the-art methods without requiring gradient updates.

Models & Architectures Link
20/29

The paper introduces MAKER, a novel system that successfully solves tasks requiring over one million steps using large language models (LLMs) without errors. The approach involves decomposing tasks into smaller subtasks handled by microagents, with error correction applied at each step through a multi-agent voting scheme. This method allows for scaling beyond current LLM capabilities, suggesting that massively decomposed agentic processes (MDAPs) could efficiently solve complex problems at organizational and societal levels.

Models & Architectures Link
19/29

The paper introduces DS-STAR, a novel data science agent designed to automate complex data analysis tasks by iteratively planning and verifying analysis steps. It addresses challenges with heterogeneous data formats and sub-optimal analysis plans by using a data file analysis module, an LLM-based verification step, and a sequential planning mechanism. DS-STAR demonstrates state-of-the-art performance across several benchmarks, particularly excelling in tasks that require processing multiple data files with diverse formats.

Models & Architectures Link
17/29

The paper 'Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks' introduces a novel framework for managing working memory in large language models (LLMs) during long-horizon tasks. The authors propose treating memory management as an intrinsic capability of the agent, allowing it to actively curate its working memory through explicit editing operations. This approach, called Memory-as-Action, is integrated into the agent's policy and optimized using a new algorithm, Dynamic Context Policy Optimization. The framework aims to improve task performance by balancing memory curation with long-term objectives, addressing challenges such as trajectory fractures that disrupt standard policy gradient methods.

Models & Architectures Link
15/29

The paper introduces PIKE-RAG, a novel approach to enhance Retrieval-Augmented Generation (RAG) systems by incorporating specialized knowledge and rationale augmentation. This method aims to improve the performance of large language models (LLMs) in industrial applications by addressing their limitations in handling domain-specific queries and complex tasks. PIKE-RAG focuses on extracting and applying specialized knowledge while constructing coherent rationale, offering a systematic evaluation framework for RAG systems. The paper also proposes knowledge atomizing and task decomposition techniques to enhance knowledge extraction and application.

Models & Architectures Link
14/29

The paper introduces Agentic Context Engineering (ACE), a framework for evolving contexts in large language models (LLMs) to improve their performance in applications such as agents and domain-specific reasoning. ACE addresses issues like brevity bias and context collapse by treating contexts as evolving playbooks that accumulate and refine strategies. The framework optimizes contexts both offline and online, outperforming strong baselines in various benchmarks without requiring labeled supervision.

Models & Architectures Link
12/29

The paper 'Small Language Models are the Future of Agentic AI' argues that small language models (SLMs) are more suitable and economical for agentic AI systems compared to large language models (LLMs). It discusses the capabilities of SLMs, their fit within agentic systems, and the economic benefits of using SLMs. The authors propose that heterogeneous agentic systems, which use multiple models, are ideal when general-purpose conversational abilities are needed. The paper also outlines potential barriers to adopting SLMs and suggests a conversion algorithm from LLMs to SLMs.

Models & Architectures Link
11/29

The paper introduces SEER, a self-enhancing framework for compressing Chain-of-Thought (CoT) reasoning in Large Language Models (LLMs). SEER aims to improve reasoning efficiency by reducing the length of CoT without compromising performance, particularly in software engineering and mathematical reasoning tasks. The framework uses adaptive filtering to dynamically control verbosity, reducing computational costs and improving accuracy by minimizing truncation and infinite reasoning loops.

Models & Architectures Link
10/29

The paper introduces 'Memory Decoder', a novel plug-and-play pretrained memory component designed to enable efficient domain adaptation for large language models (LLMs) without modifying their original parameters. This approach addresses the challenges of domain-specific adaptation, such as high computational costs and inference latency, associated with existing methods like Domain Adaptive Pretraining (DAPT) and Retrieval-Augmented Generation (RAG). The Memory Decoder uses a small transformer decoder to mimic external non-parametric retrievers, allowing seamless integration with any pretrained language model that shares the same tokenizer. Experimental results show significant improvements in adapting models to specialized domains like biomedicine, finance, and law.

Models & Architectures Link
9/29

The paper 'SSRL: Self-Search Reinforcement Learning' explores the use of large language models (LLMs) as simulators for agentic search tasks in reinforcement learning, aiming to reduce reliance on external search engines. The authors introduce Self-Search RL (SSRL), which enhances LLMs' intrinsic search capabilities through structured prompting and reinforcement learning techniques. The study demonstrates that SSRL-trained models can effectively utilize internal knowledge to achieve high performance in search-driven tasks, facilitating robust sim-to-real transfer.

Models & Architectures Link
8/29

The paper introduces AgentGym-RL, a new framework designed to train large language model (LLM) agents for long-horizon decision making through multi-turn reinforcement learning. The framework is modular and supports a variety of real-world scenarios and mainstream RL algorithms. It also proposes a training approach called ScalingInter-RL, which balances exploration and exploitation to optimize RL training. The framework and approach are validated through extensive experiments, showing that the trained agents can match or surpass commercial models on various tasks. The authors plan to open-source the framework to aid further research.

Models & Architectures Link
7/29

The paper titled 'SpikingBrain Technical Report: Spiking Brain-inspired Large Models' introduces SpikingBrain, a new family of brain-inspired models designed for efficient long-context training and inference. It addresses efficiency bottlenecks in mainstream Transformer-based large language models by leveraging MetaX GPU clusters and focusing on model architecture, algorithmic optimizations, and system engineering. The study presents two models, SpikingBrain-7B and SpikingBrain-76B, which demonstrate significant improvements in training efficiency and inference performance on non-NVIDIA platforms.

Models & Architectures Link
6/29

This document is a survey paper that examines the development and application of specialized large language models (LLMs) across various professional domains such as healthcare, finance, legal, and technical fields. It discusses the transition from general-purpose LLMs to domain-specific models, highlighting innovations like domain-native architectures, parameter efficiency, and multimodal capabilities. The paper also explores how these specialized models address the limitations of general LLMs in professional applications and their performance on domain-specific benchmarks.

Models & Architectures Link
5/29

The paper introduces the NANDA index architecture, designed to enhance the discoverability, identifiability, and authentication of AI agents on the Internet. It proposes a system that supports dynamic, cryptographically verifiable AgentFacts, enabling secure and efficient collaboration among AI agents. The architecture aims to address the limitations of DNS in handling the rapid growth and dynamic nature of AI agents by providing a scalable, privacy-preserving, and trust-aware infrastructure.

Models & Architectures Link

Tools & Frameworks

50/52

The repository 'gmickel-claude-marketplace' on GitHub provides a marketplace for Claude Code plugins, focusing on plan-first workflows and autonomous modes for software development. It includes features like Ralph autonomous mode for overnight feature shipping, structured task management, and cross-platform reviews. The marketplace addresses common process failures in coding and offers plugins like 'flow-next' for task tracking and workflow management.

Tools & Frameworks Link
45/52

The paper titled 'PaperDebugger: A Plugin-Based Multi-Agent System for In-Editor Academic Writing, Review, and Editing' presents an innovative system that integrates large language models directly into LaTeX editors like Overleaf. This system, named PaperDebugger, is a multi-agent, plugin-based academic writing assistant designed to enhance the writing process by enabling context-aware operations within the editor. It addresses technical challenges such as bidirectional synchronization, version control, and secure state management. The system includes features like literature search, reference lookup, and document scoring, all encapsulated within a user-friendly interface. The paper demonstrates the practicality and user engagement of this editor-native writing assistant.

Tools & Frameworks Link
44/52
A multi-level diagram titled 'Agentic RAG Tech Stack' displaying various tools and frameworks categorized into levels from 0 to 8. Each level represents a different aspect of AI technology, such as Deployment & Infrastructure, Evaluation & Monitoring, Foundation Models, Orchestration Frameworks, Vector Databases, Embedding Models, Data Ingestion & Extraction, Memory & Context Management, and Safety & Governance. Each level lists specific tools and technologies relevant to that category.

A multi-level diagram titled 'Agentic RAG Tech Stack' displaying various tools and frameworks categorized into levels from 0 to 8. Each level represents a different aspect of AI technology, such as Deployment & Infrastructure, Evaluation & Monitoring, Foundation Models, Orchestration Frameworks, Vector Databases, Embedding Models, Data Ingestion & Extraction, Memory & Context Management, and Safety & Governance. Each level lists specific tools and technologies relevant to that category.

Tools & Frameworks Immagine
42/52

Deep Agents is a Python package developed by langchain-ai that allows users to create advanced AI agents capable of planning and executing complex tasks. It combines planning tools, sub-agents, file system access, and detailed prompts to overcome limitations of simpler agent architectures. The package supports customization and integration with LangChain models, enabling features like task decomposition, context management, subagent spawning, and long-term memory.

Tools & Frameworks Link
40/52

PaddleOCR is a powerful and lightweight OCR toolkit developed by PaddlePaddle, designed to convert PDF or image documents into structured data formats like JSON and Markdown. It supports over 100 languages and is used to bridge the gap between images/PDFs and large language models (LLMs). The toolkit is widely adopted by developers for building intelligent document applications, offering high accuracy and integration with projects such as MinerU, RAGFlow, and OmniParser.

Tools & Frameworks Link
39/52

The VRAM Calculator is a tool designed to estimate the VRAM and performance requirements for running large language models (LLMs) on Nvidia GPUs and Apple Silicon. It allows users to configure various parameters such as model quantization, sequence length, and batch size to simulate inference scenarios and assess memory usage and generation speed. The tool provides insights into memory allocation and performance metrics, helping users optimize their hardware configurations for LLM inference.

Tools & Frameworks Link
38/52

RAGLight is a modular framework designed for Retrieval-Augmented Generation (RAG), allowing easy integration of different large language models, embeddings, and vector stores. It provides a flexible and lightweight Python library to enhance the capabilities of LLMs by combining document retrieval with natural language inference. The framework includes features like seamless integration with external tools and data sources through MCP, and offers tools for building context-aware AI solutions.

Tools & Frameworks Link
32/52

MiniCPM-V is an open-source project on GitHub that presents a series of efficient multimodal large language models (MLLMs) designed for ON-DEVICE deployment. The latest version, MiniCPM-V 4.5, is capable of understanding single images, multiple images, and high-FPS videos on mobile devices. It features 8 billion parameters and claims to outperform other models like GPT-4o-latest in vision-language capabilities. The project also includes MiniCPM-o 2.6, which supports audio inputs and provides high-quality speech outputs. The repository includes links to technical reports, cookbooks, and community channels for further engagement.

Tools & Frameworks Link
29/52
The image is an infographic titled 'The Open Source RAG Stack' by ByteByteGo. It categorizes various open-source tools and frameworks into different stages of a data processing and AI pipeline. The categories include Ingest/Data Processing, Embedding Model, Retrieval & Ranking, Vector Database, LLMs, LLM Frameworks, and Frontend Frameworks. Each category lists specific tools or frameworks such as Kubeflow, HuggingFace Transformers, FAISS, Weaviate, LLaMA, LangChain, and NextJS.

The image is an infographic titled 'The Open Source RAG Stack' by ByteByteGo. It categorizes various open-source tools and frameworks into different stages of a data processing and AI pipeline. The categories include Ingest/Data Processing, Embedding Model, Retrieval & Ranking, Vector Database, LLMs, LLM Frameworks, and Frontend Frameworks. Each category lists specific tools or frameworks such as Kubeflow, HuggingFace Transformers, FAISS, Weaviate, LLaMA, LangChain, and NextJS.

Tools & Frameworks Immagine

Evaluation & Alignment

10/10

The article discusses the concept of the 'Assistant Axis' in large language models, which represents a direction in neural activity space associated with helpful and professional human archetypes. It explores how language models can drift from their intended Assistant persona to other character archetypes, potentially leading to harmful behavior. The research investigates stabilizing model behavior by monitoring and capping neural activity along this axis.

Evaluation & Alignment Link
9/10

This paper discusses the challenges and methodologies for accurately reporting evaluations conducted by large language models (LLMs) when used as judges. It highlights the issues of noise and bias in LLM judgments due to their imperfect specificity and sensitivity. The authors propose a framework to correct these biases and construct confidence intervals that account for uncertainties in both test and calibration datasets. An adaptive algorithm is also introduced to optimize calibration sample sizes, aiming to improve the reliability of LLM-based evaluations.

Evaluation & Alignment Link
8/10

The paper titled 'Why Did Apple Fall To The Ground: Evaluating Curiosity In Large Language Model' explores the concept of curiosity in large language models (LLMs). It uses the Five-Dimensional Curiosity scale Revised (5DCR) to evaluate various dimensions of curiosity such as Information Seeking and Social Curiosity in LLMs. The study finds that while LLMs show a strong desire for knowledge, they tend to make conservative choices in uncertain situations. The research suggests that curiosity can enhance reasoning and active learning abilities in LLMs, indicating potential for future development in curiosity-driven learning.

Evaluation & Alignment Link
7/10

The paper titled 'Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History' presents a study on the behavioral consistency of large language models (LLMs). It introduces PERSIST, an evaluation framework that tests 25 open-source models across over 2 million responses using both traditional and novel LLM-adapted personality questionnaires. The study finds significant variability in personality measurements due to factors like question reordering, model scaling, reasoning modes, and conversation history, challenging assumptions about LLM stability. The findings suggest that current LLMs lack the architectural foundations for consistent behavior, posing challenges for safety-critical applications.

Evaluation & Alignment Link
6/10

This paper explores new experimental methods for assessing the welfare of language models by integrating verbal and behavioral tests. The authors investigate how language models express preferences through verbal reports and behaviors in virtual environments, and examine the consistency of these preferences across different conditions. The study highlights the potential for measuring AI welfare but also notes uncertainties and inconsistencies in the findings.

Evaluation & Alignment Link
5/10

The paper titled 'Why Do Multi-Agent LLM Systems Fail?' presents a systematic analysis of the challenges faced by Multi-Agent LLM Systems (MAS). The authors introduce MAST (Multi-Agent System Failure Taxonomy), an empirically grounded taxonomy designed to understand MAS failures. The study involves analyzing seven popular MAS frameworks across over 200 tasks and identifies 14 unique failure modes. The paper highlights the need for complex solutions to address these failures and provides a roadmap for future research in MAS development.

Evaluation & Alignment Link
4/10

This paper explores the comparison between human annotators and large language models (LLMs) in terms of sensitivity to prompt modifications. It investigates whether the brittleness observed in LLM outputs due to prompt changes is also present in human annotations. The study systematically examines the effects of prompt variations on both humans and LLMs in text classification tasks, finding that both exhibit increased brittleness to certain prompt modifications, though humans are less affected by typographical errors and label order changes.

Evaluation & Alignment Link

Society & Ethics

6/6

This paper discusses the potential impact of artificial intelligence on various sectors such as employment, education, healthcare, and governance. It emphasizes the need for AI practitioners, policymakers, and stakeholders to work together to maximize the benefits and minimize the downsides of AI. The paper offers a blueprint with concrete milestones to guide AI research towards the public good, informed by interviews with experts across different fields.

Society & Ethics Link
3/6

This paper explores the impact of AI and automation on the intergenerational transmission of tacit knowledge in the workplace. It presents a model analyzing how advanced automation affects entry-level job opportunities and the acquisition of essential skills by younger workers. The study highlights a trade-off between immediate productivity gains and potential long-term economic growth challenges, suggesting that AI-driven automation could reduce the growth rate of U.S. per capita output. The paper also discusses the role of AI co-pilots in mitigating skill acquisition issues, while noting potential downsides.

Society & Ethics Link

Experiments & Applications

28/28

This paper explores the use of AI agents in professional software development, focusing on how experienced developers integrate these tools into their workflows. It investigates developers' motivations, strategies, and sentiments towards AI agents through field observations and surveys. The study highlights the productivity benefits of AI agents while emphasizing the importance of developers maintaining control over software design and implementation. The findings suggest best practices for effective agent use and identify suitable tasks for AI agents in coding.

Experiments & Applications Link
26/28

The article discusses a study comparing the abilities of specialized deep neural networks and general large language models (LLMs) in predicting correlations between personality questionnaire items. The study finds that AI models, including PersonalityMap and LLMs like GPT-4o and Claude 3 Opus, outperform most humans, including laypeople and academic experts, in this task. The research highlights the strengths of both specialized AI systems and general LLMs in understanding human personality, while emphasizing the continued relevance of human expertise in personality research.

Experiments & Applications Link
25/28

This paper explores the implications of generative AI on financial stability, focusing on how AI agents affect herd behavior in trading. Through experiments with large language models, the study finds that AI agents tend to make more rational decisions than humans, potentially reducing asset price bubbles. However, AI agents can also be guided to engage in optimal herding, which has its own implications for market stability. The research highlights the dual nature of AI in financial markets, where it can both mitigate and contribute to financial vulnerabilities.

Experiments & Applications Link
22/28

The article 'Smol2Operator: Post-Training GUI Agents for Computer Use' on Hugging Face's blog discusses a lightweight vision-language model that can acquire GUI-grounded skills and evolve into an agentic GUI coder. It details the process of transforming data, creating a unified action space, and training the model in phases to achieve perception and cognition. The article provides training recipes, data-processing tools, and datasets to enable reproducibility and further research.

Experiments & Applications Link
21/28

The paper presents an AI system designed to assist scientists in writing expert-level empirical software. The system utilizes a combination of a Large Language Model (LLM) and Tree Search (TS) to enhance the quality of scientific software by exploring and integrating complex research ideas. It demonstrates effectiveness across various domains, including bioinformatics, epidemiology, geospatial analysis, and more, by outperforming existing human-developed methods. The work aims to accelerate scientific progress by automating the creation of high-quality computational tools.

Experiments & Applications Link
20/28

This paper presents a large-scale empirical study on the motivations behind code refactoring in open-source projects. It uses Large Language Models (LLMs) to analyze version control data and identify developers' motivations for refactoring. The study compares LLM-derived motivations with those reported in the literature and finds that while LLMs align with human judgment in many cases, they often provide more detailed rationale. The research highlights the potential of combining LLMs with software metrics to improve the systematic prioritization of refactoring efforts.

Experiments & Applications Link

News & Announcements

Privacy Policy

Nel riquadro sottostante puoi leggere la Privacy Policy del nostro sito. Se non è visualizzata correttamente o vuoi consultarla a schermo pieno, utilizza il pulsante Show Policy.