Title: THAI: Towards Humble and Discoverable AI
Principal Investigator: Pablo Martínez Olmos
Starting Date: 30/09/2024
Ending Date: 30/03/2026
Budget: 40.000,00
Abstract:
This project addresses concerns about overconfidence and reliability in generative AI. As AI technologies continue to reshape various sectors, the project’s primary motivation lies in mitigating the risks associated with AI’s persuasive capabilities, particularly the generation of plausible yet unfounded outputs, known as hallucinations. THAI aims to develop humble generative AI methods that reduce overconfidence by scaling back certainty without solid information. This initiative is crucial for maintaining the credibility of AI technologies and ensuring user trust by distinguishing between reliable and unreliable AI-generated content. Additionally, the project seeks to enhance security by developing robust training methods that reduce AI systems’ vulnerability to attacks, which could exploit overconfidence to extract sensitive information or generate harmful content. The project’s objectives are threefold: First, it aims to improve uncertainty calibration through constrained latent encoding, enhancing the confidence calibration of generative AI models. Second, THAI focuses on robust decoding to prevent overconfidence and increase the resilience of generative models against adversarial threats. Third, the project aims to identify discoverable traces in AI-generated content, making it easier to distinguish between real and synthetic outputs. These objectives collectively contribute to advancing more discernible, reliable, and ethically sound AI technologies, addressing foundational challenges in generative AI and paving the way for more responsible applications.
External links: