The Defense Advanced Research Projects Agency (DARPA) is soliciting innovative proposals in the technical areas of assessing and understanding the capabilities of artificial intelligence (AI) to enable mathematical guarantees on performance of generative AI. Proposed research should investigate innovative approaches that enable revolutionary advances in science, devices, or systems. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice.
Artificial Intelligence Quantified (AIQ) will develop technology to assess and understand the capabilities of AI to enable guaranteed performance. The program will test the hypothesis that mathematical methods, combined with advances in measurement and modeling, will allow guaranteed quantification of AI capabilities. Specifically, the program will address three interrelated capability levels: 1) specific problem level, 2) classes of problem level, and 3) natural class level, aiming to address the quantification and assessment challenges at each level.
None is available.
AIQ brings together two Technical Areas (TAs) and a government team to test the program hypothesis. The goal of TA1 is to provide rigorous foundations for understanding and guaranteeing capabilities across levels; teams proposing for TA1 are expected to be led by individuals with deep technical expertise, such as pure or applied mathematics, theoretical computer science, or statistics, or other relevant expertise and demonstrate relevance to AI. The goal of TA2 is to develop methods for evaluating AI models, integrating and evaluating TA1 results at scale using appropriate research datasets; teams proposing for TA2 are expected to comprise computational, cognitive, and/or behavioral scientists with expertise in AI evaluation.