Artificial intelligence (AI) aims to build intelligent systems that can interact with and assist humans. During the interaction, a system learns the requirements from the human user and adapts to the needs to complete tasks. A popular type of interactive system is retrieval-based, where the system uses a retrieval function to retrieve relevant…

Increasingly, machine learning is entertained as a mechanism for improving the efficiency of planning systems. Research in this area has generated an impressive battery of techniques and a growing body of empirical successes. Unfortunately the formal properties of these systems are not well understood. This is highlighted by a growing corpus of…

This study, part of a larger effort to develop a methodology for evaluating intelligent computer systems (Artificial Intelligence Systems), explores the use of benchmarking as an evaluation technique. Benchmarking means comparing the performance of intelligent computer systems with human performance on the same task. Benchmarking in evaluation has…