Artificial Intelligence (AI) is rapidly moving from the laboratory to application and promises to permeate every aspect of society and commerce. Recognizing the transformational aspects of this field, Johns Hopkins is making major internal investments to build programs around the assurance of AI, fundamental AI technologies, and domain-specific application. Core AI technologies function largely as black boxes, creating substantial risks in deployment—how should organizations think about these risks and manage them? Given that existing AI is built largely on data, what lessons are there to be learned for societal applications of AI from activities such as the COVID Portal?