AIQ Frequently Asked Questions

AIQ Phase IAIQ Phase 2

What is the AIQ? And how will it promote AI’s core mission?

The AIQ or “Arbitrator Intelligence Questionnaire” is a feedback questionnaire designed by AI to facilitate systematic collection of information about arbitrator case managmenet and decisionmaking. 

AI’s core mission is to promote fairness, transparency, accountability and diversity in arbitrator appointments.  By making systematically gathered data about arbitrator decisionmaking more equally available, the AIQ will help level the playing field l in arbitrator selection and make the entire process more transparent.  Meanwhile, more information about arbitrators will provide market-based accountability for arbitrators and allow newer and more diverse arbitrators a meaningful opportunity to build their reputations based on actual performance.

How was the AIQ developed and what is its current status?

AI developed the AIQ in coordination with the Penn State Survey Research Center.  Earlier versions of the AIQ were pilot tested and commented on by several external advisors and commentators, who represented a range of stakeholders and areas of expertise.

The AIQ is still a work-in-progress. On 22 February 2017, AI posted the AIQ on its website for public comment.  That period ended on 17 March 2017.  After incorporating public feedback, the AIQ has been finalized and will be launched in Singapore on 1 June 2017. 

Who can comment on the draft AIQ?

We welcome everyone with an interest in the data to be produced by the AIQ—parties, in-house counsel, party representatives, arbitrators, and arbitration institutions—to review and provide comments on the draft AIQ. Our goal is to make the AIQ easy to fill out while at the same time providing useful information to those selecting international arbitrators.

How will the AIQ be implemented?

Beginning in the Summer of 2017, the AIQ will be open for all parties and counsel involved in international arbitrations to fill out after the conclusion of their cases. In addition, AI is developing incentives to encourage systematic participation in the AIQ. For example, AI plans to enter into Collaboration Agreements with various arbitral institutions. Under these agreements, institutions will forward the AIQ to parties and counsel at the end of a case in exchange for access by institutions to AI data about arbitrators.  AI will also be inviting parties and law firms to participate in an AI Pact, under which they commit to submitting AIQ responses regularly at the conclusion of their cases. 

Who can complete the AIQ?

Once AI has rolled out the AIQ for use, it is hoping that a range of participants—including parties, their in-house counsel, and their outside or external counsel—will routinely fill out AIQs.  Multiple responses from different participants in a single case are welcome as they will facilitate increased quality control and more nuanced data assessment. Arbitrators, arbitral institutions, and arbitral secretaries, however, are not invited to participate to avoid disclosure of any confidential information.

How will information from the AIQs be made available?

Once sufficient data are collected from responses to the AIQ, those data will be made available (usually for a fee) to parties, counsel, institutions, and arbitrators through “AI Reports.”  The specific features and content of AI Reports are still being developed. But we hope to be able to provide data on how prospective arbitrators compare with other arbitrators on questions such as how frequently they order interim measures and document production, their interpretative methodologies in construing contracts and treaties,  and their timeliness in issuing awards, among others.

Why are there 2 Phases to the AIQ?

The AIQ is divided into two phases to ensure that responders can provide feedback in 15 minutes or less per phase. Phase I concentrates on objective background information about the case, and can be completed by anyone who has access to the award or case file.  Phase II contains questions that are evaluative and, in some questions, request subjective assessments. Phase II should be completed by an attorney or party who actively participated in the case.  Certain background information from Phase I questions is used to prefill the questions in Phase II to make it even easier to complete Phase II.

How does the AIQ protect the parties’ confidentiality?

To protect parties’ confidentiality, the AIQ does not seek information that would readily identify the case or the parties, but only key data that will facilitate assessment and analysis of arbitrator decisionmaking and case outcomes.  For example, the AIQ asks for the date of filing, the industry in which the dispute arose, the date of the award, and arbitrator names. The AIQ does not ask for parties’ names or the names of law firms or lawyers.  For quality assurance purposes, responders will be required to register in order to complete an AIQ, but their identities will be kept confidential and will not be published or otherwise connected with any AIQ responses.

How does the AIQ ensure that information provided will be relevant, accurate, and reliable?

The quality of information generated by the AIQ will be key to its success. For this reason, the AIQ is being specially designed in consultation with the Penn State Survey Research Center, with other experts in survey design, and after extensive public consultation. This extensive development and vetting process aims to ensure the accuracy and quality of data collected. 

At a more substantive level, the AIQ includes questions that will facilitate qualitative assessment of responses and filtering of data from AIQ responses. For example, responders must identify their perspective (in-house counsel, outside counsel, etc.), their level of experience, and whether the award was more or less favorable than expected, etc. Moreover, questions that require subjective evaluation, such as questions addressing the extent of document production ordered by the tribunal, are asked only after responders provide responses to several objective questions. Responses can be assessed against the background information about the case, such as the amount in dispute, the governing law, the seat, or the industry in which the dispute arose. Responses can also be assessed for their consistency with the objective information derived from related responses on a particular topic.

How does the AIQ ensure that feedback is fair to arbitrators?

The AIQ has a number of features designed to ensure not only the accuracy, but also the fairness of the information it provides.  While the identity of responders will confidential, they will be required to register and verify their identities before completing the AIQ.

Meanwhile, AI is working to establish editorial policies to ensure that only AIQ responses that meet minimum indicia of reliability will be incorporated into AI Reports.  Open-text feedback provided through AIQs will similarly be vetted through established editorial policies and generally published in summary form, similar to existing practices used by the International Mediation Institute and the Almanac of the Federal Judiciary for mediators and U.S. judges.  AI is also exploring other means of ensuring that feedback is fair to arbitrators.