This Note critiques the current structure and practice of the ICSID annulment mechanism by shifting away from the traditional focus on the ICSID arbitration system as a dispute settlement body and instead analyzing the annulment mechanism’s role in a progressively “judicializing” investor-state arbitration system. Recent developments in ICSID arbitration indicate that, over time, ICSID arbitral tribunals have undergone “judicialization”—that is, they have acquired domestic court–like characteristics enabling them to impact state and individual behavior prospectively, rather than merely to resolve the specific dispute at bar. These developments raise the question of whether the current annulment mechanism, which provides for cancellation of tribunal awards on a strictly limited set of grounds, is capable of accommodating this shift. Although the drafters of the ICSID Convention did not intend to allow an annulment committee, convened after the tribunal’s issuance of an award, to review the substantive merits of that tribunal’s award, annulment committees have previously based their decisions on more expansive substantive review than that permitted under the Convention. This Note argues that in a recent series of decisions, annulment committees appear to be engaging in greater substantive review of tribunals’ awards once again, a fact that triggers a renewed sense that annulment committees are still confused over the proper role of annulment in the ICSID arbitration system. Such confusion has serious implications in that it leads to the production of inconsistent decisions at the annulment level of the ICSID arbitration system, thus adding to the layer of inconsistent decisions produced at the tribunal level. These incoherent decisions may ultimately imperil the legitimacy of the ICSID arbitration system as a judicialized body for shaping prospective state and individual behavior. To strengthen the legitimacy of ICSID arbitral decisions and promote further development of coherent international investment law, I argue that it is critical for ICSID to establish a mechanism with official powers of substantive review.