On November 26, 2024, the Science, ICT, Broadcasting, and Communications Committee of the National Assembly (“SIBCC”) convened a general meeting during which the “Framework Act on the Development of Artificial Intelligence and the Establishment of Foundation for Reliability” (the “Proposed Bill”) was successfully passed. 

The Proposed Bill is a consolidation of 19 separate bills that were introduced to the 22^nd National Assembly as of November 21, 2024. The next step involves a review of the Proposed Bill’s structure and language by the Legislation and Judiciary Committee before it is forwarded to the National Assembly’s plenary session.

The Proposed Bill is drafted to advance AI development and promote self-regulation by establishing a framework of the following initiatives:
 

• Formulating a master plan for AI by the Minister of the Ministry of Science and ICT (“MSIT”), creating the National AI Committee (“NAIC”) under the President’s office, establishing the AI Policy Center, and establishing the legal foundation for the AI Safety Institute’s operations.

• Supporting industries related to the development and promotion of AI technology, including establishing standards for AI technology.

• Enacting and announcing the “AI Ethics Principles” to support self-verification and certification by AI-related organizations, thus ensuring the safety and reliability of AI, and establishing the legal basis for autonomous ethics committees in the private sector.
   

Moreover, the Proposed Bill stipulates various obligations for AI business operators as below:
 

• High-Impact AI Obligations: Operators involved with high-impact AI, either through provisions or through products/services, are required to adhere to mandates for advance notification and to implement pertinent safety and reliability measures.

• Generative AI and Content Obligations: Businesses offering generative AI products or services must fulfill obligations for advance notification and labeling. Additionally, those providing AI-generated content, such as virtual audio, images, or videos that may be indistinguishable from real content are subject to specific notification or labeling requirement.

• Compute Usage and Risk Management: Operators whose AI training compute usage surpasses a designated threshold are mandated to establish a risk management system designed to monitor and address AI-related safety incidents efficiently. Moreover, there is an obligation to report these system implementations to the MSIT.

• Non-Domestic Operators Obligation: Operators lacking a domicile or business location within Korea must appoint a domestic agent to comply with the proposed regulatory framework.

• Regulatory Oversight: The Proposed Bill empowers the Minister of MSIT with the authority to conduct fact-finding inspections and to issue suspension or corrective orders where necessary.
   

We provide below a detailed overview of the key provisions and implications of these obligations:
 

• AI Developers: Includes corporations, organizations, individuals, and national institutions involved in the development and provision of AI.

• AI Users: Encompasses corporations, organizations, individuals, and national institutions that offer AI products or services using AI developed by others.
   

The term “artificial intelligence” or “AI” is defined as the electronic manifestation of human intellectual capabilities such as learning, inference, perception, judgment, and language comprehension. The “AI system,” principal subject of regulation, is defined as an AI-based system with varying degrees of autonomy and adaptability, capable of influencing physical or virtual environments through its predictions, recommendations, and decisions. This definition excludes simple rule-based models or software, although AI utilizing machine learning may fall under the scope of the Proposed Bill.

We recommend that companies assess their potential obligations under the Proposed Bill based on these definitions. The current absence of distinct obligations for AI developers versus AI users may change as future regulations could impose differentiated responsibilities via a Presidential Decree. It is, therefore, crucial to stay informed of any such developments.
 

• Designation of a Domestic Agent: An AI business operator without a domicile or place of business in Korea, meeting certain users and sales criteria (to be determined by Presidential Decree), must appoint a person with residence or business operation in Korea as their domestic agent. The appointment must be documented in writing and reported to the Minister of the MSIT (Article 36, Paragraphs (1) and (2)).

• Duties of a Domestic Agent: Acting on behalf of the AI business operator, a domestic agent is responsible for (i) submitting compliance reports regarding obligations to ensure AI system safety for systems whose cumulative amount of calculation used for training exceeds a certain level, (ii) filing application to confirm if the operator’s AI qualifies as high-impact AI, and (iii) providing support in implementing measures to ensure the safety and reliability of high-impact AI (Article 36, Paragraph (1)).

• Consequences of Non-Compliance: (i) Failure to appoint a domestic agent shall result in an administrative fine of up to KRW 30 million (Article 43, Paragraph (1), Item 2). (ii) Any breach of the Proposed Bill relating to the domestic agent’s duties is imputable to the AI business operator (Article 36, Paragraph (3)).
   

It is advisable for AI business operators without an existing Korean domicile or business place to stay abreast of the forthcoming Presidential Decree that will outline the criteria for appointing domestic agents.
 

• High-Impact AI Definition: High-impact AI systems are those that significantly influence or pose risks to the safety and fundamental rights of individuals. These are typically employed in critical decision-making or assessments with substantial impact on someone’s rights and responsibilities. Examples include applications in medical device development, recruitment processes, loan assessments, and educational evaluations (Article 2, Item 4).

• Preliminary Review Obligation: AI business operators must assess whether their AI technology qualifies as high-impact before deployment. They may seek confirmation from the Minister of the MSIT if there is uncertainty regarding the classification of their AI system (Article 33).

• Advance Notification Obligation: AI business operators intending to deploy products or services using high-impact AI are obligated to inform users in advance (Article 31, Paragraph (1)). Non-compliance may result in an administrative fine of up to KRW 30 million (Article 43, Paragraph (1), Item 1).

• Safety and Reliability Measures: A comprehensive framework of safety and reliability measures^[1] must be implemented by operators offering high-impact AI systems to ensure these systems operate as intended without undue risk (Article 34).

• Impact Assessment Obligation: AI business operators are expected to proactively assess the potential impact of their high-impact AI on individuals’ fundamental rights. Public institutions, including national and local government entities, must prioritize AI solutions that have undergone such assessments (Article 35).

• Right to Explanation: Individuals affected by high-impact AI systems have the right to request clear explanations of the logic and principles behind AI-generated outcomes, to the extent that this is technically and reasonably feasible (Article 3, Paragraph (4)).
   

In summary, AI business operators providing high-impact AI or related products and services must adhere to obligations for advance notifications, safety and reliability measures, and possibly conduct impact assessments, especially when working with public institutions. Moreover, operators should be prepared to provide explanations of AI decisions when requested by users. It is crucial for companies planning to deliver high-impact AI solutions to review these regulatory requirements in advance to ensure compliance.
 

• Definition of Generative AI: This term refers to AI systems that produce content such as text, audio, images, and other outputs by mimicking the structure of input data (Article 2, Item 6).

• Advance Notification Obligation: AI business operators must notify users in advance that their products or services are powered by generative AI (Article 31, Paragraph (1)). Non-compliance may result in an administrative fine of up to KRW 30 million (Article 43, Paragraph (1), Item 1).

• Labeling Obligation: Products or services must be clearly labeled as being created by generative AI (Article 31, Paragraph (2)).

• Deepfake Content: AI business operators providing virtual outputs that may be mistaken for real (often referred to as “deepfakes”), must ensure these are clearly labeled. If labeled content qualifies as artistic or creative expression, the manner of labeling should not hinder its appreciation (Article 31, Paragraph (3)).

• Compliance Guidance: The specifics of notification and labeling, including potential exceptions, will be detailed in a forthcoming Presidential Decree (Article 31, Paragraph (4)).
   

AI business operators are strongly encouraged to review these obligations to ensure compliance, particularly concerning deepfake content.
 

^[1]   These measures include (i) establishment and operation of a risk management plan, (ii) establishment and implementation of plans to explain the AI-derived final outcomes to the extent technically feasible, key standards used to derive AI’s final outcomes, and outline of the training data used to develop and utilize AI, (iii) establishment and operation of a user protection plan, (iv) human management and supervision of high-impact AI, (v) preparation and storage of documents to confirm measures to secure safety and reliability, and (vi) other matters deliberated and resolved by the NAIC to secure safety and reliability of high-impact AI. The Minister of the MSIT may determine and publicly notify the details of such measures.

[Korean Version]