Decoding the Debate: Open Source in the EU AI Act
Navigating the Crossroads of Open Source Ideals and AI Governance
In the context of the EU AI Act, two opposing viewpoints concerning the application of open source to advanced AI models are emerging.
For the GovAI piece on open-sourcing highly capable foundation models, the potential risks and the technological complexity of future AI models make open-sourcing a non-viable option.
For the Mozilla open letter on open-source AI models, however, open-sourcing is essential to prevent proprietary control over emerging AI systems and an excessive concentration of power, which might result in negative consequences.
Main Arguments
According to the GovAI piece, future foundation models will be much different than those currently available in terms of potential capabilities and impact. Hence, a precautionary approach must be adopted to ensure risk mitigation.
The document lists a series of risks that these models could bring forth in the event that they are provided as open source. Ranging from cases of terrorism to new weapons development to population surveillance enacted by malicious actors, open sourcing these models might result in net negative outcomes for society.
Finally, several alternatives for ensuring safety while allowing the pursuit of open source goals are presented at the end. Among these, developing risk assessment systems, enhancing government oversight, and gradual model release are some of the proposed options.
According to the Mozilla open joint statement, as has happened multiple times in history, proprietary technologies that are kept hidden from public oversight can fail to ensure transparency and are more prone to developing unpredictable negative consequences. On the contrary, technologies and processes that are accessible and subject to public scrutiny have been demonstrated to be more safe and to respect agreed standards.
The open letter proposes several solutions to ensure that accountability and openness go hand in hand with safety. Among these are: advocating for regulations that prevent concentration of power; supporting the design and development of several multi-stakeholder approaches to enable anyone to access and scrutinize the conduct of AI development companies; and facilitating use and integration by anyone in society.
Core Differences
The two documents include various divergent elements.
First of all, they adopt different perspectives on whether open source can provide net benefits in the long term if applied to AI models.
On the one hand, according to the Mozilla letter, open source has historically demonstrated the ability to provide net benefits compared to proprietary IP.
On the other hand, according to the GovAI piece, despite the successful track record, open source presents features that might enhance the risks associated with the uncontrolled deployment of advanced AI models to the general public.
Secondly, a noticeable discrepancy exists between the positions outlined in the two documents concerning risk assessment and management. Despite both advocating for a collaborative multi-stakeholder system that defines standards and transparency systems, the similarities end here.
GovAI’s piece advocates for a number of recommendations concerning safety. It proposes the development of safety mechanisms such as risk assessment before open-source models are developed. It also encourages governments to have a solid oversight of development and the power to restrict the use of AI by developers if standards are not being respected.
Mozilla’s open letter, on the other side, does not mention any sort of initiative, apart from “investing in a spectrum of approaches”, leaving ample space for debate on what “open” signifies and thus inviting the entire scientific community to be involved in this process.
Finally, the two documents present divergent perspectives concerning the stage at which public participation in this technology takes place, with the Mozilla open letter advocating for immediate full and free access for all to such AI models and the GovAI piece advocating for a gradual staged deployment given a defined standard.
Personal Position
I personally believe that the stance of Mozilla is more concrete and based on a solid track record.
First, it recognizes the role of open source in advancing collective understanding of such technologies and promotes a multistakeholder approach to prevent emerging risks from AI systems whose capabilities are still unexplored.
Second, despite being a time-consuming and potentially risky approach, it poses the basis for a wider sense of responsibility across all stakeholders and leverages the collective knowledge of the developer and scientist communities across the globe.
Finally, it prevents companies from putting transformative technologies behind a paywall that only a few actors are able to afford, streamlining benefits to the entire ecosystem.
Overall, I believe this stance is more pragmatic because it recognizes that at this current stage, every actor has an interest in developing a safe and transparent system for AI solutions to be developed, to build debate around that, and to find democratic solutions.
Evidence that would cause me to reconsider
I would be inclined to reconsider my position in the case of one of two major events.
First, there is compelling evidence of the unregulated development and deployment of open-source AI models that fail to be mitigated by comprehensive safety mechanisms based on a multistakeholder approach.
Second, there is a failure to establish regulations and standards for open source AI development. This might be caused by the inability of international actors to establish international agreements and oversight bodies with the democratic participation of all stakeholders or by a sudden shift in public opinion and engagement.
In these two cases, I will reconsider my initial position.
EU AI Act Implications
Over the last decades, open source has become one of the major drivers of innovation at the European level. According to a 2021 OpenForum Europe study, open source contributed up to €95 billion to the EU’s GDP, injecting enormous value into the economy.
As open source frameworks are currently being used to train AI and develop ChatGPT plugins, such estimates will only rise with the widespread adoption of AI technologies.
Yet, the proposal in its current form fails to take into account the importance and fragility of the open-source ecosystem, making open-source GPAI models legally liable. The consequences of such a decision will hinder the involvement of developers in open source projects and thus hurt innovation at the EU level.
Open source is in fact run by volunteers, not corporations that have the power to deal with advanced legal compliance. This will in turn concentrate power around companies able to defend themselves in cases of legal controversy, consequently preventing public access to such technologies.
Overall, the proposed policy recommendation would be for the EU AI Act to recognize open innovation in AI and exempt open-source developers from the legal liabilities currently foreseen. Such a change will demonstrate EU acknowledgement of the role of open source as a vehicle for responsible innovation, safety, competition, and transparency.