FLOSS Research

Members Newsletter – September 2024

Open Source Initiative - Wed, 2024-09-04 13:35
September 2024 Members Newsletter

It’s been a busy couple of months, and things are going to stay that way as we approach All Things Open in October. Version 0.0.9 of the Open Source AI Definition has been released after collecting months of community feedback.

We’re continuing our march towards a stable release by the end of October 2024, at All Things Open. Get involved by joining the discussion on the forum, finding OSI staff around the world, and online at the weekly town halls. The community continues iterating through drafts after meeting diverse stakeholders at the worldwide roadshow, collecting feedback and carefully looking for new arguments in dissenting opinions. All thanks to a grant by the Alfred P. Sloan Foundation. We also need to decide how to best address the reviews of new licenses for datasets, documentation and the agreements governing model parameters. 

The lively conversations will continue at conferences, town halls, and online. The first two stops were at AI_dev and Open Source Congress. Other events are planned to take place in Africa, South America, Europe and North America.

On a separate delightful note, the Open Source community got some welcome news on August 29, as Elastic returned to the community by adding the AGPL licensing option for Elasticsearch and Kibana. This decision is confirmation that shipping software with licenses that comply with the Open Source Definition is valuable—to the maker, to the customer, and to the user. Elastic’s choice of a strong copyleft license signals the continuing importance of that license and its dual effect: one, it’s designed to preserve the user’s freedoms downstream, and two, it also grants strong control over the project by the single-vendor developers.

We’re encouraged to see Elastic return to the Open Source community. And who knows… maybe others will follow suit!

Stefano Maffulli

Executive Director, OSI 

I hold weekly office hours on Fridays with OSI members: book time if you want to chat about OSI’s activities, if you want to volunteer or have suggestions.

News from the OSI Community input drives the new draft of the Open Source AI Definition

From the Research and Advocacy program

The Open Source AI Definition v0.0.9 has been released and collaboration continues at in-person events and in the online forums. Read what changes have been made, what to do next and how to get involved. Read more.

Three things I learned at KubeCon + AI_Dev China 2024

From the Research and Advocacy program

KubeCon China 2024 was a whirlwind of innovation, community and technical deep dives. As it often happens at these community events, I was blown away by the energy, enthusiasm and sheer amount of knowledge being shared. Read more.

Highlights from our participation at Open Source Congress

From the Research and Advocacy program

The Open Source Initiative (OSI) proudly participated in the Open Source Congress 2024, held from August 25-27 in Beijing, China. This event was a pivotal gathering for key individuals in the Open Source nonprofit community, aiming to foster collaboration, innovation, and strategic development within the ecosystem. Read more.

OSI in the news Elasticsearch is open source, again

OSI at elastic.co

“Being able to call Elasticsearch and Kibana Open Source again is pure joy.” — Shay Banon, Elastic Founder and CTO. Read more.

Meta is accused of bullying the open source community

OSI at The Economist

Purists are pushing back against Meta’s efforts to set its own standard on the definition of open-source AI. Stefano Maffulli, head of the OSI, says Mr Zuckerberg “is really bullying the industry to follow his lead”. Read more.

Debate over “open source AI” term brings new push to formalize definition

OSI at Ars Technica

The Open Source Initiative (OSI) recently unveiled its latest draft definition for “open source AI,” aiming to clarify the ambiguous use of the term in the fast-moving field. The move comes as some companies like Meta release trained AI language model weights and code with usage restrictions while using the “open source” label. This has sparked intense debates among free-software advocates about what truly constitutes “open source” in the context of AI. Read more.

Other Highlights Other news News from OSI affiliates News from OpenSource.net Voices of the Open Source AI Definition

The Open Source Initiative (OSI) is running a series of stories about a few of the people involved in the Open Source AI Definition (OSAID) co-design process.

2024 Generative AI Survey

This survey aims to understand the deployment, use, and challenges of generative AI technologies in organizations and the role of open source in this domain. Take survey here.

Events Upcoming events CFPs Thanks to our sponsors New members and renewals
  • Mercado Libre

Interested in sponsoring, or partnering with, the OSI? Please see our Sponsorship Prospectus and our Annual Report. We also have a dedicated prospectus for the Deep Dive: Defining Open Source AI. Please contact the OSI to find out more about how your company can promote open source development, communities and software.

Support OSI by becoming a member!

Let’s build a world where knowledge is freely shared, ideas are nurtured, and innovation knows no bounds! 

Join the Open Source Initiative!

Categories: FLOSS Research

Highlights from our participation at Open Source Congress 2024

Open Source Initiative - Wed, 2024-09-04 13:30

The Open Source Initiative (OSI) proudly participated in the Open Source Congress 2024, held from August 25-27 in Beijing, China. This event was a gathering for key individuals in the Open Source nonprofit community, aiming to foster collaboration, innovation, and strategic development within the ecosystem. Here are some highlights from OSI’s participation at the event.

Panel: Collaboration between Open Source Organizations

Stefano Maffulli, OSI’s Executive Director, played an important role in the panel on “Collaboration between Open Source Organizations.” This session, moderated by Daniel Goldscheider (Executive Director, OpenWallet Foundation) and Chris Xie (Board Advisor, Linux Foundation Research), brought together influential leaders, including Keith Bergelt (CEO, Open Invention Network), Bryan Che (Advisory Board Member, Software Heritage Foundation), Mike Milinkovich (Executive Director, Eclipse Foundation), Rebecca Rumbul (Executive Director, Rust Foundation), Xiaohua Xin (Deputy Secretary-General, OpenAtom Foundation), and Jim Zemlin (Executive Director, Linux Foundation). The panel discussed the importance of collaboration in addressing the challenges faced by the Open Source ecosystem and explored ways to strengthen inter-organizational ties.

Fireside Chat: Datasets, Privacy, and Copyright

Stefano Maffulli also led a fireside chat on “Datasets, Privacy, and Copyright” in the context of Open Source AI along with Donnie Dong (Steering Committee Member, Digital Asia Hub; Senior Partner, Hylands Law Firm). This session was particularly relevant given the growing concerns around AI and the legal implications of creating and distributing large datasets. The discussion provided valuable insights into how these issues intersect with Open Source principles and what steps the community can take to address them responsibly. Some questions addressed included the use of copyrighted material in training datasets; fair use in the context of AI training and content generation; and China’s AI regulatory framework.

Talk: The Open Source AI Definition

OSI’s involvement was further highlighted by Stefano Maffulli’s talk on “The Open Source AI Definition,” where he announced version 0.0.9 of the Open Source AI Definition (OSAID), a significant milestone resulting from a multi-year, global, and multi-stakeholder process. This version reflects the collective input of a diverse range of experts and community members who participated in extensive co-design workshops and public consultations, ensuring that the definition is robust, inclusive, and aligned with the principles of openness. Maffulli emphasized the importance of the “4 Freedoms of Open Source AI”—Use, Study, Modify, and Share—as foundational principles guiding the development of AI technologies. The session was particularly crucial for gathering feedback from the community in China, providing a platform for discussing the practical implications of the OSAID in different cultural and regulatory contexts.

Panel: The Future of Open Source Congress

Deborah Bryant, OSI’s US Policy Director, moderated a pivotal panel discussion on “The Future of Open Source Congress: Converting Ideas to Shared Action.” This session focused on how the community can transform discussions into actionable strategies, ensuring the continued growth and impact of Open Source globally.

Other highlights from the event

The “Unlocking Innovation: Open Strategies in Generative AI” panel led by Anni Lai (Chair of Generative AI Commons; Board member of LF AI & Data; Head of Open Source Operations, Futurewei) explored how openness is essential for advancing Generative AI innovation, democratizing access, and ensuring ethical AI practices. Panelists Richard Sikang Bian (Outreach Chair, LF AI & Data; Head of OSPO, Ant Group), Richard Lin (Member, OpenDigger Community; Head of Open Source, 01.ai), Ted Liu (Co-founder, KAIYUANSHE), and Zhenhua Sun (China Workgroup Chair, OpenChain; Open Source Legal Counsel, ByteDance) delved into the challenges of the Open Source generative AI landscape, such as “open washing,” inconsistent definitions, and the complexities of licensing. They highlighted the need for clear, standardized frameworks to define what truly constitutes Open Source AI, emphasizing that openness fosters transparency, accelerates learning, and mitigates biases. The panelists called for increased collaboration among stakeholders to address these challenges and further develop Open Source AI standards, ensuring that AI technologies are transparent, ethical, and widely adoptable.

In her closing keynote at the Open Source AI track, Amreen Taneja, Standards Lead at the Digital Public Goods Alliance (DPGA), emphasized the critical role of Open Source AI in advancing public good and supporting the Sustainable Development Goals (SDGs). She explained that Digital Public Goods (DPGs) are digital technologies made freely available to benefit society and highlighted the importance of OSAI in democratizing access to powerful AI technologies. Taneja outlined the DPGA’s efforts to align AI with public interests, including updating the DPG Standard to better accommodate AI, ensuring transparency in AI development, and promoting responsible AI practices that prioritize privacy and avoid harm. She stressed the need for rigorous evaluation, clear ownership, open licensing, and platform independence to drive the adoption of AI DPGs, ultimately aiming to create AI systems that are ethical, transparent, and beneficial for all.

Quotes from OSI Board and affiliates

Attending the Open Source Congress was really inspiring. Over two days, we participated in intensive discussions and exchanges with dozens of Open Source foundations and organizations worldwide, which was incredibly beneficial. I believe this will foster broader cross-community collaboration globally. I hope the conclusion of the second Open Source Congress marks the beginning of ongoing cooperation, allowing our “community of communities” to maintain regular communication and exchange. 

Nadia Jiang – Board Chair of KAIYUANSHE

Open Source development experience is all about two words: consensus and antifragile decision-making process. The most valuable part of this event is seeing and listening to all the executive directors, open-source leaders in the room, and being very comfortable with the information density and the constructiveness of the discussions. Towards the end of the day, what people care about are not fundamentally different and there are indeed really difficult questions to resolve. I feel the world becomes slightly better after this OSC, and that means a lot to have an event like this.

Richard Bian – Head of Ant Group OSPO; Outreach Chair, Linux Foundation AI & Data

Open Source is the cornerstone of innovation, transparency, and collaboration, driving solutions that benefit everyone. The Open Source Congress 2024 represented a significant step forward in fostering alignment and building consensus within the open source community. By bringing together diverse voices and ideas, it amplified our collective efforts to create a more open, inclusive, and impactful digital ecosystem for the future.

Amreen Taneja – Standards Lead, Digital Public Goods Alliance

Stefano Maffulli with Board Directors of KAIYUANSHE: Emily Chen, Nadia Jiang (photo credits), and Ted Liu. Conclusion

OSI’s active participation in the Open Source Congress 2024 reinforced its leadership role in the global Open Source community. By engaging in critical discussions, leading panels, and contributing to the future direction of Open Source initiatives, OSI continues to shape the landscape of Open Source development, ensuring that it remains inclusive, innovative, and aligned with the values of the global community.

This event marked another successful chapter in OSI’s ongoing efforts to drive collaboration and innovation in the Open Source world. We extend our sincere thanks to the organizers of OSC and the Open Source community in China for creating a platform that brought together a diverse and dynamic group of stakeholders, enabling meaningful discussions and progress. We look forward to continuing these conversations and turning ideas into action in the years to come.

Categories: FLOSS Research

Open Source AI Definition – Weekly update September 2nd

Open Source Initiative - Mon, 2024-09-02 10:17
Share your thoughts about draft v0.0.9
  • @mkai added concerns about how OSI will address AI-generated content from both open and closed source models, given current legal rulings that such content cannot be copyrighted. He also suggests clarifying the difference between licenses for AI model parameters and the model itself within the Open Source AI Definition.
  • @shujisado added that while media coverage of the OSAID v0.0.9 release is encouraging, he is not supportive of the idea of an enforcement mechanism to flag false open source AI. He believes this approach differs from OSI’s traditional stance and suggests it may be a misunderstanding.
  • @jplorre added that while LINAGORA supports the proposed definition, they propose clarifying the term “equivalent system” to mean systems that produce the same outputs given identical inputs. They also suggest removing the specific reference to “tokenizers” in the definition, as it may not apply to all AI systems.
    • @shujisado agreed with the need for clarification on “equivalent system” but noted that identical outputs cannot always be guaranteed in general LLMs. He suggests that this clarification might be better suited for the checklist rather than the OSAID itself

Draft v.0.0.9 of the Open Source AI Definition is available for comments

  • @adafruit reconnects with @webmink and proposes updates to the Open Source AI Definition, including adding requirements for prompt transparency and data access during AI training. These updates aim to enhance the ability to audit, replicate, and modify AI models by providing detailed logs, documentation, and public access to prompts used during the training phase.
    • @webmink appreciates the proposal but points out that it seems specific to a single approach, suggesting that it may need broader applicability.
  • @thesteve0 criticizes the current definition, arguing that it does not grant true freedom to modify AI models because the weights, which are essential for using the model, cannot be reproduced without access to both the original data and code. He suggests that models sharing only their weights, especially when built on proprietary data, should be labeled as “open weights” rather than “open source.” He also expresses concern about the misuse of the “open source” label by some AI models, citing specific examples where the term is being abused.
Open-washing and unspoken assumptions of OSS
  • @pranesh added that it might be helpful to explicitly state that the governance of open-source AI is out of scope for OSAID, but also notes that neither the OSD nor the free software definition explicitly mention governance, so it may not be necessary.
  • @kjetilk added that while governance issues have traditionally been unspoken, this unspoken nature is a key problem that needs addressing. He suggests that OSI should explicitly declare governance out of scope to allow others to take on this responsibility.
  • @mjbommar added support for making an official statement that OSI does not intend to control governance, noting concerns that some might fear OSI is moving towards a walled governance approach. He references past regrets about not controlling the “open source” trademark as a means to combat open-washing.
  • @nick added assurance that OSI has no intention of creating a walled governance garden, reaffirming the organization’s long-standing position against such control.
  • @shujisado added that there seems to be a consensus within the OSAID process that governance is out of scope, and notes that related statements have already been moved to the FAQ section in recent versions.
Explaining the concept of Data information
  • @pranesh mentions that, from a legal perspective, the percentage of infringement matters, citing the “de minimis” doctrine and defenses like “fair use” that consider the amount and purpose of infringement. He emphasizes that copyright laws in different jurisdictions vary, and not all recognize the same defenses as in the US.
  • @mjbommar argues that the scale and nature of AI outputs make the “de minimis” defense irrelevant, especially when AI models generate significant amounts of copyrighted content. He stresses that the economic impact of AI-generated content is a key factor in determining whether it qualifies as transformative or infringes copyright.
  • @shujisado highlights that in Japan, using copyrighted works for AI training is generally treated as an exception under copyright law, a stance that is also being adopted by neighboring East Asian countries. He suggests that approaches like the EU Directive are unlikely to become mainstream in Asia.
  • @mjbommar acknowledges the global focus on US/EU laws but points out that many commonly used models are developed by Western organizations. He questions how Japan’s updated copyright laws align with international treaties like WCT/DMCA, expressing concern that they may allow practices that conflict with these agreements.
    • @shujisado responds by stating that Japan’s copyright laws, including Article 30-4, were carefully crafted to comply with international standards, such as the Berne Convention and the WIPO Copyright Treaty, ensuring that they meet the required legal frameworks.
Welcome diverse approaches to training data within a unified Open Source AI Definition
  • @arandal emphasizes the importance of the Open Source Definition (OSD) as a unifying framework that accommodates diverse approaches within the open-source community. She argues that AI models, being a combination of source code and training data, should have their diversity in handling data explicitly recognized in the Open Source AI Definition. She proposes specific text changes to the draft to clarify that while some developers may be comfortable with proprietary data, others may not, and both approaches should be supported to ensure the long-term success of open-source AI.
  • @mjbommar appreciates the spirit of Arandal’s proposal but adds that the OSI currently lacks specific licenses for data, which is why it is crucial for the OSI to collaborate with Creative Commons. Creative Commons maintains the ecosystem of “data licenses” that would be necessary under the proposed revisions to the Open Source AI Definition.
  • @arandal agrees with the need for collaboration with organizations like Creative Commons, noting that this coordination is already reflected in checklist v. 0.0.9. She suggests that such collaboration is necessary even without the proposed revisions to ensure the definition accurately addresses data licensing in AI.
  • @nick acknowledges the importance of working with organizations like Creative Commons and mentions that OSI is in ongoing communication with several relevant organizations, including MLCommons, the Open Future Foundation, and the Data and Trust Alliance. He highlights the recent publication of the Data Provenance Standards by the Data and Trust Alliance as an example of the kind of collaborative work that is being pursued.
  • @mjbommar reiterates the need for explicit coordination with Creative Commons, arguing that the OSI cannot realistically finalize the Open Source AI Definition without such collaboration. He also suggests that the OSI should explore AI preference signaling and work with Creative Commons and SPDX/LF to establish shared standards, which should be part of the OSAID standard’s roadmap.

Join this week’s town hall to hear the latest developments, give your comments and ask questions.

Register for the townall
Categories: FLOSS Research

Ezequiel Lanza: Voices of the Open Source AI Definition

Open Source Initiative - Wed, 2024-08-28 09:38

The Open Source Initiative (OSI) is running a blog series to introduce some of the people who have been actively involved in the Open Source AI Definition (OSAID) co-design process. The co-design methodology allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support and contribution from a significant and broad group of stakeholders is imperative to the Open Source process and is proven to bring diverse issues to light, deliver swift outputs and garner community buy-in.

This series features the voices of the volunteers who have helped shape and are shaping the Definition.

Meet Ezequiel Lanza

What’s your background related to Open Source and AI?

I’ve been working in AI for more than 10 years (Yes, before ChatGPT!). With a background in engineering, I’ve consistently focused on building and supporting AI applications, particularly in machine learning and data science. Over the years, I’ve contributed to and collaborated on various projects. A few years ago, I decided to pursue a master’s in data science to deepen my theoretical knowledge and further enhance my skills. Open Source has also been a significant part of my work; the frameworks, tools and community have continually drawn me in, making me an active participant in this evolving conversation for years.

What motivated you to join this co-design process to define Open Source AI?

AI owes much of its progress to Open Source, and it’s essential for continued innovation. My experience in both AI and Open Source spans many years, and I believe this co-design process offers a unique chance to contribute meaningfully. It’s not just about sharing my insights but also about learning from other professionals across AI and different disciplines. This collective knowledge and diverse perspectives make this initiative truly powerful and enriching, to shape the future of Open Source AI together.

Can you describe your experience participating in this process? What did you most enjoy about it, and what were some of the challenges you faced?

Participating in this process has been both rewarding and challenging. I’ve particularly enjoyed engaging with diverse groups and hearing different perspectives. The in-person events, such as All Things Open in Raleigh in 2023, have been valuable for fostering direct collaboration and building relationships. However, balancing these meetings with my work duties has been challenging. Coordinating schedules and managing time effectively to attend all the relevant discussions can be demanding. Despite these challenges, the insights and progress have made the effort worthwhile.

Why do you think AI should be Open Source?

We often say AI is everywhere, and while that’s partially true, I believe AI will be everywhere, significantly impacting our lives. However, AI’s full potential can only be realized if it is open and accessible to everyone. Open Source AI should also foster innovation by enabling developers and researchers from all backgrounds to contribute to and improve existing models, frameworks and tools, allowing freedom of expression. Without open access, involvement in AI can be costly, limiting participation to only a few large companies. Open Source AI should aim to democratize access, allowing small businesses, startups and individuals to leverage powerful tools that might otherwise be out of reach due to cost or proprietary barriers.

What do you think is the role of data in Open Source AI?

Data is essential for any AI system. Initially, from my ML bias perspective, open and accessible datasets were crucial for effective ML development. However, I’ve reevaluated this perspective, considering how to adapt the system while staying true to Open Source principles. As AI models, particularly GenAI like LLMs, become increasingly complex, I’ve come to value the models themselves. For example, Generative AI requires vast amounts of data, and gaining access to this data can be a significant challenge.

This insight has led me to consider what I—whether as a researcher, developer or user—truly need from a model to use/investigate it effectively. While understanding the data used in training is important, having access to specific datasets may not always be necessary. In approaches like federated learning, the model itself can be highly valuable while keeping data private, though understanding the nature of the data remains important. For LLMs, techniques such as fine-tuning, RAG and RAFT emphasize the benefits of accessing the model rather than the original dataset, providing substantial advantages to the community.

Sharing model architecture and weights is crucial, and data security can be maintained through methods like model introspection and fine-tuning, reducing the need for extensive dataset sharing.

Data is undoubtedly a critical component. However, the essence of Open Source AI lies in ensuring transparency, then the focus should be on how data is used in training models. Documenting which datasets were used and the data handling processes is essential. This transparency helps the community understand the origins of the data, assess potential biases and ensure the responsible use of data in model development. While sharing the exact datasets may not always be necessary, providing clear information about data sources and usage practices is crucial for maintaining trust and integrity in Open Source AI.

Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

Of course, it changed and evolved – that’s what a thought process is about. I’d be stubborn if I never changed my perspective along the way. I’ve often questioned even the most fundamental concepts I’ve relied on for years, avoiding easy or lazy assumptions. This thorough process has been essential in refining my understanding of Open Source AI. Engaging in meaningful exchanges with others has shown me the importance of practical definitions that can be implemented in real-world scenarios. While striving for an ideal, flawless definition is tempting, I’ve found that embracing a pragmatic approach is ultimately more beneficial.

What do you think the primary benefit will be once there is a clear definition of Open Source AI?

As I see it, the Open Source AI Definition will support the growth, and it will be the first big step. The primary benefit of having a clear definition of Open Source AI will be increased clarity and consistency in the field. This will enhance collaboration by setting clear standards and expectations for researchers, developers and organizations. It will also improve transparency by ensuring that AI models and tools genuinely follow Open Source principles, fostering trust in their development and sharing.

A clear definition will create standardized practices and guidelines, making it easier to evaluate and compare different Open Source AI projects.

What do you think are the next steps for the community involved in Open Source AI?

The next steps for the community should start with setting up a certification process for AI models to ensure they meet certain standards. This could include tools to help automate the process. After that, it would be helpful to offer templates and best practice guides for AI models. This will support model designers in creating high-quality, compliant systems and make the development process smoother and more consistent.

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the forum: share your comment on the drafts.
  • Leave comment on the latest draft: provide precise feedback on the text of the latest draft.
  • Follow the weekly recaps: subscribe to our monthly newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: we’re increasing the frequency to weekly meetings where you can learn more, ask questions and share your thoughts.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
Categories: FLOSS Research

Three things I learned at KubeCon + AI_Dev China 2024

Open Source Initiative - Tue, 2024-08-27 14:10

KubeCon China 2024 was a whirlwind of innovation, community and technical deep dives. As it often happens at these community events, I was blown away by the energy, enthusiasm and sheer amount of knowledge being shared. Here are three key takeaways that stood out to me:

1. The focus on AI and machine learning

AI and machine learning are increasingly integrated into cloud-native applications. At KubeCon China, I saw numerous demonstrations of how these technologies are being used to automate tasks, optimize resource utilization and improve application performance. From AI-powered observability tools to machine learning-driven anomaly detection, the potential for AI and ML in the cloud-native space is astounding.

Mer Joyce and Anni Lai introduced the new draft of the Open Source AI Definition (v.0.0.9) and the Model Openness Framework. 

We also saw a robot on stage demonstrating that teaching a robotic arm to use a spoon to help disabled people is not a programming issue but a data issue. This was probably my biggest learning moment: A robot can be “taught” to execute tasks by imitating humans. Follow Xavier Tao and the dora-rs project.

2. The growing maturity of cloud-native technologies

It’s clear that cloud-native technologies have come of age. From Kubernetes adoption to the rise of serverless platforms and edge computing, the ecosystem is thriving. In his keynote, Chris Aniszczyk announced over 200 projects are hosted by the Cloud Native Computing Foundation and half of the contributors are not in the US. The conference showcased a wide range of tools, frameworks and use cases that demonstrate the versatility and scalability of cloud-native architectures.

The presentation by Kevin Wang (Huawei) and Saint Jiang (NIO) showed how Containerd, Kubernetes and KubeEdge power the transition to electric vehicles. Modern cars are computers… no, cars are full datacenters on wheels, a collection of sensors feeding distributed applications to optimize battery usage, feeding into centralized programs to constantly improve the whole mobility system.

3. AI technology is removing the language barrier

I was absolutely amazed by being able to follow the keynote sessions delivered in Chinese. I don’t speak Chinese but I could read the automatic translation in real time superimposed on the slides behind the speakers. This technology is absolutely jaw-droppingly amazing! Within a few years, there won’t be a career for simultaneous translators or for live transcribers.

Final thoughts

KubeCon + AI_Dev China was a testament to the power of Open Source collaboration hosted in one of the most amazing regions of the world. The conference brought together developers, operators and end-users from around the world to share their experiences, best practices and contributions to Open Source projects. This collaborative spirit is essential for driving innovation and ensuring the long-term success of cloud-native technologies.

Categories: FLOSS Research

Open Source AI – Weekly update August 26

Open Source Initiative - Mon, 2024-08-26 15:33
Week 34 summary  Share your thoughts about draft v0.0.9

As we move toward the release of the first-ever Open Source AI Definition in October at All Things Open, the publication of the 0.0.9 draft brings us one step closer to realizing this goal.

  • OSAID 0.0.9 draft definition is live! 
  •   Changelog includes:
    • New Feature: Clarified Open Source Models and Weights
      • Added a new paragraph under “What is Open Source AI” to define “system” as including both models and weights.
      • Clarified that all components of a larger system must meet the standard.
      • Updated paragraph after the “share” bullet to emphasize this point.  
    • New Section: Open Source Models and Open Source Weights
      • Added descriptions of components for both models and weights in machine learning systems.
      • Edited subsequent paragraphs to eliminate redundancy.
    • Training Data: Defined as a Benefit, Not a Requirement
      • Defined open, public, and unshareable non-public training data.
      • Explained the role of training data in studying AI systems and understanding biases.
      • Emphasized extra requirements for data to advance openness, especially in private-first areas like healthcare.
    • Separation of Checklist
      • The Checklist is now a separate document from the main Definition.
      • Fully aligned Checklist content with the Model Openness Framework (MOF).
    • Terminology Changes
      • Replaced “Model” with “Weights” under “Preferred form to make modifications” for consistency.
    • Explicit Reference to Recipients of the Four Freedoms
      • Added specific references to developers, deployers, and end users of AI systems.
    • Credits and References
      • Incorporated credit to the Free Software Definition.
      • Added references to conditions of availability of components, referencing the Open Source Definition.
  • Initial reactions on the forum: 
    • @shujisado praises the updates in version 0.0.9, particularly the decision to separate the checklist from the main document, which clarifies the intent behind OSAID. He also supports the separation of “code” and “weights,” noting that in Japan, “code” clearly falls under copyright, making this distinction logical. He acknowledges revisions in the checklist that consider the importance of complete datasets, even though he disagrees with making datasets mandatory. 
  • Comments on the draft on HackMD
    • @Joshua Gay adds that instead of narrowing the focus to machine-learning systems, the emphasis should be on “parameters” as a whole since weights are just one type of parameter. He suggests a rewrite that highlights making model parameters, such as weights and other settings, available under OSI-approved terms, with examples across various AI models.
      • He further suggests using broader language that covers more AI systems instead of narrower terminology. Specifically, he proposes replacing “Open Source models and Open Source weights” with “Open Source models and Open Source parameters,” and using “AI systems” instead of “machine learning systems.” Additionally, he recommends redefining an AI model to include architecture, parameters like weights and decision boundaries, and inference code, while referring to AI parameters as configuration settings that produce outputs from inputs.
    • Under “Open Source models and Open Source weights”, @shujisado adds that the last paragraph titled “Open Source models and Open Source weights” actually explains “AI model” and “AI weights,” leading to a mismatch between the title and content, and notes that these terms are not used elsewhere in the definition.
    • Under “Preferred form to make modifications to machine-learning systems”, @shujisado suggests some grammatical corrections.
  • Next steps
    • The OSI has recently presented at the following events: 
    • Iterate Drafts: Continue refining drafts with feedback from the worldwide roadshow, considering new dissenting opinions.
    • Review Licenses: Decide on the best approach for reviewing new licenses for datasets, documentation, and model parameters.
    • Enhance FAQ: Continue improving the FAQ to address emerging questions.
    • Post-Stable Release Plan: Establish a process for reviewing and updating future versions of the Open Source AI Definition.
 Explaining the concept of Data information
  •  @Kjetilk points out the legal distinction between using copyrighted works for AI training (reproduction) and incorporating them into publishable datasets, questioning the fairness of allowing exploitative models without compensation while potentially banning those that benefit society.
  • @Shujisadoclarifies that compensation for copyrighted works used in AI training is possible for both open source and closed models, distinguishing it from “royalty,” and notes that Japan’s copyright law exempts such uses for machine learning.
    • @Kjetilk reiterates the relevance of “royalty” for compensation in closed, non-published models, suggesting it makes sense under copyright law if required, but if not, it could benefit science and the arts.
Open Source AI Definition Town Hall
  • The slides and recording from the town hall meeting held on August 23, 2024 are available here.
  • The next town hall meeting will be held on September 6th. Sign up for the event here.
Categories: FLOSS Research

Members Newsletter – August 2024

Open Source Initiative - Mon, 2024-08-26 11:27
August 2024 Members Newsletter

The lively conversation about the role of data in building and modifying AI systems will continue as the OSI travels to China this month for AI_dev (August 21-23 in Hong Kong) and Open Source Congress (August 25-27 in Beijing). The OSI has been able to chime in on news stories on the topic, several of which are linked here in the newsletter.

Last month the OSI was at the United Nations in New York City for OSPOs for Good, an event that covered key areas of open source policy, as well as emerging examples of ‘Open Source for good’ from across the globe. I participated in a panel on Open Source AI.

Creating an Open Source AI Definition has been an arduous task over the past couple of years, but we know the importance of creating this standard so the freedoms to use, study, share and modify AI systems can be guaranteed. Those are the core tenets of Open Source, and it warrants the dedicated work it has required. Please read about the people who have played key roles in bringing the Definition to life in our Voices of Open Source AI Definition on the blog.

Stefano Maffulli

Executive Director, OSI 

I hold weekly office hours on Fridays with OSI members: book time if you want to chat about OSI’s activities, if you want to volunteer or have suggestions.

News from the OSI OSI at the United Nations OSPOs for Good

From the Research and Advocacy program

Earlier this month the Open Source Initiative participated in the “OSPOs for Good” event promoted by the United Nations in NYC. Read more.

The Open Source Initiative joins CMU in launching Open Forum for AI: A human-centered approach to AI development

From the Research and Advocacy program

The Open Source Initiative (OSI) is pleased to share that we are joining the founding team of Open Forum for AI (OFAI), an initiative designed by Carnegie Mellon University (CMU). Read more

GUAC adopts license metadata from ClearlyDefined

From the License and Legal program

The software supply chain just gained some transparency thanks to an integration of the Open Source Initiative (OSI) project, ClearlyDefined, into GUAC (Graph for Understanding Artifact Composition), an OpenSSF project from the Linux Foundation. Read more.

Better identifying conda packages with ClearlyDefined

From the License and Legal program

ClearlyDefined now provides a new harvester implementation for conda, a popular package manager with a large collection of pre-built packages for various domains, including data science, machine learning, scientific computing and more. Read more.

OSI in the news Can AI even be open source? It’s complicated

OSI at ZDNet

AI can’t exist without open source, but the top AI vendors are unwilling to commit to open-sourcing their programs and data sets. To complicate matters further, defining open-source AI is a messy issue that has yet to be settled. Read more.

Open Source AI: What About Data Transparency?

OSI at The New Stack

AI uses both code and data, and this combination continues to be a challenge for open source, said experts at the United Nations OSPOs for Good Conference. Read more.

A new White House report embraces open-source AI

OSI at ZDNet

The National Telecommunications and Information Administration (NTIA) issued a report supporting open-source and open models to promote innovation in AI, while emphasizing the need for vigilant risk monitoring. Read more.

With Open Source Artificial Intelligence, Don’t Forget the Lessons of Open Source Software

OSI at CISA

While there is not yet a consensus on the definition of what constitutes “open source AI”, the Open Source Initiative, which maintains the “Open Source Definition” and a list of approved OSS licenses, has been “driving a multi-stakeholder process to define an ‘Open Source AI’”. Read more.

Meta inches toward open source AI with new LLaMA 3.1

OSI at ZDNet

Is Meta’s 405 billion parameter model really open source? Depends on who you ask. Here’s how to try out the new engine for yourself​. Read more.

Other news News from OSI affiliates News from OpenSource.net Voices of the Open Source AI Definition

The Open Source Initiative (OSI) is running a series of stories about a few of the people involved in the Open Source AI Definition (OSAID) co-design process.

7th annual OSPO and Open Source Management Survey

The TODO Group and Linux Foundation Research, in partnership with Cisco, NGINX, Open Source Initiative, InnerSource Commons, and CHAOSS, are excited to be launching the 7th annual OSPO and Open Source Management survey! Take survey here.

2024 Open Source Software Funding Survey

This survey tries to better understand how organizations fund, contribute to, and support open source software projects. This survey is a collaboration between GitHub, Inc., the Linux Foundation, and researchers from Harvard University. Take survey here.

Events Upcoming events Thanks to our sponsors New members and renewals
  • Cisco
  • Microsoft
  • Bloomberg
  • SAS
  • Intel
  • Look to the right

Interested in sponsoring, or partnering with, the OSI? Please see our Sponsorship Prospectus and our Annual Report. We also have a dedicated prospectus for the Deep Dive: Defining Open Source AI. Please contact the OSI to find out more about how your company can promote open source development, communities and software.

Support OSI by becoming a member!

Let’s build a world where knowledge is freely shared, ideas are nurtured, and innovation knows no bounds! 

Join the Open Source Initiative!

Categories: FLOSS Research

Community input drives the new draft of the Open Source AI Definition

Open Source Initiative - Thu, 2024-08-22 08:30

A new version of the Open Source AI Definition has been released with one new feature and a cleaner text, based on comments received from public discussions and recommendations. We’re continuing our march towards having a stable release by the end of October 2024, at All Things Open. Get involved by joining the discussion on the forum, finding OSI staff around the world and online at the weekly town halls. 

New feature: clarified Open Source model and Open Source weights
  • Under “What is Open Source AI,” there is a new paragraph that (1) identifies both models and weights/parameters as encompassed by the word “system” and (2) makes it clear that all components of a larger system have to meet the standard. There is a new sentence in the paragraph after the “share” bullet making this point.
  • Under the heading “Open Source models and Open Source weights,” there is a description of the components for both of those for machine learning systems. We also edited the paragraph below those additions to eliminate some redundancy.
Training data in the preferred form to make modifications

The role of training data is one of the most hotly debated parts of the definition. After long deliberation and co-design sessions we have concluded that defining training data as a benefit, not a requirement, is the best way to go.

Training data is valuable to study AI systems: to understand the biases that have been learned, which can impact system behavior. But training data is not part of the preferred form for making modifications to an existing AI system. The insights and correlations in that data have already been learned.

Data can be hard to share. Laws that permit training on data often limit the resharing of that same data to protect copyright or other interests. Privacy rules also give a person the rightful ability to control their most sensitive information, such as decisions about their health. Similarly, much of the world’s Indigenous knowledge is protected through mechanisms that are not compatible with later-developed frameworks for rights exclusivity and sharing.

  • Open training data  (data that can be reshared) provides the best way to enable users to study the system, along with the preferred form of making modifications.
  • Public training data  (data that others can inspect as long as it remains available) also enables users to study the work, along with the preferred form.
  • Unshareable non-public training data  (data that cannot be shared for explainable reasons) gives the ability to study some of the systems biases and demands a detailed description of the data – what it is, how it was collected, its characteristics, and so on – so that users can understand the biases and categorization underlying the system.

OSI believes these extra requirements for data beyond the preferred form of making modifications to the AI system both advance openness in all the components of the preferred form of modifying the AI system and drive more Open Source AI in private-first areas such as healthcare.

Other changes
  • The Checklist is separated into its own document. This is to separate the discussion about how to identify Open Source AI from the establishment of general principles in the Definition. The content of the Checklist has also been fully aligned with the Model Openness Framework (MOF), allowing for an easy overlay.
  • Under “Preferred form to make modifications,” the word “Model” changed to “Weights.” The word “Model”  was referring only to parameters, and was inconsistent with how the word “model” is used in the rest of the document.
  • There is an explicit reference to the intended recipients of the four freedoms: developers, deployers and end users of AI systems.
  • Incorporated credit to the Free Software Definition.
  • Added references to conditions of availability of components, referencing the Open Source Definition.
Next steps
  • Continue iterating through drafts after meeting diverse stakeholders at the worldwide roadshow, collect feedback and carefully look for new arguments in dissenting opinions. 
  • Decide how to best address the reviews of new licenses for datasets, documentation and the agreements governing model parameters.
  • Keep improving the FAQ.
  • Prepare for post-stable-release: Establish a process to review future versions of the Open Source AI Definition.
Collecting input and endorsements

We will be taking draft v.0.0.9 on the road collecting input and endorsements, thanks to a grant by the Sloan Foundation. The lively conversation about the role of data in building and modifying AI systems will continue at multiple conferences from around the world, the weekly town halls and online throughout the Open Source community. 

The first two stops are in Asia: Hong Kong for AI_dev August 21-23, then Beijing for Open Source Congress August 25-27. Other events are planned to take place in Africa, South America, Europe and North America. These are all steps toward the conclusion of the co-design process that will result in the release of the stable version of the Definition in October at All Things Open. 

Creating an Open Source AI Definition is an arduous task over the past two years, but we know the importance of creating this standard so the freedoms to use, study, share and modify AI systems can be guaranteed. Those are the core tenets of Open Source, and it warrants the dedicated work it has required. You can read about the people who have played key roles in bringing the Definition to life in our Voices of Open Source AI Definition on the blog. 

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the forum: share your comment on the drafts.
  • Leave comment on the latest draft: provide precise feedback on the text of the latest draft.
  • Follow the weekly recaps: subscribe to our monthly newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: we’re increasing the frequency to weekly meetings where you can learn more, ask questions and share your thoughts.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
One of the many OSAID workshops organized by the OSI around the world
Categories: FLOSS Research

Mark Collier: Voices of the Open Source AI Definition

Open Source Initiative - Fri, 2024-08-16 10:06

The Open Source Initiative (OSI) is running a blog series to introduce some of the people who have been actively involved in the Open Source AI Definition (OSAID) co-design process. The co-design methodology allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support and contribution from a significant and broad group of stakeholders is imperative to the Open Source process and is proven to bring diverse issues to light, deliver swift outputs and garner community buy-in.

This series features the voices of the volunteers who have helped shape and are shaping the Definition.

Meet Mark Collier

What’s your background related to Open Source and AI?

I’ve worked in Open Source most of my career, over 20 years, and have found it to be one of the greatest, if not the greatest, drivers of economic opportunity. It creates new markets and gives people all over the world access to not only use but to influence the direction of technologies. I started the OpenStack project and then the OpenStack Foundation, and later the Open Infrastructure Foundation. With members of our foundation from over 180 countries, I’ve seen firsthand how Open Source is the most efficient way to drive innovation. You get to crowdsource ideas from people all over the world, that are not just in one company or just in one country. We’ve certainly seen that with infrastructure in the cloud computing/edge computing era. AI is the next generation wave, with people investing literally trillions of dollars in building infrastructure, both physical and the software being written around it. This is another opportunity to embrace Open Source as a path to innovation. 

Open Source drives the fastest adoption of new technologies and gives the most people an opportunity to both influence it and benefit economically from it, all over the world. I want to see that pattern repeat in this next era of AI.

What motivated you to join this co-design process to define Open Source AI?

I’m concerned about the potential of there not being credible Open Source alternatives to the big proprietary players in this massive next wave of technology. It will be a bad thing for humanity if we can only get state-of-the-art AI from two or three massive players in one or two countries. In the same way we don’t want to see just one cloud provider or one software vendor, we don’t want any sort of monopoly or oligopoly in AI; That really slows innovation. I wanted to be part of this co-design process because it’s actually not trivial to apply the concept of Open Source to AI. We can carry over the principles and freedoms that underlie Open Source software, like the freedom to use it without restriction and the ability to modify it for different use cases, but an AI system is not just software. A whole debate has been stirred up about whether data needs to be released and published under an Open Source friendly license to be considered Open Source AI. That’s just one consideration of many that I wanted to contribute to. 

We have a very impressive group of people with a lot of diverse backgrounds and opinions working on this. I wanted to be part of the process not because I have the answers, but because I have some perspective and I can learn from all the others. We need to reach a consensus on this because if we don’t, the meaning of Open Source in the AI era will get watered down or potentially just lost all together, which affects all of Open Source and all of technology. 

Can you describe your experience participating in this process? What did you most enjoy about it and what were some of the challenges you faced?

The process started as a mailing list of sorts and evolved to more of an online discussion forum. Although it hasn’t always been easy for me to follow along, the folks at OSI that have been shepherding the process have done an excellent job summarizing the threads and bringing key topics to the top. Discussions are happening rapidly in the forum, but also in the press. There are new models released nearly every day it seems, and the bar for what are called Open Source models is causing a lot of noise. It’s a challenge for anybody to keep up but overall I think it’s been a good process.

Why do you think AI should be Open Source?

The more important a technology is to the future of the economy and the more a technology impacts our daily lives, the more critical it is that it be Open Source. For economic and participation reasons, but also for security. We have seen time and time again that transparency and openness breeds better security. With more mysterious and complex technologies such as AI, Open Source offers the transparency to help us understand the decisions the technology is making. There have been a number of large players who have been lobbying for more regulation, making it more difficult to have Open Source AI, and I think that shows a very clear conflict of interest. 

There is legislation out there that, if it gets passed, poses a real danger to not just Open Source AI, but Open Source in general. We have a real opportunity but also a real risk of there being conscious concentrations of power if state-of-the-art AI doesn’t fit a standard definition of Open Source. Open Source AI continues to be neck and neck with the proprietary models, which makes me optimistic.

Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

My personal definition of Open Source AI is not set in stone even having been through this process for over a year. Things are moving so quickly, I think we need to be careful that perfect doesn’t become the enemy of good. Time is of the essence as the mainstream media and the tech press report on models that are trained on billions of dollars worth of hardware, claiming to be Open Source when they clearly are not. I’ve become more willing to compromise on an imperfect definition so we can come to a consensus sooner.

What do you think the primary benefit will be once there is a clear definition of Open Source AI?

All the reasons people love Open Source are inherently the same reasons why people are very tempted to put an Open Source label on their AI; trust, transparency, they can modify it and build their business on it, and the license won’t be changed. Once we finalize and ratify the definition, we can start broadly using it in practice. This will bring some clarity to the market again. We need to be able to point to something very clear and documented if we’re going to challenge a technology that has been labeled Open Source AI. Legal departments of big companies working on massive AI tools and workloads want to know that their license isn’t going to be pulled out from under them. If the definition upholds the key freedoms people expect from Open Source, it will lead to faster adoption by all.  

What do you think are the next steps for the community involved in Open Source AI?

I think Stefano from the OSI has done a wonderful job of trying to hit the conference circuit to share and collect feedback, and virtual participation in the process is still key to keeping it inclusive. I think the next step is building awareness in the press about the definition and market testing it. It’s an iterative process from there. 

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the working groups: be part of a team to evaluate various models against the OSAID.
  • Join the forum: support and comment on the drafts, record your approval or concerns to new and existing threads.
  • Comment on the latest draft: provide feedback on the latest draft document directly.
  • Follow the weekly recaps: subscribe to our newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: participate in the online public town hall meetings to learn more and ask questions.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
Categories: FLOSS Research

Update from the board of directors

Open Source Initiative - Wed, 2024-08-07 12:13

The Chair of the Board of the OSI has acknowledged the resignation offered by Secretary of the Board, Aeva Black. The Chair and the entire Board would like to thank Black for their invaluable contribution to the success of OSI, as well of the entire Open Source Community, and for their service as board member and officer of the Initiative.

Categories: FLOSS Research

Jean-Pierre Lorre: Voices of the Open Source AI Definition

Open Source Initiative - Wed, 2024-08-07 08:43

The Open Source Initiative (OSI) is running a blog series to introduce some of the people who have been actively involved in the Open Source AI Definition (OSAID) co-design process. The co-design methodology allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support and contribution from a significant and broad group of stakeholders is imperative to the Open Source process and is proven to bring diverse issues to light, deliver swift outputs and garner community buy-in.

This series features the voices of the volunteers who have helped shape and are shaping the Definition.

Meet Jean-Pierre Lorre

What’s your background related to Open Source and AI?

I’ve been using Open Source technologies since the very beginning of my career and have been directly involved in Open Source projects for around 20 years.

I graduated in artificial intelligence engineering in 1985. Since then I have worked in a number of applied AI research structures in fields such as medical image processing, industrial plant supervision, speech recognition and natural language processing. My knowledge covers both symbolic AI methods and techniques and deep learning.

I currently lead a team of around fifteen AI researchers at LINAGORA. LINAGORA is an Open Source company.

What motivated you to join this co-design process to define Open Source AI?

The team I lead is heavily involved in the development of LLM generative models, which we want to distribute under an open license. I realized that the term Open Source AI was not defined and that the definition we had at LINAGORA was not the same as the one adopted by our competitors.

As the OSI is the leading organization for defining Open Source and there was a project underway to define the term Open Source AI, I decided to join it.

Can you describe your experience participating in this process? What did you most enjoy about it and what were some of the challenges you faced?

I participated in two ways: firstly, to provide input for the definition currently being drafted; and secondly, to evaluate LLM models with regard to the definition (I contributed to Bloom, Falcon and Mistral).

For the first item, my main difficulty was keeping up with the meandering discussions, which were very active. I didn’t manage to do so completely, but I was able to appreciate the summaries provided from time to time, which enabled me to follow the overall thread.

The second difficulty concerns the evaluation of the models: the aim of the exercise was to evaluate the consistency of OSAID version 0.8 on models that currently claim to be “Open Source.” Implementing the definition involves looking for information that is sometimes non-existent and sometimes difficult to find. 

Why do you think AI should be Open Source?

Artificial intelligence models are expected to play a very important role in our professional lives, but also in our everyday lives. In this respect, the need for transparency is essential to enable people to check the properties of the models. They must also be accessible to as many people as possible, to avoid widening the inequalities between those who have the means to develop them and those who will remain on the sidelines of this innovation. Similarly, they might be adapted for different uses without the need for authorization.

The Open Source approach makes it possible to create a community such as the one created by LINAGORA, OpenLLM-Europe. This is a way for small players to come together to build the critical mass needed not only to develop models but also to disseminate them. Such an approach, which may be compared to that associated with the digital commons, is a guarantee of sovereignty because it allows knowledge and governance to be shared.

In short, they are the fruit of work based on data collected from as many people as possible, so they must remain accessible to as wide an audience as possible.

What do you think is the role of data in Open Source AI?

Data provides the basis for training models. It is therefore the pool of information from which the knowledge displayed by the model and the applications deduced from it will be drawn. In the case of an open model, the dissemination of as many elements as possible to qualify this data is a means of transparency that facilitates the study of the model’s properties; indeed, this data is likely to include cultural bias, gender, ethnic origin, skin color, etc. It is also a means of facilitating the study of the model’s properties. It also makes it easier to modify the model and its outputs.

Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

Yes, we initially thought that the provision of training data was a sine qua non condition for the design of truly Open Source models. Our basic assumption was that the model may be seen as a work derived from the data and that therefore the license assigned to the data, in particular the non-commercial nature, had an impact on the license of the model. As the discussions progressed, we realized that this condition was very restrictive and severely limited the possibility of developing models.

Our current analysis is that the condition defined in version 0.8 of the OSAID is sufficient to provide the necessary guarantees of transparency for the four freedoms and in particular the freedom to study the model underlying access to data. With regard to the data, it stipulates that “sufficiently detailed information about the data used to train the system, so that a skilled person can recreate a substantially equivalent system using the same or similar data” must be provided. Even if we can agree that this condition seems difficult to satisfy without providing the data sets, other avenues may be envisaged, in particular the provision of synthetic data. This information should make it possible to carry out almost all of the model’s studies.

What do you think the primary benefit will be once there is a clear definition of Open Source AI?

Having such a definition with clear, implementable rules will provide model suppliers with a concrete framework for producing models that comply with the ethics of the Open Source movement.

A collateral effect will be to help sort out the “wheat from the chaff.” In particular, to detect attempts at “Open Source washing.” This definition is therefore a structuring element for a company such as LINAGORA, which wants to build a sustainable business model around the provision of value-added AI services.

It should also be noted that such a definition is necessary for regulations such as the European IA Act, which defines exceptions for Open Source generative models. Such legislative construction cannot be satisfied with a fuzzy basis.

What do you think are the next steps for the community involved in Open Source AI?

The next steps that need to be addressed by the community concern firstly the definition of a certification process that will formalize the conformity of a model; this process may be accompanied by tools to automate it.

In a second phase, it may also be useful to provide templates of AI models that comply with the definition, as well as best practice guides, which would help model designers.

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the working groups: be part of a team to evaluate various models against the OSAID.
  • Join the forum: support and comment on the drafts, record your approval or concerns to new and existing threads.
  • Comment on the latest draft: provide feedback on the latest draft document directly.
  • Follow the weekly recaps: subscribe to our newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: participate in the online public town hall meetings to learn more and ask questions.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
Categories: FLOSS Research

GUAC adopts license metadata from ClearlyDefined

Open Source Initiative - Tue, 2024-08-06 12:23

The software supply chain just gained some transparency thanks to an integration of the Open Source Initiative (OSI) project, ClearlyDefined, into GUAC (Graph for Understanding Artifact Composition), an OpenSSF project from the Linux Foundation. GUAC provides a comprehensive mapping of software packages, dependencies, vulnerabilities, attestations, and more, allowing organizations to achieve better compliance and security of their software supply chain.

GUAC offers the full view of the supply chain

Software supply chain attacks are on the rise. Many tools are available to help generate software bills of materials (SBOMs), signed attestations and vulnerability reports, but they stop there, leaving users to figure out how they all fit together. GUAC provides an aggregated, queryable view across the whole software supply chain, not just one SBOM at a time.

GUAC is for developers, operations and security practitioners who need to identify and address problems in their software supply chain, including proactively managing dependencies and responding to vulnerabilities. GUAC provides supply chain observability with a graph view of the software supply chain and tools for performing queries to gain actionable insights.

GUAC enhanced with ClearlyDefined integration

The latest version of GUAC (v0.8.0) now provides support for ClearlyDefined. GUAC will query the ClearlyDefined license metadata store to discover license information for packages, even when the SBOM does not include that information.

A ClearlyDefined certifier will listen on collector-subscriber for any pkg/src strings, then convert to ClearlyDefined coordinates, then query the API service for the definition. The user agent will be the same as existing outgoing GUAC requests GUAC/<version> (e.g. GUAC/v0.1.0).

A CertifyLegal node will be created using the “licensed” “declared” field from the definition. The expression will be copied and any license identifiers found will result in linked License noun nodes, created if needed. Type will be “declared”. Justification will be “Retrieved from ClearlyDefined”. Time will be the current time the information was retrieved from the API.

Similarly a node will be created using the “licensed” “facets” “core” “discovered” “expressions” field. Multiple expressions will be “AND”ed together. Type will be “discovered”, and other fields the same (Time, Justification, License links, etc).

The “licensed” “facets” “core” “attribution” “parties” array will be concatenated and stored in the Attribution field on CertifyLegal.

Optionally, “described” “sourceLocation” can be used to create a HasSourceAt GUAC node.

Thanks to the community

Although licenses don’t directly impact security, they are an important part of understanding the software supply chain. We would like to thank Parth Patel (Kusari), Jeff Mendoza (Kusari), Ben Cotton (Kusari), and Qing Tomlinson (SAP) for their support to get this feature implemented in GUAC. The ClearlyDefined community looks forward to working together with the GUAC community to help organizations worldwide to better achieve compliance and security of their software supply chain.

Categories: FLOSS Research

Deshni Govender: Voices of the Open Source AI Definition

Open Source Initiative - Thu, 2024-08-01 14:14

The Open Source Initiative (OSI) is running a blog series to introduce some of the people who have been actively involved in the Open Source AI Definition (OSAID) co-design process. The co-design methodology allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support and contribution from a significant and broad group of stakeholders is imperative to the Open Source process and is proven to bring diverse issues to light, deliver swift outputs and garner community buy-in.

This series features the voices of the volunteers who have helped shape and are shaping the Definition.

Meet Deshni Govender

What’s your background related to Open Source and AI?

I am the South Africa country focal point for the German Development Cooperation initiative “FAIR Forward – Artificial Intelligence for All” and the project strives for a more open, inclusive and sustainable approach to AI on an international level. More significantly, we seek to democratize the field of AI, to enable more robust, inclusive and self-determined AI ecosystems. Having worked in private sector and then now being in international development, my attention has been drawn to the disparity between the power imbalances of proprietary vs open and how this results in economic barriers for global majority, but also creates further harms and challenges for vulnerable populations and marginalized sectors, especially women. This fuelled my journey of working towards bridging the digital divide and digital gender gap through democratizing technology.

Some projects I am working on in this space include developing data governance models for African NLP (with Masakhane Foundation) and piloting new community-centered, equitable license types for voice data collection for language communities (with Mozilla).

What motivated you to join this co-design process to define Open Source AI?

I have experienced first hand the power imbalances that exist in geo-politics, but also in the context of economics where global minority countries shape the ‘global trajectory’ of AI without global voices. The definition of open means different things to different people / ecosystems / communities, and all voices should be heard and considered. Defining open means the values and responsibilities attached to it should be considered in a diverse manner, else the context of ‘open’ is in and of itself a hypocrisy.

Why do you think AI should be Open Source?

An enabling ecosystem is one that benefits all the stakeholders and ecosystem components. Inclusive efforts must be outlaid to explore and find tangible actions or potential avenues on how to reconcile the tension between openness, democracy and representation in AI training data whilst preserving community agency, diverse values and stakeholder rights. However, the misuse, colonization and misinterpretation of data continues unabated. Much of African culture and knowledge is passed down generations by story telling, art, dance and poetry and is done so verbally or through different ways of documentation, and in local manners and nuances of language. It is rarely digitized and certainly not in English. Language is culture and culture is context, yet somehow we find LLMs being used as an agent for language and context. Solutions and information are provided about and for communities but not with those communities, and the lack of transparency and post-colonial manipulation of data and culture is both irresponsible and should be considered a human rights violation.

Additionally, Open Source and open systems enable nations to develop inclusive AI policy processes so that policymakers from Global South countries can draw from peer experience on tackling their AI policies and AI-related challenges to find their own approaches to AI policy. This will also challenge dependence from and domination by western centric / Global North countries on AI policies to push a narrative or agenda on ‘what’ and ‘how’; i.e. Africa / Asia / LATAM must learn from us how to do X (since we hold the power, we can determine the extent and cost – exploitative). We aim for government self-determination and to empower countries, so that they may collectively have a voice on the global stage.

Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

My personal definition has not changed but it has been refreshing to witness the diverse views on how open is defined. The idea that behavior (e.g. of tech oligopolies) could reshape the way we define an idea or concept was thought-provoking. It means therefore that as emerging technology evolves, the idea of ‘open’ could change still in the future, depending on the trajectory of emerging technology and the values that society holds and attributes.

What do you think the primary benefit will be once there is a clear definition of Open Source AI?

A clear and more inclusive definition of Open Source AI would commerce a wave towards making data injustice, data invisibility, data extractivism, and data colonialism more visible and for which there exists repercussions. It would spur open, inclusive and responsible repositories of data, data use, and more importantly accuracy of use and interpretation. I am hoping that this would also spur innovative ways on how to track and monitor / evaluate use of Open Source data, so that local and small businesses are encouraged to develop in an Open Source while still being able to track and monitor players who extract and commercialize without giving back.

Ideally it would begin the process (albeit transitional) of bridging the digital divide between source and resource countries (i.e. global majority where data is collected from versus those who receive and process data for commercial benefit).

What do you think are the next steps for the community involved in Open Source AI?

If we make everything Open Source, it encourages sharing and use in developing and deploying, offers transparency and shared learning but enables freeriding. However the corollary is that closed models such as copyright prioritize proprietary information and commercialisation but can limit shared innovation, and does not uphold the concept of communal efforts, community agency and development. How do we quell this tension? I would like to see the Open Source community working to find practical and actionable ways in which we can make this work (open, responsible and innovative but enabling community benefit / remuneration).

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the working groups: be part of a team to evaluate various models against the OSAID.
  • Join the forum: support and comment on the drafts, record your approval or concerns to new and existing threads.
  • Comment on the latest draft: provide feedback on the latest draft document directly.
  • Follow the weekly recaps: subscribe to our newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: participate in the online public town hall meetings to learn more and ask questions.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
Categories: FLOSS Research

Hailey Schoelkopf: Voices of the Open Source AI Definition

Open Source Initiative - Thu, 2024-07-25 07:45

The Open Source Initiative (OSI) is running a blog series to introduce some of the people who have been actively involved in the Open Source AI Definition (OSAID) co-design process. The co-design methodology allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support and contribution from a significant and broad group of stakeholders is imperative to the Open Source process and is proven to bring diverse issues to light, deliver swift outputs and garner community buy-in.

This series features the voices of the volunteers who have helped shape and are shaping the Definition.

Meet Hailey Schoelkopf What’s your background related to Open Source and AI?

One of the main reasons I was able to get more deeply involved in AI research was through open research communities such as the BigScience Workshop and EleutherAI, where discussions and collaboration were available to outsiders. These opportunities to share knowledge and learn from others more experienced than me were crucial to learning about the field and growing as a practitioner and researcher.

I co-lead the training of the Pythia language models (https://arxiv.org/abs/2304.01373), some of the first fully-documented and reproducible large-scale language models with as many related artifacts as possible released Open Source. We were happy and lucky to see these models fill a clear need, especially in the research community, where Pythia has since contributed to a large amount of studies attempting to build our understanding of LLMs, including interpreting their internals, understanding the process by which these models improve over training, and disentangling some of the effects of the dataset contents on these models’ downstream behavior.

What motivated you to join this co-design process to define Open Source AI?

There has been a significant amount of confusion induced by the fact that not all ‘open-weights’ AI models released are released under OSI-compliant licenses-–or impose restrictions on their usage or adaptation-–so I was excited that OSI was working on reducing this confusion by producing a clear definition that could be used by the Open Source community. I more directly joined the process by helping discuss how the Open Source AI Definition could be mapped onto the Pythia language models and the accompanying artifacts we released.

Can you describe your experience participating in this process? What did you most enjoy about it and what were some of the challenges you faced?

Deciding what counts as sufficient transparency and modifiability to be Open Source was an interesting problem. Although public model weights are very beneficial to the Open Source community, releasing model weights without sufficient detail to understand the model and its development process to make modifications or understand reasons behind its design and resulting characteristics can hinder understanding or prevent the full benefits of a completely Open Source model from being realized.

Why do you think AI should be Open Source?

There are clear advantages to having models that are Open Source. Access to such fully-documented models can help a much, much broader group of people–trained researchers and also many others–who can use, study, and examine these models for their own purposes. While not every model should be made Open Source under all conditions, wider scrutiny and study of these models can help increase our understanding of AI systems’ behavior, raise societal preparedness and awareness of AI capabilities, and improve these models’ safety by allowing more people to understand them and explore their flaws.

With the Pythia language models, we’ve seen many researchers explore questions around the safety and biases of these models, including a breadth of questions we’d not have been able to study ourselves, or many that we could not even anticipate. These different perspectives are a crucial component in making AI systems safer and more broadly beneficial.

What do you think is the role of data in Open Source AI?

Data is a crucial component of AI systems. Transparency around (and, potentially, open release of) training datasets can enable a wide range of extended benefits to researchers, practitioners, and society at large. I think that for a model to be truly Open Source, and to derive the greatest benefits from its openness, information on training data must be shared transparently. This information also importantly allows various members of the Open Source community to avoid replicating each other’s work independently. Transparent sharing about motivations and findings with respect to dataset creation choices can improve the community’s collective understanding of system and dataset design for the future and minimize overlapping, wasted effort.

Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

An interesting perspective that I’ve grown to appreciate is that the Open Source AI definition includes public and Open Source licensed training and inference code. Actually making one’s Open Source AI model effectively usable by the community and practitioners is a crucial step of promoting transparency, though not often enough discussed.

What do you think the primary benefit will be once there is a clear definition of Open Source AI?

Having a clear definition of Open Source AI can make it clearer where existing currently “open” systems fall, and potentially encourage future open-weights models to be released with more transparency. Many current open-weights models are shared under bespoke licenses with terms not compliant with Open Source principles–this creates legal uncertainty and also makes it less likely that a new open-weights model release will benefit practitioners at large or contribute to better understanding of how to design better systems. I would hope that a clearer Open Source AI definition will make it easier to draw these lines and encourage those currently releasing open-weights models to do so in a way more closely fitting the Open Source AI standard.

What do you think are the next steps for the community involved in Open Source AI?

An exciting future direction for the Open Source AI research community is to explore methods for greater control over AI model behavior; attempting to explore approaches to collective modification and collaborative development of AI systems that can adapt and be “patched” over time. A stronger understanding of how to properly evaluate these systems for capabilities, robustness, and safety will also be crucial. I hope to see the community direct greater attention to evaluation in the future as well.

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the working groups: be part of a team to evaluate various models against the OSAID.
  • Join the forum: support and comment on the drafts, record your approval or concerns to new and existing threads.
  • Comment on the latest draft: provide feedback on the latest draft document directly.
  • Follow the weekly recaps: subscribe to our newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: participate in the online public town hall meetings to learn more and ask questions.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
Categories: FLOSS Research

OSI at the United Nations OSPOs for Good

Open Source Initiative - Wed, 2024-07-24 18:57

Earlier this month the Open Source Initiative participated in the “OSPOs for Good” event promoted by the United Nations in NYC. Stefano Maffulli, the Executive Director of the OSI, participated in a panel moderated by Mehdi Snene about Open Source AI alongside distinguished speakers Ashley Kramer, Craig Ramlal,  Sasha Luccioni, and Sergio Gago. Please find below a transcript of Stefano’s presentation.

Mehdi Snene  

What is Open Source in AI? What does it mean? What are the foundational pieces? How far along is the data? There is mention of weights, and data skills. How can we truly understand what Open Source in AI is? Today, joining us, we’ll have someone who can help us understand what Open Source in AI means and where we are heading. Stefano, can you offer your insights?

Stefano Maffulli  

Thanks. We have some thoughts on this. We’ve been pondering these questions since they first emerged when GPT started to appear. We asked ourselves: How do we transfer the principles of permissionless innovation and the immense value created by the Open Source ecosystem into the AI space?

After a little over two years of research and global conversations with multiple stakeholders, we identified three key elements. Firstly, permissionless innovation needs to be ported to AI, but this is complex and must be broken down into smaller components.

We realized that, as developers, users, and deployers of AI systems, we need to understand how these systems are built. This involves studying all components carefully, being able to run them for any purpose without asking for permission (a basic tenet of Open Source), and modifying them to change outputs based on the same inputs. These basic principles include being able to share these modifications with others.

To achieve this, you need data, the code used for training and cleaning the data (e.g., removing duplicates), the parameters, the weights, and a way to run inference on those weights. It’s fairly straightforward. However, the challenge lies in the legal framework.

Now, the complicated piece is how Open Source software has had a very wonderful run, based on the fact that the legal framework that governs Open Source is fairly simple and globally accepted. It’s built on copyright, a system that has worked wonderfully in both ways. It gives exclusive rights to the content creators, but also the same mechanism can be used to grant rights to anyone who receives the creation.

With data, we don’t have that mechanism. That is a very simple and dramatic realization. When we talk about data, we should pay attention to what kind of data we’re discussing. There is data as content created, and there is data as facts; like fires, speed limits, or traces of a road. Those are facts, and they have different ways of being treated. There is also private data, personal information, and various other kinds of data, each with different rules and regulations around the world.

Governments’ major role in the future will be to facilitate permissionless innovation in data by harmonizing these rules. This will level the playing field, where currently larger corporations have significantly more power than Open Source developers or those wishing to create large language models. Governments should help create datasets, remove barriers, and facilitate access for academia, smaller developers, and the global south.

Mehdi Snene  

We already have open data and Open Source. Now, we need to create open AI and open models. Are we bringing these two domains together and keeping them separate, or are we creating something new from scratch when we talk about open AI?

Stefano Maffulli 

This is a very interesting and powerful question. I believe that open data as a movement has been around for quite a while. However, it’s only recently that data scientists have truly realized the value they hold in their hands. Data is fungible and can be used to build new things that are completely different from their original domains.

We need to talk more about this and establish platforms for better interaction. One striking example is a popular dataset of images used for training many image generation AI tools, which contained child sexual abuse images for many years. A research paper highlighted this huge problem, but no one filed a bug report, and there was no easy way for the maintainers of this dataset to notice and remove those images.

There are things that the software world understands very well, and things that data scientists understand very well. We are starting to see the need for more space for interactions and learning from each other.

The conversation is extremely complicated. Alex and I have had long discussions about this. I don’t want to focus entirely on this, but I do want to say that Open Source has never been about pleasing companies or specific stakeholders. We need to think of it as an ecosystem where the balances of power are maintained.

While Open Source software and Open Source AI are still evolving, the necessary ingredients—data, code, and other components—are there. However, the data piece still needs to be debated and finalized. Pushing for radical openness with data has clear drawbacks and issues. It’s going to be a balance of intentions, aiming for the best outcome for the general public and the whole ecosystem.

Mehdi Snene  

Thank you so much. My next question is about the future. What are your thoughts on the next big technology?

Stefano Maffulli 

From the perspective of open innovation, it’s about what’s going to give society control over technology. The focus of Open Source has always been to enable developers and end-users to have sovereignty over the technology they use. Whether it’s quantum computers, AI, or future technologies, maintaining that control is crucial.

Governments need to play a role in enabling innovation and ensuring that no single power becomes too dominant. The balance between the private sector, public sector, nonprofit sector, and the often-overlooked fourth sector—which includes developers and creators who work for the public good rather than for profit—must be maintained. This balance is essential for fostering an ecosystem where all stakeholders have equal interests and influence.


If you would like to listen to the panel discussion in its entirety, you can do so here (the Open Source AI panel starts at 1:00:00 approximately).

Categories: FLOSS Research

Better identifying conda packages with ClearlyDefined

Open Source Initiative - Tue, 2024-07-23 19:17

ClearlyDefined, an Open Source project that helps organizations with supply chain compliance,  now provides a new harvester implementation for conda, a popular package manager with a large collection of pre-built packages for various domains, including data science, machine learning, scientific computing and more.

Conda provides package, dependency and environment management for any language and is very popular with Python and R. It allows users to manage and control the dependencies and versions of packages specific to each project, ensuring reproducibility and avoiding conflicts between different software requirements.

ClearlyDefined crawls both the main conda package and the source code for licensing metadata. The main conda package is hosted on the conda channels themselves and contains all necessary licensing information, compilers, environment configuration scripts and dependencies that are needed to make the package work. The source code from which the conda package is created oftentimes is hosted in an external website such as GitHub.

The conda crawler uses the following coordinates:

  • type (required): conda or condasource
  • provider (required): channel on which the package will be crawled, such as conda-forge, anaconda-main or anaconda-r
  • namespace (optional): architecture and OS of the package to be crawled, i.e. win64, linux-aarch64 or any if no architecture is specified.
  • package name (required): name of the package
  • revision (optional): package version and optional build version

For example, the popular numpy package is represented as shown below.

With the increased importance of data science, machine learning and scientific computing, this support for conda packages in ClearlyDefined is extremely important. It will allow organizations to better manage the licenses of their conda packages for compliance. This work was led by Basit Ayantunde from CodeThink with the stewardship from Qing Tomlison from SAP. We would like to thank them and all those involved in the development and testing of this implementation.

Categories: FLOSS Research

Cailean Osborne: voices of the Open Source AI Definition

Open Source Initiative - Thu, 2024-07-18 13:09

The Open Source Initiative (OSI) is running a series of stories about a few of the people involved in the Open Source AI Definition (OSAID) co-design process. Today, we are featuring Cailean Osborne, one of the volunteers who has helped to shape and are shaping the OSAID.

Question: What’s your background related to Open Source and AI?

My interest in Open Source AI began around 2020 when I was working in AI policy at the UK Government. I was surprised that Open Source never came up in policy discussions, given its crucial role in AI R&D. Having been a regular user of libraries like scikit-learn and PyTorch in my previous studies. I followed Open Source AI trends in my own time and eventually I decided to do a PhD on the topic. When I started my PhD back in 2021, Open Source AI still felt like a niche topic, so it’s been exciting to watch it become a major talking point over the years. 

Beyond my PhD, I’ve been involved in Open Source AI community as a contributor to scikit-learn and as a co-developer of the Model Openness Framework (MOF) with peers from the Generative AI Commons community. Our goal with the MOF is to provide guidance for AI researchers and developers to evaluate the completeness and openness of “Open Source” models based on open science principles. We were chuffed that the OSI team chose to use the 16 components from the MOF as the rubric for reviewing models in the co-design process. 

Question: What motivated you to join this co-design process to define Open Source AI?

The short answer is: to contribute to establishing an accurate definition for “Open Source AI” and to learn from all the other experts involved in the co-design process. The longer answer is: There’s been a lot of confusion about what is or is not “Open Source AI,” which hasn’t been helped by open-washing. “Open source” has a specific definition (i.e. the right to use, study, modify, and redistribute source code) and what is being promoted as “Open Source AI” deviates significantly from this definition. Rather than being pedantic, getting the definition right matters for several reasons; for example, for the “Open Source” exemptions in the EU AI Act to work (or not work), we need to know precisely what “Open Source” models actually are. Andreas Liesenfeld and Mark Dingemanse have written a great piece about the issues of open-washing and how they relate to the AI Act, which I recommend reading if you haven’t yet. So, I got involved to help develop a definition and to learn from all the other experts involved. It hasn’t been easy (it’s a pretty divisive topic!), but I think we’ve made good progress.

Question: Can you describe your experience participating in this process? What did you most enjoy about it and what were some of the challenges you faced?

First off, I have to give credit to Stef and Mer for maintaining momentum throughout the process. Coordinating a co-design effort with volunteers scattered around the globe, each with varying levels of availability and (strong) opinions on the matter, is no small feat. So, well done! I also enjoyed seeing how others agreed or disagreed when reviewing models. The moments of disagreement were the most interesting; for example, about whether training data should be available versus documented and if so, in how much detail… Personally, the main challenge was searching for information about the various components of models that were apparently “Open Source” and observing how little information was actually provided beyond weights, a model card, and if you’re lucky an arXiv preprint or technical report.

Question: Why do you think AI should be Open Source?

When talking about the benefits of Open Source AI, I like to point folks to a 2007 paper, in which 16 researchers highlighted “The Need for Open Source Software in Machine Learning” due to basically the complete lack of OSS for ML/AI at the time. Fast forward to today, AI R&D is practically unthinkable without OSS, from data tooling to the deep learning frameworks used to build LLMs. Open source and openness in general have many benefits for AI, from enabling access to SOTA AI technologies and transparency which is key for reproducibility, scrutiny, and accountability to widening participation in their design, development, and governance. 

Question: What do you think is the role of data in Open Source AI?

If the question is strictly about the role of data in developing open AI models, the answer is pretty simple: Data plays a crucial role because it is needed for training, testing, aligning, and auditing models. But if the question is asking “should the release of data be a condition for an open model to qualify as Open Source AI,” then the answer is obviously much more complicated. 

Companies are in no rush to share training data due to a handful of reasons: be it competitive advantage, data protection, or frankly being sued for copyright infringement. The copyright concern isn’t limited to companies: EleutherAI has also been sued and had to take down the Books3 dataset from The Pile. There are also many social and cultural concerns that restrict data sharing; for example, the Kōrero Kaitiakitanga license has been developed to protect the interests of indigenous communities in New Zealand. So, the data question isn’t easy and perhaps we shouldn’t be too dogmatic about it.  

Personally, I think the compromise in v. 0.0.8, which states that model developers should provide sufficiently detailed information about data if they can’t release the training dataset itself, is a reasonable halfway house. I also hope to see more open pre-training datasets like the one developed by the community-driven BigScience Project, which involved open deliberation about the design of the dataset and provides extensive documentation about data provenance and processing decisions (e.g. check out their Data Catalogue). The FineWeb dataset by Hugging Face is another good example of an open pre-training dataset, which they released with pre-processing code, evaluation results, and super detailed documentation.

Question: Has your personal definition of Open Source AI changed along the way? What new perspectives or ideas did you encounter while participating in the co-design process?

To be honest, my personal definition hasn’t changed much. I am not a big fan of the use of “Open Source AI” when folks specifically mean “open models” or “open-weight models”. What we need to do is raise awareness about appropriate terminology and point out “open-washing”, as people have done, and I must say that subjectively I’ve seen improvements: less “Open Source models” and more “open models”. But I will say that I do find “Open Source AI” a useful umbrella term for the various communities of practice that intertwine in the development of open models, including OSS, open data, and AI researchers and developers, who all bring different perspectives and ways of working to the overarching “Open Source AI” community.

Question: What do you think the primary benefit will be once there is a clear definition of Open Source AI?

We’ll be able to reduce confusion about what is or isn’t “Open Source AI” and more easily combat open-washing efforts. As I mentioned before, this clarity will be beneficial for compliance with regulations like the AI Act which includes exemptions for “Open Source” AI.  

Question: What do you think are the next steps for the community involved in Open Source AI?

We still have many steps to take but I’ll share three for now.

First, we urgently need to improve the auditability and therefore the safety of open models. With OSS, we know that (1) the availability of source code and (2) open development enable the distributed scrutiny of source code. Think Linus’ Law: “Given enough eyeballs, all bugs are shallow.” Yet open models are more complex than just source code, and the lack of openness of many key components like training data is holding back adoption because would-be adopters can’t adequately run due diligence tests on the models. If we want to realise the benefits of “Open Source AI,” we need to figure out how to increase the transparency and openness of models —we hope the Model Openness Framework can help with this. 

Second, I’m really excited about grassroots initiatives that are leading community-driven approaches to developing open models and open datasets like the BigScience project. They’re setting an example of how to do “Open Source AI” in a way that promotes open collaboration, transparency, reproducibility, and safety from the ground up. I can still count such initiatives with my fingers but I am hopeful that we will see more community-driven efforts in the future.
Third, I hope to see the public sector and non-profit foundations get more involved in supporting public interest and grassroots initiatives. France has been a role model on this front: providing a public grant to train the BigScience project’s BLOOM model on the Jean Zay supercomputer, as well as funding the scikit-learn team to build out a data science commons.

Categories: FLOSS Research

The Open Source Initiative joins CMU in launching Open Forum for AI: A human-centered approach to AI development

Open Source Initiative - Mon, 2024-07-15 16:32

The Open Source Initiative (OSI) is pleased to share that we are joining the founding team of Open Forum for AI (OFAI), an initiative designed by Carnegie Mellon University (CMU) to foster a human-centered approach to artificial intelligence. OFAI aims to enhance our understanding of AI and its potential to augment human capabilities while promoting responsible development practices.

The missions of OSI and OFAI are well-aligned; at the heart of OFAI is a commitment to ensuring that AI development serves the public interest. With the support of renowned partners like Omidyar Network, NobleReach Foundation, and internal CMU funding, OFAI is positioned to serve as a pivotal platform for shaping AI strategies and policies that prioritize safety, privacy, and equity.

The OSI is proud to be part of this project. Stefano Mafulli and Deb Bryant from the OSI will participate in OFAI, integrating their efforts toward a standard Open Source AI Definition through a collaborative process involving stakeholders from the Open Source community, industry, and academia as well as their contributions to public policy.  

A collective effort

The success of OFAI hinges on the diverse expertise it convenes. Leading this initiative is Sayeed Choudhury, Associate Dean for Digital Infrastructure at CMU and a member of the OSI Board. Alongside him, a team of CMU faculty members and external advisors will contribute knowledge in ethics, computational technologies, and inclusive AI research. 

Notable participants like Michele Jawando from Omidyar Network and Arun Gupta from NobleReach Foundation have emphasized the importance of Open Source AI in driving innovation and inclusivity as well as the need for a human-centered, trust-based approach to AI development.

OFAI’s ambitious goals

OFAI aims to influence AI policy by coordinating research and policy objectives and advocating for transparent and inclusive AI development. The initiative will focus on five key areas: 

  • Research
  • Technical prototypes
  • Policy recommendations
  • Community engagement
  • Talent for service

Deb Bryant will lead Community Engagement, building in part upon the broad community of interest gathered through the public process of OSI’s Defining Open Source AI.

One of OFAI’s foundational projects is the creation of an “Openness in AI” framework, which seeks to make AI development more transparent and inclusive. This framework will serve as a vital resource for policymakers, researchers, and the broader community.

Looking ahead

With the OSI set to deliver a stable version of the Open Source AI Definition at All Things Open in October, the launch of OFAI magnifies the importance of this work to bring together diverse stakeholders to ensure AI technologies align with societal values and public interests.

Categories: FLOSS Research

Open Source AI Definition – Weekly update July 15

Open Source Initiative - Mon, 2024-07-15 15:26

It has been quiet over the 4th of July weekend on the forums and OSI has been speaking at different events:

Why and how to certify Open Source AI
  • @jberkus expresses concern about the extensive resources required to certify AI systems, estimating that it would take weeks of work per system. This scale makes it impractical for a volunteer committee like License Review.
  • @shujisado reflects on past controversies over license conformity, noting that Open Source AI has the potential for a greater economic impact than early Open Source” He acknowledges the need for a more robust certification process given this increased significance. He suggests that cooperation from the machine learning community or consortia might be necessary to address technical issues and monitor the certification process neutrally. He offers to help spread the word about OSAID within the Japanese ML/LLM development community.

@jberkus clarifies that the OSI would need full-time paid staff to handle the certifications, as the work cannot be managed by volunteers alone.

Categories: FLOSS Research

Mer Joyce: voices of the Open Source AI Definition

Open Source Initiative - Wed, 2024-07-10 09:36

The Open Source Initiative (OSI) is running a series of stories about a few of the people involved in the Open Source AI Definition (OSAID) co-design process. We’ll be featuring the voices of the volunteers who have helped shape and are shaping the Definition.

The OSI started researching the topic in 2022, and in 2023 began the co-design process of a new definition of Open Source that applies to AI. The OSI hired Mer Joyce, founder and principal of Do Big Good, as an independent consultant to lead the co-design process. She has worked for over a decade at the intersection of research, policy, innovation and social change.

Mer Joyce, process facilitator for the Open Source AI Definition About co-design

Co-design, also called participatory or human-centered design, is a set of creative methods used to solve communal problems by sharing knowledge and power. The co-design methodology addresses the challenges of reaching an agreed definition within a diverse community (Costanza-Chock, 2020: Escobar, 2018: Creative Reaction Lab, 2018: Friedman et al., 2019). 

As noted in MIT Technology Review’s article about the OSAID, “[t]he open-source community is a big tent… encompassing everything from hacktivists to Fortune 500 companies…. With so many competing interests to consider, finding a solution that satisfies everyone while ensuring that the biggest companies play along is no easy task.” (Gent, 2024). 

The co-design method allows for the integration of diverging perspectives into one just, cohesive and feasible standard. Support from such a significant and broad group of people also creates a tension to be managed between moving swiftly enough to deliver outputs that can be used operationally and taking the time to consult widely to understand the big issues and garner community buy-in. Having Mer as facilitator of the OSAID co-design, with her in-depth experience, has been important in ensuring the integrity of the process. 

The OSAID co-design process

The first step of the OSAID co-design process was to identify the freedoms needed for Open Source AI. After various online and in-person activities and discussions, including five workshops across the world, the community adopted the four freedoms for software, now adapted for AI systems:

  • Freedom to Use the system for any purpose and without having to ask for permission.
  • Freedom to Study how the system works and inspect its components.
  • Freedom to Modify the system for any purpose, including to change its output.
  • Freedom to Share the system for others to use with or without modifications, for any purpose.

The next step was the formation of four working groups to initially analyze four different AI systems and their components. To achieve better representation, special attention was given to diversity, equity and inclusion. Over 50% of the working group participants are people of color, 30% are black, 75% were born outside the US, and 25% are women, trans or nonbinary.

These working groups discussed and voted on which AI system components should be required to satisfy the four freedoms for AI. The components adopted are described in the Model Openness Framework developed by the Linux Foundation.

The vote compilation was performed based on the mean total votes per component (μ). Components that received over 2μ votes were marked as “required,” and between 1.5μ and 2μ were marked “likely required.” Components that received between 0.5μ and μ were marked as “likely not required,” and less than 0.5μ were marked “not required.”

After the working groups evaluated legal frameworks and legal documents for each component, each working group published a recommendation report. The end result is the OSAID with a comprehensive definition checklist encompassing a total of 17 components. More working groups are being formed to evaluate how well other AI systems align with the Definition.

OSAID multi-stakeholder co-design process: from component list to a definition checklist Meet Mer Joyce Video recorded by Ezequiel Lanza, Open Source AI Evangelist at Intel

I am the process facilitator for the Open Source AI Definition, the Open Source Initiative project creating a definition of Open Source AI that will be a part of the stable public infrastructure of Open Source technology that everyone can benefit from, similar to the Open Source Definition that OSI currently stewards. The co-design of the Open Source AI Definition involves consulting with global stakeholders to ensure their vast range of needs are represented while integrating and weaving together the variety of different perspectives on what Open Source AI should mean.

If you would like to participate in the process, we’re currently on version 0.0.7. We will have a release candidate in June and a stable version in October. There is a public forum at discuss.opensource.org where anyone can create an account and make comments. As different versions are created, updates about our process are released here as well. I am available, as is the executive director of the OSI, to answer questions at bi-weekly town halls that are open for anyone to attend.

How to get involved

The OSAID co-design process is open to everyone interested in collaborating. There are many ways to get involved:

  • Join the working groups: be part of a team to evaluate various models against the OSAID.
  • Join the forum: support and comment on the drafts, record your approval or concerns to new and existing threads.
  • Comment on the latest draft: provide feedback on the latest draft document directly.
  • Follow the weekly recaps: subscribe to our newsletter and blog to be kept up-to-date.
  • Join the town hall meetings: participate in the online public town hall meetings to learn more and ask questions.
  • Join the workshops and scheduled conferences: meet the OSI and other participants at in-person events around the world.
One of the many OSAID workshops organized by Mer Joyce around the world
Categories: FLOSS Research

Pages