Earnie
ARTICLEENGINEERING

Standardising crypto: an open data journey

How a customer request for crypto detection became an open dataset, and then the starting point for an SPDX list.

Julián Coccia
CEO
7 min read
An orange line running from a CC0 file to the SPDX list, with contributions joining along it

TL;DR: In 2021 there was no common way to identify, describe or categorise cryptographic algorithms in code. What began as a customer request became an open CC0 dataset in 2024, grew to around 120 algorithms with customer contributions, and in 2025 became the starting point for the SPDX Cryptographic Algorithm List. The datasets have since moved to the Software Transparency Foundation for neutral stewardship. This is how the team behind Earnie's knowledge base got there, and what it learned.

This article was first published by SCANOSS, the company behind the knowledge base Earnie is built on.

Open collaboration can transform isolated challenges into industry standards. That's precisely what happened when we applied our "open source virtuous circle" approach to cryptographic algorithms. What began as a customer request has evolved into a community-led effort to define, classify and detect algorithms consistently, benefiting developers, regulators, and security teams across the software ecosystem.

For a condensed overview, watch Matias D'aloia's OSS EU 2025 talk, Know Your Crypto: Standardising and Detecting Crypto Algorithms the Open Source Way.

Our open data journey prior to crypto_algorithms_open_dataset

We provide a subscription to our knowledge base alongside fully open source tooling and open APIs: our GitHub tools and public API.

Many customers integrate our knowledge base across the SDLC for compliance, security, and export control use cases. As a data company, we treat openly shared datasets with care: releasing truly open datasets (e.g., CC0) creates clear benefits but must be balanced with intellectual-property stewardship.

In 2021, the Software Transparency Foundation (STF), in collaboration with our team as its Strategic Member, launched osskb.org. The service provides free, privacy-friendly access to a knowledge base and the same file and snippet detection mechanisms used in our products, enriched with licence metadata. It is integrated with tools such as FOSSology, ORT, FOSSLight, Theia IDE, and our open source tools. osskb.org enables developers and compliance experts to create accurate, standardised SBOMs faster, improving transparency across the software supply chain. It is intended for academic use and sole contributors only, and is not suitable for commercial use due to usage limitations.

The following year, we released purl2cpe, our first CC0-licensed open dataset mapping Package URLs to Common Platform Enumerations. This dataset supports large-scale vulnerability management and is now widely used by several tools.

However, contribution volume was lower than expected, a useful signal that we needed a clearer engagement model for contributors. Together with osskb.org, purl2cpe validated the business case for selective open datasets and set the stage for our next focus area: cryptographic algorithms.

From customer demands to open source

In 2021, several customers were asking for a new capability: to detect cryptographic algorithms within their codebases and dependencies, the basis of what is now called a CBOM. At the time, there was no common way across the industry to identify, describe, or categorise algorithms, let alone detect them efficiently at scale.

When designing a solution, we quickly realised that this wasn't a problem any single vendor could solve sustainably. True progress required community alignment. Following the open source playbook, we saw that the best path forward was open data collaboration. The characterisation and declaration of cryptographic algorithms needed to come from shared expertise rather than a single company's catalogue. No vendor could possibly maintain complete coverage of all algorithms, metadata, and use cases alone.

Between 2022 and 2023, we released cryptographic algorithm detection as part of our product offering (Encryption Dataset). The response was highly positive, validating both market demand and our technical direction. Once the commercial value was proven, we were ready to open up the commodity layer of that feature, its underlying dataset.

In early 2024, we released crypto_algorithms_open_dataset, under a CC0 licence, hosted publicly on GitHub. The dataset included:

  • A starting list of around 60 cryptographic algorithms
  • A simple taxonomy inspired by SPDX to standardise identification
  • Keyword associations for each algorithm to support open source tooling developers

For us, this was more than just another dataset, it was an invitation to collaborate. Customers, experts, and community contributors could now participate in shaping a shared foundation for cryptographic visibility.

Together with the STF, we promoted the project through industry events such as FOSSNorth (2024 and 2025), OCX'24, OSPO Summit 2024, OSS Japan 2024, OCS 2024, FOSDEM 2025, and OSS EU 2025. Beyond conferences, we and the STF also engaged actively with online communities such as OpenChain, contributing to discussions and sharing progress updates to encourage broader participation. These combined efforts helped expand awareness and build early momentum around cryptographic algorithm transparency.

By late 2024, contributions to crypto_algorithms_open_dataset were coming directly from customers, experts in security, auditing, and export control whose input strengthened both the dataset and our own detection capabilities. Our outreach at events and within online communities also led to new research and commercial collaborations, including work on areas like quantum readiness.

By early 2025, the dataset had grown to roughly 120 algorithms, thanks mainly to customer contributions. The taxonomy became more structured, the documentation more comprehensive, and the keyword mappings more accurate, reflecting real-world feedback from our users.

Still, direct community contributions, particularly around taxonomy and algorithm declarations, fell short of expectations, highlighting the need for broader participation. So we took the next step in our open data journey: transferring the crypto_algorithms_open_dataset to a vendor-neutral environment to encourage broader collaboration. After evaluating several options, we agreed with the STF that it would become the long-term steward of this and our other open datasets. To enable this transition, STF began designing the necessary collaboration framework, contribution processes, and tooling to host and grow open data projects under open source principles. As a strategic member, we committed additional technical and organisational support to help establish this foundation for sustainable community participation.

From open dataset to industry standard

At FOSS North 2024, Alexios Zavras, lead of the SPDX Outreach Working Group, approached us with a proposal: to use the crypto_algorithms_open_dataset as a starting point for a new SPDX Cryptographic Algorithms List, similar to the well-known SPDX Licence List.

We immediately recognised the value of this initiative.

  • SPDX is an ISO standard SBOM format already supported by our open source tooling.
  • It aligned our work with community efforts to standardise cryptographic algorithms.
  • Standardising algorithm definitions and taxonomy would improve interoperability across the SCA industry and strengthen our own product for customers.
  • It would also allow us to focus engineering effort on differentiation, specifically on advancing cryptographic algorithm detection.

This collaboration expanded as we and the STF agreed to contribute to the emerging SPDX Cryptographic Algorithm List, laying the foundation for an industry-wide standard.

Work on the list began in mid 2025. The initial version included 122 algorithms and their identifiers, derived directly from the crypto_algorithms_open_dataset (CC0). The first draft of the SPDX cryptographic algorithms list is now available as a foundation for further contributions. We began adopting the list as our upstream reference within crypto_algorithms_open_dataset, narrowing the focus of this open dataset to the characterisation of those algorithms included in the list through keywords. This integration ensures that community contributions flow seamlessly into our detection, turning open collaboration into continuous improvement.

For additional context on the SPDX Cryptographic Algorithms List, see Agustín Benito Bethencourt's blog post. As an active contributor to the list, his perspective offers valuable insight into its goals and early progress.

In 2025, STF began preparing to host our open datasets, providing a neutral and transparent environment for their long-term maintenance. The datasets have since been donated to the STF and are free for anyone to use.

Looking ahead, we will continue to contribute to the SPDX Cryptographic Algorithm List, directly (SPDX Cryptography Group) and by supporting STF contributions; support crypto_algorithms_open_dataset with additional keywords, in collaboration with other players; collaborate with CycloneDX to ensure that the list is compatible with both standards; and bring all these innovations to customers through Earnie.

The journey to standardise cryptographic algorithms has shown how open collaboration can drive both innovation and resilience. From this journey, several lessons stand out:

  1. Open collaboration reveals demand. Working in the open surfaces real use cases early and opens new markets.
  2. Standards reduce cost and risk. Shared taxonomies and identifiers speed integration, limit lock-in, and support compliance.
  3. Focus engineering where it adds value. By sharing foundational datasets like the SPDX Cryptographic List, we can direct effort toward higher-value innovation in detection and workflow accuracy.
  4. The virtuous circle compounds. Establish it, consolidate it, then scale it; be disciplined about where investment creates ROI.
  5. Neutral governance unlocks scale. Trusted hosts (e.g., SPDX, STF, OpenChain) increase credible, sustained participation.
  6. Crypto visibility is a must-have. Demand for cryptographic detection is rising across sectors; agility beats size.
  7. Sustainability needs reciprocity. The benefits to contributors must flow back, measure and reinvest, not just contribute.
  8. Cross-industry collaboration multiplies impact. Strategic open datasets create ecosystem value and tangible business outcomes.

About Software Transparency Foundation. STF promotes transparency, security, and compliance in software development across supply chains. STF provides osskb.org, the back-end service where some of the most popular open source SCA tools integrate to enhance their open source software detection capabilities at both file and snippet level, for licence compliance purposes. Learn more at softwaretransparency.org.

About SPDX. SPDX (Software Package Data Exchange) https://spdx.dev is an open standard used worldwide to communicate software bill of materials (SBOM) information, including components, licences, security references, and now cryptographic algorithms. Maintained by the Linux Foundation, SPDX enables transparency, interoperability, and trust across the software supply chain.

RELATED READING

See it on your kind of codebase.

Request a demo
Standardising crypto: an open data journey
Earnie
ARTICLEENGINEERING

Standardising crypto: an open data journey

How a customer request for crypto detection became an open dataset, and then the starting point for an SPDX list.

Julián Coccia
CEO
7 min read
An orange line running from a CC0 file to the SPDX list, with contributions joining along it

TL;DR: In 2021 there was no common way to identify, describe or categorise cryptographic algorithms in code. What began as a customer request became an open CC0 dataset in 2024, grew to around 120 algorithms with customer contributions, and in 2025 became the starting point for the SPDX Cryptographic Algorithm List. The datasets have since moved to the Software Transparency Foundation for neutral stewardship. This is how the team behind Earnie's knowledge base got there, and what it learned.

This article was first published by SCANOSS, the company behind the knowledge base Earnie is built on.

Open collaboration can transform isolated challenges into industry standards. That's precisely what happened when we applied our "open source virtuous circle" approach to cryptographic algorithms. What began as a customer request has evolved into a community-led effort to define, classify and detect algorithms consistently, benefiting developers, regulators, and security teams across the software ecosystem.

For a condensed overview, watch Matias D'aloia's OSS EU 2025 talk, Know Your Crypto: Standardising and Detecting Crypto Algorithms the Open Source Way.

Our open data journey prior to crypto_algorithms_open_dataset

We provide a subscription to our knowledge base alongside fully open source tooling and open APIs: our GitHub tools and public API.

Many customers integrate our knowledge base across the SDLC for compliance, security, and export control use cases. As a data company, we treat openly shared datasets with care: releasing truly open datasets (e.g., CC0) creates clear benefits but must be balanced with intellectual-property stewardship.

In 2021, the Software Transparency Foundation (STF), in collaboration with our team as its Strategic Member, launched osskb.org. The service provides free, privacy-friendly access to a knowledge base and the same file and snippet detection mechanisms used in our products, enriched with licence metadata. It is integrated with tools such as FOSSology, ORT, FOSSLight, Theia IDE, and our open source tools. osskb.org enables developers and compliance experts to create accurate, standardised SBOMs faster, improving transparency across the software supply chain. It is intended for academic use and sole contributors only, and is not suitable for commercial use due to usage limitations.

The following year, we released purl2cpe, our first CC0-licensed open dataset mapping Package URLs to Common Platform Enumerations. This dataset supports large-scale vulnerability management and is now widely used by several tools.

However, contribution volume was lower than expected, a useful signal that we needed a clearer engagement model for contributors. Together with osskb.org, purl2cpe validated the business case for selective open datasets and set the stage for our next focus area: cryptographic algorithms.

From customer demands to open source

In 2021, several customers were asking for a new capability: to detect cryptographic algorithms within their codebases and dependencies, the basis of what is now called a CBOM. At the time, there was no common way across the industry to identify, describe, or categorise algorithms, let alone detect them efficiently at scale.

When designing a solution, we quickly realised that this wasn't a problem any single vendor could solve sustainably. True progress required community alignment. Following the open source playbook, we saw that the best path forward was open data collaboration. The characterisation and declaration of cryptographic algorithms needed to come from shared expertise rather than a single company's catalogue. No vendor could possibly maintain complete coverage of all algorithms, metadata, and use cases alone.

Between 2022 and 2023, we released cryptographic algorithm detection as part of our product offering (Encryption Dataset). The response was highly positive, validating both market demand and our technical direction. Once the commercial value was proven, we were ready to open up the commodity layer of that feature, its underlying dataset.

In early 2024, we released crypto_algorithms_open_dataset, under a CC0 licence, hosted publicly on GitHub. The dataset included:

  • A starting list of around 60 cryptographic algorithms
  • A simple taxonomy inspired by SPDX to standardise identification
  • Keyword associations for each algorithm to support open source tooling developers

For us, this was more than just another dataset, it was an invitation to collaborate. Customers, experts, and community contributors could now participate in shaping a shared foundation for cryptographic visibility.

Together with the STF, we promoted the project through industry events such as FOSSNorth (2024 and 2025), OCX'24, OSPO Summit 2024, OSS Japan 2024, OCS 2024, FOSDEM 2025, and OSS EU 2025. Beyond conferences, we and the STF also engaged actively with online communities such as OpenChain, contributing to discussions and sharing progress updates to encourage broader participation. These combined efforts helped expand awareness and build early momentum around cryptographic algorithm transparency.

By late 2024, contributions to crypto_algorithms_open_dataset were coming directly from customers, experts in security, auditing, and export control whose input strengthened both the dataset and our own detection capabilities. Our outreach at events and within online communities also led to new research and commercial collaborations, including work on areas like quantum readiness.

By early 2025, the dataset had grown to roughly 120 algorithms, thanks mainly to customer contributions. The taxonomy became more structured, the documentation more comprehensive, and the keyword mappings more accurate, reflecting real-world feedback from our users.

Still, direct community contributions, particularly around taxonomy and algorithm declarations, fell short of expectations, highlighting the need for broader participation. So we took the next step in our open data journey: transferring the crypto_algorithms_open_dataset to a vendor-neutral environment to encourage broader collaboration. After evaluating several options, we agreed with the STF that it would become the long-term steward of this and our other open datasets. To enable this transition, STF began designing the necessary collaboration framework, contribution processes, and tooling to host and grow open data projects under open source principles. As a strategic member, we committed additional technical and organisational support to help establish this foundation for sustainable community participation.

From open dataset to industry standard

At FOSS North 2024, Alexios Zavras, lead of the SPDX Outreach Working Group, approached us with a proposal: to use the crypto_algorithms_open_dataset as a starting point for a new SPDX Cryptographic Algorithms List, similar to the well-known SPDX Licence List.

We immediately recognised the value of this initiative.

  • SPDX is an ISO standard SBOM format already supported by our open source tooling.
  • It aligned our work with community efforts to standardise cryptographic algorithms.
  • Standardising algorithm definitions and taxonomy would improve interoperability across the SCA industry and strengthen our own product for customers.
  • It would also allow us to focus engineering effort on differentiation, specifically on advancing cryptographic algorithm detection.

This collaboration expanded as we and the STF agreed to contribute to the emerging SPDX Cryptographic Algorithm List, laying the foundation for an industry-wide standard.

Work on the list began in mid 2025. The initial version included 122 algorithms and their identifiers, derived directly from the crypto_algorithms_open_dataset (CC0). The first draft of the SPDX cryptographic algorithms list is now available as a foundation for further contributions. We began adopting the list as our upstream reference within crypto_algorithms_open_dataset, narrowing the focus of this open dataset to the characterisation of those algorithms included in the list through keywords. This integration ensures that community contributions flow seamlessly into our detection, turning open collaboration into continuous improvement.

For additional context on the SPDX Cryptographic Algorithms List, see Agustín Benito Bethencourt's blog post. As an active contributor to the list, his perspective offers valuable insight into its goals and early progress.

In 2025, STF began preparing to host our open datasets, providing a neutral and transparent environment for their long-term maintenance. The datasets have since been donated to the STF and are free for anyone to use.

Looking ahead, we will continue to contribute to the SPDX Cryptographic Algorithm List, directly (SPDX Cryptography Group) and by supporting STF contributions; support crypto_algorithms_open_dataset with additional keywords, in collaboration with other players; collaborate with CycloneDX to ensure that the list is compatible with both standards; and bring all these innovations to customers through Earnie.

The journey to standardise cryptographic algorithms has shown how open collaboration can drive both innovation and resilience. From this journey, several lessons stand out:

  1. Open collaboration reveals demand. Working in the open surfaces real use cases early and opens new markets.
  2. Standards reduce cost and risk. Shared taxonomies and identifiers speed integration, limit lock-in, and support compliance.
  3. Focus engineering where it adds value. By sharing foundational datasets like the SPDX Cryptographic List, we can direct effort toward higher-value innovation in detection and workflow accuracy.
  4. The virtuous circle compounds. Establish it, consolidate it, then scale it; be disciplined about where investment creates ROI.
  5. Neutral governance unlocks scale. Trusted hosts (e.g., SPDX, STF, OpenChain) increase credible, sustained participation.
  6. Crypto visibility is a must-have. Demand for cryptographic detection is rising across sectors; agility beats size.
  7. Sustainability needs reciprocity. The benefits to contributors must flow back, measure and reinvest, not just contribute.
  8. Cross-industry collaboration multiplies impact. Strategic open datasets create ecosystem value and tangible business outcomes.

About Software Transparency Foundation. STF promotes transparency, security, and compliance in software development across supply chains. STF provides osskb.org, the back-end service where some of the most popular open source SCA tools integrate to enhance their open source software detection capabilities at both file and snippet level, for licence compliance purposes. Learn more at softwaretransparency.org.

About SPDX. SPDX (Software Package Data Exchange) https://spdx.dev is an open standard used worldwide to communicate software bill of materials (SBOM) information, including components, licences, security references, and now cryptographic algorithms. Maintained by the Linux Foundation, SPDX enables transparency, interoperability, and trust across the software supply chain.

RELATED READING
ARTICLE · PQC / CRYPTO READINESS
What is a CBOM (Cryptography Bill of Materials)?
ARTICLE · ENGINEERING
The critical role of version specificity in SBOMs

See it on your kind of codebase.

Request a demo