Paper deep dive

Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)

Dick A. H. Blankvoort, Harshvardhan J. Pandit, Maximilian Gahntz

Year: 2026Venue: arXiv preprintArea: cs.CYType: PreprintEmbeddings: 100

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 8:35:28 AM

Summary

This paper introduces a quality assessment framework for the public summaries of training content required by Article 53(1)(d) of the EU AI Act for General-Purpose AI (GPAI) models. The framework evaluates summaries based on two primary dimensions: Transparency (Clarity, Completeness, Consistency, Correctness) and Usefulness (Accessibility, Comprehension). The authors applied this framework to assess five public summaries published as of January 2026, aiming to provide actionable recommendations for providers and guidance for the AI Office to ensure high-quality documentation that facilitates rights enforcement and accountability.

Entities (16)

EU AI Act · legislation · 98%Article 53(1)(d) · legalclause · 97%AI Office · organization · 96%GPAI · technology · 95%Public Summary of Training Content · document · 94%Quality Assessment Framework · methodology · 93%Transparency · concept · 92%Usefulness · concept · 92%Accessibility · metric · 90%Clarity · metric · 90%Completeness · metric · 90%Comprehension · metric · 90%Consistency · metric · 90%Correctness · metric · 90%Mozilla.org · organization · 85%Trinity College Dublin · organization · 85%

Relation Signals (15)

Article 53(1)(d) → partof → EU AI Act

confidence 98% · The AI Act’s Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary

EU AI Act → requires → Public Summary of Training Content

confidence 98% · The AI Act’s Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary about the content used for training

AI Office → provides → Public Summary of Training Content

confidence 96% · The European Artificial Intelligence Office (AI Office)... is responsible for providing a common template to allow implementation of the obligation.

Quality Assessment Framework → assesses → Public Summary of Training Content

confidence 95% · This paper provides a quality assessment framework to assess the public summary across two key dimensions

Quality Assessment Framework → evaluates → Transparency

confidence 94% · This paper provides a quality assessment framework to assess the public summary across two key dimensions: transparency

Quality Assessment Framework → evaluates → Usefulness

confidence 94% · and usefulness regarding whether the provision of the document and the contents can be effectively utilised by stakeholders

Transparency → includesmetric → Completeness

confidence 92% · we selected 6 metrics: Clarity, Completeness, Consistency, and Correctness for representing Transparency

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The AI Act's Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary about the content used for training based on a template provided by the AI Office. The stated goal of this obligation is to increase transparency regarding the data used for training GPAI models, and to enable relevant stakeholders to exercise their rights, especially regarding IP, copyright, and data protection. This paper provides a quality assessment framework to assess the public summary across two key dimensions: \textit{transparency} regarding information being provided in a clear, comprehensive, and sufficiently detailed manner; and \textit{usefulness} regarding whether the provision of the document and the contents can be effectively utilised by stakeholders to carry out rights related actions. This framework enables identification of key issues in public summaries, and provides a structured and research-based method to compare practices across public summaries and providers. It also enables authorities such as the AI Office to identify potential issues that could emerge and provides actionable recommendations and guidelines for providers to develop public summaries with high quality. The paper provides an assessment of 5 public summaries published as of 12th January 2026 which were found through an exhaustive search process. To disseminate these findings as a public resource, the paper also describes the development of a website where the assessments, outcomes, and methodologies will be shared.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

98,332 characters extracted from source content.

Expand or collapse full text

Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d) DICK A. H. BLANKVOORT ∗ , AI Accountability Lab (AIAL), ADAPT Centre, Trinity College Dublin, Ireland HARSHVARDHAN J. PANDIT ∗ , AI Accountability Lab (AIAL), ADAPT Centre, Trinity College Dublin, Ireland MAXIMILIAN GAHNTZ, Mozilla.org, USA The AI Act’s Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary about the content used for training based on a template provided by the AI Office. The stated goal of this obligation is to increase transparency regarding the data used for training GPAI models, and to enable relevant stakeholders to exercise their rights, especially regarding IP, copyright, and data protection. This paper provides a quality assessment framework to assess the public summary across two key dimensions: transparency regarding information being provided in a clear, comprehensive, and sufficiently detailed manner; and usefulness regarding whether the provision of the document and the contents can be effectively utilised by stakeholders to carry out rights related actions. This framework enables identification of key issues in public summaries, and provides a structured and research-based method to compare practices across public summaries and providers. It also enables authorities such as the AI Office to identify potential issues that could emerge and provides actionable recommendations and guidelines for providers to develop public summaries with high quality. The paper provides an assessment of 5 public summaries published as of 12th January 2026 which were found through an exhaustive search process. To disseminate these findings as a public resource, the paper also describes the development of a website where the assessments, outcomes, and methodologies will be shared. Additional Key Words and Phrases: Generative AI, Model Documentation, EU Law, Consumer Rights Enforcement, Quality Assessment 1 Introduction The EU’s Artificial Intelligence Act (AI Act) [3] is the world’s first regulation that specifically targets AI technologies. Drafted over a period of 3 years, the regulation saw drastic changes due to rapid advancements made in the field of General Purpose AI (GPAI) – in particular the launch of powerful large language models (LLMs) and their risk to rights. The AI Act itself mentions the potential for GPAI models to pose ‘systemic risks’ to public health and safety, democratic processes, and economic security (Recital 110). In addition to these, it also mentions the challenges presented by GPAI models to artists and other stakeholders due to the inclusion of intellectual property and copyrighted material in training data (Recital 105). It also notes the responsibilities of GPAI providers and requires transparency measures, including those that involve the public release of material (Recital 107). While most requirements are expected to be complied with through internal and confidential mechanisms and documenta- tions, one of the publicly accessible obligations includes the requirement to publish “a sufficiently detailed summary about the content used for training of the general-purpose AI model” (Article 53(1)(d)). The justification for this, as noted in Recital 107, is to increase transparency regarding the data used in the pre-training and training phases of model development, and to facilitate rights enforcement associated with IP, copyright, and other laws. The European Artificial Intelligence Office (AI Office), established within the European Commission for implementation of the AI Act, is responsible for providing a common template to allow implementation of the obligation. The public summary of training content is an important document for stakeholders, as emphasised in the explanatory notice, because of its role in ensuring accountability of GPAI providers and in enabling exercise of fundamental rights investigations and enforcement actions under EU laws. Therefore, it is important for the document to be of a sufficiently high quality to enable these intended investigations and accountability measures. The AI Office published the Template for the Public Summary of Training Content for general-purpose AI models on July 24th, 2025 [6], along with an explanatory notice that described the objectives of the obligation, the role of the provided template, and the considerations regarding provider’s need for confidentiality being balanced with the interests of stakeholders. However, neither the AI Office’s explanatory note nor its guidelines [9] provide an indication of how stakeholders can assess and ensure that the public summary is sufficiently transparent and useful. In addition, despite the template being published over six months prior at the time of writing, and many major GPAI providers having published updated models, none have yet published their public summaries [26]. Thus, achieving the desired transparency, accountability, and rights empowerments through the public summary risk are at risk ∗ Both authors contributed equally to this work. Authors’ Contact Information: Dick A. H. Blankvoort, dick.blankvoort@ru.nl, AI Accountability Lab (AIAL), ADAPT Centre, Trinity College Dublin, Dublin, Ireland; Harshvardhan J. Pandit, me@harshp.com, AI Accountability Lab (AIAL), ADAPT Centre, Trinity College Dublin, Dublin, Ireland; Maximilian Gahntz, maximilian@mozillafoundation.org, Mozilla.org, California, USA. 1 arXiv:2603.13270v1 [cs.CY] 26 Feb 2026 2Blankvoort & Pandit & Gahntz for avoision tactics [28], including bad faith implementations, intentional obfuscations, and delaying investigations and enforcements. The public summary obligation has also received little attention in academic research related to the AI Act, with no studies or analysis available that can guide how the public summaries should be published or how to assess adherence to the AI Act’s principles. Providers thus also may not have sufficient understanding of how to ensure their public summaries are of a sufficient quality and can achieve the intended outcomes. Simultaneously, the AI Office might prioritise (other) urgent obligations related to prohibited and high-risk AI systems and the integration of harmonised standards, and not consider public summaries a key focus in terms of assigning resources or proactive investigations. An understanding of how the published summaries should be filled so as to fulfil the intended objectives outlined in the AI Act would thus benefit both the Providers as well as enforcers like the AI Office. To assist with the upcoming enforcement of AI Act and ensure the intended goals of requiring public transparency and accountability are achieved, we provide a ‘documentation quality assessment framework’. This framework allows Providers to assess their implementations against a comprehensive set of metrics and ensure the document is of high quality. It also allows auditors, evaluators, and enforcers – including the AI Office – to objectively evaluate published summaries in a structured and comprehensive manner to identify deficiencies. Additionally, the framework also serves as a tool to compare published summaries across models and providers, and in doing so to identify specific trends that can guide future revisions to the template made by the AI Office. In summary, we provide three core contributions: RC1A Documentation Quality Assessment Framework for evaluating the public summaries of training content as required by AI Act Article 53(1)(d) for achieving transparency, accountability, and rights empowerment; RC2An evaluation of five published summaries using the developed framework that shows which aspects of the template are filled in sufficiently while also exposing which parts providers face difficulties in; RC3A discussion on the development and application of our framework that provides recommendations for Providers to improve their documentation practices and suggestions for the AI Office for improving future templates. The rest of the paper is structured as follows: Section 2 presents background information regarding the AI Act Article 53(1)(d) and template published by the AI Office, and a literature review on documentation quality assessments. Section 3 presents the methodology and the completed documentation quality assessment framework. Section 4 presents findings from applying the framework to published public summaries. Section 5 discusses our findings and providers recommendations for Providers and the AI Office based on our experiences in developing and utilising the framework. Section 6 concludes the paper. 2 Background & Related Work 2.1 AI Act Article 53(1)(d) and Template for Public Summary The AI Act’s Article 53(1)(d) puts an obligation on General Purpose AI (GPAI) model providers (henceforth just ‘Providers’) to publish a summary of training content used to develop the model (henceforth just ‘summary’). The obligation also requires the summary to based on the common template provided by the AI Office. The rationale for this, as stated in Recital 107, is ‘to facilitate parties with legitimate interests ... to exercise and enforce their rights’, and for which the provided summary must be ‘generally comprehensive in its scope instead of technically detailed’. As with the rest of the AI Act, the obligation to publish the public summary of training content will be applicable in a staggered manner [3] from 2nd August 2025. However, the AI Office will start enforcement actions only from 2nd August 2026, providing a one year leeway for providers to implement the obligation. For GPAI models ‘placed on the market’ prior to this date, the summaries must be made available before 2nd August 2027. In practical terms, providers who publish new models must also publish the public summary of training content in accordance with the AI Act, or face enforcement actions and potential penalties once enforcement begins. In addition, the AI Act requires the summaries to be made available in the ‘same context’ as the model, which in practical terms would mean in the same website or location, in a way that would be reasonable for stakeholders to discover. An explanatory notice and template for the public summary of training content for GPAI models required by Article 53(1)(d) of the AI Act was published on 24th July 2025 [6]. The announcement stated it to be a complement to the guidelines on the scope of the rules for general-purpose AI models published earlier on 18th July 2025 [6] and the General-Purpose AI Code of Practice released on 10th July 2025 [8]. The template forms a necessary part of the obligation to implement Article 53(1)(d), while the guideline and code of conduct are intended as voluntary measures by Articles 95 and 96. The explanatory note accompanying the public summary template contains 34 paragraphs that outline the objective of the template, its development Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)3 process, scope of details for filling in the template, balancing of business confidentiality, guidelines for modifications and updates to models, and procedural aspects for the implementation of the AI Act. Paragraph 34 states the Commission (through the AI Office) will monitor the implementation of the template and has the option of conducting a review prior to the start of enforcement actions on August 2nd 2026 based on technical developments and practical experiences. The explanatory note also provides insight into how the template was drafted (§5) and its intended objectives that guided the final design (§15,§16). The public summary template (henceforth just “template") is a 6-page Office Open XML Document (.docx file used in word processing applications such as Microsoft Word and LibreOffice Writer). It contains three sections: Section 1 (General Information) asks about provider identification, model details, and broad information about training data modalities; Section 2 (List of Data Sources) contains 6 subsections that enquire about data sources based on their characteristics as public or non-public sources, data crawled and scraped from online sources, reuse of user data, and synthetic data; and Section 3 (Data processing aspects) asks about data processing aspects relevant to text and data mining limitations and removal of illegal content. 2.2 AI & Technical Documentation Quality The assessment of the summary requires an approach that considers the role of the summary as documentation of the GPAI model alongside its utility as a notice for relevant stakeholders to enforce their rights and protect their legitimate interests in accordance with the AI Act. Therefore in addition to assessing the ‘quality’ of information within the summary, we must also assess the manner in which the summary is provided and assess the broader ‘document quality’. For this, we searched for existing approaches that provided guidance or assessment resources related to the AI Act. We found efforts such as those by HuggingFace [23] to assess model cards in a repository based on section titles that reflected industry best practices in terms of maintaining and assessing documentation. Liang et al. [19]provided a comprehensive and systematic analysis of 32,111 AI model documentation pages on HuggingFace and found only 44.2% models had a corresponding model card, and that there is an overwhelming lack of conformity regarding structure, contents, and expected information as compared to established guidelines. The paper also discusses resources including tools to generate model cards, guidelines for filling in model cards, and educational materials. Lucaj et al. [21]provided templates for AI Act documentation, which draw from the experience of dealing with model cards, but only focus on inter-organisational documentation. From this literature we draw the conclusion that current industry practices do not include good quality documentation, especially those that are to be made publicly available, and that this represents a risk regarding the availability and the utility of public summaries to be published by the same providers. Of note, we did not find any work specifically related to Article 53(1)(d) public summaries. Therefore, we looked towards other broader approaches regarding quality assessment associated with technical documen- tation, in particular those associated with software. To enhance our understanding on both topics, we utilised authoritative sources of well-established practices such as the ISO Quality Management Systems (QMS) [1] and ISO/IEC Systems and software Quality Requirements and Evaluation (SQuaRE) [2] standards, as well as the Eurostat Handbook on Data Quality Assessment Methods and Tools [14]. A simplified explanation of how quality assessment frameworks are developed is to identify a particular set of ‘dimensions’ (similar to principles) that best represent the quality in context of the objectives of the assessments, and to then identify specific ‘metrics’ (e.g. questions) that should be evaluated to score each dimension. We therefore conducted a literature review of technical/software documentation quality assessments and data quality assessments for identifying established dimensions and their utility based on empirical results and user studies. Surveying papers from 2010 to 2025, we identified 54 papers in total, of which we chose 7 papers as being relevant. Garousi et al. [18]described the application of Action Research (AR) methodology within an industrial setting to assess maintenance and application of documented information by using the Goal-Question-Metric paradigm. Ding et al. [16]presented a comprehensive literature review of 60 articles to derive 12 quality attributes, four cost categories, and nine benefit categories. Plosch et al. [22]conducted a survey with 88 experts and identified the most important quality attributes for documentation as accuracy (as a synonym for correctness), clarity, consistency, readability, structuredness, and understandability. Treude et al. [25] conducted a similar study where they asked technical editors to rate quality metrics for different types of documentation. This study provides important insights for the assessment of public summaries as narrative documentation with technical information. Aghajani et al. [13]analysed 878 documentation artefacts and developed a taxonomy of (common) documentation issues that was mapped to specific quality attributes. Tang and Nadi[24]presented a framework of 9 metrics developed through design co-creation process with developers and use this to automate the assessment and verification of software documentation. Zaveri et al. [29]presented a comprehensive survey of linked data quality assessment, which provided specific metrics and dimensions for assessment of the public summary as a document linked to the model as well as the use of 4Blankvoort & Pandit & Gahntz Table 1. Overview of dimensions identified from survey of literature related to technical documentation, software documentation, and data quality assessment frameworks DimensionGarousi et al Ding et alPlosch et al Treude et al Aghajani et al Tang and Nedi Zaveri et al Accessibilityxxxx Accuracyxxxxx Availabilityxx Clarityxxxx Concisenessxxxx Completenessxxxxxx Consistencyxxxxxxx Correctnessxxxxx Readabilityxxxxx Relevancyxxx Structurexxxxx Traceabilityxxxx Understandabilityxxxxx Up to date-nessxxxx Usabilityxxxx Usefulnessxxxx hyperlinked information within it. We also considered the FAIR Principles [27] which are considered best practice within the broader scientific community for the open sharing of information. Table 1 summarises our findings regarding demonstrably useful dimensions for assessing and representing quality of technical documentation, software documentation, and data(sets). For convenience, we consolidated dimensions based on their provided definitions, e.g. we combined comprehension and understandability based on equivalence in their definitions. We only included metrics that were used in at least 3 frameworks to ensure consistency across frameworks. Doing so resulted in 16 metrics that represent useful and validated indicators for quality of documentation, and which are suitable for use in assessing the public summary. 3 Documentation Quality Assessment Framework 3.1 Development Methodology To develop our quality assessment framework, we first assessed the contents and structure of the template by discussing the explanatory note and the structure and contents of the template. We then established an understanding of what information would be filled into the template to create the public summary based on our existing experiences in analysis of GPAI models and their documentation as well as understanding of the goals and requirements stated in the AI Act. For practical reasons, we identified two use-cases that guided our design process: ‘good faith’ implementations where all fields are filled in and it is clear from the information that the providers have made a conscientious effort; and ‘bad faith’ implementations where not all fields may be filled in or where there is unclear, ambiguous, or otherwise unhelpful information in the public summary. Our objective is to support the ‘good faith’ public summaries in improving their quality while highlighting deficiencies present in the ‘bad faith’ implementations. Following an iterative development process, we first identified the high-level categories through which the quality of the public summary should be represented. Based on the rationale provided by the European Commission in its explanatory note as well as the role of the summary as defined in the AI Act, we identified Transparency and Usefulness as good overall indicators for intended stakeholders to understand the quality of the public summary. In this, transparency refers to the extent of information provided through the public summary, and usefulness represents the utility of provided information as well as its potential to be actionable by specific stakeholders. We selected transparency due to its existing and established application in the context of legal documentation, for example as a principle in the GDPR, as well as being listed as an objective of methods such as model cards. And we chose usefulness, which is well-defined as a quality dimension in literature, as a broad measure of whether the provided information is sufficient to take meaningful actions and ensure accountability. Using our framework, a ‘high-quality’ public summary is described as a document that provides information in a clear, structured, accurate, and consistent manner (transparency) such that it allows relevant stakeholders to understand the training process and Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)5 Table 2. Chosen quality dimensions and their categorisation under Transparency and Usefulness as overall quality indicators. DimensionGroupDescription ClarityTransparencyinformation is unambiguous, easy to understand, and avoids misinterpretation CompletenessTransparencyrelevant and necessary information is provided ConsistencyTransparencyterminology, format, and structure is consistent within and across documents CorrectnessTransparencyinformation is accurate, validated, and reliable AccessibilityUsefulnessinformation is easy to obtain, navigate, interact, & includes accessibility standards ComprehensionUsefulnessinformation is understandable and interpretable for the intended audience Table 3. Sections; consolidated metrics; total metrics Section in our QA frameworkPublic Summary Template SectionsNo. of metrics Document-30 General informationSection 154 Public Data SourcesSection 2.126 Private Data SourcesSection 2.227 Scraped/Crawled DataSection 2.346 User DataSection 2.414 Synthetic & Other DataSection 2.5 & 2.628 Data ProcessingSection 317 Total242 to take relevant actions where necessary (usefulness). Public summaries that do not meet these requirements are considered as ‘low-quality’, and are potentially in breach of the template and the corresponding Article 53(1)(d) obligations. To quantify how transparency and usefulness should be assessed for public summaries, we deliberated on each dimension identified in literature regarding its applicability and interpretation for the public summary. This was an iterative process where we also discussed whether the chosen dimensions would be clear or confusing for various audiences, and how they would be useful to indicate transparency and usefulness. The outcome of this was the selection of 6 metrics: Clarity, Completeness, Consistency, and Correctness for representing Transparency, and Accessibility and Comprehension for representing Usefulness. In addition to representing the quality of the public summary through these dimensions, we also identified the need to consider the quality dimensions of specific sections of the template. For example, relevant stakeholders may be more interested in Section 2.4 User data as compared to 2.5 Synthetic data, and thus would benefit from an dedicated quality score for how transparent and useful specific sections of the summary are for their particular needs. To implement this, we divided the template into 8 sections (see table 2) and assessed each section independently using the chosen dimensions. Of these, 7 sections are as they appear in the template: General information (Section 1), Public Data Sources (Section 2.1), Private Data Sources (Section 2.2), Scraped/Crawled Data (Section 2.3), User Data (Section 2.4), Synthetic & Other Data (Section 2.5 & 2.6), and Data Processing (Section 3). In addition to these, we also added Document as a section to represent assessments regarding metadata, provision, accessibility, and adherence of template structure of the public summary document. To evaluate the public summary using our chosen dimensions, we utilised the Goal-Question-Metric paradigm (GQM) [15], which was also used in some of the analysed quality assessment frameworks from literature. In GQM, we start with high-level goals, which in our case are assessing the 6 quality dimensions for each of the 8 sections, for a total of 42 goals. For each goal, we then identify specific metrics (i.e. questions or requirements) that are assessed and scored, and where each metric represents an independent assessment of a piece of information required by the template, and is associated with exactly one dimension. For example, though Section 1.1 has a field asking about provider name and contact, we developed two metrics to check whether name and contact are filled in independently as part of the completeness dimension. For provenance of how each metric was developed, we created and attached identifiers for each information element in the template. For example, we assigned the identifier ‘1.1.a’ to the provider name and contact information present in the template, and then further split these into name (‘F1.1.a.1’) and contact (‘F1.1.a.2’). Using this method, we created a comprehensive set of 242 metrics for a thorough analysis of any public summary based on the template. 3.2 Assessment Procedure To assess the quality of a given public summary, the evaluator first scores each metric individually. To do this, the evaluator assesses the corresponding information field and element, and records a value that reflects their findings as: sufficient (scored 6Blankvoort & Pandit & Gahntz as 1) to indicate the provided information meets or satisfies the requirement e.g. by being clear, or being complete; insufficient (scored as 0) to indicate either an absence of information or where the information does not meet the criteria of the metric e.g. by being obfuscated or being incomprehensible; and partially sufficient (scored as 0.5) to indicate that information partially satisfies the requirement. As not all questions in the template carry the same importance or impact, we assigned an ‘importance score’ for each question (called ‘weight’ in quality assessment literature). For example, key questions in the data sources section that ask for specific details have a higher score due to their perceived importance as compared to generic and optional information. The final score for each metric is the assigned score (0...1) multiplied by the weight. By utilizing weighted scores in this manner, the quality framework proportionately rates public summaries that provide key information as being of better quality while penalizing those that don’t as lower quality. Once all metrics are scored as above, each dimension within a section is scored by taking the average of all metrics associated with that dimension and section. The transparency and usefulness scores are then calculated for each section by taking the average of the scores of their respective dimensions. The overall transparency and usefulness score for the public summary are then calculated as the average of the transparency and usefulness scores for each section. Finally, a letter grade system from A as the highest quality and F as the lowest quality is assigned based on the scores to enable stakeholders to understand quality implications at a glance. The template contains optional sections which we counted as mandatory to reward those providers who do provide additional information in their summaries. Without this approach, a summary that does not provide additional information would score equally to a summary that does provide additional information. Whereas by counting optional information we incentivise providers to go beyond the minimum information requirements and provide additional details. The template also contains fields that are conditional, for example, Section 2.5 has fields that should only be filled in if the first question asks whether synthetic data was used for training the model. If the answer is yes, the rest of the fields are necessary, and should be assessed. However, if the answer is no, then not filling in these fields should not be penalised. We took this into account by only counting those metrics that are applicable during the scoring of dimensions, so that a summary where some sections do not apply is not penalised in comparison to other summaries where the sections are required. Given that this means different public summaries may have different sections filled in, we normalise the scores as a percentage to enable comparing them along the same dimensions. This methodology is fairly similar to the one used by the European Data Portal where a variety of datasets are comparable through normalised scores [11]. 4 Quality Assessment of Published Summaries To assess the utility of our framework, we sought to apply it in practice to published summaries. The AI Act only requires the public summaries to be published ‘in the same context’ as the model, which does not lead to a specific or consistent location for where to find them. We therefore utilised a combination of search engines (using queries such as “public summary of training content" and “Article 53(1)(d) public summary”) and utilising experimental web-search features of GenAI services such as Google Search and ChatGPT. We also undertook a manual discovery process where we looked at: (1) The webpage or README in the repository where the model weights or model card are provided; (2) The legal or compliance page on the model provider’s webpage; (3) The technical report or paper where the model’s architecture and functioning are described. We expected the public summary to be explicitly indicated as such (i.e. titled ’public summary’) or by referring to AI Act’s Article 53(1)(d) obligations or broadly referring to AI Act compliance. We first prioritised Anthropic, Google, OpenAI, Meta, Mistral, and xAI as high-profile GPAI model providers based on popularity, reach, and as signatories of the Code of Practice 1 . We did not find any public summaries published by these providers based on the method described earlier. We then utilised the European Open Source AI Index [20] to identify additional models and providers. In total, we identified four public summaries as of 20th January 2026 – HuggingFace for their SmolLM3-3B model, Swiss National AI Initiative for their Apertus model family, Speakleash for the Bielik v3 11B Instruct model, and Bria AI for the Bria 3.2 model. We also found a document titled “data information” in the repo of Microsoft’s Phi-4 model on HuggingFace whose structure was similar to the template, but which was not explicitly mentioned as a public summary. Though there was no indication as to this being a public summary under Article 53(1)(d), we opted to assess it due to the similarity in structure of the template, as well as with the understanding that this would be a valid expectation from stakeholders looking for the public summary within the model’s repository. Euractiv’s reporting in January confirms the lack of public summaries, and also includes responses from Google and OpenAI that they are working on understanding the requirements and that Mistral did not respond [17]. We thus could analyse only the found published summaries. 1 We note that xAI is only signatory to the Security chapter of the Code of Practice, and has deliberately opted to not sign the Transparency chapter which is relevant to public summaries. Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)7 Table 4. Scores for published public summaries of training content. Percentages represent normalised scores, N/A represents the dimension is not applicable and therefore was not scored. Colours are used to draw attention to severity of scores with green indicating a higher quality, orange indicating moderate quality, and red indicating low quality. SectionDimensionSmolLMApertusBielikPhiBria DocumentTransparency68 .97%87 .04%88 .89%87 .04%70 .37% Usefulness58 .00%84 .21%84 .21%81 .58%68 .42% General InformationTransparency73 .53%74 .62%87 .03%70 .59%69 .73% Usefulness90 .91%93 .75%93 .75%100 .00%92 .86% Public Data SourcesTransparency96 .06%100 .00%86 .91%3.70%100 .00% Usefulness100 .00%100 .00%100 .00%0.00%N/A Private Data SourcesTransparency100 .00%100 .00%100 .00%53 .85%100 .00% UsefulnessN/AN/AN/AN/A100 .00% Scraped/Crawled DataTransparency100 .00%100 .00%83 .78%0.00%100 .00% UsefulnessN/AN/A51 .46%0.00%100 .00% User DataTransparency100 .00%100 .00%100 .00%0.00%100 .00% UsefulnessN/AN/AN/AN/AN/A Synthetic & Other DataTransparency60 .00%100 .00%100 .00%8.89%100 .00% Usefulness0.00%N/A100 .00%0.00%N/A Data ProcessingTransparency92 .00%100 .00%98 .72%52 .00%100 .00% Usefulness100 .00%100 .00%71 .43%0.00%100 .00% Overall ScoresTransparency82 .50%92 .90%88 .02%33.30%86 .97% Usefulness86 .01%97 .14%71 .11%24.54%94 .74% Overall GradesTransparencyB+AB+DB+ UsefulnessB+A+C+FA For each identified public summary, we first created a locally archived copy (PDF) for the specific version we annotated in case they were updated. We then read through the public summary to familiarise ourselves with its structure, content, and implications. One evaluator then scored the public summary using developed metrics and recorded the outcomes in a spreadsheet. The other evaluator then worked with the first annotator to verify the assessment and make clarifications where necessary. We opted for this approach as the public summary is a new form of documentation and we had no examples when developing the initial framework, and only found the summaries closer to completion stages of the work. In this process, there was no disagreement recorded. To ensure consistency, the same persons followed the roles of evaluator and verifier for each public summary. We then calculated the scores for each section and the overall transparency and usefulness scores through the method described earlier. As these were the first instances of public summary assessment, we then compared the scores and the contents of the public summaries to understand whether our scores were fair and an appropriate indicator of what was provided in the public summaries. We did not find any pertinent issues and assessed the scores to be a fair approximation of our own subjective impressions. The scores for the five identified models are described in Table 4, with the full assessment provided in Appendix A. Below we discuss the contents, our scoring, and implications for each public summary. 4.1 SmolLM by HuggingFace The public summary for HuggingFace’s SmolLM3-3B model is available on its platform as a webpage [10]. In general, the structure of the template was preserved with clear indication of section numbers, consistent field labels, and explicitly indicating yes/no where applicable. Other than issues with accessibility, we found no major issues with the public summary. It scored 82.50% and received a grade of B+ for transparency, and 86.01% with grade B+ for usefulness, which meant that the summary was of a moderate but good quality. SmolLM scored particularly well on its disclosure of public data sources, a key disclosure demanded by Article 53(1)(d), and a good indicator of transparency in model development process. The SmolLM model described in the summary was modified from a previous model whose details were not provided in this summary. We assumed this was because of the previous model being published before the cut-off date for publishing summaries, but were uncertain regarding the correct legal interpretation for modified models where the published summaries for base model are not available. SmolLM’s summary was not penalised in this regard. One major issue we found that could be easily rectified was regarding the Section 2.5 Synthetic data, where an entire field was missing (GPAI models available on market) and its information was instead put in the next field (GPAI models not available on market). This resulted in deduction of points regarding structure and for the incorrectly filled fields. Another issue we faced was regarding accessibility, where 8Blankvoort & Pandit & Gahntz despite the summary following the structure of the template correctly, accessing and exporting a copy was made particularly difficult due to the ‘dynamic’ nature of the webpage. This resulted in a lower document accessibility score. 4.2 Apertus by Swiss AI Initiative The public summary provided by the Swiss AI Initiative covers multiple models – Apertus-8B, Apertus-8B-Instruct, Apertus- 70B, and Apertus-70B-Instruct from the Apertus model family [4]. Like SmolLM, the public summary was also published on HuggingFace. However, instead of a dynamic webpage it was provided as a PDF closely following the template. Each field was filled in, including explicitly marking sections as not applicable, though some fields included superfluous information not relevant to the topic or question which caused a few points deduction. We assessed its score to be 92.90% with Grade A for transparency, and 97.14% with Grade A+ for usefulness, which were the highest of all assessed summaries. 4.3 Bielik by Speakleash The open collaboration project SpeakLeash published a public summary for its Bielik v3 11B Instruct model [5] on its website and also linked it in its model card. The public summary follows the structure of the template, and describes Bielik as a fine-tuned version of Mistral. Of note, Bielik utilised the optional additional comments field in Section 1.3 to describe the purposes, regional, and linguistic characteristics of the dataset used as based on Polish and EU legal administrative domains and the inclusion of Silesian and Kashubian language Wikipedia pages. For their list of public data sources, however, the description referenced external documents (‘preprints’) which resulted in a deduction of points as relevant information was not provided in the document. We assessed its score to be 88.02% with grade B+ for transparency, and 71.11% with grade C+ for usefulness. 4.4 Phi by Microsoft We did not find an explicitly published summary for Microsoft’s Phi-4 model, but found a file in its repository on HuggingFace called “data_summary_card.md”, whose structure closely matched that of the template [7]. The title used in the file was “Data Summary”, which was also title used by HuggingFace in its SmolLM public summary. We are uncertain as to whether this represents an industry practice of co-opting of the formal title of “public summary of training content" present in the template with a broader and vague title that could be confused with other data-related documentation also provided with GPAI models. We opted to assess this model regardless based on the assumption of stakeholders viewing the document as fulfilling the template requirements, as explained earlier. The assessment showed that fields provided in the document are filled in to some extent, but that the document itself is quite sparse and does not provide many details and also had sections missing. We further found that section numbers and questions had a significant mismatch from what was provided in the template. The document also suffered from subjective non-relevant statements, such as question 2.3.1 on whether public data was used to train the model being answered with “Microsoft follows all relevant laws and regulations pertaining to personal information" – which is neither relevant nor clear. We also found the document did not provide required information significantly, for example subsequent parts that enquire about the source and uses of personal data (2.4 User data in template) were found completely missing. Our assessment reflects these systematic major issues in the outcomes where the document scored 33.30% with a Grade D for transparency, and 24.54% with a Grade F for usefulness, which is the lowest score amongst all assessed summaries. 4.5 Bria by Bria AI The public summary for the Bria 3.2 model was published by Bria AI on its website under the ‘Security & Compliance’ section as a link to a PDF hosted on Google Drive [12]. The summary closely followed the structure of the template, but which provided contact details for the authorised representative but not for the company itself. The document also lacked some metadata such as version information and links to documentation. The summary describes training only involving licensed data, which requires relatively little disclosure through the template. We assessed its score as 86.97% with a grade of B+ for transparency, and 94.74% with a grade of A for usefulness. Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)9 5 Discussion 5.1 Status of Published Summaries Of the five summaries we assessed, three —SmolLM, Apertus, and Bria— had a sufficiently high quality to be considered as having made a good effort. All three had relatively minor issues that we felt did not impact the objectives of the template, and which should be easily fixable. Of these, Apertus scored the highest, and the evaluators consider it as an exemplary example for guiding subsequent public summaries to be published by other providers. Of the remaining two – Bielik and Phi – only Bielik’s public summary was explicitly labelled as such. While Bieliks’s summary also seemed to be in good faith, it did sufficiently provide details in required forms and as a result suffered from a lower scores. We consider this to also be fixable - though it would require the Provider to be willing to provide this information. In the case of Phi, the document was unclear and shows several systemic issues which would require major fixes. We are therefore unclear as to whether Microsoft intended this document to be a public summary, and hope that a revised and improved version would be provided soon. As we are at an early stage of the AI Act’s implementation, and given the rather small amount of published summaries available, we have requested the four Providers who have provided their public summaries to discuss our findings. Through this, we aim to improve our own understanding of Provider’s perspectives in completing the template, and to inform these Providers on how their summaries can be improved in quality. We consider this to be an important step as early practices in publishing the summaries might shape those that come later, and establishing a high level of quality from the onset would also be beneficial to encourage larger Providers to publish their summaries with sufficient details. Of the four Providers, only Apertus responded at the time of writing this. In our meeting, Apertus confirmed the identified issues and mentioned to us that they will take this into account in their next public summary which they are planning to be published soon. They also informed us about their perception of this work as being potentially helpful to have for small/open providers to help fill in the summary. 5.2 Author’s Perspectives and Self-Reflection The public summary is a powerful obligation considering its potential to mandate transparency regarding the data used to train GPAI models. The motivations of the authors in taking on this work were rooted in concerns around the understanding and enforcement of regulations as well as a desire to support the objectives of the AI Act in upholding rights and promoting responsible innovation. In creating the documentation quality assessment framework and carrying out the assessments, our experiences were shaped by the understanding of the intended objectives of the template, as well as practical limitations that might be faced by Providers. Therefore, in this section we share key observations to assist other evaluators, in particular the AI Office, in their assessment of public summaries. Starting with the template, initially we found that it was sufficiently legible from a technical perspective as we were familiar with earlier efforts such as data sheets and model cards. However, when we attempted to understand which information would be ‘ideal’ for each field and how it could be assessed across summaries, we faced barriers in achieving sufficient understanding. This stemmed primarily from not knowing the precise rationale for why specific fields were chosen for inclusion while others which seemed conceivable were not present. In particular, we were conscious of the final template development process occurring behind closed doors, and that the discussion and choices that ultimately influenced the design and structure of the template were inaccessible to us. Existing literature on quality assessments, both for technical documentation and data quality, places a strong emphasis on requiring structure as a way to ensure quality and ease assessments. The narrative structure of the template thus created difficulties in developing our framework and assessing it in a consistent manner as we had to take into account significant variation in the form in which documentation was provided. This complexity largely influenced how we chose metrics and combined them in dimensions to assess quality. If the template were more structured, or provided more guidance that restricted information to expected forms, it would have vastly simplified the framework development and assessment process. In addition, the template, despite aiming to provide a common format and structure to data summaries, was found to result in heterogenous implementations which deviated from the goal of providing the document in a standardised format. This further increased the labour involved in assessment. Lastly, we encountered difficulties in discovering published summaries, as no defined common mechanism for publishing summaries exists, nor is there an established practice on where and how to provide the summary alongside models. This also resulted in a significant challenge as we had to expend considerable efforts to carry out an exhaustive search with the concern of missing published summaries. These experiences motivated us to formulate recommendations for GPAI Providers 10Blankvoort & Pandit & Gahntz to publish their public summaries in a manner that reduces friction for other stakeholders, and for the AI Office to consider improvements to the template in subsequent revisions. 5.3 Recommendations 5.3.1 Recommendations for GPAI Providers. As the most prominent challenge we surfaced through this work is the lack of available public summaries, We provide the following recommendations for GPAI providers: P.1.Ensure the public summary document adheres closely, ideally exactly, to the template provided by the AI office. P.2.Mention the public summary document in relevant webpages and documents, for example the README in a repository, and under the correct title as utilised by the template. P.3.For sections in the template for which no information is available or where the fields are not applicable, explicitly state this to avoid ambiguity arising from open-world assumptions. P.4. Utilise correct and high-quality SEO metadata to help surface published summaries in search engine results. P.5.Provide a common area or location for all relevant legal documentation through which stakeholders can also access the published summaries. P.6. Maintain versions of summaries and clearly indicate where to find these in the document. P.7.Ensure the public summaries contain all pertinent information, and refrain from directing stakeholders to other locations to access this information. Links to external resources that expand upon the details provided in the public summary are helpful but should not be used to avoid providing the information in the public summary. P.8. To ensure the public summaries are filled in to a high degree, having the correct and up to date information about datasets used in the model development process is necessary, which in turn requires good data governance. 5.3.2 Recommendations for AI Office to improve the template. : R.1.Adding more structured questions, i.e. non-narrative, related to provenance and metadata of the document will assist stakeholders in use of summary (e.g. regarding version, published date, link to previous versions). R.2.A broad document-level field that indicates what to expect, for example, whether the published summary relates to a new model, or a modified model; and if the base model’s published summary is available (if yes, a link to it), and if not, then a justification. Ideally, this could be implemented through different versions of the template. R.3.A guide attached to the template that describes which information should be expected for specific fields would be helpful. Though this information is provided in part within the ’help text’ part of the field itself, we found that this could be expanded to cover more use-cases and offer greater guidance. R.4.Fields as currently should be tested for use by specific stakeholders, for example through a workshop that identifies whether information is clear, comprehensible, useful, etc. R.5.Specific sections could benefit obligations for other laws beyond AI Act. For example, Section 2.4 User Data regarding GDPR Articles 13 and 14 where providers could provide information (e.g. purpose, legal basis) or a link to external resource (e.g. privacy policy). R.6.The public summaries being difficult to discover is a barrier for stakeholders. The AI Office could consider making a centralised portal that hosts all public summaries, and which would includes an online form for guiding providers in providing specific information. This would strengthen and simplify transparency and accountability for public summaries and would simplify the enforcement of this obligation at scale. 5.4 Public dissemination Our objective of assisting GPAI Providers in filling in the template with the highest possible quality, as well as highlighting the deficiencies in published summaries, motivates us to further publish our work in a publicly visible manner. For this, we have published our methodology, assessments, and helpful resources such as the recommendations through a public website at https://aial.ie/research/gpai-training-transparency. The website also acts as a common resource and provides visibility for GPAI Providers to identify other summaries and thereby improve their own, and benefits stakeholders by collecting identified public summaries in a single location (with links to their source). Our assessments provide both convenient (e.g. letter grades, percentage) and quantitative (i.e. detailed metrics) scores that are helpful to a variety of stakeholders, and the website can provide the necessary information depending on the role and interests of the stakeholder. In order to ensure long-term sustainability of the work, the AI Accountability Lab in Trinity College Dublin will host and oversee the publishing and maintenance of the website and associated resources, which provides a degree of sustainability and longevity to this work. Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)11 The intended audience of the website consists of three key stakeholders: first, the GPAI model providers who can use the website to find examples of public summaries and identify best practices for implementing their own based on the quality assessments; second, the authorities and enforcers such as the AI Office for the AI Act as well as GDPR authorities, who will benefit from a common repository of public summaries and their analysis in a manner that aligns with their investigatory concerns; third, the organisations, representatives, and citizens who are concerned regarding their rights and want to understand how the public summaries could provide transparency and meaningful opportunities for them to take action. To allow the intended audiences to meet their objectives, we identified functional requirements regarding what information must be provided to stakeholders (e.g. metrics used in assessments, outcomes), the granularity and comprehensibility of presenting this information (e.g. overview of overall scores, followed by section-level scores, followed by a link to the full report), traceability (e.g. a link to the public summary, access to archive copies), and support (e.g. a link to AI Office guidance, relevant literature). We also identified non-functional requirements related to ease of use (e.g. design of webpages, density of information), accessibility (e.g. desktops, smartphones), and discoverability (e.g. SEO, search engines). For maintainability and sustainability, we open open the work as a community resource via GitHub at https://github.com/AIAccountabilityLab/gpai- training-transparency where we will host our reports of all model providers in a transparent and trackable manner, and also invite contributions, discussions, and bug fixes. 5.5 Limitations Our work takes place in the early stages of AI Act implementation, as is evident by the limited number of published summaries. As such, our results may not be indicative of the general practice as enforcement takes effect. However, the documentation quality assessment framework we have developed is based on robust practices that have been established within the quality assessment literature. To mitigate this, we have consciously chosen a methodological approach that permits future modifications and corrections without a large amount of repeated work to update the metrics and scores. We are thus aware and have taken into account that the developed framework will need revisions as more summaries are published and as complexities arise in understanding and using the information. It is conceivable that our metrics and assessments may be questioned, especially by specific Providers. This can happen for many reasons, for example disagreements regarding what is of more importance, the perceived fairness of the scoring system, or even to avoid exposing problems in published documents. As with all quality assessments, ours is one possible method for evaluating the published summaries. We have provided sufficient information to justify our approach as being informed in literature, clarified our objectives as based in pragmatic implementation of the AI Act’s objectives, and identified socio-technical approaches to improve the use of the templates and published summaries through our recommendations. However, we invite future discussions regarding the implementation of our documentation quality assessment framework as we perceive the work to require a community effort in the future based on the number of GPAI Providers and published summaries to be assessed. We also acknowledge the lack of stakeholder participation or validation as a limitation of this work. In particular, we identify the need to assess our work with Providers, stakeholders with specific rights interests such as IP, copyright, and GDPR that want to use the template, civil society organisations, and the AI Office and other enforcers. However, we consider this as an intended and planned iteration of our quality assessment framework, where the adjustments made to the metrics and scores will reflect the growing discussion and perception of transparency and usefulness within the public summaries. We have shared this work through the website and engaged with the Providers who have published their public summaries as a way to start this process. 6 Conclusion The requirement for public summaries in AI Act’s Article 53(1)(d) is intended to facilitate stakeholders in understanding the implications of GPAI models and to enforce their rights under EU law. As such, the quality of public summaries is a determining factor for whether stakeholders can carry out relevant investigations and take corrective actions. The documentation quality assessment framework we have developed is the first approach to consider the practicalities of this obligation and how it is implemented and assessed. Our work demonstrates both the specifics of the information demanded by the public summary, and provides valuable guidance on how the public summary and its implementations can be provided so as to achieve the stated objectives of transparency and rights empowerment. We show how the industry is yet to react significantly to Article 53(1)(d), with only a handful of GPAI model providers having yet published their training data summaries. Of the providers which published a summary, three were small organizations 12Blankvoort & Pandit & Gahntz with an emphasis on data transparency, including two based in EU. While major GPAI providers are yet to publish their summaries, this paper provides sufficient evidence that training data summaries can be provided in high-quality, and that the onus is on model providers to ensure that sufficient quality standards are met. Our work also provides potentially valuable insights for the AI Office by providing a methodological tool that can show how different summaries compare in terms of transparency and usefulness. The structured analysis we developed is particularly useful for identifying whether deficiencies or issues are widespread or merely isolated to a few actors. To assist both model providers and the AI Office, we provide actionable recommendations based on our experiences, and aim to publish training data summary assessments as a public resource. Finally, by publishing our assessments of public summaries publicly, we assist stakeholders by providing all summaries in a single centralised location, and also encourage providers to provide high-quality data summaries in fulfilment of European regulations. Acknowledgements Funding. This work was funded by Mozilla.org. Additional Funding Declarations. The AI Accountability Lab is funded under the AI Collaborative, an Initiative of the Omidyar Group; the Bestseller Foundation; and the John D. and Catherine T. MacArthur Foundation. The ADAPT Research Ireland Centre for Digital Media Technology is funded by Science Foundation Ireland through the SFI Research Centres Programme and is co-funded under the European Regional Development Fund (ERDF) through Grant#13/RC/2106_P2. Generative AI Usage Statement. The authors hereby declare that no form of generative AI was used in the writing of this paper. Authors. This report was authored by Dick Blankvoort and Harshvardhan J. Pandit. The authors thank Maximillian Gahntz for their role in provisioning and supervision of the project, and Zuzanna Warso, Prof. Abeba Birhane and the AI Accountability Lab in Trinity College Dublin for supporting the work. Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)13 References [1] 2015. ISO 9001:2015. https://w.iso.org/standard/62085.html [2] 2022. ISO/IEC TS 25052-1:2022. https://w.iso.org/standard/81467.html [3] 2024. Regulation - EU - 2024/1689 - EN - EUR-Lex. https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng [4]2025. Apertus_EU_Public_Summary.Pdf·Swiss-Ai/Apertus-70B-2509 at Main. https://huggingface.co/swiss-ai/Apertus-70B-2509/blob/main/Apertus_EU_ Public_Summary.pdf [5] 2025. Bielik 11B v3 EU Public Summary. https://bielik.ai/downloads/Bielik%2011B%20v3%20EU%20Public%20Summary.pdf [6] 2025. Commission presents template for General-Purpose AI model providers to summarise the data used to train their model | Shaping Europe’s digital future. https://digital-strategy.ec.europa.eu/en/news/commission-presents-template-general-purpose-ai-model-providers-summarise-data-used-train-their [7] 2025. Data_summary_card.Md· Microsoft/Phi-4 at Main. https://huggingface.co/microsoft/phi-4/blob/main/data_summary_card.md [8] 2025. The General-Purpose AI Code of Practice | Shaping Europe’s digital future. https://digital-strategy.ec.europa.eu/en/policies/contents-code-gpai [9] 2025. Guidelines on the scope of obligations for providers of general-purpose AI models under the AI Act | Shaping Europe’s digital future. https://digital- strategy.ec.europa.eu/en/library/guidelines-scope-obligations-providers-general-purpose-ai-models-under-ai-act [10] 2025. SmolLM3-3B Public Summary of Training Content. https://huggingface.co/spaces/hfmlsoc/smollm3-eu-data-transparency [11] 2026. Metadata Quality Assessment Methodology. https://data.europa.eu/mqa/methodology?locale=en [12] 2026. Security & Compliance. https://bria.ai/security-and-compliance [13] Emad Aghajani, Csaba Nagy, Olga Lucero Vega-Marquez, Mario Linares-Vasquez, Laura Moreno, Gabriele Bavota, and Michele Lanza. 2019. Software Documentation Issues Unveiled. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, Montreal, QC, Canada, 1199–1210. doi:10.1109/ICSE.2019.00122 [14]Mats Bergdahl, Manfred Ehling, Eva Elvers, Erika Földesi, Thomas Körner, Andrea Kron, Peter Lohauß, Kornelia Mag, Vera Morais, Anja Nimmergut, Hans Viggo Sæbø, Ulrike Timm, and Maria João Zilhão. 2007. Handbook on Data Quality Assessment: Methods and Tools. Eurostat, European Commission. [15]Victor R Basili-Gianluigi Caldiera and H Dieter Rombach. 1994. Goal Question Metric Paradigm. Encyclopedia of software engineering 1, 528-532 (1994), 6. [16]Wei Ding, Peng Liang, Antony Tang, and Hans van Vliet. 2014. Knowledge-Based Approaches in Software Documentation: A Systematic Literature Review. Information and Software Technology 56, 6 (June 2014), 545–567. doi:10.1016/j.infsof.2014.01.008 [17] Euractiv. 2026. AI Companies Are Not Being as Transparent as the EU Wants. [18]Golara Garousi, Vahid Garousi, Mahmoud Moussavi, Guenther Ruhe, and Brian Smith. 2013. Evaluating Usage and Quality of Technical Software Documentation: An Empirical Study. In Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering. ACM, Porto de Galinhas Brazil, 24–35. doi:10.1145/2460999.2461003 [19] Weixin Liang, Nazneen Rajani, Xinyu Yang, Ezinwanne Ozoani, Eric Wu, Yiqun Chen, Daniel Scott Smith, and James Zou. 2024. Systematic Analysis of 32,111 AI Model Cards Characterizes Documentation Practice in AI. Nature Machine Intelligence 6, 7 (July 2024), 744–753. doi:10.1038/s42256-024-00857-z [20]Andreas Liesenfeld, Dick Blankvoort, Nityaa Kalra, and Mark Dingemanse. 2025. European Open Source AI Index database. doi:10.5281/ZENODO.15386042 [21]Laura Lucaj, Alex Loosley, Håkan Jonsson, Urs Gasser, and Patrick van der Smagt. 2025. TechOps: Technical Documentation Templates for the AI Act. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 8. 1647–1660. [22] Reinhold Plosch, Andreas Dautovic, and Matthias Saft. 2014. The Value of Software Documentation Quality. In 2014 14th International Conference on Quality Software. IEEE, Allen, TX, USA, 333–342. doi:10.1109/QSIC.2014.22 [23]society ethics. 2024. Model Card Regulatory Check - a Hugging Face Space by society-ethics. https://huggingface.co/spaces/society-ethics/model-card- regulatory-check [24]Henry Tang and Sarah Nadi. 2023. Evaluating Software Documentation Quality. In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR). IEEE, Melbourne, Australia, 67–78. doi:10.1109/MSR59073.2023.00023 [25] Christoph Treude, Justin Middleton, and Thushari Atapattu. 2020. Beyond Accuracy: Assessing Software Documentation Quality. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, Virtual Event USA, 1509–1512. doi:10.1145/3368089.3417045 [26] Zuzanna Warso and Paul Keller. 2025. 5 + 3 + 3 = 0 transparency. https://openfuture.eu/blog/5-3-3-0-transparency [27]Mark D. Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E. Bourne, Jildau Bouwman, Anthony J. Brookes, Tim Clark, Mercè Crosas, Ingrid Dillo, Olivier Dumon, Scott Edmunds, Chris T. Evelo, Richard Finkers, Alejandra Gonzalez-Beltran, Alasdair J. G. Gray, Paul Groth, Carole Goble, Jeffrey S. Grethe, Jaap Heringa, Peter A. C. ’t Hoen, Rob Hooft, Tobias Kuhn, Ruben Kok, Joost Kok, Scott J. Lusher, Maryann E. Martone, Albert Mons, Abel L. Packer, Bengt Persson, Philippe Rocca-Serra, Marco Roos, Rene van Schaik, Susanna-Assunta Sansone, Erik Schultes, Thierry Sengstag, Ted Slater, George Strawn, Morris A. Swertz, Mark Thompson, Johan van der Lei, Erik van Mulligen, Jan Velterop, Andra Waagmeester, Peter Wittenburg, Katherine Wolstencroft, Jun Zhao, and Barend Mons. 2016. The FAIR Guiding Principles for Scientific Data Management and Stewardship. Scientific Data 3, 1 (March 2016), 160018. doi:10.1038/sdata.2016.18 [28]Rui-Jie Yew, Bill Marino, and Suresh Venkatasubramanian. 2025. Red Teaming AI Policy: A Taxonomy of Avoision and the EU AI Act. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (Athens Greece, 2025-06-23). ACM, 404–415. doi:10.1145/3715275.3732028 [29]Amrapali Zaveri, Anisa Rula, Andrea Maurino, Ricardo Pietrobon, Jens Lehmann, and Sören Auer. 2015. Quality Assessment for Linked Data: A Survey: A Systematic Literature Review and Conceptual Framework. Semantic Web 7, 1 (March 2015), 63–93. doi:10.3233/SW-150175 14Blankvoort & Pandit & Gahntz A Detailed score cards for evaluated models Table 5. The detailed score cards for each of the evaluated models. SmolLMClarityCompletenessConsistencyCorrectnessAccessibilityComprehensionTransparencyUsefulness Document68 .18%100 .00%68 .75%50 .00%65 .38%50 .00%68 .97%58 .00% General information52 .50%83 .08%100 .00%73 .91%N/A90 .91%73 .53%90 .91% Public Data Sources 84 .62%100 .00%100 .00%100 .00%100 .00%100 .00%96 .06%100 .00% Private Data SourcesN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Scraped/Crawled DataN/A100 .00%N/A100 .00%N/AN/A100 .00%N/A User Data N/A100 .00%N/A100 .00%N/AN/A100 .00%N/A Synthetic & Other Data67 .57%61 .29%100 .00%41.94%N/A0.00%60 .00%0.00% Data Processing100 .00%83 .33%100 .00%91 .67%100 .00%100 .00%92 .00%100 .00% Sum73 .10%89 .12%95 .10%76 .74%90 .63%84 .17%82 .50%86 .01% B+B+ ApertusClarityCompletenessConsistencyCorrectnessAccessibilityComprehensionTransparencyUsefulness Document81 .82%100 .00%100 .00%70 .00%100 .00%66 .67%87 .04%84 .21% General information47.62%82 .26%100 .00%100 .00%100 .00%90 .91%74 .62%93 .75% Public Data Sources 100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00% Private Data SourcesN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Scraped/Crawled Data N/A100 .00%N/A100 .00%N/AN/A100 .00%N/A User DataN/A100 .00%N/A100 .00%N/AN/A100 .00%N/A Synthetic & Other DataN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Data Processing100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00% Sum 82 .25%94 .39%100 .00%98 .48%100 .00%95 .56%92 .90%97 .14% A+ BielikClarityCompletenessConsistencyCorrectnessAccessibilityComprehensionTransparencyUsefulness Document81 .82%100 .00%100 .00%80 .00%100 .00%66 .67%88 .89%84 .21% General information78 .89%86 .59%100 .00%100 .00%100 .00%90 .91%87 .03%93 .75% Public Data Sources100 .00%74 .29%100 .00%100 .00%100 .00%100 .00%86 .91%100 .00% Private Data Sources N/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Scraped/Crawled Data91 .60%81 .48%100 .00%77 .88%0.00%67 .95%83 .78%51 .46% User DataN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Synthetic & Other Data100 .00%100 .00%100 .00%100 .00%N/A100 .00%100 .00%100 .00% Data Processing 94 .74%100 .00%100 .00%100 .00%0.00%86 .96%98 .72%71 .43% Sum92 .74%82 .48%100 .00%88 .65%46.67%80 .51%88 .02%71 .11% B+C+ PhiClarityCompletenessConsistencyCorrectnessAccessibilityComprehensionTransparencyUsefulness Document81 .82%100 .00%100 .00%70 .00%95 .00%66 .67%87 .04%81 .58% General information 50 .00%78 .95%100 .00%53 .85%N/A100 .00%70 .59%100 .00% Public Data Sources 0.00%1.67%100 .00%5.00%0.00%0.00%3.70%0.00% Private Data SourcesN/A40.00%100 .00%40.00%N/AN/A53 .85%N/A Scraped/Crawled DataN/A0.00%N/A0.00%N/AN/A0.00%N/A User DataN/A0.00%N/A0.00%N/AN/A0.00%N/A Synthetic & Other Data 0.00%3.33%100 .00%3.33%N/A0.00%8.89%0.00% Data Processing100 .00%0.00%100 .00%50 .00%0.00%0.00%52 .00%0.00% Sum28.89%28.65%100 .00%27.08%21.11%26.98%33.30%24.54% DF Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)15 BriaClarityCompletenessConsistencyCorrectnessAccessibilityComprehensionTransparencyUsefulness Document81 .82%25.00%100 .00%40.00%70 .00%66 .67%70 .37%68 .42% General information81 .25%60 .26%100 .00%71 .43%N/A92 .86%69 .73%92 .86% Public Data SourcesN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Private Data Sources100 .00%100 .00%100 .00%100 .00%N/A100 .00%100 .00%100 .00% Scraped/Crawled Data N/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A User DataN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Synthetic & Other DataN/A100 .00%100 .00%100 .00%N/AN/A100 .00%N/A Data Processing 100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00%100 .00% Sum90 .59%79 .76%100 .00%90 .82%88 .00%96 .30%86 .97%94 .74% B+A 16Blankvoort & Pandit & Gahntz B List of Metrics Identified for Assessment of Public Summaries Table 10. The metrics identified for template assessment, along with their weights IDMetricSection FieldDimensionWeight D1All information must be provided within a single document0n/aAccessibility3 D2Document must be easy to find0n/aAccessibility1 D3Document must be accessible0n/aAccessibility1 D4Document should be comprehensible0n/aComprehension3 D5Document should have the correct structure0n/aClarity1 D6Document must be readable0n/aComprehension3 D7Document should have clear provenance0n/aClarity1 D8Document should have assured integrity0n/aCorrectness2 D9Document should be in a well-defined, structured, and interoperable format 0n/aClarity1 D10Document should support sharing and exporting0n/aAccessibility1 D11Document language should be consistent with the language used in other external documentation 0n/aConsistency1 D12Document should be provided in the same context as the model0n/aAccessibility3 D13Document should be consistent with other documents provided else- where representing the same model version(s) and summary 0n/aConsistency3 D14Document should be consistent across versions (across updates)0n/aConsistency3 D15Document should clearly indicate changes from previous version0n/aComprehension2 D16Document should clearly indicate its current status, in particular whether it is the latest version or if it is outdated and a replacement is made available 0n/aClarity1 D17Document should indicate where notice of updates or changes will be provided 0n/aComprehension1 D18Document should provide link to authoritative source of the document0n/aCorrectness1 D19Document should provide link to all versions of the document0n/aClarity1 D20Document with updated information must be provided in the same context as the earlier document 0n/aClarity3 D21Document should be provided in a timely manner0n/aClarity3 D22Document export preserves links to external resources0n/aAccessibility1 Version of the Summary D23Is a version for the document provided?00.aCompleteness3 D24Are link(s) to previous versions of the document provided, where appli- cable? 00.aAccessibility3 D25If link(s) to previous versions are provided, does each version have a unique version number? 00.aConsistency1 D26If each version has a unique version number, do these version numbers adhere to a consistent format so that it is clear which version comes prior and which one comes next? 00.aCorrectness1 D27If link(s) to previous versions are provided, are the links accessible for intended stakeholders? 00.aComprehension3 Last update D28Does the document have a date of last update?00.bCompleteness1 D29Is this date accurate?00.bCorrectness1 D30Is the date format correct?00.bCorrectness1 1General information Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)17 IDMetricSection FieldDimensionWeight F1.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 11Consistency3 1.1Provider identification 1.1.aProvider name and contact details F1.1.a.1Is the provider name given?11.1.aCompleteness5 F1.1.a.2Is this provider name sufficiently detailed? (I.e. can be traced back to individual organizations) 11.1.aComprehension5 F1.1.a.3Are contact details provided?11.1.aCompleteness5 F1.1.a.4Do these contact details refer to a non-generic contact point? (E.g. not the address used for general correspondence with the company) 11.1.aClarity5 1.1.bAuthorised representative name and contact details F1.1.b.1Is the name of the authorised representative given, if required?11.1.bCompleteness3 F1.1.b.2If the name is required, is this name sufficiently detailed? (i.e. can be traced back to individual representatives) 11.1.bComprehension3 F1.1.b.3If required, are contact details for the authorised representative pro- vided? 11.1.bCompleteness3 F1.1.b.4If required, do these contact details refer to a non-generic contact point? (E.g. not the address used for general correspondence with the company) 11.1.bClarity3 1.2Model identification 1.2.aVersioned model name(s) F1.2.a.1Are the identifier(s) provided for each specific model(s)?11.2.aCompleteness5 F1.2.a.2Are the provided identifiers traceable and point to a single specific model? 11.2.a Comprehension5 F1.2.a.3Are links to additional publicly available documentation provided for the model(s)? 11.2.aCompleteness5 F1.2.a.4If links are provided, do these links lead to relevant locations?11.2.aAccessibility5 F1.2.a.5Is the content for all provided model version(s) identical?11.2.aConsistency5 1.2.bModel dependencies F1.2.b.1Are dependencies indicated in cases where the model is fine-tuned from another model? 11.2.bCompleteness5 F1.2.b.2If dependencies are indicated, is it clear for which models they apple?11.2.bClarity5 F1.2.b.3If dependencies exist, are these dependencies accurate?11.2.bCorrectness5 F1.2.b.4If dependencies exist and there exist Summary(ies) for them, are these linked? 11.2.bCompleteness5 F1.2.b.5If Summary(ies) are linked, do these links lead to relevant locations?11.2.bAccessibility5 1.2.cDate of placement of the model on the Union market F1.2.c.1Is/are the date(s) on which the model(s) was/were placed on the Union market given? 11.2.cCompleteness1 F1.2.c.2Are dates given accurate?11.2.cCorrectness1 1.3Modalities, overall training data size and other char- acteristics 1.3.aModality F1.3.a.1Are any modalities checked?11.3.aCompleteness3 F1.3.a.2Are the checked modalities accurate?11.3.aCorrectness3 1.3.bTraining Data Size 18Blankvoort & Pandit & Gahntz IDMetricSection FieldDimensionWeight F1.3.b.1For the regular modalities, is exactly only of the checkboxes ticked, or alternatively is the approximate size provided? 11.3.bCompleteness3 F1.3.b.2For the'Text' modality, if an approximate size is provided, is the unit of measurement sensible? 11.3.bClarity3 F1.3.b.3For the regular modalities, if the unit of measurement is sensible or one of the checkboxes is ticked, is the number of tokens accurate? 11.3.bCorrectness3 F1.3.b.4For the'Other' modality, are the other modalities listed sensibly?11.3.bClarity3 F1.3.b.5For the'Other' modality, are the units of measurement provided sensi- bly? 11.3.bClarity3 F1.3.b.6For the'Other' modality, are the measurements provided accurate?11.3.bCorrectness3 F1.3.b.7Does the'Other' modality cover all modalities used in the training of the model outside of'Text','Image','Audio', and'Video'? 11.3.bCompleteness3 1.3.cTypes of content F1.3.c.1Are any types of content provided?11.3.cCompleteness10 F1.3.c.2If yes, does the description cover all types of content?11.3.cCompleteness10 F1.3.c.3Does the description restrict itself to only describing the types of con- tent? In particular, does it refrain from describing the goals for which the content is included? 11.3.cClarity10 1.3.dLatest date of data acquisition/collection for model training F1.3.d.1Is the latest date of data collection/attainment provided?11.3.dCompleteness1 F1.3.d.2Is this date provided exactly (i.e. exact month of latest acquisi- tion/collection) rather than as a loose description? 11.3.dCorrectness1 F1.3.d.3If yes, does this date have the proper format?11.3.dClarity1 F1.3.d.4Is the date provided accurate?11.3.dCorrectness1 F1.3.d.5Is it indicated whether the model is continuously trained on new or dynamic data after the date provided? 11.3.dComprehension1 F1.3.d.6If yes, is this information true?11.3.dCorrectness1 1.3.eDescription of the linguistic characteristics of the overall train- ing data F1.3.e.1Are the languages covered by the training data described?11.3.eCompleteness6 F1.3.e.2If yes, are these languages described exactly (i.e. as a list of languages rather than as the number covered)? 11.3.eClarity6 F1.3.e.3Are the EU languages which are covered mentioned?11.3.eCompleteness6 F1.3.e.4Is provided information relating to the linguistic characteristics of the overall training data accurate? 11.3.eCorrectness6 F1.3.e.5Does the field not contain any superfluous information?11.3.eClarity6 1.3.fOther relevant characteristics of the overall training data F1.3.f.1Are national/regional specificities of the training data provided?11.3.fCompleteness6 F1.3.f.2Are demographic specificities of the training data provided?11.3.fCompleteness6 F1.3.f.3Is any other relevant information relating to the characteristics of the overall training data provided? 11.3.fClarity6 F1.3.f.4Is the provided data accurate?11.3.fCorrectness6 F1.3.f.5Is all provided information relevant? In particular, does the field not contain any information which should be provided elsewhere in the template? 11.3.fClarity6 1.3.gAdditional comments (optional) F1.3.g.1Is information regarding the compression methodologies applied for the data size calculation supplied? 11.3.gCompleteness2 F1.3.g.2Is information regarding the tokenization methodologies applied for the data size calculation supplied? 11.3.gCompleteness2 Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)19 IDMetricSection FieldDimensionWeight F1.3.g.3If audio or video content is provided, is information provided regarding the sampling frequency or rate plays? 11.3.gCompleteness2 F1.3.g.4Is any other relevant information disclosed?11.3.gCompleteness2 F1.3.g.5Is all information contained within this field relevant?11.3.gClarity2 2SECTION 2 2.1Publicly available datasets F2.1.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.12.1Consistency3 2.1.aHave you used publicly available datasets to train the model? F2.1.a.1Is this field filled in?2.12.1.aCompleteness1 F2.1.a.2Is this field accurate?2.12.1.aCorrectness1 2.1.bIf yes, specify the modality(ies) of the content covered by the datasets concerned F2.1.b.1Are any modalities supplied?2.12.1.bCompleteness3 F2.1.b.2Are all provided modalities accurate?2.12.1.bCorrectness3 2.1.cList of large publicly available datasets F2.1.c.1Does this field only provide a list of large publicly available datasets, with explanations for selecting part of the datasets where necessary? 2.12.1.cCompleteness20 F2.1.c.2Are no large publicly available datasets which were used left out?2.12.1.cCompleteness20 F2.1.c.3Is a link provided for each identifier/name listed?2.12.1.cAccessibility20 F2.1.c.4If yes, does each identifier/name match the identifier/name of the dataset linked? 2.12.1.cConsistency20 F2.1.c.5Are specific versions of the datasets listed?2.12.1.cClarity20 F2.1.c.6Is a general approach to selecting part of the datasets provided wherever part of a dataset was selected? 2.12.1.cClarity20 F2.1.c.7Are the descriptions clearly understandable for the approaches to se- lecting part of the datasets? 2.12.1.cClarity20 2.1.dGeneral description of other publicly available datasets not listed above F2.1.d.1Are general descriptions provided for publicly available datasets which were used and not listed above? 2.12.1.dCompleteness16 F2.1.d.2Does the field only contain such descriptions?2.12.1.dClarity16 F2.1.d.3Do the descriptions contain information relating to the types of modality contained in the datasets? 2.12.1.dCompleteness16 F2.1.d.4Do the descriptions contain information relating to the nature of the content contained in the datasets? 2.12.1.dCompleteness16 F2.1.d.5Do the descriptions contain information relating to the linguistic char- acteristics of the datasets, where applicable? 2.12.1.dCompleteness16 F2.1.d.6Do the descriptions contain information relating to the approximate start and end dates of the data collection, or otherwise contain that such information is “not known"? 2.12.1.dCompleteness16 F2.1.d.7Do the descriptions contain any other relevant information?2.12.1.dCompleteness16 F2.1.d.8Are the descriptions provided clearly readable, such that someone with- out expertise could understand them? 2.12.1.dComprehension16 F2.1.d.9Do the descriptions limit themselves to relevant information?2.12.1.dClarity16 F2.1.d.10Are all descriptions accurate?2.12.1.dCorrectness16 2.1.eAdditional comments (optional) F2.1.e.1Is information relating to the size of the datasets provided?2.12.1.eCompleteness3 F2.1.e.2Are other relevant details provided in this field?2.12.1.eCompleteness3 20Blankvoort & Pandit & Gahntz IDMetricSection FieldDimensionWeight F2.1.e.3Is all information provided in this field relevant?2.12.1.eComprehension3 F2.1.e.4Is all information provided in this field accurate?2.12.1.eCorrectness3 2.2Private non-publicly available datasets obtained from third parties F2.2.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.22.2Consistency3 2.2.1Datasets commercially licensed by rightsholders or their representatives 2.2.1.aHave you concluded transactional commercial licensing agree- ment(s) with rightholder(s) or with their representatives? F2.2.1.a.1Is this field filled in?2.22.2.1.aCompleteness1 F2.2.1.a.2Is this field accurate?2.22.2.1.aCorrectness1 2.2.1.bIf yes, specify the modality(ies) of the content covered by the datasets concerned F2.2.1.b.1Are any modalities supplied?2.22.2.1.bCompleteness3 F2.2.1.b.2Are all provided modalities accurate?2.22.2.1.bCorrectness3 2.2.2 Private datasets obtained from other third parties 2.2.2.aHave you obtained private datasets from third parties that are not licensed as described in Section 2.2.1., such as data obtained from providers of private databases, or data intermediaries? F2.2.2.a.1Is this field filled in?2.22.2.2.aCompleteness1 F2.2.2.a.2Is this field accurate?2.22.2.2.aCorrectness1 2.2.2.bIf yes, specify the modality(ies) of the content covered by the datasets concerned F2.2.2.b.1Are any modalities supplied?2.22.2.2.bCompleteness3 F2.2.2.b.2Are all provided modalities accurate?2.22.2.2.bCorrectness3 2.2.2.cIf publicly known, list private datasets obtained from other third parties F2.2.2.c.1Does this field only provide a list of publicly known private datasets obtained from other third parties without license? 2.22.2.2.cClarity20 F2.2.2.c.2Are any publicly known private datasets obtained from third parties without license, which were used for this model, left out? 2.22.2.2.cCorrectness20 F2.2.2.c.3Are specific versions of the datasets listed?2.22.2.2.cClarity20 F2.2.2.c.4Are links to relevant information provided?2.22.2.2.cAccessibility20 2.2.2.dGeneral description of non-publicly known private datasets ob- tained from third parties F2.2.2.d.1Are general descriptions provided for all non-publicly known, non- licensed private datasets obtained from third parties which were used? 2.22.2.2.dCompleteness16 F2.2.2.d.2Does the field only contain such descriptions?2.22.2.2.dClarity16 F2.2.2.d.3Do the descriptions contain information relating to the types of modality contained in the datasets? 2.22.2.2.dCompleteness16 F2.2.2.d.4Do the descriptions contain information relating to the nature of the content contained in the datasets? 2.22.2.2.dCompleteness16 F2.2.2.d.5Do the descriptions contain information relating to the linguistic char- acteristics of the datasets, where applicable? 2.22.2.2.dCompleteness16 F2.2.2.d.6Do the descriptions contain any other relevant information?2.22.2.2.dCompleteness16 Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)21 IDMetricSection FieldDimensionWeight F2.2.2.d.7Are the descriptions provided clearly readable, such that someone with- out expertise could understand them? 2.22.2.2.dComprehension16 F2.2.2.d.8Do the descriptions limit themselves to relevant information?2.22.2.2.dClarity16 F2.2.2.d.9Are all descriptions accurate?2.22.2.2.dCorrectness16 2.2.2.e Additional comments (optional) F2.2.2.e.1Is information relating to the period of data collection provided?2.22.2.2.eCompleteness3 F2.2.2.e.2Is information relating to the size of the datasets provided?2.22.2.2.eCompleteness3 F2.2.2.e.3Are other relevant details provided in this field?2.22.2.2.eCompleteness3 F2.2.2.e.4Is all information provided in this field relevant?2.22.2.2.eClarity3 F2.2.2.e.5Is all information provided in this field accurate?2.22.2.2.eCorrectness3 2.3Data crawled and scraped from online sources F2.3.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.32.3Consistency3 2.3.aWere crawlers used by the provider or on behalf of? F2.3.a.1Is this field filled in?2.32.3.aCompleteness3 F2.3.a.2Is this field accurate?2.32.3.aCorrectness3 2.3.bIf yes, specify crawler name(s)/identifier(s) F2.3.b.1Does the field only contain crawler name(s)/identifier(s)?2.32.3.bClarity15 F2.3.b.2Can these name(s)/identifier(s) each be used to identify specific crawlers? 2.32.3.bClarity15 F2.3.b.3Were all crawlers which are listed indeed used?2.32.3.bCorrectness15 F2.3.b.4Are all crawlers which were used described?2.32.3.bCompleteness15 F2.3.b.5Is the information provided in a clearly-structured form?2.32.3.bClarity15 2.3.cPurposes of the crawler(s) F2.3.c.1Are descriptions relating to the purposes of the crawler(s) provided?2.32.3.cCompleteness10 F2.3.c.2Are descriptions provided for each crawler?2.32.3.cCompleteness10 F2.3.c.3Are these descriptions understandable, such that someone without specialised expertise can understand the purposes of the crawlers? 2.32.3.c Comprehension10 F2.3.c.4Does the field only contain information relating to the purposes of the crawlers listed? 2.32.3.cClarity10 F2.3.c.5Are all of the descriptions accurate?2.32.3.cCorrectness10 2.3.dGeneral description of crawler behaviour F2.3.d.1Is a general description of crawler behavior provided?2.32.3.dCompleteness10 F2.3.d.2If a description is provided, does it contain information relating to the respect of captchas by crawlers? 2.32.3.dClarity10 F2.3.d.3If a description is provided, does it contain information relating to the handling of password protected websites by crawlers? 2.32.3.dClarity10 F2.3.d.4If a description is provided, does it contain information relating to the handling of paywalls by crawlers? 2.32.3.dClarity10 F2.3.d.5If a description is provided, does it contain information relating to the respect of robots.txt by crawlers? 2.32.3.dClarity10 F2.3.d.6If a description is provided, does it contain information relating to the respect of protocols outside of robots.txt by crawlers? 2.32.3.dClarity10 F2.3.d.7Does the description contain other relevant information?2.32.3.dCompleteness10 F2.3.d.8Is the description clearly understandable, such that someone without detailed technical knowledge can understand the crawler behavior? 2.32.3.d Comprehension10 F2.3.d.9Does the description limit itself to relevant information?2.32.3.dClarity10 F2.3.d.10Is the description accurate?2.32.3.dCorrectness10 22Blankvoort & Pandit & Gahntz IDMetricSection FieldDimensionWeight 2.3.ePeriod of data collection F2.3.e.1Is a period of data collection provided?2.32.3.eCompleteness7 F2.3.e.2If a period of data collection is provided, is it provided for each crawler?2.32.3.eClarity7 F2.3.e.3If a period of data collection is provided, are exact dates given?2.32.3.eClarity7 F2.3.e.4If exact dates are given, are these in the appropriate format (M/Y)?2.32.3.eCorrectness7 F2.3.e.5Is all information provided accurate?2.32.3.eCorrectness7 2.3.fComprehensive description of the type of content and online sources crawled F2.3.f.1Is information relating to the geographical characteristics of the crawled content provided? 2.32.3.fCompleteness13 F2.3.f.2Is information relating to the linguistic characteristics of the crawled content provided? 2.32.3.fCompleteness13 F2.3.f.3Is information relating to the demographic characteristics of the crawled content provided? 2.32.3.fCompleteness13 F2.3.f.4Is an indication given regarding which type(s) of websites are scraped?2.32.3.fCompleteness13 F2.3.f.5Is any other pertinent information relating to the types of content crawled provided? 2.32.3.fCompleteness13 F2.3.f.6Is all provided information relevant?2.32.3.fComprehension13 F2.3.f.7Is all provided information accurate?2.32.3.fCorrectness13 2.3.gType of modality covered F2.3.g.1Are any modalities supplied?2.32.3.gCompleteness3 F2.3.g.2Are all provided modalities accurate?2.32.3.gCorrectness3 2.3.hSummary of the most relevant domain names crawled F2.3.h.1Is a list of the most relevant internet domains provided as per the requirements? 2.32.3.hCompleteness25 F2.3.h.2Is the provided list accurate?2.32.3.hCorrectness25 F2.3.h.3Is the provided list easily accessible?2.32.3.hAccessibility25 F2.3.h.4Is the provided list provided in a straightforwardly readable format?2.32.3.hComprehension25 2.3.iAdditional comments (optional) F2.3.i.1In this field, are more domains disclosed than those required in the list above? 2.32.3.iClarity20 F2.3.i.2Are the URLs and sources of individual works provided in this field?2.32.3.iClarity20 F2.3.i.3Are other relevant details provided in this field?2.32.3.iClarity20 F2.3.i.4Is all information provided in this field relevant?2.32.3.iComprehension20 F2.3.i.5Is all information provided in this field accurate?2.32.3.iCorrectness20 2.4User data F2.4.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.42.4Consistency3 2.4.aWas data from user interactions with the AI model (e.g. user input and prompts) used to train the model? F2.4.a.1Is this field filled in?2.42.4.aCompleteness3 F2.4.a.2Is this field accurate?2.42.4.aCorrectness3 2.4.bWas data collected from user interactions with the provider's other services or products used to train the model? F2.4.b.1Is this field filled in?2.42.4.bCompleteness5 F2.4.b.2Is this field accurate?2.42.4.bCorrectness5 Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)23 IDMetricSection FieldDimensionWeight 2.4.cIf yes, provide a general description of the provider's services or products that were used to collect the user data F2.4.c.1Does this field only contain the relevant services or products by the provider used to train the model? 2.42.4.cClarity15 F2.4.c.2Are the services or products each clearly identified?2.42.4.cComprehension15 F2.4.c.3Are all involved services or products by the provider listed?2.42.4.cCompleteness15 F2.4.c.4Is the description provided in a readable format?2.42.4.cAccessibility15 2.4.dType of modality covered F2.4.d.1Are any modalities supplied?2.42.4.dCompleteness3 F2.4.d.2Are all provided modalities accurate?2.42.4.dCorrectness3 2.4.eAdditional comments (optional) F2.4.e.1Are relevant details provided in this field?2.42.4.eCompleteness1 F2.4.e.2Is all information provided in this field relevant?2.42.4.eClarity1 F2.4.e.3Is all information provided in this field accurate?2.42.4.eCorrectness1 2.5Synthetic data F2.5.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.52.5Consistency3 2.5.aWas synthetic AI-generated data created by the provider or on their behalf to train the model? F2.5.a.1Is this field filled in?2.52.5.aCompleteness1 F2.5.a.2Is this field accurate?2.52.5.aCorrectness1 2.5.bIf yes, modality of the synthetic data F2.5.b.1Are any modalities supplied?2.52.5.bCompleteness3 F2.5.b.2Are all provided modalities accurate?2.52.5.bCorrectness3 2.5.cIf yes, specify the general-purpose AI model(s) used to generate the synthetic data if available on the market F2.5.c.1If yes was ticked, are models provided?2.52.5.cCompleteness12 F2.5.c.2If models are provided, are these uniquely identifiable?2.52.5.cClarity12 F2.5.c.3Are the correct models provided?2.52.5.cCorrectness12 F2.5.c.4Is a link to the Summary(ies) of the relevant models provided where such a summary is available? 2.52.5.c Comprehension12 F2.5.c.5Does the field only contain the list of GPAI models used and a link to their Summary(ies) where available? 2.52.5.cClarity12 2.5.dInformation about other AI models, includer provider's own AI model(s) not available on the market, used to generate synthetic data to train the model to which this Summary applies F2.5.d.1Is information provided for each AI model used to generate synthetic data which is not available on the market? 2.52.5.dCompleteness12 F2.5.d.2Does the field only contain information about the relevant group of AI models? 2.52.5.dClarity12 F2.5.d.3Does the information provided include a general description of each model's training data if known and necessary as described? 2.52.5.dComprehension12 F2.5.d.4Is any other relevant information about the relevant group of AI models provided? 2.52.5.dClarity12 F2.5.d.5Is all provided information accurate?2.52.5.dCorrectness12 2.5.eAdditional comments (optional) F2.5.e.1Is relevant information provided in this field?2.52.5.eCompleteness1 24Blankvoort & Pandit & Gahntz IDMetricSection FieldDimensionWeight F2.5.e.2Is all information provided in this field relevant?2.52.5.eClarity1 F2.5.e.3Is all information provided in this field accurate?2.52.5.eCorrectness1 2.6Other sources of data F2.6.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 2.52.6Consistency3 2.6.aHave data sources other than those described in Sections 2.1 to 2.5. been used to train the model? F2.6.a.1Is this field filled in?2.52.6.aCompleteness2 F2.6.a.2Is this field accurate?2.52.6.aCorrectness2 2.6.bIf yes, provide a narrative description of these data sources and the data F2.6.b.1Does this field contain relevant information relating to the miscella- neous data sources and their data? 2.52.6.bCompleteness17 F2.6.b.2Does this field restrict itself to relevant information relating to the miscellaneous data sources and their data? 2.52.6.bClarity17 F2.6.b.3Is all information accurate?2.52.6.bCorrectness17 F2.6.b.4Is the information provided in a clearly readable format?2.52.6.bAccessibility17 2.6.cAdditional comments (optional), for other sources of data F2.6.c.1Is relevant information provided in this field?2.52.6.cCompleteness1 F2.6.c.2Is all information provided in this field relevant?2.52.6.cCorrectness1 F2.6.c.3Is all information provided in this field accurate?2.52.6.cAccessibility1 3SECTION 3 F3.1Is the information provided in this field consistent with other sections and other documentation provided for the same model(s)? 33Consistency3 3.1Respect of reservation of rights from text and data mining exception or limitation 3.1.aAre you a Signatory to the Code of Practice for general-purpose AI models that includes commitments to respect reservations of rights from the TDM exception or limitation? F3.1.a.1Is this field filled in?33.1.aCompleteness3 F3.1.a.2Is this field accurate?33.1.aCorrectness3 3.1.bDescribe the measures implemented before model training to respect reservations of rights from the TDM exception or limi- tation before and during data collection, including the opt-out proticols and solutions honoured by the provider or, as applica- ble, by third parties from which datasets have been obtained F3.1.b.1Does the provider describe how they respect robots.txt?33.1.bComprehension18 F3.1.b.2Does the provider describe how they use datasets with opt-out mecha- nisms, if applicable? 33.1.bComprehension18 F3.1.b.3Does the provider describe how they ensure that they are up to date with user rights requests? 33.1.bComprehension18 F3.1.b.4Is all other information relevant?33.1.bClarity18 F3.1.b.5Is all information provided accurate?33.1.bCorrectness18 3.1.cAdditional comments (optional) F3.1.c.1Is a summary of the providers copyright policy provided, if this policy is made publicly available? 33.1.cCompleteness5 Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)25 IDMetricSection FieldDimensionWeight F3.1.c.2Is all other information provided relevant?33.1.cClarity5 3.2Removal of illegal content 3.2.aGeneral decription of measures taken F3.2.a.1Are any measures described to avoid or remove illegal content under Union law from the training data? 33.2.aCompleteness15 F3.2.a.2If measures are described, are these measures described in an easily understandable manner? 33.2.aComprehension15 F3.2.a.3If measures are described, is the information provided accurate?33.2.aCorrectness15 F3.2.a.4Is all information provided pertinent? In particular, does the informa- tion not include an information relating to data selection practices, for example to increase the capability of the model? 33.2.aAccessibility15 3.3Other information (optional) 3.3.aOther relevant information about data processing (optional) F3.3.a.1Is any additional information provided?33.3.aCompleteness1 F3.3.a.2If yes, is this relevant to data processing aspects and measures taken before or after training model that his relevant for the respect and exercise of rights protected under Union law? 33.3.aClarity1 F3.3.a.3If yes, is this information accurate?33.3.aCorrectness1

Instant research discovery