Systima Comply: EU AI Act Compliance Scanning for CI/CD Pipelines

Consider this example:

A pull request lands on a Tuesday afternoon. It adds a new feature to your lending platform; an AI-powered affordability check that calls GPT-5.4, scores the applicant, and writes the result to Postgres.

The developer who wrote it is excellent. The code is clean, the tests pass, and the review approves it by Wednesday morning.

Nobody notices that this change just made the system high-risk under Annex III, Category 5(b) of the EU AI Act ("AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score, with the exception of AI systems used for the purpose of detecting financial fraud").

Nobody notices because there is no mechanism to notice:

• The linter does not know about Article 6.
• The test suite does not check for Article 12 logging.
• The CI pipeline does not care whether AI output flows into a database that influences decisions about natural persons.

By the time a compliance review catches it, the feature has been in production for four months, the architecture has calcified around it, and retrofitting the seven engineering obligations from Articles 9 through 15 is now a project, not a task.

This is the problem that Systima Comply solves.

What Comply does

Comply is an open-source static analysis tool that scans your codebase for AI framework usage, validates your declared risk classification against what it finds in the code, checks your EU AI Act obligation compliance, and reports the results directly in your pull requests.

• It runs in CI.
• It requires no API keys, no external services, no telemetry.
• It is fully deterministic; the same codebase and the same configuration produce identical results every time.

Think of it as the regulatory equivalent of what Snyk does for dependency vulnerabilities or what ESLint does for code quality; except the rules come from the EU AI Act, not a style guide.

It ships as an npm package (@systima/comply) with a CLI, a GitHub Action (systima-ai/comply@v1), and a programmatic TypeScript API. It is Apache 2.0 licensed.

What it detects

Comply's scanner has four layers.

Import scanning

Using the TypeScript Compiler API for TypeScript and JavaScript, and web-tree-sitter for Python, Comply performs AST-based import detection across your codebase.

It recognises 37 AI/ML frameworks:

• LLM providers (OpenAI, Anthropic, Google Generative AI, Cohere, Mistral, Groq, Replicate, AWS Bedrock, Azure OpenAI).
• ML frameworks (TensorFlow, PyTorch, scikit-learn, XGBoost, LightGBM).
• Agentic frameworks (LangChain, LlamaIndex, AutoGen, CrewAI, Mastra, Haystack, DSPy).
• Computer vision libraries (face_recognition, DeepFace, InsightFace, MediaPipe).
• NLP/embeddings libraries (Hugging Face Transformers, spaCy, Sentence Transformers).
• and AI infrastructure (Vercel AI SDK, MLflow, Weights & Biases).

Every import is traced to a specific file and line number.

Dependency scanning

Comply parses package.json, requirements.txt, pyproject.toml, Pipfile, and setup.cfg to detect AI frameworks declared as dependencies, including development dependencies (which are flagged but not treated as production AI usage).

Configuration scanning

Environment files (.env, .env.production), Docker Compose configurations, and Terraform modules are scanned for AI service references: API keys (OPENAI_API_KEY, ANTHROPIC_API_KEY), container images (tensorflow/serving, vllm), and cloud AI resources (aws_sagemaker, google_vertex_ai, azurerm_openai).

Call-chain analysis

This is where Comply goes beyond simple detection. It traces the return values of AI API calls through your code: following assignments, destructuring, property access chains, and function returns within each file. It identifies four regulated decision patterns:

• Conditional branching: AI output used in if statements or switch cases (potential automated decision-making)
• Database persistence: AI output written to a database (potential scoring or classification of natural persons)
• UI rendering: AI output rendered to a user interface without disclosure (Article 50 transparency gap)
• Downstream API calls: AI output passed to another service (integration into a regulated decision chain)

These patterns are the ones that determine whether your system is merely using AI or is making decisions with AI; a distinction that Article 6 of the EU AI Act treats as the boundary between limited-risk and high-risk.

Risk-tiered reporting

Comply understands context through the domain field in its configuration.

When you declare your system, you specify both its risk level and its domain:

systems:
  - id: support-chatbot
    name: "Customer Support Chatbot"
    classification:
      risk_level: limited
      domain: customer_support

Non-regulated domains (general_purpose, customer_support, internal_tooling, content_generation) cause call-chain findings to be treated as informational advisory notes.

Regulated domains (creditworthiness, employment, insurance, education, legal, law_enforcement, migration, biometric) cause the same findings to be treated as critical.

The compliance score is scoped accordingly:

A limited-risk system is only assessed against the obligations that actually apply to it:

• Article 5 (prohibited practices).
• Article 50 (transparency).

High-risk-only obligations from Articles 9 through 15 are shown in a collapsible advisory section; visible for the interested reader, but not counting against the score or creating false urgency.

The result is that the same scanning engine produces contextually appropriate output depending on what the system does, not just what frameworks it imports.

A real scan: Vercel's AI chatbot

To demonstrate what this looks like in practice, we ran Comply against Vercel's official AI chatbot template; a 19,800-star Next.js project using the Vercel AI SDK (ai, @ai-sdk/gateway, @ai-sdk/react).

We declared it as a limited-risk customer support chatbot and ran the scan via the GitHub Action.

The PR comment:

✅ 1 system(s) scanned | 2/2 obligations met | Score: 100%

Your Obligations: AI Chatbot (limited-risk, customer_support)

| Article | Obligation                              | Status |
|---------|-----------------------------------------|--------|
| Art. 5  | No prohibited practice indicators       | ✅ Pass |
| Art. 50 | AI interaction disclosure detected      | ✅ Pass |

Both applicable obligations passed:

• Article 5 passed because the chatbot uses no biometric or social scoring frameworks.
• Article 50 passed because Vercel's chatbot already has AI disclosure patterns in its UI code.

Below the obligation table, a collapsible advisory section noted:

• Three call-chain patterns: AI output persisted to a database in lib/ai/tools/create-document.ts:35, and conditional branching on AI output in app/(chat)/api/chat/route.ts at lines 259 and 261
• Five documentation gaps that would apply if the system were reclassified as high-risk (Articles 9, 10, 11, 13, 14)

These are informational because the system is declared as customer_support.

If someone forked this chatbot and turned it into a credit assessment tool, changing domain: customer_support to domain: creditworthiness would immediately surface those same patterns as critical findings, the score would drop, and the PR would fail.

That is the point. The compliance posture changes when the use case changes, not when the code changes. Comply encodes this distinction.

The seven commands

Comply ships with a CLI that covers the full compliance workflow:

comply scan runs the full analysis and outputs results to the terminal, a file, or a PR comment. Supports JSON, SARIF (for GitHub Code Scanning integration), Markdown, and plain text output. The --fail-on flag controls the exit code: none, warning, fail, or critical.

comply init interactively creates a .systima.yml configuration file. It scans your codebase first, reports which AI frameworks it detected, then asks you about your organisation, system name, risk level, domain, and scope paths. For CI environments, --template generates a commented template without interactive prompts.

comply scaffold generates template documentation files for every path declared in your .systima.yml. Each template includes the section headings required by the corresponding EU AI Act article: risk identification, risk estimation, risk evaluation, and risk mitigation for Article 9; data sources, data quality, and bias for Article 10; and so on. The templates are starting points, not finished documents; but they tell you exactly what each article requires.

comply doctor validates your .systima.yml without running a full scan. It checks YAML syntax, schema validity, whether scope patterns match any files, whether documentation paths exist, whether Annex III categories are consistent with risk levels, and whether any system scopes overlap.

comply report --format pdf generates a branded A4 compliance report suitable for legal and compliance teams. It includes an executive summary with the overall compliance score, per-system obligation tables with colour-coded status cells, detected framework lists, remediation guidance, and a disclaimer. The PDF is generated locally via pdfmake; no external service involved.

comply baseline and comply diff support compliance tracking over time. Save a baseline after your first clean scan, then compare subsequent scans against it to see new findings, resolved findings, classification changes, and compliance score deltas.

Ecosystem integration

Comply does not exist in isolation. It is the detection and reporting layer in a three-part open-source compliance stack:

@systima/aiact-audit-log provides Article 12 compliant structured, tamper-evident audit logging with Vercel AI SDK middleware. When Comply scans a codebase that has @systima/aiact-audit-log installed, it recognises the package and reports Article 12 as passing. When the package is absent, the remediation guidance includes the exact npm install command and a link to the setup documentation.

@systima/aiact-docs generates Annex IV technical documentation by scanning your codebase and filling in a structured template. When Comply finds documentation gaps, the remediation text suggests running npx @systima/aiact-docs generate to auto-generate the missing files.

The three tools are independent; you can use any of them alone. But together, they form a workflow: Comply detects what is missing, aiact-docs generates the documentation, and aiact-audit-log provides the logging infrastructure. The gap between "scan" and "fix" becomes a single command.

The GitHub Action

For most teams, the primary interface with Comply will be the GitHub Action. Add this workflow to your repository:

name: EU AI Act Compliance
on:
  pull_request:
    types: [opened, synchronize, reopened]
permissions:
  contents: read
  pull-requests: write
jobs:
  compliance:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: systima-ai/comply@v1
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        with:
          fail-on: warning
          output-format: comment

Every pull request receives a compliance comment that is updated on each push. The comment shows your obligations, your score, any critical findings, and the collapsible advisory section. The Action outputs compliance-score, risk-level, findings-count, and badge-url for use in downstream steps. It writes a job summary to the Actions tab. It can optionally generate a compliance badge SVG for your README.

The scan completes in under 10 seconds on a 20,000-star repository. It runs on ubuntu-latest with Node 20. It requires no secrets beyond the default GITHUB_TOKEN.

What Comply is not

Comply is a static analysis tool, not a legal opinion. It checks what it can observe in your codebase: imports, dependencies, configuration files, call chains, documentation existence, and content structure.

It cannot determine whether your risk classification is correct in the regulatory sense; that requires understanding the full deployment context, the affected population, and the nature of the decisions being made.

It cannot audit your data governance practices, evaluate your model's accuracy, or assess whether your human oversight procedures are effective.

What it can do is ensure that the engineering foundations are in place: that the right frameworks are declared, that documentation exists where it should, that logging infrastructure is present, that AI disclosure is in the UI, and that the code patterns in your repository are consistent with your declared risk classification.

It catches the Tuesday afternoon pull request that quietly changes your system's risk profile before it becomes a four-month-old architectural problem.

Getting started

npx @systima/comply init
npx @systima/comply scan

The source code is on GitHub. The npm package is published. The GitHub Action is available as systima-ai/comply@v1.

For a comprehensive treatment of the engineering obligations Comply checks against, see our EU AI Act Engineering Compliance Guide. For the specific Article 12 logging requirements and how @systima/aiact-audit-log addresses them, see Article 12 for Engineers.

If your team is building AI systems that will be subject to the EU AI Act and you need help designing the compliance architecture, Systima's AI Governance and Compliance practice works with engineering teams to design and implement the full stack.