gRPC API

CommonLayer offers a gRPC API, package commonlayer.v1, for machine clients: CI pipelines, clctl and integrations. It calls the same application services as the REST API, so data, validation and authorization are identical; only the encoding differs. The browser portal uses REST.

The API reference is the set of protobuf files in proto/commonlayer/v1/, one file per service plus common.proto. Every service, RPC, message, field and enumeration value carries a comment, and every RPC names its access rule and the REST operation it corresponds to. The running server also describes itself through server reflection. There are 12 services and 59 RPCs: the 58 of the gRPC work package plus RegistryService.GetPullStats, restored in WP-70b once a pull-statistics read existed.

What was tested: every service has integration tests against an in-process server and a real database (crates/commonlayer-grpc/tests/it/, 25 tests plus the pull-statistics test, and 8 unit tests), run locally and by ci / rust on GitHub runners; a manual run used generated clients and clctl, because grpcurl could not be installed in the build sandbox. The grpcurl examples below follow the reflection service's documented behaviour and have not been run. No deployed endpoint behind TLS has been called.

Endpoint

The API process serves gRPC on server.grpc_addr (default 0.0.0.0:50051; the development backend uses localhost:50051) next to REST on server.http_addr. Both start together and stop together on SIGINT/SIGTERM: the health service switches to NOT_SERVING, in-flight calls finish within server.shutdown_grace_secs, and the process exits. Deployments terminate TLS in front of the API (ingress or load balancer with HTTP/2 to the backend). The Helm chart exposes it as the api Service's grpc port (appProtocol: kubernetes.io/h2c) and, optionally, through its own Ingress host (ingress.grpc.*); that was rendered and unit-tested, not installed on a cluster. In the Compose stack it is published on CL_STACK_GRPC_PORT (default 50051).

Limits:

• request messages are limited to server.body_limit_bytes (1 MiB by default, the same as REST bodies); larger requests fail with OUT_OF_RANGE before they are decoded;
• responses are limited to 16 MiB;
• each call is limited to server.request_timeout_secs (30 s by default) and fails with CANCELLED/DEADLINE_EXCEEDED beyond it; a client's own grpc-timeout applies when shorter.

Services

The server also runs the standard grpc.health.v1.Health service (the empty service name and each of the twelve services above report SERVING) and server reflection (grpc.reflection.v1 and grpc.reflection.v1alpha).

Operations that need an interactive session — creating or revoking API tokens, password and email changes, settings, retention plans and preservation status (platform.settings.manage), payout approval and resolution, ledger adjustments, maintainer assignment approval, appeal resolution and eligibility overrides — have no RPC or are refused over gRPC: API tokens can never hold those permissions or make those decisions (see what an API token can never do and the permission matrix). Checkout stays a browser flow with the payment provider.

Authentication

Send an API token in the authorization metadata:

authorization: Bearer clt_<prefix>_<secret>

Create tokens in the portal (or POST /api/v1/users/me/tokens with a session). The token's scopes intersect with its owner's roles exactly as for REST (read, artifacts:submit, projects:write, security:write, full, …), and a token may be restricted to one project or organization. WhoAmI returns the effective permissions per scope.

• No authorization metadata: the call is anonymous. RPCs marked "Access: public" in the proto files work anonymously (with the public projection only); every other RPC fails with UNAUTHENTICATED (auth.required), where REST answers 401.
• A clt_… token that is unknown, expired, revoked or belongs to a suspended account fails every call with UNAUTHENTICATED (auth.invalid_token), public RPCs included.
• Authorization happens in the application services, as for REST: a caller without the permission gets PERMISSION_DENIED (authz.forbidden), or NOT_FOUND where the resource must not be revealed (non-public projects, private organizations, other users' notifications). The gRPC layer adds no checks of its own and removes none.
• GitHub Actions OIDC tokens are accepted by the REST submission endpoints only; over gRPC, CI submits with an API token holding artifacts:submit.

Rate limits are the REST API's buckets, shared between both transports: authenticated_api per principal and anonymous_api per client IP (failed token authentications count against the anonymous bucket), plus the submission buckets on SubmitArtifact (submissions_per_project for callers allowed to submit, keyed by the resolved project; submissions_denied per principal for everyone else). A limited call fails with RESOURCE_EXHAUSTED (rate_limited), a google.rpc.RetryInfo detail and retry-after metadata in seconds. The client IP is the TCP peer, or the right-most untrusted x-forwarded-for entry when the peer is in server.trusted_proxies. Health and reflection need no token and are not limited.

Every response carries x-request-id metadata: the caller's own value when it sends a well-formed x-request-id (1–128 characters of [A-Za-z0-9._-]), otherwise a generated one. It appears in the server's logs, traces and audit entries.

Error model

Errors use the standard status codes, mapped from the platform's error kinds:

The status message is the human-readable message. Machine-readable details travel in two places:

1. Status details (grpc-status-details-bin, a google.rpc.Status), readable with any rich-error library (tonic-types, google.golang.org/genproto/googleapis/rpc, grpcio-status, …):
   • google.rpc.ErrorInfo — reason is the stable CommonLayer error code (funding.insufficient_funds, project.not_found, validation.failed, …), domain is commonlayer.dev;
   • google.rpc.BadRequest — one field violation per invalid field: field (proto field path, page.page_size, update_mask, preferences.0.channel), reason (field code such as required, invalid, too_long, out_of_range) and description;
   • google.rpc.RetryInfo — for rate limits and other retry hints;
   • google.rpc.RequestInfo — request_id of the call.
2. Metadata for clients without rich-error support: commonlayer-error-code (the stable code) and, with a retry hint, retry-after (seconds).

Enumerations: every enum has an …_UNSPECIFIED = 0 value. Sending it (or an unknown number) where a value is required is INVALID_ARGUMENT naming the field. A server never sends UNSPECIFIED for a known value; a client that receives a number it does not know (a newer server) must treat it as unknown rather than fail.

Pagination

List RPCs take PageRequest page and return PageResponse page:

• page_size: 1–100, 0 means the default (25), more than 100 is INVALID_ARGUMENT (page.page_size);
• page_token: empty for the first page, then the previous response's next_page_token;
• next_page_token: empty on the last page.

Tokens are opaque, bound to the request's other parameters, and must not be constructed or stored long-term. Lists that REST pages by keyset (projects, findings, cycles, ledger, audit log, jobs, notifications, …) use the same cursors. Lists that are bounded by nature and returned complete by the service (a project's artifacts or versions, an organization's members, the caller's tokens or memberships, rewards, a digest's vulnerabilities) are paged by the gRPC layer with offset tokens; clients treat both the same way.

Calling the API with grpcurl

Reflection makes the proto files unnecessary on the client side (grpcurl):

# what is there
grpcurl -plaintext localhost:50051 list
grpcurl -plaintext localhost:50051 describe commonlayer.v1.ProjectService.GetProject
grpcurl -plaintext localhost:50051 describe commonlayer.v1.GetProjectResponse

# health
grpcurl -plaintext -d '{"service": "commonlayer.v1.ProjectService"}' \
  localhost:50051 grpc.health.v1.Health/Check

# public calls need no token
grpcurl -plaintext -d '{"slug": "nginx"}' localhost:50051 commonlayer.v1.ProjectService/GetProject
grpcurl -plaintext -d '{"page": {"page_size": 10}, "sort": "PROJECT_SORT_NAME"}' \
  localhost:50051 commonlayer.v1.ProjectService/ListProjects

# authenticated calls send the API token
grpcurl -plaintext -H "authorization: Bearer $CLCTL_TOKEN" \
  localhost:50051 commonlayer.v1.AuthService/WhoAmI

# rich error details are printed with -v (or decode grpc-status-details-bin)
grpcurl -plaintext -v -d '{"username": "nobody"}' localhost:50051 commonlayer.v1.UserService/GetUser

Behind TLS, drop -plaintext. Without reflection access (or to generate clients), use the files in proto/ directly: grpcurl -import-path proto -proto commonlayer/v1/projects.proto ….

clctl

• clctl submit --grpc --project <slug> --file submission.json [--wait] submits a built digest with ArtifactService.SubmitArtifact and polls GetSubmission; the JSON body is the REST SubmitArtifact document. The token comes from CLCTL_TOKEN (--token-env).
• clctl ledger verify --remote runs AdminService.VerifyLedger on a running server (needs platform.ledger.read) instead of connecting to the database.
• Both take --grpc-url (or CLCTL_GRPC_URL; default http://<server.grpc_addr> with an unspecified address replaced by loopback). https:// endpoints use TLS with the system's root certificates.

Compatibility policy

commonlayer.v1 changes only compatibly:

• fields, enum values, messages, RPCs and services are added, never renamed, renumbered, retyped or removed; a field that must go is marked deprecated and its number and name are reserved before removal in a future major version;
• new enum values may appear in responses at any time (see "Error model" for how clients treat unknown values); new optional request fields default to the previous behaviour;
• a new error code may be added; the gRPC status code of an existing situation does not change;
• an incompatible change ships as commonlayer.v2 alongside v1, and v1 keeps working for at least two minor releases after v2 is published (announced in the changelog).

proto/buf.yaml configures buf lint (STANDARD and COMMENTS rules) and buf breaking (FILE rules). The ci workflow runs buf lint proto on every push and, on pull requests, buf breaking against the pull request's base commit:

buf lint proto
buf breaking proto --against '.git#branch=main,subdir=proto'   # locally, against main

The Rust code is generated during cargo build by crates/commonlayer-proto/build.rs with protox; no system protoc is needed.

Observability

Each call is counted in commonlayer_grpc_requests_total{service, method, code} and timed in commonlayer_grpc_request_duration_seconds{service, method} (labels only for methods the server serves; anything else is unknown), and runs in a grpc.request span with rpc.service, rpc.method, request_id, principal and code (metrics).