Managing ClusterNest Managed Prometheus Credentials

Every cluster has three static Basic-auth credentials, each scoped to a different slice of the API:

Username	Access
admin	Everything the other two can do, plus rule management and the alertmanager config API/UI.
readonly	Queries and rule evaluation state only (GET/POST on /prometheus/* and the legacy /api/prom/api/v1/*). Can't push, manage rules, or reach the alertmanager config API.
write	Pushing metrics only (POST on /api/v1/push and the legacy /api/prom/push). Can't query anything back.

Hand write to a remote-write agent and readonly to anything that only needs to query (a Grafana datasource, a dashboard). Reserve admin for cluster administration - it's the only one of the three that can manage alert rules or reach the alertmanager config API, see Alert rules & Alertmanager.

Fetching credentials

Console

Shown once, automatically, right when the cluster finishes creating - save them then. The console has no way to view them again later; use Terraform or the raw API for that.

Terraform

data "clusternest_prometheus_credentials" "metrics" {
  cluster_id = clusternest_prometheus.metrics.id
}

output "write_password" {
  value     = data.clusternest_prometheus_credentials.metrics.write
  sensitive = true
}

See the data source reference for every field.

Raw API

curl https://api.clusternest.com/cluster/prometheus/$CLUSTER_ID/credentials \
  -H "Authorization: Bearer $ACCESS_TOKEN"

Rotating credentials

Rotation generates a fresh password for all three users at once. It isn't instantaneous - the new config has to propagate to the cluster (a Kubernetes ConfigMap update, then a file reload inside the proxy) - so expect roughly 30 seconds where the old passwords may still work and the new ones may not yet, then a clean cutover to the new set. Update every client before rotating, and don't assume the swap has landed until you've confirmed the new password actually works.

curl -X POST https://api.clusternest.com/cluster/prometheus/$CLUSTER_ID/credentials \
  -H "Authorization: Bearer $ACCESS_TOKEN"

The response has the same shape as the fetch above, with new passwords. Raw API only - there's no console or Terraform action for rotation.

An alternative to rotation: JWT federation

If static passwords are awkward to distribute and rotate (a fleet of remote-write agents, a Kubernetes workload), your own identity provider - or Kubernetes cluster's own apiserver - can authenticate directly instead, with no shared secret to rotate at all. See JWT/OIDC federation.