Two ways the failure shows up#
The deployment health check starts one instance of the new revision and waits for its startup probe to pass. If the probe fails, Cloud Run marks the revision unhealthy and routes no traffic to it, so the previous revision keeps serving. Deployment health check
- The deploy fails. Cloud Run's troubleshooting page records two wordings. One is
Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable.The other isRevision REVISION_NAME is not ready and cannot serve traffic. The user provided container failed to start and listen on port defined by PORT=8080 environment variable within the allocated timeout. This can happen if the port is misconfigured or if the timeout is too short. The healthcheck timeout can be extended. - The deploy succeeded, and requests fail later. A revision deployed with
--no-deploy-health-checkfirst runs its startup probe when a request or a minimum-instances setting starts an instance. Requests that need that instance return HTTP 500 withThe request failed because the instance could not start successfully.Cloud Run names a Secret Manager secret the container cannot read as a common cause. Troubleshoot Cloud Run ·--deploy-health-check
Both messages name the port, but a container that never reached its listener produces them as well. The question to settle is whether the new container cannot start, or whether it starts and then waits on something it calls before it listens. When the deploy failed its health check, a rollback of traffic is not the answer, because the old revision still has the traffic. When the deploy skipped the health check and traffic moved, the failing revision may already be serving; if the previous revision's instances still start and serve, move traffic back to it first (test the previous revision with a tag). The answer decides who fixes it, and whether the old revision is exposed too: if a startup dependency is broken, the old revision's next new instance will wait on it in the same way.
What the startup probe allows#
The ingress container must listen on 0.0.0.0, not 127.0.0.1, on the port Cloud Run passes in the PORT environment variable, 8080 by default. Without an explicit startup probe, Cloud Run configures a TCP probe with timeoutSeconds: 240, periodSeconds: 240 and failureThreshold: 1. A configured probe may run for failureThreshold times periodSeconds, at most 600 seconds, before Cloud Run shuts the container down. Container runtime contract · Default TCP startup probe
Read the failed revision's probe and port before reading its logs, since a custom HTTP probe changes what "ready" means:
gcloud run revisions describe REVISION \
--project PROJECT --region REGION \
--format='yaml(status.conditions, status.imageDigest, spec.containers[0].ports, spec.containers[0].startupProbe)'
The revision's conditions include ContainerHealthy, which turns true when the readiness check completes, and status.imageDigest records the digest Cloud Run resolved when it created the revision. Revision fields
Check 1: read the new revision's logs in order#
Cloud Run attaches the revision to every container and request log entry, and each container entry carries labels.instanceId. System entries in varlog/system record when an instance started and why; the deployment health check's instance starts with the reason DEPLOYMENT_ROLLOUT. Read from a minute before the deploy, oldest first. --freshness works only with descending order, so put the start time in the filter. Cloud Run logs · gcloud logging read
gcloud logging read \
'resource.type="cloud_run_revision"
AND resource.labels.location="REGION"
AND resource.labels.service_name="SERVICE"
AND resource.labels.revision_name="REVISION"
AND timestamp>="START_TIME"' \
--project PROJECT --order=asc --limit=300 \
--format='value(timestamp, logName, labels.instanceId, textPayload, jsonPayload.message)'
Two facts matter: the last thing the instance wrote, and how long after its start the failure came.
| What the logs show | Time from instance start to failure | Points to |
|---|---|---|
| The process exits with an error or stack trace | Seconds | The container: its code, image or a setting the code requires |
The process says it listens on another port, or on 127.0.0.1 | The whole probe window | The container's port binding |
| The process logs a connection to a database, secret or API, then nothing | The whole probe window | A call the container makes before it listens |
| No container output, only system entries | Varies | The image cannot run, or a secret exposed as an environment variable could not be read, in which case the instance does not start |
A failure that takes the whole probe window and stops at an outbound call is the case the next check exists for. The call might hang because the new code calls something new, with a new client setting, or because the dependency stopped answering new connections for everyone.
Check 2: start the previous image under today's conditions#
Take the digest from the last revision that served, then deploy that image as a new revision that receives no traffic. Every new revision copies the service's current configuration and changes only what the command sets, so this revision runs the old code with the new revision's environment, secrets and network settings. Settings carry to later revisions · Roll out with no traffic
gcloud run revisions describe PREVIOUS_REVISION \
--project PROJECT --region REGION --format='value(status.imageDigest)'
gcloud run deploy SERVICE \
--project PROJECT --region REGION \
--image IMAGE_URL@sha256:DIGEST --no-traffic --tag old-image
This is a deploy, not a read. --no-traffic pins the traffic that followed the latest revision to the revision that was latest before the deploy, and later deploys stop sending traffic to the latest revision until you run gcloud run services update-traffic SERVICE --to-latest. Remove the tag with --remove-tags old-image when you finish. --no-traffic · Remove a tag
| Old image under today's configuration | Reading |
|---|---|
| Starts and passes its probe | The new image is the cause, provided both instances read the same secret values: both reference the same pinned versions, or the last change to each latest secret's versions, which version history and Cloud Audit Logs record, came well before the failing instance started, since reads through latest can take minutes to converge. Otherwise a secret repaired between the two starts, or still converging, looks like a bad image. Fix the release. The old revision keeps its own configuration, so before relying on it to scale, check that its secret references and startup settings still resolve, for example that a pinned secret version is still enabled. |
| Hangs at the same outbound call and fails after the same window | Usually not the new code: either the dependency stopped accepting new connections, or a configuration change both revisions now share is to blame. The exception is startup work that already changed the dependency, such as a migration holding a lock or retrying instances holding connections; check the dependency's sessions and locks from the failing instance's start, and its connection attempts from the service, which a startup retry loop raises without adding sessions. |
To split the second row, compare the failed revision's configuration with the last good one. If nothing but the image differs, the dependency changed. Diff the two revisions
A worked fixture: one error message, two causes#
The fixture is illustrative. A fictional service checkout runs checkout-00041. The deploy of checkout-00042 starts at 14:00:05 UTC, and both incidents end with the same not-ready error.
Incident A. Logs for checkout-00042:
| Time (UTC) | Log | Entry |
|---|---|---|
| 14:00:09 | varlog/system | New instance started, reason DEPLOYMENT_ROLLOUT |
| 14:00:10 | stderr | config: required setting PAYMENTS_URL is empty |
The process wrote nothing after that line, about one second after the instance started. The old image deployed with --no-traffic as checkout-00043 logs listening on 0.0.0.0:8080 three seconds after it starts and passes its probe. The revision's secrets have had no version created, disabled or enabled since the previous day, so both instances read the same values. The new code needs a setting the revision never defined. Fix the release. checkout-00041 references the same secrets through latest and the same network settings, so it can keep serving and scaling.
Incident B. Same service, same deploy:
| Time (UTC) | Log | Entry |
|---|---|---|
| 14:00:09 | varlog/system | New instance started, reason DEPLOYMENT_ROLLOUT |
| 14:00:11 | stdout | connecting to orders-db at 10.20.0.5:5432 |
| 14:04:09 | varlog/system | The default TCP startup probe failed and the instance was shut down |
The process waited 240 seconds at its database connection, the length of the default probe. The old image as checkout-00043 stops at the same line and fails at the same interval, and the failed revision's configuration matches checkout-00041 except for the image. checkout-00041 still serves because its instances connected hours ago. The database's sessions show no locks or extra connections from checkout-00042's instance. The database, or the network path to it, is not completing new connections. Look at its connection count and the network path from the service before you retry the deploy, and treat any scale-out of checkout-00041 as at risk. Timeouts only from new tasks or revisions · Connection budgets during rolling deployments
Where the checks mislead#
Starting the old image runs its startup code against production. If that code applies migrations, warms caches by writing, or registers itself somewhere, run it only when that is acceptable, or use the application's own switch, if it has one, to skip that step in this revision.
A custom startup probe that calls the database moves the dependency into the probe itself. The container may be listening, and the revision still fails because the probe's check fails. Read the probe's path before you read the timing. Health checks that call a dependency
A secret exposed as an environment variable is read before the instance starts, so a missing or disabled version fails the instance with no container output, for old and new images alike. That pattern follows when instances start, not which revision they belong to. A secret rotated during the rollout
One clean start of the old image does not prove the dependency healthy. An intermittent fault can let one connection through. If the old image starts but the new one fails slowly at the same call, look for what the new code does differently with that call, such as a shorter timeout, a different host or more connections at startup. For the wider method, see bad release or unhealthy dependency.
Questions#
Why does Cloud Run say the container failed to listen on PORT when my port is correct?
The message reports what the startup probe saw: nothing accepted a connection on the port within the probe window. A container that is still waiting on a database or API call before it opens its listener produces the same message as one bound to the wrong port. The logs and the time from instance start to failure tell them apart.
How long does Cloud Run wait for a new revision to start?
Without an explicit startup probe, Cloud Run uses a TCP probe with a 240-second timeout, a 240-second period and a failure threshold of 1. A configured probe can allow up to 600 seconds, the product of its failure threshold and period.
Does a failed Cloud Run deploy take the service down?
No. With the deployment health check on, a revision that fails its startup probe receives no traffic and the previous revision keeps serving. Its new instances run the same startup calls, though, so a broken dependency can still affect it when it scales out.