Deployment history

Every deploy of a project keeps a timeline. It records when the deploy was requested, when the new version could take calls, when new calls moved over to it, and when the previous version finished its calls and stopped. Each step has a timestamp from the VoiceThere servers, shown in UTC to the second.

Timelines are kept for 90 days. After that they are deleted, and the deploy itself stays in the list with its start, end and status only.

What each step means

These are the names you see in the dashboard, the API and the CLI.

StepWhat happened
Deploy requestedVoiceThere accepted your deploy request.
Deploy refusedThe request was turned away. The reason is shown next to the step; see the two reasons below.
Rollout startedVoiceThere began starting the new version.
New version readyThe new version is up and can take calls.
New calls go to the new versionFrom this moment, new sessions start on the new version.
Previous version finishing its callsCalls already in progress stay on the previous version until they end on their own.
Calls on the previous version endedA deploy with Force ended the calls that were still running on the previous version. The step shows how many runners were affected.
Ending calls on the previous version failedA deploy with Force tried to end those calls and could not. The error is shown next to the step.
Deploy finishedThe rollout is done. With a drain deploy the previous version can still be finishing calls at this point.
Deploy failedThe rollout did not complete. The reason is shown under the step; see When a deploy fails.
Previous version stoppedThe last call on the previous version ended and it shut down. The step shows how long that took after the new version took over.

A drain deploy keeps going after it finishes: the previous version stays up until its last call ends, so Previous version stopped can appear minutes after Deploy finished. A long call makes that gap long. A deploy with Force ends those calls instead of waiting for them.

When a deploy fails

A failed deploy says whose problem it is, so you know whether to change your agent or wait for us. The reason appears under the failed step on Deployments and in the Cloud rollout panel on the project Overview. In the table, a failed deploy has a small label next to its status: Fix needed, Refused or On our side. The API returns the same reason as failure, and the CLI prints it too.

  • Fix needed in your agent. The cause is in your code or your configuration. The notice links to the agent logs from the time you requested the deploy, which is where the error is. The previous version keeps serving calls, so nothing goes down while you fix it. See How a deploy checks your agent.
  • Deploy refused. An earlier deploy is still working, so VoiceThere turned yours away. Nothing is wrong with your agent, and the notice has no logs link. See Refused deploys.
  • Problem on our side. Nothing you changed caused it. The notice has an Open a ticket button that starts an email to support@voicethere.io.
  • Retrying. Some problems on our side pass by themselves, such as a shortage of capacity. VoiceThere runs the deploy again on its own, and the notice reads Retrying (attempt 3 of 9)while it does. You don't need to do anything. If the last attempt fails, the deploy ends as one of the two kinds above.

Failure codes

Every failed deploy carries one of these codes, in the API as failure.code and in the ticket email.

CodeKindWhat it meansWhat to do
AGENT_START_FAILEDFix neededYour agent threw an error while it was loading, so the new version never started.The previous version keeps serving calls, and a first deploy stays not deployed. Open the agent logs, fix the error and deploy again.
PREVIOUS_VERSION_DRAININGRefusedAn earlier version is still finishing its calls, so the deploy was refused.Deploy again once it has stopped (up to 9 minutes), or deploy with Force.
FORCED_ROLLOUT_SHUTTING_DOWNRefusedA forced deploy is still ending calls on older versions.Try again in a few seconds.
CLUSTER_CAPACITYOn our sideWe didn't have room to start your new version.We retry on our own. If the last attempt fails too, open a ticket from the failed deploy.
IMAGE_PULL_FAILEDOn our sideWe couldn't load the software your new version needs to start.We retry on our own. If the last attempt fails too, open a ticket from the failed deploy.
RUNNER_START_FAILEDOn our sideThe new version stopped while starting, for a reason that isn't in your agent.Open a ticket from the failed deploy.
READY_TIMEOUTOn our sideThe new version started but didn't become ready in time.We retry on our own. If the last attempt fails too, open a ticket from the failed deploy.
BUILD_FILES_MISSINGOn our sideWe couldn't find the files of the build you deployed.Build again and deploy the new build. If that fails too, open a ticket from the failed deploy.
DEPLOY_STEP_FAILEDOn our sideA step of the rollout failed on our side.Some of these are retried on their own. If the deploy still ends as failed, open a ticket.
DEPLOY_QUEUE_STALLEDOn our sideThe deploy didn't finish in the time we expect.Deploy again. If it happens again, open a ticket.
DEPLOY_FAILEDOn our sideThe deploy failed and we couldn't tell more. Older deploys without a code show this too.Deploy again. If it fails again, open a ticket.

What the ticket button sends

Open a ticket opens your email app with a message to support@voicethere.io already filled in. It holds the organization, project, deployment and build ids, the deploy mode, the failure code and its message, the times the deploy was requested, started and failed in UTC, a trace id, and a link to the deployment in the dashboard. It never includes your logs, your code, your secrets or your environment variables. You can read and edit the email before you send it.

In the CLI

voicethere deploy --wait (CLI 0.16.8 and later) prints the same reason when a deploy fails. For a failure in your agent it points you to the agent logs. For a failure on our side it prints the support address and the ids to put in your email.

How a deploy checks your agent

Before a new version takes calls, VoiceThere loads your agent once, the way it loads it for a call: in the same sandbox and with the project's environment variables. The check covers loading, so it finds a module that throws while it is imported or while its top-level code runs.

Where the check runs depends on the deploy. When a deploy starts a new copy of your agent (the warm pool is on, or calls are live during a redeploy), that copy checks itself while it starts. When a deploy starts none, because the project scales to zero and no call is live, VoiceThere runs a short one-off check first. Those deploys take a few seconds longer, 90 seconds at most.

What the check decides

  • The agent loads. The deploy goes on.
  • The agent throws while loading. The deploy fails with AGENT_START_FAILED, shown as Fix needed in your agent. The previous version keeps serving existing and new calls, and the error text is in the agent logs.
  • The check is unclear. It was slow to start, or it could not start at all. The deploy goes on, there is nothing for you to fix, and calls behave as they did before.

A first deploy that fails the check leaves the project not deployed, so no call reaches a broken version. Deploy a fixed build to bring the project up.

Check it before you deploy

You can reproduce the load error on your machine with npm run verify:start, which loads the bundle in the same sandbox the platform uses. Projects created by the CLI or from a template already have that script.

What the check does not cover

Code that fails only during a call, such as inside a handler, is not caught here, so make a test call once a deploy of changed agent code has finished. And because the check loads your agent, its top-level code runs one extra time for each check. Keep side effects, such as writes to an outside service, out of the top level of your module and inside your handlers.

Refused deploys

A deploy can be turned away while an earlier one is still working. The step Deploy refused carries one of two reasons:

  • The previous version is still finishing its calls. Try again when it stops, or deploy with Force.
  • A forced deploy is still ending calls on the previous version. Try again in a few seconds.

Who started a deploy

Each deploy shows who requested it. A person who deployed from the dashboard or the CLI appears by email. A deploy made with a personal API key appears under the key's name, and one made with an organization API key under that key's name. This is the same text the Audit log shows for the deploy, and the Audit log is the complete record of who did what: Deployments links each deploy to its entries there (the deploy request, a refusal, and the finished or failed result), and each of those entries links back with View deploy. A deploy from before the Audit log recorded it shows no name.

Deploys without a full timeline

Deploys from before the history was recorded have no stored steps. For those, VoiceThere shows the request and the result, taken from the deploy itself, and says that only the start and end are recorded.

Dashboard

Open Project → Deployments (in the Build group, next to Code), or use Deployment history on the project Overview. The table lists deploys newest first with the start time in UTC, the build, the mode, the status, how long the deploy took, who requested it, and the last recorded step. Open a row to see its timeline, with the time of each step, the time since the request, and small chips for counts, reasons and durations. Hover a time to see it in your local zone. Load older goes back further than the first 20 deploys.

CLI

Available in @voicethere/cli 0.16.6 and later. See the CLI reference for every option and example output.

voicethere deployments list prints the same table, and voicethere deployments show <id> prints one timeline. From CLI 0.16.7 the list has a By column and the timeline starts with a Requested by line, so you can see who started each deploy. Both use the project in .voicethere/config.json, or --project <id>. Use --limit and --before to page through the list, and --json to get the API response unchanged.

voicethere deployments list --limit 20
voicethere deployments show <deploymentId>

API

Both endpoints take the same API key as the rest of the control plane API and only return deploys of the project in the path.

GET /api/v1/projects/{projectId}/deployments?limit=20&before=<ISO time>
GET /api/v1/projects/{projectId}/deployments/{deploymentId}/events
  • The list returns deployments (id, build_id, mode, status, error, created_at, completed_at, failure, trace_id, started_at, last_event_at, event_count, requested_by) and next_before. limit is 1 to 50 and defaults to 20. Pass next_before as before to read the next page; it is null on the last page. requested_by has a kind (user, api_key, personal_api_key, system or unknown) and a label.
  • failure is null unless the deploy failed. When it is set it has class (customer or platform), code and message from the table above, plus support_email for a failure on our side. GET /api/v1/deployments/{id} also returns failure while a deploy is being retried, with class transient, attempt and max_attempts. Read failure rather than error; error stays for older clients and its text can change.
  • The events endpoint returns the deployment, its events (id, type, occurred_at, details) in order, and derived, which is true when only the start and end are shown.
  • A deploy that is not in that project gets a 404, even if its id is valid in another project.

← All documentation