Deployment history
Every deploy of a project keeps a timeline. It records when the deploy was requested, when the new version could take calls, when new calls moved over to it, and when the previous version finished its calls and stopped. Each step has a timestamp from the VoiceThere servers, shown in UTC to the second.
Timelines are kept for 90 days. After that they are deleted, and the deploy itself stays in the list with its start, end and status only.
What each step means
These are the names you see in the dashboard, the API and the CLI.
| Step | What happened |
|---|---|
| Deploy requested | VoiceThere accepted your deploy request. |
| Deploy refused | The request was turned away. The reason is shown next to the step; see the two reasons below. |
| Rollout started | VoiceThere began starting the new version. |
| New version ready | The new version is up and can take calls. |
| New calls go to the new version | From this moment, new sessions start on the new version. |
| Previous version finishing its calls | Calls already in progress stay on the previous version until they end on their own. |
| Calls on the previous version ended | A deploy with Force ended the calls that were still running on the previous version. The step shows how many runners were affected. |
| Ending calls on the previous version failed | A deploy with Force tried to end those calls and could not. The error is shown next to the step. |
| Deploy finished | The rollout is done. With a drain deploy the previous version can still be finishing calls at this point. |
| Deploy failed | The rollout did not complete. The reason is shown under the step; see When a deploy fails. |
| Previous version stopped | The last call on the previous version ended and it shut down. The step shows how long that took after the new version took over. |
A drain deploy keeps going after it finishes: the previous version stays up until its last call ends, so Previous version stopped can appear minutes after Deploy finished. A long call makes that gap long. A deploy with Force ends those calls instead of waiting for them.
When a deploy fails
A failed deploy says whose problem it is, so you know whether to change your agent or wait for us. The reason appears under the failed step on Deployments and in the Cloud rollout panel on the project Overview. In the table, a failed deploy has a small label next to its status: Fix needed, Refused or On our side. The API returns the same reason as failure, and the CLI prints it too.
- Fix needed in your agent. The cause is in your code or your configuration. The notice links to the agent logs from the time you requested the deploy, which is where the error is. The previous version keeps serving calls, so nothing goes down while you fix it. See How a deploy checks your agent.
- Deploy refused. An earlier deploy is still working, so VoiceThere turned yours away. Nothing is wrong with your agent, and the notice has no logs link. See Refused deploys.
- Problem on our side. Nothing you changed caused it. The notice has an Open a ticket button that starts an email to support@voicethere.io.
- Retrying. Some problems on our side pass by themselves, such as a shortage of capacity. VoiceThere runs the deploy again on its own, and the notice reads Retrying (attempt 3 of 9)while it does. You don't need to do anything. If the last attempt fails, the deploy ends as one of the two kinds above.
Failure codes
Every failed deploy carries one of these codes, in the API as failure.code and in the ticket email.
| Code | Kind | What it means | What to do |
|---|---|---|---|
AGENT_START_FAILED | Fix needed | Your agent threw an error while it was loading, so the new version never started. | The previous version keeps serving calls, and a first deploy stays not deployed. Open the agent logs, fix the error and deploy again. |
PREVIOUS_VERSION_DRAINING | Refused | An earlier version is still finishing its calls, so the deploy was refused. | Deploy again once it has stopped (up to 9 minutes), or deploy with Force. |
FORCED_ROLLOUT_SHUTTING_DOWN | Refused | A forced deploy is still ending calls on older versions. | Try again in a few seconds. |
CLUSTER_CAPACITY | On our side | We didn't have room to start your new version. | We retry on our own. If the last attempt fails too, open a ticket from the failed deploy. |
IMAGE_PULL_FAILED | On our side | We couldn't load the software your new version needs to start. | We retry on our own. If the last attempt fails too, open a ticket from the failed deploy. |
RUNNER_START_FAILED | On our side | The new version stopped while starting, for a reason that isn't in your agent. | Open a ticket from the failed deploy. |
READY_TIMEOUT | On our side | The new version started but didn't become ready in time. | We retry on our own. If the last attempt fails too, open a ticket from the failed deploy. |
BUILD_FILES_MISSING | On our side | We couldn't find the files of the build you deployed. | Build again and deploy the new build. If that fails too, open a ticket from the failed deploy. |
DEPLOY_STEP_FAILED | On our side | A step of the rollout failed on our side. | Some of these are retried on their own. If the deploy still ends as failed, open a ticket. |
DEPLOY_QUEUE_STALLED | On our side | The deploy didn't finish in the time we expect. | Deploy again. If it happens again, open a ticket. |
DEPLOY_FAILED | On our side | The deploy failed and we couldn't tell more. Older deploys without a code show this too. | Deploy again. If it fails again, open a ticket. |
What the ticket button sends
Open a ticket opens your email app with a message to support@voicethere.io already filled in. It holds the organization, project, deployment and build ids, the deploy mode, the failure code and its message, the times the deploy was requested, started and failed in UTC, a trace id, and a link to the deployment in the dashboard. It never includes your logs, your code, your secrets or your environment variables. You can read and edit the email before you send it.
In the CLI
voicethere deploy --wait (CLI 0.16.8 and later) prints the same reason when a deploy fails. For a failure in your agent it points you to the agent logs. For a failure on our side it prints the support address and the ids to put in your email.
How a deploy checks your agent
Before a new version takes calls, VoiceThere loads your agent once, the way it loads it for a call: in the same sandbox and with the project's environment variables. The check covers loading, so it finds a module that throws while it is imported or while its top-level code runs.
Where the check runs depends on the deploy. When a deploy starts a new copy of your agent (the warm pool is on, or calls are live during a redeploy), that copy checks itself while it starts. When a deploy starts none, because the project scales to zero and no call is live, VoiceThere runs a short one-off check first. Those deploys take a few seconds longer, 90 seconds at most.
What the check decides
- The agent loads. The deploy goes on.
- The agent throws while loading. The deploy fails with
AGENT_START_FAILED, shown as Fix needed in your agent. The previous version keeps serving existing and new calls, and the error text is in the agent logs. - The check is unclear. It was slow to start, or it could not start at all. The deploy goes on, there is nothing for you to fix, and calls behave as they did before.
A first deploy that fails the check leaves the project not deployed, so no call reaches a broken version. Deploy a fixed build to bring the project up.
Check it before you deploy
You can reproduce the load error on your machine with npm run verify:start, which loads the bundle in the same sandbox the platform uses. Projects created by the CLI or from a template already have that script.
What the check does not cover
Code that fails only during a call, such as inside a handler, is not caught here, so make a test call once a deploy of changed agent code has finished. And because the check loads your agent, its top-level code runs one extra time for each check. Keep side effects, such as writes to an outside service, out of the top level of your module and inside your handlers.
Refused deploys
A deploy can be turned away while an earlier one is still working. The step Deploy refused carries one of two reasons:
- The previous version is still finishing its calls. Try again when it stops, or deploy with Force.
- A forced deploy is still ending calls on the previous version. Try again in a few seconds.
Who started a deploy
Each deploy shows who requested it. A person who deployed from the dashboard or the CLI appears by email. A deploy made with a personal API key appears under the key's name, and one made with an organization API key under that key's name. This is the same text the Audit log shows for the deploy, and the Audit log is the complete record of who did what: Deployments links each deploy to its entries there (the deploy request, a refusal, and the finished or failed result), and each of those entries links back with View deploy. A deploy from before the Audit log recorded it shows no name.
Deploys without a full timeline
Deploys from before the history was recorded have no stored steps. For those, VoiceThere shows the request and the result, taken from the deploy itself, and says that only the start and end are recorded.
Dashboard
Open Project → Deployments (in the Build group, next to Code), or use Deployment history on the project Overview. The table lists deploys newest first with the start time in UTC, the build, the mode, the status, how long the deploy took, who requested it, and the last recorded step. Open a row to see its timeline, with the time of each step, the time since the request, and small chips for counts, reasons and durations. Hover a time to see it in your local zone. Load older goes back further than the first 20 deploys.
CLI
Available in @voicethere/cli 0.16.6 and later. See the CLI reference for every option and example output.
voicethere deployments list prints the same table, and voicethere deployments show <id> prints one timeline. From CLI 0.16.7 the list has a By column and the timeline starts with a Requested by line, so you can see who started each deploy. Both use the project in .voicethere/config.json, or --project <id>. Use --limit and --before to page through the list, and --json to get the API response unchanged.
voicethere deployments list --limit 20 voicethere deployments show <deploymentId>
API
Both endpoints take the same API key as the rest of the control plane API and only return deploys of the project in the path.
GET /api/v1/projects/{projectId}/deployments?limit=20&before=<ISO time>
GET /api/v1/projects/{projectId}/deployments/{deploymentId}/events- The list returns
deployments(id, build_id, mode, status, error, created_at, completed_at, failure, trace_id, started_at, last_event_at, event_count, requested_by) andnext_before.limitis 1 to 50 and defaults to 20. Passnext_beforeasbeforeto read the next page; it is null on the last page.requested_byhas akind(user, api_key, personal_api_key, system or unknown) and alabel. failureis null unless the deploy failed. When it is set it hasclass(customer or platform),codeandmessagefrom the table above, plussupport_emailfor a failure on our side.GET /api/v1/deployments/{id}also returnsfailurewhile a deploy is being retried, with classtransient,attemptandmax_attempts. Readfailurerather thanerror;errorstays for older clients and its text can change.- The events endpoint returns the
deployment, itsevents(id, type, occurred_at, details) in order, andderived, which is true when only the start and end are shown. - A deploy that is not in that project gets a 404, even if its id is valid in another project.