AACsearch
Operations & Reliability

Troubleshooting

A decision tree for diagnosing the most common AACsearch failures before opening a ticket.

Troubleshooting

Most production problems with AACsearch fall into one of seven patterns. This page is a decision tree: start at the top, follow the question that matches, and you'll land at either a fix or a "this is genuinely a support escalation" page with the right diagnostics already gathered.

For escalating after this page, see Support escalation.

The decision tree

Is search returning unexpected results?               → Skip to "Wrong results"
Is search returning 4xx errors?                       → Skip to "Auth / origin / rate-limit errors"
Is search returning 5xx errors?                       → Skip to "Server-side errors"
Is ingest not arriving in search?                     → Skip to "Stale or missing documents"
Are documents arriving but disappearing later?        → Skip to "Documents disappear"
Is a reindex stuck or failed?                         → Skip to "Reindex problems"
Is the dashboard slow or showing wrong numbers?       → Skip to "Dashboard problems"

Wrong results

SymptomMost likely cause
"I searched for X and got nothing"Typo tolerance is off, or the document field isn't in query_by.
"Results are missing a brand-new document"Document is in the ingest buffer but hasn't flushed yet. Check Diagnostics.
"Results include documents I deleted yesterday"Delete is in the buffer but hasn't flushed; reindex is in progress; or the delete was per-document while the application searches a soft-deleted view.
"Results respect the wrong tenant"Scoped token's filterBy doesn't include the tenant field, or application code is sending an OR-filter that broadens scope. See Tenant isolation.
"Sort order changed after reindex"default_sorting_field was changed in the new schema. Check the schema diff in the audit log.
"Synonyms aren't being applied"Synonym set wasn't included in the reindex's synonymSets array. Re-apply.

If the document looks right in the dashboard but searches don't return it, suspect the schema's index: true or the field's absence from query_by. The dashboard stores the document; the engine doesn't necessarily search every field.

Auth / origin / rate-limit errors

HTTPBodyCauseFix
401missing_bearer_tokenNo Authorization header.Send Authorization: Bearer ss_search_…
401invalid_or_revoked_keyKey was revoked, deleted, or is malformed.Check API keys for status. Create a new key if needed.
401invalid_or_expired_scoped_tokenToken's exp is in the past, or HMAC doesn't verify.Mint a fresh token. If the issue persists immediately after minting, the server's BETTER_AUTH_SECRET may have rotated.
403origin_not_allowedRequest's Origin header isn't in the key's allow-list.Add the origin to the key. See Origin allow-list.
403scope_deniedKey doesn't have the scope required for this endpoint.Use a key with the required scope. Search ≠ ingest ≠ admin.
403tenant_mismatchIndex lives in a different organization than the verified key.Make sure the key belongs to the org that owns the index.
429rate_limitedKey's bucket is exhausted.Wait for the next clock minute. See Rate limits.

Server-side errors

HTTPBodyCauseFix
500internal_error + requestIdUnhandled exception. Always escalate with the request ID.Open a ticket with the request ID and timestamp.
502none / HTMLEdge proxy couldn't reach the upstream.Check status. If green, retry; if not, wait for the incident.
503degradedUpstream cluster is degraded; we're rejecting some traffic to recover.Retry with backoff. Status page will show why.
504noneSearch took longer than 10 seconds. Almost always a malformed query.Check the query. A filter_by with no index hint can scan the whole collection.

Stale or missing documents

A document you uploaded N seconds ago doesn't appear in search. The ingest pipeline is:

Your application
     │  POST /api/orpc/searchDocument.bulkUpsert

SearchIngestBuffer (PostgreSQL row)
     │  Worker picks it up

Typesense bulkUpsert
     │  Document searchable

Audit + analytics

Where the document might be stuck:

  1. Application → API — your request returned 4xx and you didn't retry.
  2. API → buffer — buffer row exists but processedAt is null (worker hasn't run yet).
  3. Worker → Typesense — buffer row has failedAt and an error message.

Diagnostics:

  1. Dashboard → Diagnostics — find the document by docId and read the timeline.
  2. Buffer status — the dashboard surfaces this; the procedure is searchIndex.getIngestStatus.
  3. Retry — the retry worker re-attempts failed rows up to 5 times. If a row has attempts >= 5 and failedAt set, the message in lastError is your fix.

If everything in the buffer says "succeeded" and the document still doesn't appear, suspect Typesense memory pressure or a swap that completed but didn't pick up recent writes — open a ticket with the document ID.

Documents disappear

A document was searchable yesterday and isn't today. Possibilities, in order:

  1. You deleted it. Audit log: action = bulk_delete_documents or delete_documents_by_filter.
  2. A reindex dropped it. A reindex pulls from your source of truth. If the document is gone from your source, the reindex won't carry it forward.
  3. A connector deleted it. PrestaShop/Bitrix/Shopify connectors mirror your CMS. A delete in the CMS becomes a delete here.
  4. Typesense restart from snapshot. Extremely rare. If you suspect this, request a backup verification report.

Action: find the audit log row for the deletion. If you don't see one, it's option 4 — escalate.

Reindex problems

SymptomCause / fix
Reindex appears stuck at X %Progress is updated every 1 000 documents. For a 10 M index, a "stuck at 50 %" reading may mean it's actively pushing 4 999 docs. Wait 60 s, refresh.
Reindex completes but new fields are missingSource query didn't include them. Check the source-of-truth payload.
Reindex completes but doc count is lower than expectedSource filter excludes deleted/inactive rows you wanted to include.
Reindex fails halfwaySee Reindexing — When reindex fails.

Dashboard problems

SymptomCause / fix
Sign-in loops back to sign-in pageBrowser blocking third-party cookies for the auth subdomain. Allow cookies for app.aacsearch.com.
404 on a project you can access from a colleagueYou're in the wrong organization. Use the org switcher.
Indexes list is emptySame as above, or the key's project isn't selected. Check the project switcher.
Analytics shows different numbers than the APIAnalytics is eventually consistent — up to 5 minutes lag.

When to escalate

If your problem isn't in this tree, or you've followed the tree and reached a "this is genuinely a support issue" leaf, open a ticket per Support escalation. Specifically include:

  1. The request ID (x-request-id header) from a failed request.
  2. The timestamp in UTC.
  3. The organization ID, project ID, and index slug.
  4. What you've already checked from this page.

See also

On this page