Order search times out unless the search text is common

Component: Orders screen search (support's guess)

From: Dana K., customer support, Cartwheel · Received: Tue 27 Oct, 9:21 am (UTC)

Hi, for a while now I use the search box on the staff Orders screen to find a customer's order when they write in ("I ordered a mug three weeks ago, where is it"). It used to be fine; I think it got worse as the shop grew. Now it only works some of the time. If I type something that lots of orders have, like "ceramic mug" or a common surname, the results come back at once. If I type something specific, like part of one customer's email address, or the code of a product we hardly sell, the screen spins for half a minute and then says Gateway timeout. Same when I mistype and nothing should match. That is exactly when I need it: a specific customer, a specific product.

This morning I wrote down what I typed and what happened (attached), and Ibrahim in ops pulled the access log for the same minutes. The shop has roughly 60,000 orders now. The other screens are fine, only the search.

Thanks, Dana

Support note (Ibrahim Y.): I can confirm the 504s come from the proxy giving up after 30 seconds (the production machine is slower than a laptop and busy in the mornings), the shop server itself is not down and the same screens answer in well under 100 ms. Nobody deployed anything to the shop this week except a new marketing banner. The search only works on orders (customer, item name or SKU, order number).

Ask: find out why searching for something rare or something that matches nothing is so much slower than searching for something common, and fix it, so that every search answers quickly on a shop of this size without changing which orders are found or how they are listed. Everything else in the shop must keep working exactly as today.