Roadmap 0.10, Performance roadmap — wave 3
PERF.50 Website: the Library at 1000x (1 M items, 2.8 M versions, 10 M files): what breaks and the design that does not
Scaling the measured plans linearly from 13,846 items / 27,940 versions / 108k files / 41k metric rows: browse sort plus COUNT plus categoryCounts 64+30+30 ms become 4.6+2.2+2.2 s per page (breaks at about 100x); LIKE search 52 ms becomes 3.7 s while FULLTEXT stays tens of ms (index about 1 GB at 1 M items with descriptions: budget it); the per-card N+1 stays about 450 statements per page but 450 round trips is the floor of every page; publicStats / inUse / makers / forUser full reads 42 ms become 3 s each; possibleDuplicates 159 ms becomes 16 s per /admin/library view; library_metrics at 2 M rows a day needs the daily rollup and a 90-day raw retention; /sitemap (PERF.11) must become a sitemap index of 50k-URL files written nightly, never rendered per request; hot data 5 to 8 GB against a 1 GB server-wide buffer pool (the MySQL settings row). Fine as is: PK and unique lookups (uq_slug, uq_uuid, uq_sha256, uq_item_version), int unsigned ids, the content-addressed file store, the events feed and the notification mailer (idx_feed, idx_mail keep them O(result)). This row is the design pass that ties the Library rows together and sets the data budget before the data arrives (docs/0.9-sessions/perf-wave3/sql-audit.md 12).
Community comments
No comments yet
Add detail as we go: how you would use it, files or steps that show the problem, what other programs do. The team reads every comment.
Sign in or create a free account to add a comment.