Roadmap 0.10, Performance roadmap — wave 3

PERF.20 Scale: every text format reader holds the whole file as one JS string (DXF, LandXML, OBJ, DAE, ASCII STL, USDA, the native JSON documents)

  • Fix
  • Planned
  • CadCraft
  • ModelCraft
  • CivilCraft

Measured: a 1 GB DXF and a 692 MB LandXML die in TextDecoder.decode ("Cannot create a string longer than 0x1fffffe8"); a 358 MB LandXML needs 4.9 GB and 27 s; a 10 MB DXF needs 364 MB (38 bytes of heap per file byte, parsed twice through dxf-model.js:65). Sites: dxf-decode.js:35-49 and dxf.js:68-74, civilcraft.js:2485 and xml.js:27, modelcraft io/obj.js:84, and JSON.parse(await file.text()) for every .modelcraft/.detailcraft/.civilcraft document (modelcraft.js:3210). The caps that exist (CivilCraft 200 MB, ModelCraft 200 MB, OBJ 64 MB) are honest and stay until the readers stream. Change: byte-level tokenizers over Uint8Array fed from Blob.slice with lazily decoded values and Float64Array/Uint32Array output, the way lib/ifc scanStep already works. DXF first (the reader is 1,563 lines, L), then LandXML (M), OBJ and ASCII STL (S each). Target: 100 MB DXF under 1 GB of memory, 1 GB files open with a progress bar instead of an error.

Add a comment

Community comments

No comments yet

Add detail as we go: how you would use it, files or steps that show the problem, what other programs do. The team reads every comment.

Sign in or create a free account to add a comment.