How PDF Layout Detection & Spatial Parsing Works

PDF files are not formatted as text streams with paragraphs or lines. A PDF is an explicit 2D graphical display instruction program. Text characters are drawn at arbitrary coordinate points (e.g., 'Draw glyph at X=72, Y=540'). The PDF specification does not define where a paragraph begins, whether two words belong to the same sentence, or what constitutes a table cell.

Text scaling in PDFs is defined by a 2x2 affine transformation matrix [scaleX, skewY, skewX, scaleY]. To compute the actual rendered font size in points, our engine calculates the Euclidean norm: fontSize = sqrt(scaleX^2 + skewY^2). This allows us to deterministically map rendered font sizes to # H1, ## H2, and ### H3 headings.

The engine clusters characters along the vertical Y-axis using a ±5px proximity tolerance. Glyphs sharing the same baseline are sorted left-to-right along the X-axis. If the horizontal gap between characters exceeds the average glyph width, a whitespace delimiter is inserted.

The parser computes a 1D histogram of text box coordinates projected onto the X-axis. Valleys with zero character density identify vertical column gutters. The parser then traverses each column sequentially from top to bottom before moving to the next column.