Audiobook Producer
7f0b598441
Root cause: clean-up found paragraph breaks from the layout indent of pdftotext -layout
(columns, relative to the smallest indent in the whole chapter). Chapter 1 happened to
have a clean left margin; later pages are skewed and drift 1-2 columns, drop caps push
the first lines right by varying amounts, and some pages wrap text around a label, so
continuation lines were taken as paragraph starts (a blank line inside a sentence).
Fix (all deterministic, all logged to warnings.log):
- pdftext: read each line's position in points (pdftotext -bbox-layout) and attach it to
the layout line it belongs to (units() now carries 'geom'; merged/partial lines get none).
- cleanup: a paragraph starts when a line sits >= 4.5 pt right of the line before it, or
>= 2.5 pt lower than the page's normal line pitch. A line without a position or starting
in lower case never starts a paragraph. Text wrapped beside a drop cap is not a start.
Without positions the old indent rule is still used.
- cleanup: drop cap + small caps on ONE layout line ('F OR MANY YEARS') is repaired like
the two-line form (chapters 4 and 5).
- pageparts.header_zone: the running header is found even when text follows it without a
blank line, and the right-hand header (chapter title, page number sometimes misread as
't5' or 'a') is recognised from the chapter title.
- cleanup: all-capitals lines below the text are captions (removed); an all-capitals line
above the text is kept as a section heading in title case ('The Deferred Preface');
'Right: MARK TWIN' style captions removed; '5. JOY' labels set inside a sentence removed.
- cleanup: paragraph starting with a lone quote mark + lower-case word joins the paragraph
before it; paragraph-leading ' + capital becomes '"' (not elisions such as 'Tis); a bullet
in place of an opening quote becomes '"'; closing quote with no opening quote, and a '1'
that is probably the letter I, are logged only.
- cleanup: mid-line 'worth- while' joined only when certain, otherwise logged.
- tests: new _pipeline/tests/test_cleanup.py (36 tests, 27 fail on the previous code);
test_pipeline.py helper test_piece renamed make_test_piece so pytest stops collecting it.
Chapter 1 text is byte-identical to the approved pilot text.
Re-run for chapters 1-6: cleanup, figures, script, license. Text, script, warnings and
book.json for chapters 2-6 added.
150 lines
3.0 KiB
JSON
150 lines
3.0 KiB
JSON
{
|
|
"title": "The Man Who Tapped the Secrets of the Universe",
|
|
"author": "Glenn Clark",
|
|
"source_pdf": "The Man Who Tapped the Secrets of the Universe - by Glenn Clark (Walter Russell).pdf",
|
|
"sha256": "84900fe216252a084989694ca578f9ce2ea605a9c1d6a3121e747d8e2198dad8",
|
|
"pdf_pages": 34,
|
|
"pages_without_text_layer": [
|
|
1,
|
|
18
|
|
],
|
|
"coauthors": [],
|
|
"publisher": "",
|
|
"edition": "",
|
|
"series": "",
|
|
"summaries": {},
|
|
"chapters": [
|
|
{
|
|
"number": 1,
|
|
"title": "We Go Seeking",
|
|
"first_unit": 7,
|
|
"last_unit": 9,
|
|
"pdf_pages": [
|
|
4,
|
|
5
|
|
],
|
|
"book_pages": [
|
|
1,
|
|
3
|
|
],
|
|
"switches": {},
|
|
"toc_page": 1,
|
|
"text_file": "text/chapter-01.txt",
|
|
"words": 754,
|
|
"paragraphs": 16
|
|
},
|
|
{
|
|
"number": 2,
|
|
"title": "We Meet the Man",
|
|
"first_unit": 10,
|
|
"last_unit": 16,
|
|
"pdf_pages": [
|
|
6,
|
|
9
|
|
],
|
|
"book_pages": [
|
|
4,
|
|
10
|
|
],
|
|
"switches": {},
|
|
"toc_page": 4,
|
|
"text_file": "text/chapter-02.txt",
|
|
"words": 1940,
|
|
"paragraphs": 29
|
|
},
|
|
{
|
|
"number": 3,
|
|
"title": "We Meet the Artist",
|
|
"first_unit": 17,
|
|
"last_unit": 29,
|
|
"pdf_pages": [
|
|
9,
|
|
15
|
|
],
|
|
"book_pages": [
|
|
11,
|
|
23
|
|
],
|
|
"switches": {},
|
|
"toc_page": 11,
|
|
"text_file": "text/chapter-03.txt",
|
|
"words": 3439,
|
|
"paragraphs": 51
|
|
},
|
|
{
|
|
"number": 4,
|
|
"title": "We Meet the Man of Action",
|
|
"first_unit": 30,
|
|
"last_unit": 42,
|
|
"pdf_pages": [
|
|
16,
|
|
22
|
|
],
|
|
"book_pages": [
|
|
24,
|
|
36
|
|
],
|
|
"switches": {},
|
|
"toc_page": 24,
|
|
"text_file": "text/chapter-04.txt",
|
|
"words": 3411,
|
|
"paragraphs": 51
|
|
},
|
|
{
|
|
"number": 5,
|
|
"title": "The Five Laws of Success",
|
|
"first_unit": 43,
|
|
"last_unit": 59,
|
|
"pdf_pages": [
|
|
22,
|
|
30
|
|
],
|
|
"book_pages": [
|
|
37,
|
|
53
|
|
],
|
|
"switches": {},
|
|
"toc_page": 37,
|
|
"text_file": "text/chapter-05.txt",
|
|
"words": 4717,
|
|
"paragraphs": 67
|
|
},
|
|
{
|
|
"number": 6,
|
|
"title": "The Vision From the Studio",
|
|
"first_unit": 60,
|
|
"last_unit": 67,
|
|
"pdf_pages": [
|
|
31,
|
|
34
|
|
],
|
|
"book_pages": [
|
|
54,
|
|
61
|
|
],
|
|
"switches": {},
|
|
"toc_page": 54,
|
|
"text_file": "text/chapter-06.txt",
|
|
"words": 1285,
|
|
"paragraphs": 15
|
|
}
|
|
],
|
|
"audio_files": [
|
|
"opening-credits",
|
|
"chapter-01",
|
|
"chapter-02",
|
|
"chapter-03",
|
|
"chapter-04",
|
|
"chapter-05",
|
|
"chapter-06",
|
|
"closing-credits",
|
|
"retail-sample"
|
|
],
|
|
"published_year": 1946,
|
|
"license": {
|
|
"status": "unknown",
|
|
"reason": "published 1946 (book text, pdf p.2: \"First Published in 1946\"), later than 1930, so the date rule does not clear it; confirm before sharing",
|
|
"checked": "2026-10-17"
|
|
}
|
|
}
|