Skip to content

Fix schema parsing of complexType children - #231

Open
kflemin wants to merge 3 commits into
developfrom
fix/ref-named-complex-type-children
Open

kflemin wants to merge 3 commits into
developfrom
fix/ref-named-complex-type-children

Conversation

@kflemin

@kflemin kflemin commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #229: the schema viewer showed no children for FanBased, TimeSeries, CalculationMethod and PortfolioManager. These elements are used via ref="auc:X", and the global X has a named complex type. The parser only followed that type when it was a simple type (an enumeration).

This fix adds a new branch in schema_parser.py:456 that also walks a named complex type. Only those four elements gain rows; nothing existing is removed. For 2.7.0 that’s 296 new rows.

Consequences:

  • Every live version is affected. In 2.4.0–2.7.0 all four elements are broken. In 1.0.0–2.3.1 only CalculationMethod is.
    The fix shows up only after a re-parse. Each version’s tree is stored in the database, so deploying the code alone changes nothing.
  • The existing re-parse routes are unsafe:
    reset_schema deletes the Schema row, and its use cases go with it.
    Unchecking “schema parsed” in the admin builds a second copy of the tree.
  • The new reparse_schema command re-parses safely. It keeps the Schema row, use cases and uploaded file. It carries BEDES mappings over by attribute path and enumeration value, since the database IDs change. A rehearsal on 2.7.0 carried over all 1947 term mappings and all 6131 enumeration mappings.
    The committed BEDES CSVs will now be stale and will need to be regenerated for posterity.

kflemin and others added 2 commits November 11, 2026 11:14
The schema tree viewer showed no children for elements used via
ref="auc:X" when the global element X is declared as
<xs:element name="X" type="auc:XType"/> and XType is a named
xs:complexType (e.g. FanBased, TimeSeries, CalculationMethod,
PortfolioManager). _walk_named_element only followed the element's
type when it resolved to a simpleType.

- Walk the named complexType in _walk_named_element
- Add a parser test with a minimal XSD comparing a ref'd element to a
  local name+type element of the same type

Fixes #229

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kflemin
kflemin requested a balanced review from Copilot November 17, 2026 22:24
@kflemin kflemin added the bug Something isn't working label Nov 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The reparse command overwrites live generated HTML even during --dry-run, creating filesystem changes that database rollback cannot undo and risking truncation if the write fails.

3 open issues
What changed in this PR

Fixes #229 by expanding referenced elements with named complex types, allowing their children to appear in the schema viewer. Adds a reparse command to rebuild stored trees while preserving schema records, use cases, uploaded files, and existing BEDES mappings.

Changes:

  • Extend complex-type traversal and add focused regression tests.
  • Add transactional reparsing with mapping restoration, reporting, and a dry-run option.
  • Update Docker startup and document reparsing and manual BEDES loading.
File Description
README.md Documents reparsing, preservation behavior, and BEDES CSV handling.
docker/​start_server.sh Imports the default schema and removes automatic BEDES loading.
bsyncviewer/​tests/​test_schema_parser.py Tests referenced complex-type children, paths, levels, and indexes.
bsyncviewer/​tests/​test_reparse_schema.py Tests rebuilding, mapping restoration, dry runs, and invalid versions.
bsyncviewer/​tests/​data/​test_ref_complex_type.xsd Provides a minimal referenced complex-type fixture.
bsyncviewer/​management/​commands/​reparse_schema.py Adds tree rebuilding and BEDES mapping restoration.
bsyncviewer/​lib/​schema_parser.py Traverses named complex types reached through element references.

Review effort: Balanced

Comment on lines +121 to +123
# 3. Re-parse: the Schema post_save signal parses the file when schema_parsed is False
schema.schema_parsed = False
schema.save()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the reparse command to call process_schema() directly and save with schema_parsed=True, avoiding documentation generation during dry runs. Commit: a9eeb28.

Comment thread README.md Outdated
python manage.py reparse_schema --schema_version 2.7.0 --schema_version 2.6.0
python manage.py reparse_schema --all
```
Re-parsing gives every attribute and enumeration a new database ID. The BEDES CSV files in `bsyncviewer/lib/bedes/` reference those IDs, so after a re-parse they no longer match the database and `bedes --save_to_db` skips their rows. The mappings already in the database are carried over, so the CSVs only need regenerating (with `bedes --schema_version <version> --bedes_version v2.5`) to bring the files back in sync or to map attributes the re-parse added.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the guidance to warn that BEDES mapping CSVs must be regenerated against the target database before loading. Commit: a9eeb28.

Comment thread docker/start_server.sh Outdated
Comment on lines +14 to +16
# BEDES mappings are not loaded here: the CSVs in bsyncviewer/lib/bedes reference database
# IDs from the database they were generated in, so they only match that database. Load
# them manually with `./manage.py bedes --schema_version=<version> --save_to_db` if needed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the startup guidance with regeneration and loading commands using --bedes_version=v2.5 for schema 2.7.0. Commit: a9eeb28.

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Schema viewer omits children of ref'd global elements whose type is a named complexType

3 participants