Ingestion and the neutral IR

Hardware design files come in many formats: Altium and Cadence binaries, IPC-2581 and ODB++ interchange files, KiCad s-expressions, netlists, SPICE. Each vendor spells the same ideas differently. Agni has readers for EDIF, KiCad, IPC-2581, ODB++, Cadence Allegro and Altium Designer boards, xschem, gEDA and Telesis today. They normalize these formats into one neutral intermediate representation (IR), and everything else works from that. This page covers how the reading works and what the IR looks like once a design is in it.

Many readers, one IR ¶

The architecture is the one a compiler or a document converter uses. A compiler like LLVM has many frontends that all lower into one IR, then many backends that emit from it. Pandoc does the same for documents: every input format parses into one central AST, and every output format writes from that AST. Agni does the same for hardware designs. It has one reader per format, each reader normalizes into a single IR, and writers emit back out of it.

The reader layer is heterogeneous by necessity, because the formats are. Entries marked (planned) have no reader yet:

Format class	Examples	Reader approach
Binary natives	Cadence Allegro .brd (16.6, 17.2); Altium Designer .PcbDoc (binary); OrCAD Capture .DSN	a binary deserializer over a reverse-engineered layout (inside a Microsoft compound file for Altium and Capture), held to an extract, export or netlist the vendor’s tool wrote from the same design
Text grammars	EDIF, KiCad s-expr, xschem, gEDA, Telesis; SPICE, IBIS, Touchstone (planned)	hand-written Go parsers
XML	IPC-2581	Go’s encoding/xml
Record/tabular	ODB++	custom record reader over the job’s directory tree, and an XML stream reader for ODB++X (ODX) that fills the same structures

They all target one IR. Diff, rules, analysis and everything else downstream read that IR and never touch a format directly. Adding a reader therefore makes every existing analysis work on the new format for free.

EDIF was the first reader written, but the IR is named for concepts, not for EDIF’s spelling of them.

The neutral IR’s two layers ¶

The IR is a single neutral model of a design. Readers normalize into it and writers emit from it. It has two layers.

• The semantic layer holds the normalized messages, such as Design, PartType, Component and Net. Diff, rules and analysis consume them. It is cross-format, so an Altium net and a Cadence net look identical here. Think of it as the clean domain model the rest of the system programs against.
• The fidelity layer is a FidelityFragment list. A lossless reader keeps its raw source there, keyed back to a node by provenance, so a write-back can be byte-for-byte or semantically identical to the input. A reader that is only lossy-bounded leaves this list empty, and the current EDIF netlist reader is one. Every node also carries an attributes string map as a fallback for named properties that have not been promoted into first-class fields.

Provenance links the two layers. Programming-language tooling does the same with full-fidelity syntax trees, such as Roslyn’s red-green trees, rust-analyzer’s rowan and tree-sitter CSTs. There a concrete layer retains everything, including ordering, trivia and fields not modeled, beside the clean semantic view that analysis uses. The normalized layer abstracts detail away, and the fidelity layer keeps that detail.

Three things come from provenance spans at once: lossless reconstruction, “this finding maps to exactly this line or figure” for trust, and surgical edits (rewrite only the changed region and leave the rest byte-identical). The IR also carries unmodeled fields opaquely. So write-back never drops a field the reader did not understand, and a reader can still be lossless where its own coverage is incomplete.

Two tiers of maturity ¶

The schema is split by how well-exercised it is.

• Tier 1 (netlist) is frozen. It holds Design, PartLibrary, PartType, Pin, Component, ComponentSection, Net, Connection, Sheet, exercised by the EDIF reader and verified on a real 3980-component board.
• Tier 2 (physical) is provisional. It holds Footprint, Layer, Stackup, Constraint, BomLine. The IPC-2581 reader fills Footprint, Layer, Stackup and BomLine, and the KiCad board reader fills Footprint and Constraint. They were designed from the published IPC-2581 and ODB++ schemas and are unvalidated until more than one reader agrees on them. Names and shapes may change. They ship anyway, marked provisional in the proto, so consumers can plan against them.

Neutral vocabulary ¶

The names are grounded in a survey of how EDIF, KiCad, IPC-2581, and ODB++ express the same concepts. Where the IR name differs from EDIF’s word, EDIF’s word is format-specific:

IR name	EDIF term	Why not EDIF’s word
PartLibrary / PartType / Pin	library / cell / port	Only EDIF says “cell”. KiCad, IPC-2581, and ODB++ do not.
Component + ComponentSection	one instance each	A ref-des is not unique per instance. Sections keep multi-gate and bank detail.
Connection	portRef	Neutral: component_ref, pin_ref.
Provenance.native_id	rename &id	The &id is one kind of native id, not the universal key.

The domain in plain terms (for software readers) ¶

The vocabulary is domain-specific. In terms a programmer already has:

• A Component is a physical part placed in the design (one resistor, one chip), like an object instance. Identified by its ref_des (R1, U3), which is its unique instance name.
• A PartType is the kind of part a component is, like the class the instance is of.
• A Pin is a connection point on a part (a leg of a resistor, a pad of a chip), like a port. Identified by a designator (1, 2, A7).
• A Net is a set of pins that are all electrically the same point, wired together. “These pins are one node.” In software terms, a net is a wire generalized to many endpoints.
• A Footprint is the physical package, the copper pads the part solders to. (Provisional tier.)
• The schematic is the logical drawing (what connects to what), and the layout is the physical board (where parts sit, how copper routes). Both encode the same connectivity. The IR takes logical structure from the schematic and exact connectivity from the layout.

The IR as a graph ¶

The IR is a graph. It has a containment tree of nodes, plus cross-references where one node points at another by a stable key. The cross-references make it a netlist rather than a flat document. The proto is the source of truth for the schema. Containment (what owns what):

• Design
  • PartLibrary* → PartType* → Pin* : the definition side
  • Component* → ComponentSection* : the placed side (sections = units)
  • Net* → Connection* : connectivity
  • Sheet* : logical page refs (geometry is a sidecar)
  • Footprint*, Layer*, Stackup(→StackupLayer*), Constraint*, BomLine*: physical tier (provisional)
• Cross-cutting on every node: Provenance (source locator) plus an attributes map. Design also carries the FidelityFragment list.

Relationships (who references whom, and by which key):

erDiagram
    Design ||--o{ PartLibrary : contains
    PartLibrary ||--o{ PartType : contains
    PartType ||--o{ Pin : defines
    Design ||--o{ Component : places
    Component ||--o{ ComponentSection : "units"
    Design ||--o{ Net : has
    Net ||--o{ Connection : joins
    Design ||--o{ Sheet : has
    Design ||--o{ Footprint : "has (provisional)"

    ComponentSection }o..|| PartType : "part_ref -> name"
    Connection }o..|| Component : "component_ref -> ref_des"
    Connection }o..|| Pin : "pin_ref -> designator"
    Component }o..o| Footprint : "footprint_ref -> name"

Solid edges are containment, dashed edges are references by key. The reference keys are always the semantic identifiers (names, designators), never the native id.

Reference (field)	Targets	Meaning
ComponentSection.part_ref	PartType.name	which part definition a section instantiates
ComponentSection.library_ref	PartLibrary.name	that part’s library
Connection.component_ref	Component.ref_des	which component a pin sits on
Connection.pin_ref	Pin.designator (on that component’s PartType)	which pin
Component.footprint_ref	Footprint.name	physical package (provisional)

The geometry sidecar joins back into this graph the same way, by the same semantic keys: a placement’s ref_des joins to Component.ref_des, a wire’s net joins to Net.name, a pin point’s port_ref joins to Pin.designator. So a renderer attaches drawing coordinates, and geometry never enters the core IR.

A voltage divider in the IR ¶

The smallest useful circuit is two resistors in series from an input to ground, where the midpoint is a divided-down output.

VIN ──[ R1 ]──┬── MID (output)
              │
            [ R2 ]
              │
             GND

That circuit as an IR instance, the abstract boxes above filled in:

Entity	Instance	Relationships
Design	divider
PartType	Device:R	has Pins 1, 2
Component	R1	part_ref → Device:R, Value = 10k
Component	R2	part_ref → Device:R, Value = 10k
Net	VIN	connections: R1.1
Net	MID	connections: R1.2, R2.1
Net	GND	connections: R2.2

Both Components point at the one PartType Device:R, two instances of the same class. The Net MID has two Connections, R1.2 and R2.1, and that pair is the junction in the middle of the divider. Follow any Connection’s component_ref and pin_ref and you reach a specific pin of a specific component.

One physical part, several sections ¶

Some chips hold several independent circuits in one package. A 74LS00 is a single 14-pin chip containing four separate NAND gates. On the board it is one part (ref_des U1), but logically it is four gates (units). The IR models it as one Component U1 with four ComponentSections:

Entity	Instance	Note
Component	U1	one physical chip
ComponentSection	U1 index 0	gate A, part_ref → 74xx:74LS00
ComponentSection	U1 index 1	gate B
ComponentSection	U1 index 2	gate C
ComponentSection	U1 index 3	gate D

A ref-des is not unique per section, since all four gates share U1. So Component groups the sections, and the reader does not emit U1 four times. In software terms it is one object, identified once, exposing several independent sub-units that happen to share a package. Sections may reference different PartTypes (a heterogeneous part). Diff compares the set of part references over sections rather than a single value, so a change to one section is reported without false positives from ordering.

Provenance and identity ¶

Provenance{ source_file, Span span, native_id, native_id_kind } is a neutral locator. native_id is a format-native id, and native_id_kind names its kind (edif-rename-id, kicad-uuid, ipc2581-xpath, and so on). A native id is not assumed stable across exports. EDIF regenerates its &id on every export.

The stable key for cross-revision diff and for the IR-to-geometry join is always the semantic key (ref-des, net name, pin designator), never the native id. Matching on a native id would report the entire design as changed on every export. Span holds a byte offset and length plus a line and column. It exists for lossless reconstruction and surgical edits. The EDIF s-expr reader does not populate byte offsets yet.

Placeholder designators ¶

A semantic key can be absent while still being present. Before a designer runs annotation, every unassigned part carries a placeholder designator: R?, C?, REF**, or a partly-assigned C?1845. That is annotation state, not a name. Every unannotated resistor on the sheet reads R?, so the string is a label saying “no identity yet” while occupying the field where an identity goes.

internal/refdes.IsPlaceholder is the single definition. It has its own package because several layers must agree on it and they cannot import each other. Readers take nothing from core, and core takes nothing from a reader. When those layers disagree about what counts as a designator, nothing fails loudly, and they quietly answer different questions about the same design.

Consumers decline rather than merge. A board reader skips a placeholder-referenced footprint, because a REF** on a board is usually a fiducial or a mechanical artifact rather than a part. A schematic reader keeps the part, because those parts are real circuitry and nobody has named them yet, and dropping them would make the design read short. The check model declines to assert pin uniqueness over one, because (R?, 1) does not name a pin. On one export 176 distinct resistors shared that key, so the pin index saw a single pin sitting on 129 nets. Reporting that as malformed input would say something false, because the netlist is fine.

Declining also means a placeholder is not a claimed designator, so two symbols reading R? are not two parts fighting over one name and the schematic reader does not report them as a reader.ref_des_collision. That finding is an error, and its remedy is to rename one of them. Renaming is the wrong instruction when nobody has annotated the sheet yet. It would also report the same two placements a second time under a different heading.

In return, a reader that KEEPS unannotated parts has to say so. The parts are drawn and connected, so by the time the IR reaches a rule they look like ordinary components with odd names. The reader is the only layer that ever knew. internal/refdes.Unannotated builds the diagnostic from the same predicate, so the reader that keeps them cannot group them by a rule of its own.

The format decides which half a reader is on, not the reader’s taste, so the shipped answer is uniform. kicad (board), ipc2581, odb, allegro and altium skip, and edif, kicad (schematic), geda and xschem keep and report. For a new reader, ask first which half it is on. The two halves are mutually exclusive, and doing both would report a part the design does not contain.

Do not assume a reader groups by designator, because they disagree and the diagnostic must not. internal/refdes.Unannotated groups by designator before it walks sections, so every spelling produces one entry per placeholder carrying every placement, and “2 parts are still called R?” means the same thing everywhere.

How far the readers actually diverge, and why repairing the identity is worse

kicad folds every symbol sharing a designator into one component; geda folds only when the source says slot=, so unslotted duplicates remain separate components with one designator between them; xschem does not fold at all. Two unannotated symbols are therefore one component or two depending on the format. The shared predicate exists to remove that divergence, and a reader deriving its own grouping would bring it back.

It is tempting to repair the identity instead and key those parts on their native id, which really is unique. The identity model forbids that, because the id is regenerated per export, so every unannotated part would read as changed on every revision diff. The part has no identity yet, and unannotated_components below records that absence.

Geometry is a keyed sidecar ¶

Render data does not live in the core IR. Render data means symbol shapes, placements, wire routing and pin coordinates. It lives in a separate artifact that references the core IR by stable keys (ref_des, net name, port_ref, plus provenance), joined at render time. Diff, rules, and simulation never carry graphics. The renderer loads the core IR plus the geometry sidecar and joins them.

Two things follow. Heavy graphics are kept off the analysis hot paths, and geometry can come from a different source than connectivity. A design’s connectivity can be read from an EDIF .edn netlist while its geometry comes from the .eds schematic export. The geometry and rendering page covers the sidecar in full.

Fidelity per reader ¶

“Lossless” is a property of an individual reader, not a blanket platform promise. Each reader declares what it preserves. An IPC-2581 reader can be lossless. An ODB++ reader is lossless with respect to ODB++. An extractor-based reader is lossy and documents what it drops. The round-trip oracle below then applies only where a reader claims losslessness.

Input diagnostics ¶

A read can succeed and still be worth complaining about. A reader records what it saw in InputDiagnostics, so a condition it noticed does not vanish once the IR is built:

Field	What it records
dangling_endpoints	a wire end touching nothing
no_junction_endpoints	a wire end dropped mid-span of another with no junction dot
joined_taps	a wire end dropped mid-span of another WITH something joining them
ref_des_collisions	one designator claimed by placements that are not sections of one part
unmodeled_buses	a bus construct recognized but not expanded into member nets
unresolved_symbols	a symbol file that failed to open, so its placements carry no pins
resolved_symbols	a symbol reference that DID open, with the pin count it supplied
unannotated_components	parts whose designator is still a placeholder
unexpanded_hierarchy	a hierarchical block (an EDIF sub-cell) whose contents the read did not extract, with its instance count

They exist so that silence does not read as coverage. Every one of these makes the design report less rather than reporting an error. An unresolved symbol yields a smaller netlist, and connectivity rules then pass cleanly over the gap. An unexpanded bus leaves members merged or off-net. Un-annotated parts are fully drawn and connected, so nothing looks wrong at all. In each case a clean run is indistinguishable from a design that genuinely had none of the problem, and the reader is the only layer that ever knew the difference.

Recording is therefore separate from judging. The reader states what it observed; a thin rule in the catalog turns each into a finding with severity and remedy. With that split, the diagnostics are per-reader without being per-reader policy. A format that cannot detect a condition contributes nothing, and the rule is then silent for a knowable reason rather than an accidental one.

Two consequences follow. A field is not populated by every reader, and an empty one means “this reader does not detect this”, not “the design is clean”. EDIF contributes no ref_des_collisions on purpose, because it represents a multi-gate part as several instances sharing a designator and carries nothing to tell that legitimate grouping from a duplicate. And a reader must build the struct once. Assigning a fresh InputDiagnostics per signal silently drops whatever was recorded before it, and the bug only appears when the second signal is added.

This page stated that first consequence and nothing enforced it. Only the KiCad reader populated the field, so duplicate-ref-des read as a clean pass on every EDIF, IPC-2581, gEDA and xschem design (agni issue 309). So a reader now DECLARES what it computed, in supplied, whether or not it found any:

InputDiagnostics{ RefDesCollisions: nil, Supplied: []string{"ref_des_collisions"} }   // looked, found none
InputDiagnostics{ RefDesCollisions: nil }                                             // never looked

Some rules have a diagnostic as their whole subject. Such a rule gates on the declaration (check.CapRefDesCollisions), so the second case reports not-applicable with a reason instead of a pass. Declaring is unconditional. Suppose a reader declared only when it found something. The clean read would then be ambiguous again, and the clean read is the common case.

When you write a new reader, detect what your format can express, declare what you detected, and leave out what your format cannot tell apart. EDIF leaves this one out and the report says so.

A diagnostic that is not a finding ¶

No rule turns unexpanded_hierarchy into a finding (agni issue 707), and it is the only field in the table with no rule. A design being hierarchical is not a defect, so a verdict would state something false about it. The field records a limit of the READ, because the EDIF reader scopes extraction to the top cell (WS1-004), so every count and every rule covers the top level alone. Every surface states it in one sentence, report.HierarchyNote in core/report (agni issue 717). The CLI prints it as a note on stderr and agni stats adds a count line. A check run carries the list on CheckDesignResponse.unexpanded_hierarchy and a results or review document on DesignRef.unexpanded_hierarchy, so the check report and the checklist page put the sentence in their header without reading the design again. GetDesignResponse carries the list and the sentence (hierarchy_note). The viewer shows the sentence as a strip over the canvas, and its Properties panel lists the skipped cells at the design level. It is in the same register as the note that names a companion when a read pulls one in.

A snapshot can ask for the blocks instead (Snapshot.flatten_hierarchy, --flatten-hierarchy on the CLI, agni issue 718). A block is a cell whose contents hold instances. The EDIF reader then walks from the top cell into every block it places. Inside a block, a net can name one of the block’s own ports, and in the parent another net names that port on the block’s instance. The two are one net, named by its shallowest piece. Some nets inside a block reach no port. Such a net keeps its own name when no other net in the design carries that name and its block is placed once. Otherwise it is qualified by the block’s instance path (/A1/MID). Sheets reuse local and generated names, and joining them by name would merge two nets. A part keeps its designator from the file. Parts sharing a designator are sections of one component wherever they sit, since a multi-gate part split across sheets is one part. Sometimes the design places a block more than once. A part reached through that block carries its path in its ref-des (B2/R1) and in its locator, so two placements are two sets of parts. A file with no designators names each part by its instance name, and that name is unique only within its cell. So a name used in more than one cell instance is qualified by its block’s path in each block (SA/R1), and the top cell keeps its own name (agni issue 1105). A multi-gate part split across sheets then reads as two parts. A reader can see that error, while merging two blocks’ R1 would hide one part.

Blocks are not part types. A flattened read lists a block only when a cycle kept the walk out of it. The choice changes what the design contains. So it travels on the overlay into ReadOptions and into the overlay’s identity, and the parse cache keys on it too. TestFlattenedSnapshotIsOneModelAcrossServices holds this over one cache serving both reads. A design declares it once with flatten_hierarchy: true in its design.yaml, and every surface honours that. The viewer asks for one page with ?flatten=1, and the strip’s Read every block button adds it (agni issue 1102). Each listed block carries a reason. It is NOT_WALKED on a top-level read, CYCLE when a block placing itself stopped a flattened read, and FILE_MISSING when an OrCAD block’s own file could not be opened. report.HierarchyNote says a different sentence for each, since “only its top cell was read” is false after a walk. report.FlattenReadsMore says whether reading every block would take in a listed block, and GetDesignResponse.flatten_reads_more carries it, so the strip offers its button only for a NOT_WALKED block (agni issue 1406).

The instance count makes it actionable. “This design is hierarchical” says nothing about how much is missing; “SUB holds 212 instances and none were read” does. Once a reader learns to descend into a block, it stops listing that block, so a shrinking list shows the hierarchy walk working. The xschem and gEDA readers skip type=subcircuit and source= in the same way (agni issue 134), and the field is shaped so they can populate it too.

Record what went right, not only what went wrong ¶

Most of the table above names a failure, and a rule reading a failure list can only ever report failures. Its silence on a clean design is then indistinguishable from its silence on a design nobody looked at. supplied closes the same ambiguity one level up. unmodeled_buses never had the problem. It holds every bus construct the reader saw, and the rule partitions that list, so a bus whose members are already nets is a countable pass.

resolved_symbols is unresolved_symbols given the same shape (agni issue 418). The resolve walk already decided per reference and threw the successes away. Keeping them lets symbol-unresolved state a considered set. The pin count tells the two cases apart. “The symbol resolved” reads identically on a 100-pin FPGA and on a stale library entry that answered with an empty stub. The second costs the netlist exactly what a missing file does. A count moves when the library does.

joined_taps is the same move applied to the T-tap diagnostic (agni issue 420). There the silence cost most.

Why a T-tap is the worst of these silences, and what recovers the set

A T-tap with no junction dot is drawn as a connection, but the netlist does not have it. So the author, the reviewer and the plot all agree while the board ships with the connection missing. Before this, a sheet whose every tap carried its dot reported the same as a sheet with no tap in it. A format that cannot see wire geometry reported the same again.

The reader had the answer. splitWiresAt runs at every junction dot and mid-span label BEFORE the detection pass, so a joined tap is an endpoint of both wires by the time anything looks. Running the same detection once more before the split recovers the set.

Copy two things when adding a reader. Declare resolved_symbols in supplied only on the branch that actually opened a symbol, because xschem and gEDA have a no-opener path where the caller deliberately asked for a symbol-free read, and declaring there would turn “we did not look” into “we looked and found nothing missing”. And keep the two lists a partition, since a reference on both would let a rule report one subject as passed and failed at once. A guard in readers/formats checks that a reader recording unresolved symbols also declares the resolved set.

Tiered writers ¶

Not every emitter needs to be built in. The write path is tiered.

1. Agni builds in emitters for open formats and writes them end to end with no external dependency. IPC-2581 and the EDIF netlist ship today, and ODB++, Gerber and KiCad are planned.
2. Drive the native tool to write its own format. Xpedition writes Xpedition and Altium writes Altium, via their automation APIs or by importing IPC-2581 or ODB++ that Agni emitted. Agni never emits a vendor’s binary directly. This is also the clean way to produce a native format, using the vendor’s own tool as intended.
3. Community or optional emitters fill gaps over time.

Like Pandoc, the set of writers is broad, and it does not promise every possible target.

Round-trip as a test oracle ¶

For unchanged input, parse then emit should be the identity function. Run the whole real-file corpus through read-then-write and assert byte-identical, or semantically identical, output. That is property-based and differential testing applied to the reader layer. It catches parser regressions and silent breakage from format churn automatically, rather than when a customer file fails to load.

Two nuances about “lossless” ¶

1. “Lossless” is relative to the ingestion source, not the native file. ODB++ and an extractor’s output are both exports, and each has already dropped some of the native binary’s information. A reader can be perfectly lossless with respect to the interchange format it reads while still lacking native-only detail. Lossless-to-ODB++ is achievable. Lossless-to-Altium-native is not, unless the native file is read directly.
2. Write-back into a native tool goes through interchange or the API, never by emitting the vendor’s binary. The way back is IR to open interchange to tool (the tool imports the emitted ODB++ or IPC-2581), or the IR drives the tool’s automation API, with provenance carrying surgical fidelity. This is still lossless along the path under Agni’s control.

How formats get ingested ¶

Agni puts reading a proprietary binary directly, by reverse-engineering it, low on the priority list, mostly for legal reasons.

The four reasons, none of which is legal advice

• EULAs typically prohibit reverse engineering. Breaching one risks contract claims.
• Anti-circumvention law applies if any encryption or protection is involved (encrypted SPICE, protected files), a separate and worse category.
• Reverse engineering risks trade-secret and copyright exposure. Clean-room interop has some footing but is jurisdiction-dependent and expensive.
• Procurement review. Aerospace, defense, and industrial buyers run legal review, and a legally questionable ingestion method gets rejected there regardless of actual litigation risk.

So the ingest ordering prefers sanctioned paths, in this order:

1. Official automation API or export (vendor-blessed).
2. Open standards and documented formats (IPC-2581, ODB++ spec, KiCad, Gerber).
3. Official extractors.
4. Community reverse-engineered parsers. Fine for prototyping, but they inherit the legal questions, so relying on them commercially imports that risk.
5. In-house reverse engineering, as a last resort where no sanctioned path exists, and only after a deliberate legal check.

The technical argument points the same way. Ingesting via a documented export or API is both the easier path and the legally clean one, so the two reinforce each other rather than competing.

Guarding against overfitting ¶

Two mechanisms keep the schema from drifting toward whatever format is read most.

1. The promotion rule admits a field to the semantic layer only once a second format’s reader would populate it. Until then it lives in the attributes map or the fidelity layer. This blocks both EDIF-shaped fields and speculative ones, and keeps the IR format-neutral, since no reader adds a field another format could not fill.
2. Reader reconciliation means every new reader reconciles its concepts against the cross-format map before adding fields. A concept the map missed updates the map. A key that recurs in attributes across two readers becomes a promotion candidate. A first-class rename updates the proto and the map together. So when a new format brings a new concept, the IR absorbs it over time rather than accreting a one-off field.

Derived fields, and the tiers that fill them ¶

Some fields no reader populates. Two examples are a net’s role and a component’s device class. Roles include a rail, a ground, and the four that mean “named after a rail without being one” (see naming conventions). A shared, format-neutral pass derives these fields from the already-read IR, so every format gets them the same way rather than each reader inventing its own answer.

The roles themselves are the ir.Role enum, so Go and TypeScript both generate the vocabulary from one definition, and neither language keeps its own copy of the list.

Those passes originally all ran at ingestion, and the rule was one shared pass per field. That assumed a derived field could find all its inputs at read time. The assumption broke when the datasheet tier arrived. Seeded parameters attach at model construction, after the read. So an ingestion pass cannot fill a field derivable from a vendor’s pin table at all.

So a derived field is now filled by one pass per evidence tier, under two conditions that keep the original guarantees intact.

The field records which tier established each value. ir.Net.roles is the worked example: each role carries a RoleSource. So a consumer can tell whether a role came from a naming convention, from the source format’s own declaration, or from a datasheet’s pin function. Without that, the tiers union into a flat set, and nobody at the point of use can answer “how do we know this”. For as long as that was true, a naming convention and a vendor fact looked identical. ir.Component.device_classes followed, each tag carrying a ClassSource. It had to. The datasheet tier used to enrich the check model privately, and the IR never carried its answer. So a query said a part was a tvs while a drawing of the same design called it a diode. Without a field for its evidence, the datasheet tier had nowhere to write at all.

A later tier may only add, never remove or downgrade. So admitting a tier is safe. Switching one on can reveal more, and every value an earlier tier found survives it. So a design read without the datasheet tier classifies exactly as it did before that tier existed, and no tier’s absence can silently narrow an answer.

The two instances today are classify.StampClassesFromSpecs (a datasheet’s declared device class) and enrichRolesFromParams (a datasheet’s pin functions establishing rail and ground). They share one rule, including what a duplicate means, and it has one home each in classify.AddClassTag and classify.AddNetRole.

The class pass runs wherever a corpus is present rather than only at model construction. The Loader runs it when the read carries one, and check.Model runs the same pass again when it is handed a corpus the read did not have. Running it twice is free, since it adds and never replaces. In exchange the design itself carries the answer, so a drawing, a query and a rule are reading one set rather than three derivations of it. Nothing else can reach the classes at stake. One is a bare crystal against a ceramic resonator. The keyword path refuses to guess between them, because vendor part text is unreliable for exactly that split. The other is an ideal-diode controller, which is a FET plus a bias network that no netlist labels.

The two passes must run in order, and getting the order wrong fails silently. The convention pass REPLACES the set it writes, so it has to run first. The datasheet pass joins on the MPN, and another pass fills the MPN, so the datasheet pass has to run after that one. The order is set in formats.DeriveDesign, the configured half of Loader.ReadDesign.

Before the datasheet tier, a net was a rail because of its NAME, and the built-in vocabulary is start-anchored, so a project naming rails function-first had to declare its own lexicon or the engine could not see its rails. On a real 1700-net board that was the difference between 13 rails and 91, with no error and no warning. A vendor pin table settles it without consulting any name.

Versioning ¶

Design.ir_version and Design.source_format are stamped on every document. Proto3 retains unknown fields on parse. Together with the attributes map and the fidelity fragments, that gives forward-compatible carry-through across schema changes.

Caveats ¶

• Connectivity is in both files. You draw it in the schematic (as wires), and the layout carries it explicitly on every pad. The two must agree. The reader takes exact connectivity from the layout and logical structure (part types, sections) from the schematic.
• Match on the semantic key, never the native id. A component is matched across revisions by its ref-des, a net by its name. The tool’s internal id is regenerated on every export, and matching on it would report the entire design as changed.
• Geometry is a separate artifact. Where a symbol or trace is drawn lives in the geometry sidecar, joined by the same keys, never in the core IR. Diff, rules, and analysis carry no pixels.
• The physical tier is provisional. Footprint, Layer, Stackup, and BomLine are filled by the IPC-2581 reader, and Footprint by the KiCad board reader too, but the shapes may still change. Treat them as a sketch until a second PCB-fabrication reader agrees on them.
• “Same net name” is ambiguous in a diff. A net that keeps its connectivity but changes name is a rename. A net that keeps its name but changes connectivity is a rewire. A semantic diff has to tell those apart, because they are very different edits.

A lossless IR with provenance enables surgical write-back, automated fixes and transformations, beyond read-only review. Diff, rules, analysis, the corpus and everything else build on this one representation.