Architecture

PetroScript is a standalone Python library that holds a reservoir model as one typed, frozen value, compiles it to a simulator’s deck, reads decks back into the same value, and reads the results of a run by name. It stays independent from OPM Flow internals and proprietary ECLIPSE SDKs: it meets a simulator only through the files it writes and reads and a subprocess that runs it.

This page is a contributor orientation: the layers, the plugin tables, the model value, the validation contract and the snapshot harness. Every Python block on it runs, in order, in one fresh process (tests/test_docs_run.py runs them).

Four layers

The package is organized as four layers, from the bottom up. Each depends only on the ones below it.

src/petroscript/
  # foundation: no model knowledge
  core/errors.py        PetroScriptError and its family (InputError, ModelError,
                        CompileError, DeckImportError, RunError)
  core/issues.py        Issue, the one row every check and warning carries, and every
                        model or compile refusal (an InputError carries none)
  validation/           the issue catalog: every code and its fix line
  core/units.py         Quantity, a number with its own unit
  core/registry.py      the plugin tables: targets, front ends, catalogs, added checks

  # model layer: frozen pydantic values, one module per concern
  core/core.py          Model
  core/grid.py rock.py fluids/ regions.py wells.py groups/ schedule/ aquifers.py ...
  core/addresses.py     every fact's address: what m.get, m.set, m.find and m.diff read
  core/deck.py          Deck and Simulator: one target's files for one model
  core/results.py       Run: a finished run read by PetroScript's names
  core/printer.py       m.to_python: the Python that rebuilds a model

  # surface: the public names
  __init__.py           the root (ps.Model, its top-level parts, ps.load, ps.compile, ps.help,
                        ...) and the composition root that fills the plugin tables
  grid/ fluid.py pvt.py relperm.py initial.py wells/ groups.py aquifers.py actions.py
  lift.py polymer.py thermal.py storage.py ...    one namespace per keyword, topic or family

  # services: what reads, writes or runs a model
  checks/               the checks construction runs (checks/invariants.py) and ps.checks
  eclipse/              the deck dialect: lexer and reader, the importer behind ps.load, the
                        writer every target spells records with, the keyword catalog
  targets/opm_flow/     the OPM Flow target: compile, run, results
  bundle.py io/         the sidecar bundle and its sheets; GRDECL geometry files
  testing/              ps.testing: assert_issue, records, first_divergence
  __main__.py           the petroscript command line: load, compile, check, run

The plugin tables

petroscript.core.registry holds four tables, plain dicts read by name and only added to: targets (the simulators a model compiles for), front_ends (the formats a model is read from), catalogs (each deck dialect’s keyword catalog, which ps.help reads) and checks (the checks an author adds with ps.checks.register, never PetroScript’s own). The built-ins are declared in petroscript/__init__.py as module:attr strings and loaded on their first lookup, so import petroscript loads none of them (a contributor’s view: product code and examples stay on ps. and never import petroscript.core):

import petroscript as ps
from petroscript.core import registry

assert set(registry.targets) == {"opm-flow", "jutuldarcy"}
assert set(registry.front_ends) == {"eclipse"}

ps.compile(model, target) and deck.run() go through targets; ps.load(path) through front_ends. A new simulator is one more entry in targets: an object with a name, the version it is measured on, and check, compile, simulators, preflight (what its simulator says reading a deck before a run, for deck.check()), run and results (registry.Target). A front end has a name, load and read (a file’s keywords as load reads them, registry.FrontEnd).

The model value

ps.Model is the apex value and the whole case. Its fields: name, title, units and start; capacity (the sizes the run allocates) and dimensions (what a loaded deck states beyond the other fields: phase and option cards, size slots capacity does not name); grid, rock, fluid, relperm, regions, passive_tracers and physics (the family packages); initial, summary, output and numerics; the grid program in edits, then transmissibility, faults and non_neighbor_connections; the wells, groups, lift_tables and network; the dated changes in schedule; aquifers and boundary_conditions; and acknowledged_warnings, the warnings the author acknowledges, each row then demoted to information: a physics warning’s code, or one of the three target.* warnings that name what the target cannot run (target.brine.oil_water, target.restart.field_group_wells, target.thermal.multisegment_energy), as the catalog’s ACKNOWLEDGEABLE_WARNING_CODES lists them. It requires its units, start, grid and rock: there is no implicit empty model and no default run date.

Every part is a frozen pydantic model, built by keyword only, strict about types, refusing an unknown field, an infinity and a NaN. A numpy field holds a read-only copy, so a later write into the caller’s array changes nothing the model holds:

import numpy as np

porosity = np.full((2, 2, 1), 0.2)
rock = ps.Rock(porosity=porosity, permeability=100.0, vertical_ratio=0.1)
porosity[0, 0, 0] = 0.9
assert isinstance(rock.porosity, np.ndarray)       # the rock holds its own copy
assert rock.porosity[0, 0, 0] == 0.2
assert rock.replace(porosity=0.25).porosity == 0.25   # a new value, validated as it is built

A change is a new value: x.replace(**fields) on any part, and on a model the verbs m.at, m.set, m.until and m.apply. Every fact a model states has an address (wells/P1, rock/porosity, schedule/2027-01-01/P1/control), core/addresses.py maps addresses to fields, and ps.help("addresses") lists the forms.

Pydantic method-name collision

pydantic v2 takes a subclass attribute whose name is an inherited field as that field’s new default: a method named after a field is removed from the class and becomes the field’s value. This method-name collision fails silently, so never give a Model subclass a method named after a field such as schedule.

Validation contract

Validation runs at two gates, both reading one registry of checks.

Construction gate

Model’s last validator runs every check in petroscript.checks.invariants (the _RESERVOIR_MODEL_CHECKS tuple) over the new value. An error row refuses the construction with a ps.ModelError; the other rows are kept, and m.check() returns them, followed by the rows of the checks ps.checks.register added. Each check is a callable (Model) -> list[Issue] that examines one concern: units, phases, grid shapes, well controls, group membership, PVT and saturation tables, regions, and each family package. Count them with:

PYTHONPATH=src python3 -c "from petroscript.checks.invariants import _RESERVOIR_MODEL_CHECKS; print(len(_RESERVOIR_MODEL_CHECKS))"

Compile gate

ps.compile reads the model’s rows again and raises ps.CompileError on an error row before it writes anything. A model built around the validators (pydantic’s model_construct) is refused there all the same:

model = ps.Model(
    name="TINY", units="metric", start="2024-01-01",
    grid=ps.grid.Cartesian(shape=(2, 2, 1), dx=100.0, dy=100.0, dz=10.0, top=2000.0),
    rock=rock,
    fluid=ps.fluid.GasWater(
        gas=ps.pvt.DryGas(pressure=[50.0, 250.0], gas_fvf=[0.024, 0.005],
                          gas_viscosity=[0.013, 0.019]),
        water=ps.pvt.Water(reference_pressure=200.0, fvf=1.02, compressibility=4.4e-5,
                           viscosity=0.3, viscosibility=0.0),
        gas_surface_density=0.78, water_surface_density=1000.0),
    relperm=[ps.relperm.Gas(sg=[0.0, 0.8], krg=[0.0, 1.0], pc=[0.0, 0.0]),
             ps.relperm.Water(sw=[0.2, 1.0], krw=[0.0, 1.0], pc=[0.0, 0.0])],
    initial=ps.initial.Equilibrium(datum_depth=2005.0, datum_pressure=200.0,
                                   gas_water_contact=2100.0),
    schedule=ps.Schedule(end="2024-07-01"),
)
no_kz = ps.Rock.model_construct(**{**dict(rock), "vertical_ratio": None})   # kz: unstated
bypassed = ps.Model.model_construct(**{**dict(model), "rock": no_kz})   # no validator ran
ps.testing.assert_issue(lambda: ps.compile(bypassed), code="rock.permeability_z.unstated")

The target’s own refusals (what Flow cannot run) join the gate.

Issue codes and fix lines

Every check row, and every model or compile refusal, is an Issue with a stable code, a severity (error, warning or info), the address of the fact, the message and the fix. The codes and their fix lines are the catalog’s, petroscript/validation/catalog.py. Tests and agents assert on the code rather than on prose. ps.ModelError and ps.CompileError carry their rows in .issues, so tooling reads the violations without re-running the checks. The other refusals carry no rows. Among them, an InputError (a construction refused: field.unknown for a field the class does not take, field.invalid for any other field or rule it refuses, a rule’s own code, when it has one, only inside the message; or a verb’s date argument that is no date, field.invalid: m.at("not a date", ...), m.find(pattern, since="nope")), a DeckImportError (a deck ps.load cannot read) and a RunError (a run that stopped short) each state their message on the error itself, with a code and a fix where they have one.

Pydantic paradigm boundary

The frozen-pydantic paradigm applies to the model layer, petroscript/core/**, and to what the public surface builds. A few values stay outside pydantic on purpose, because they are not statements a model makes:

• core/deck.py: Deck and Simulator, frozen dataclasses: one compile’s files and one executable found;
• core/units.py: Quantity, a frozen dataclass: the number with its own unit;
• core/results.py: Run and its Cells views, plain classes that read a finished run’s files when asked, and the run’s Grid and Series, frozen dataclasses;
• targets/opm_flow/context.py, targets/opm_flow/lowerers/common.py: one compile’s context and the lowerers’ intermediate values, frozen dataclasses.

These holdouts are intentional.

Snapshot harness

tools/snapshot_decks.py is the deck-drift gate. It compiles every example module and every seed under the tests’ fixtures through ps.compile to deterministic .DATA text and byte-compares it with the committed snapshots under tools/deck_snapshots/:

python tools/snapshot_decks.py            # regenerate the snapshots and their manifest
python tools/snapshot_decks.py --check    # diff against the committed snapshots; exit 1 on drift

Run --check after any change that could move deck text. It is the fast regression net that catches a writer’s drift before a Flow run does.

Semantic API Law

Zero escape hatches. Everything a deck can say enters PetroScript as a typed statement on the model. The compiler writes the target’s keywords internally, but no public API, example, test or doc bypasses the statements with a keyword string, a raw record or a deck fragment: there is no argument that takes deck text and no passthrough of a card the model cannot mean. ps.load reads a deck onto the same statements, or refuses it whole.

External data boundary

Files beside a model are data, never deck fragments. ps.io.read_grdecl reads a GRDECL file’s corner-point geometry into a ps.grid.CornerPoint and refuses any other keyword in it by name; ps.io.read_sheet and ps.io.write_sheet read and write the bundle’s parquet sheets with their key, units and schema stamped. A cell array is a numpy array in a field that takes one (ps.Rock(porosity=np.load("poro.npy"))).

Internal compiler IR

DeckPlan, DeckChunk and KeywordBlock (targets/opm_flow/) are the compiler’s internal representation. They are inspectable in the compiler’s own tests, but no public name returns them: an author states the model and reads ps.compile(m).files.

Examples are not core source

A benchmark case such as SPE1 is a model built on the public surface, in the tests’ fixtures or in a page like spe1-walkthrough.md. It never gets a compiler path of its own, so the source holds building blocks, writers, checks and readers only.

Licensing boundary

The core package writes OPM Flow/ECLIPSE-style deck syntax and reads its output files without importing GPL OPM code or proprietary SDKs. Tighter integrations live in optional packages or external processes with explicit licensing boundaries.

NeqSim, a separate program: the runner pattern

NeqSim (Apache-2.0, a Java library) runs as a separate program, through petroscript._neqsim.runner, and is found and called the way every engine added after OPM Flow is, so a harness can relay it into a container without PetroScript knowing it is there:

• one environment variable names the runner (PETROSCRIPT_NEQSIM), or simulator= names it for one call; nothing else is searched;
• the real engine is identified by its own banner, whether the runner comes as a path or as a Simulator value;
• every argument is relative to the folder the call runs in, so a relay binds the folder anywhere;
• nothing in PetroScript knows about a harness, an image or a relay.

OPM Flow, the first engine, differs on three points: discovery has a third rung (flow on PATH, after simulator= and PETROSCRIPT_FLOW), deck.run passes the case folder as an absolute --output-dir, and a Simulator value handed in is used as given, its banner not read again.

NeqSim’s runner is a Python with jpype and NEQSIM_JAR naming NeqSim’s jar. PetroScript runs the driver it ships, driver.py, two ways, each in a folder it has copied the driver into:

• <runner> driver.py --version in a temporary folder holding only the driver; the first line must be neqsim <version>, read from the jar’s own Maven record (then the jar’s sha256, and NEQSIM_IMAGE when the runner states it);
• <runner> driver.py request.json result.json in the case folder, with exit status 0 (the result), 2 (the request refused: {"refused": ...}) or 3 (NeqSim raised: {"failed": ...}).

neqsim.run.json records the runner, its banner, the request’s (without the call’s nonce), the driver’s and the result’s sha256 and the exit code; neqsim.log holds what the runner printed. Each file a call writes is removed first and created afresh, never through a link, and the result must echo the nonce the request carries, so an earlier call’s result never reads as this one’s. The exit code is read before the result. A fluid request’s component names are E300 words (letters, digits and - _ + .); it is built through NeqSim’s own E300 reader and read back: each component’s constants, the volume shift NeqSim applies, the composition, the binary interaction parameters, the temperature, the pressure and the equation of state NeqSim built are compared with the request, and a fluid built differently is refused, naming the first number that differs. A fluid that states no volume shift, or a shift of zero, is built unshifted, as E300, OPM Flow and JutulDarcy read it: the driver turns NeqSim’s volume correction off (its E300 reader would put its own database shift on every component it knows, about 5 % on a light oil’s density), reads the effective state back, and the runner refuses a correction the request did not state; a zero beside a stated shift is refused, since NeqSim then falls back to its Rackett-Z correction for it. One more difference is NeqSim’s own rule: a water component stated with no volume shift keeps NeqSim’s database constants (647.3 K, 220.89 bar). A lift_table request states a fluid (in the fluid request’s form), a tubing survey, the axes in ps.lift.ProductionTable’s own words, a correlation (beggs_brill, hagedorn_brown, gray or mukherjee_brill, NeqSim’s standalone pipe classes) and the datum’s TVD. For each axis point the driver recombines the stated fluid’s own stock-tank phases to the point’s oil, gas and water rates, flashes the stream back to check them to 1e-6, and solves the bottom-hole pressure at the datum by its own bisection over a march up the segments (each isothermal at the survey’s temperature there; a datum at the wellhead leaves no tubing, and the pressure is the THP). The march is adaptive, each step its own run of the class: a step is halved while it moves the pressure by more than 5 % of its inlet, and taken back and halved when Ek changes by more than 0.05 across it; Ek is judged at every accepted state with the class’s holdup at that same state (the wellhead’s from a short probe run; Beggs and Brill’s from its holdup profile, since the getLiquidHoldup() it inherits returns 0), and Ek reaching 1, or a step that cannot keep under the pressure bound by 1 mm, is a choke. An Ek jump still there across a 1 mm step (a dew point, a flow-regime boundary) is localised in pressure on the class’s own state to 1e-6 bar, every state on the way judged: one with Ek at 1 or more, or one the class cannot compute, refuses the cell. The solve bisects on the BHP and then on against the THP residual until it holds within 1e-3 bar or the bracket is spent. A cell is filled only by a march that holds end to end: no state at or below the 1 bara floor, the THP reached, Ek below 1 at every state (below 0.05 where the class leaves the momentum term out) and, for the three classes that take their liquid from the oil, no state where gas, oil and water flow together by the flash there. It also refuses a zero-rate point, a segment off vertical for the two vertical correlations and a point the well cannot lift, naming the rule; refused cells are listed, never filled. ps.lift.generate is the public call that makes such a request (_neqsim/lift.py): it states the model’s numbers in the driver’s units (METRIC or FIELD; LAB is refused), the stream’s composition from the ps.fluid.Compositional it names, the datum at the well’s own BHP reference depth unless one is stated, and a THP axis left out sized from what the model states for the well (the same walk the vfp_thp_off_axis check reads, from its lowest THP to the equilibration pressure, six nodes). It reads the pressures back into the model’s units and returns an ordinary ps.lift.ProductionTable whose source names NeqSim’s banner, the correlation, the composition, the datum’s and the THP axis’s origins, the standard conditions and the request’s sha256; a point the driver refuses refuses the call, named by its axis values, since a table needs every entry. Rows that need real NeqSim carry @pytest.mark.neqsim and skip while PETROSCRIPT_NEQSIM names no runner (PETROSCRIPT_REQUIRE_NEQSIM=1 makes them fail instead).

JutulDarcy runs as a separate program

JutulDarcy (MIT, Julia) is found and called the way NeqSim is, so a harness can relay it into a container without PetroScript knowing it is there: PETROSCRIPT_JUTULDARCY names the runner (or simulator= names it for one call), the real engine is identified by its own banner, every argument is relative to the case folder, and nothing in PetroScript knows about a harness, an image or a relay. It runs through petroscript._jutuldarcy.runner, which ps.compile(m, "jutuldarcy") reaches: the deck’s check() runs the parse and setup only, its run() the whole deck. The runner is any program that runs Julia with JutulDarcy and GeoEnergyIO loadable. PetroScript writes the deck’s files, request.toml and the driver it ships, driver.jl, into the case folder and calls:

• <runner> driver.jl --version, whose first line must be jutuldarcy <version>, read from the package itself, then GeoEnergyIO’s, Jutul’s and Julia’s versions (and image <id> when the runner states JUTULDARCY_IMAGE);
• <runner> driver.jl request.toml result.json, with exit status 0 (the result), 2 (refused: {"refused": ..., "lines": [...]}) or 3 (JutulDarcy raised, or stopped short of the deck’s end: {"failed": ..., "phase": ...}).

The driver parses, sets up and simulates the deck with JutulDarcy’s messages captured. Its second layer is absolute: any parser, setup or simulation line saying “unsupported”, “not supported”, “ignored” or “partial” refuses the deck with the line quoted, so a keyword JutulDarcy skips fails loudly instead of running another model. The request can stop after the setup (setup_only). jutuldarcy.run.json records the runner, its banner, the deck’s (as Deck.digest), the driver’s and the result’s sha256 and the exit code; jutuldarcy.log holds what the runner printed. Each file a call writes is removed first and created afresh, never through a link, and the result must echo the nonce the request carries, so an earlier call’s result never reads as this one’s. The exit code is read before the result. request.toml is a flat table whose values are JSON as well (Julia’s stdlib TOML reads it); result.json is written by the driver’s own serializer, so neither side needs a package the other lacks. Rows that need real JutulDarcy carry @pytest.mark.jutuldarcy and skip while PETROSCRIPT_JUTULDARCY names no runner (PETROSCRIPT_REQUIRE_JUTULDARCY=1 makes them fail instead).