Architecture

PyMTR is a clean-room, Tkinter-based network troubleshooting application. The desktop UI, exports, help text, charting and reports all use the shared column catalog in pymtr.core.columns, while packet collection stays isolated behind TraceSession and the tokenized packet-helper subprocess.

C4 Context

        C4Context
    title PyMTR System Context
    Person(analyst, "Network analyst", "Runs route diagnostics, reviews hop metrics, and exports evidence")
    System(pymtr, "PyMTR", "Desktop MTR-style diagnostic application")
    System_Ext(target, "Target host", "DNS name or IP address being traced")
    System_Ext(dns, "DNS resolvers", "Forward target lookup and optional reverse hop lookup")
    System_Ext(osnet, "Operating system network stack", "Raw sockets, ICMP, UDP, TCP, IPv4 and IPv6")
    System_Ext(files, "Local filesystem", "PyMTR.ini, logs, temporary SQLite history, generated exports, brand assets, manual and bundled docs")
    Rel(analyst, pymtr, "Starts/stops traces, changes options, opens charts and exports")
    Rel(pymtr, target, "Sends TTL-limited probes and receives replies")
    Rel(pymtr, dns, "Resolves target and hop names")
    Rel(pymtr, osnet, "Uses socket APIs through the packet helper")
    Rel(pymtr, files, "Reads and writes local application state")
    

C4 Container

        C4Container
    title PyMTR Containers
    Person(analyst, "Network analyst")
    System_Boundary(pymtr, "PyMTR") {
        Container(gui, "Tkinter GUI", "Python/Tk", "Main window, in-window menus, options, dialogs, metric grid and hop charts")
        Container(core, "Core session and metrics", "Python", "TraceSession, HopStats, route modes, DNS orchestration and metric formatting")
        Container(helper_client, "PacketHelperBackend", "Python", "Runtime backend that starts and talks to the packet helper")
        Container(helper, "pymtr.packet_helper", "Python subprocess", "Raw-socket event loop that sends tokenized probes and writes tokenized replies")
        ContainerDb(history, "Temporary SQLite history", "SQLite", "Per-trace hop samples used by charts, percentiles and FullReport")
        Container(exports, "Export/report layer", "Python", "Clipboard, TXT, CSV, HTML, chart image/HTML/PDF and FullReport PDF")
        ContainerDb(config, "PyMTR.ini", "INI", "Runtime options, column order/visibility/widths, chart metrics, formatting rules and host history")
        Container(docs, "Bundled documentation", "Sphinx/Markdown", "Local HTML documentation opened from the About menu")
        Container(cli, "CLI/TUI", "Python/Rich", "MTR-like argparse interface, finite report mode, and live terminal table")
    }
    Rel(analyst, gui, "Uses")
    Rel(gui, config, "Loads and saves AppSettings")
    Rel(cli, core, "Uses TraceSession, AppSettings and shared column catalog")
    Rel(cli, exports, "Writes TXT, CSV, HTML and PDF exports")
    Rel(gui, core, "Creates and controls TraceSession")
    Rel(core, helper_client, "Selects ProbeBackend")
    Rel(helper_client, helper, "Sends packet commands over stdin and receives replies over stdout")
    Rel(core, history, "Records visible snapshots")
    Rel(gui, exports, "Requests copy/export/report actions")
    Rel(exports, core, "Formats current HopStats snapshot")
    Rel(exports, history, "Reads samples for percentile values and chart/report history")
    Rel(gui, docs, "Opens local HTML docs")
    

C4 Component

        C4Component
    title Main Runtime Components
    Container_Boundary(app, "Desktop process") {
        Component(app_main, "pymtr.app.main", "Entry point", "Dispatches GUI, CLI report mode, or live Rich TUI")
        Component(settings, "AppSettings", "Configuration", "Loads/saves PyMTR.ini and migrates legacy Interval seconds to IntervalMs")
        Component(texts, "TextResources", "English text resources", "Loads built-in English defaults and .env branding overrides")
        Component(window, "PyMTRWindow", "Tkinter controller", "Owns menus, trace lifecycle, settings application, exports and dialog orchestration")
        Component(grid, "MetricGrid", "Tk Canvas grid", "Renders cell-level metrics, selection, reorder, resize and conditional formatting")
        Component(options, "OptionsDialog", "Tkinter dialog", "Edits interval, packet size, LRU, theme, route mode, DNS, debug and visible columns")
        Component(cli_component, "pymtr.cli", "argparse CLI", "Parses MTR-like options, starts GUI/TUI/report mode, supports MTR letters and PyMTR column keys, and writes requested exports")
        Component(tui, "pymtr.tui", "Rich TUI", "Renders live terminal tables from TraceSession snapshots")
        Component(properties, "PropertiesDialog", "Tkinter dialog", "Shows hop identity, IPv6 when available, snapshot/live metrics and historical charts")
        Component(chart, "HistoryChartFrame", "Tk Canvas chart", "Groups metrics, persists non-empty chart selections, zooms/pans and live-follows")
        Component(session, "TraceSession", "Trace orchestration", "Schedules TTL probes, applies route mode, updates HopStats and emits snapshots")
        Component(hop, "HopStats", "Metric model", "Maintains incremental latency, loss and jitter metrics")
        Component(history_store, "HopHistoryStore", "SQLite gateway", "Creates temporary history DB, records visible snapshots and calculates percentiles")
        Component(packet_backend, "PacketHelperBackend", "ProbeBackend", "Manages packet-helper subprocess and correlates tokenized replies")
        Component(packet_helper, "PacketHelper", "Subprocess runtime", "Uses raw sockets, ProbeRegistry and protocol parsers to send probes")
        Component(exporters, "export.*", "Exporters", "Render text, CSV, HTML, chart artifacts, clipboard text and PDF reports")
    }
    Rel(app_main, settings, "loads")
    Rel(app_main, cli_component, "delegates")
    Rel(cli_component, texts, "loads for GUI and report titles")
    Rel(app_main, window, "creates")
    Rel(cli_component, tui, "runs live terminal mode")
    Rel(cli_component, session, "runs finite report cycles through the same backend/session path as GUI")
    Rel(cli_component, exporters, "writes requested CLI TXT, CSV, HTML and FullReport outputs")
    Rel(window, grid, "renders MetricGridRow data")
    Rel(window, options, "opens")
    Rel(window, properties, "opens for selected hop")
    Rel(properties, chart, "embeds when history exists")
    Rel(window, session, "starts/stops")
    Rel(session, hop, "updates")
    Rel(session, history_store, "records visible snapshots")
    Rel(session, packet_backend, "calls probe_batch")
    Rel(packet_backend, packet_helper, "starts and exchanges packet messages")
    Rel(window, exporters, "invokes")
    Rel(exporters, history_store, "reads percentiles and chart samples")
    

Trace Data Flow

        sequenceDiagram
    participant User as Analyst
    participant GUI as PyMTRWindow
    participant Session as TraceSession
    participant DNS as DNS helpers
    participant Backend as PacketHelperBackend
    participant Helper as PacketHelper subprocess
    participant Hop as HopStats
    participant History as HopHistoryStore

    User->>GUI: Enter target and Start
    GUI->>GUI: remember host and create HopHistoryStore
    GUI->>Session: start(target, settings, history_store)
    loop every max(IntervalMs, 100ms)
        Session->>DNS: resolve_target(target)
        Session->>Backend: probe_batch(target_ip, ProbeRequest per TTL)
        Backend->>Helper: send-probe token, protocol, ip, ttl, size, timeout
        Helper-->>Backend: reply / ttl-expired / no-reply / error with same token
        Backend-->>Session: ProbeResult keyed by token
        Session->>Hop: begin_probe, record_reply when RTT exists, end_probe
        Session->>Session: apply static route lock or dynamic visible-hop depth
        Session->>DNS: schedule reverse_lookup(address) when UseDNS is enabled
        Session->>History: record_snapshot(cycle, visible snapshot)
        Session-->>GUI: on_snapshot(list[HopStats])
        GUI->>GUI: queue update and redraw MetricGrid
    end
    User->>GUI: Stop or Exit
    GUI->>Session: stop and join
    GUI->>History: close(remove=true)
    

SQLite History Flow

        flowchart TD
    Start["Start trace in PyMTRWindow"] --> Store["Create HopHistoryStore(target)"]
    Store --> DbFile["runtime_data_dir()/history/pymtr-history-{pid}-{session_id}.sqlite3"]
    Store --> MetaFile["Sidecar JSON metadata: pid, session_id, started_at, target, version, sqlite_path"]
    Cycle["TraceSession.trace_once applied hop results"] --> Snapshot["TraceSession.snapshot() returns visible cloned HopStats"]
    Snapshot --> Record["HopHistoryStore.record_snapshot(cycle, hops, trace_id)"]
    Record --> Insert["INSERT rows into hop_samples"]
    Insert --> Cache["Clear percentile cache"]
    DbFile --> Details["PropertiesDialog and HistoryChartFrame"]
    DbFile --> GridPercentiles["column_value/metric_numeric_value for LP50, LP95, LP99, JP50, JP95, JP99"]
    DbFile --> FullReport["write_full_report_pdf reads all hop samples"]
    MetaFile --> CleanupCheck["close(remove=true) verifies pid and session_id"]
    CleanupCheck --> Remove["Remove only matching SQLite and metadata files"]
    

Percentile Flow

        flowchart LR
    Samples["hop_samples ordered by timestamp_epoch_ms and cycle"] --> LatencySeries["Cumulative last_ms values per hop"]
    Samples --> JitterSeries["Cumulative jitter_ms values per hop"]
    LatencySeries --> LP50["nearest_rank_percentile(last_ms, 50)"]
    LatencySeries --> LP95["nearest_rank_percentile(last_ms, 95)"]
    LatencySeries --> LP99["nearest_rank_percentile(last_ms, 99)"]
    JitterSeries --> JP50["nearest_rank_percentile(jitter_ms, 50)"]
    JitterSeries --> JP95["nearest_rank_percentile(jitter_ms, 95)"]
    JitterSeries --> JP99["nearest_rank_percentile(jitter_ms, 99)"]
    LP50 --> Grid
    LP95 --> Grid["Main grid/export current percentile via percentile_value"]
    LP99 --> Grid
    JP50 --> Grid
    JP95 --> Grid
    JP99 --> Grid
    Samples --> ChartRead["HistoryChartFrame reads full hop history first"]
    ChartRead --> CumulativeValues["samples() calculates percentile values cumulatively before filtering"]
    CumulativeValues --> Viewport["filter_samples_by_viewport keeps only visible timestamps"]
    Viewport --> Chart["Chart lines, X-axis and scrollbar use the same TimeViewport"]
    

GUI And Export Flow

        flowchart TD
    Catalog["COLUMNS in pymtr.core.columns"] --> GridLabels["MetricGrid headers and cell order"]
    Catalog --> Help["HelpDialog metric descriptions"]
    Catalog --> OptionsColumns["Options visible-column checkboxes"]
    Catalog --> Reports["TXT/CSV/HTML/FullReport column order"]
    Settings["AppSettings.active_column_order()"] --> GridLabels
    Settings --> Reports
    Snapshot["PyMTRWindow.snapshot: current visible HopStats list"] --> RenderRows["column_value and metric_numeric_value"]
    History["HopHistoryStore"] --> RenderRows
    RenderRows --> MetricGrid["MetricGrid"]
    RenderRows --> Clipboard["Copy text/html clipboard"]
    RenderRows --> TextExport["TXT export to runtime_base_dir() by default"]
    RenderRows --> CsvExport["CSV export to runtime_base_dir() by default"]
    RenderRows --> HtmlExport["HTML export to runtime_base_dir() by default"]
    Snapshot --> FullReport["FullReport PDF with all catalog columns"]
    History --> FullReport
    History --> ChartExport["Hop chart PNG, JPG, HTML or PDF from PropertiesDialog"]
    Settings --> ChartExport
    

Logical Class Diagram

        classDiagram
    class AppSettings {
        +int interval_ms
        +int ping_size
        +int max_lru
        +bool use_dns
        +int max_hops
        +int timeout_ms
        +str route_mode
        +bool debug_enabled
        +str debug_log_path
        +str theme
        +list column_order
        +list visible_columns
        +dict column_widths
        +list chart_metric_keys
        +dict conditional_formats
        +list history
        +load()
        +save()
        +remember_host()
        +active_column_order()
    }
    class ConditionalFormatRule {
        +float normal_limit
        +float medium_limit
        +str normal_color
        +str warning_color
        +str critical_color
    }
    class TextResources {
        +dict values
        +load()
        +get()
        +column_label()
        +column_full_name()
        +column_description()
        +column_interpretation()
    }
    class PyMTRWindow {
        +TextResources texts
        +AppSettings settings
        +TraceSession session
        +HopHistoryStore history_store
        +list snapshot
        +_start_trace()
        +_stop_trace()
        +_render_table()
        +_apply_runtime_options()
        +_full_report()
    }
    class MetricGrid {
        +set_rows()
        +set_display_columns()
        +set_conditional_formats()
        +set_column_widths()
        +visible_column_order()
    }
    class TraceSession {
        +str target
        +AppSettings settings
        +str route_mode
        +list hops
        +start()
        +stop()
        +join()
        +trace_once()
        +snapshot()
    }
    class HopStats {
        +int index
        +str address
        +str hostname
        +int sent
        +int received
        +float last
        +float avg
        +float jitter
        +completed
        +loss
        +drop
        +display_name
        +begin_probe()
        +record_reply()
        +end_probe()
        +clone()
    }
    class HopHistoryStore {
        +str session_id
        +int pid
        +Path path
        +Path metadata_path
        +record_snapshot()
        +samples()
        +percentile_value()
        +time_bounds()
        +close()
    }
    class HistoryChartFrame {
        +HopHistoryStore history_store
        +int hop_index
        +TimeViewport viewport
        +selected_metrics()
        +_refresh_chart()
        +_draw_chart()
    }
    class PacketHelperBackend {
        +str name
        +process
        +probe_batch()
        +ensure_supported()
        +close()
    }
    class PacketHelper {
        +ProbeRegistry registry
        +dict raw_sockets
        +dict tcp_sockets
        +run()
    }

    AppSettings "1" o-- "*" ConditionalFormatRule
    PyMTRWindow --> AppSettings
    PyMTRWindow --> TextResources
    PyMTRWindow --> MetricGrid
    PyMTRWindow --> TraceSession
    PyMTRWindow --> HopHistoryStore
    PyMTRWindow --> HistoryChartFrame
    TraceSession --> HopStats
    TraceSession --> PacketHelperBackend
    TraceSession --> HopHistoryStore
    PacketHelperBackend --> PacketHelper
    HistoryChartFrame --> HopHistoryStore
    

SQLite ERD

        erDiagram
    sessions ||--o{ hop_samples : "session_id"
    sessions {
        TEXT session_id PK
        TEXT created_at
    }
    hop_samples {
        INTEGER id PK
        TEXT session_id FK
        INTEGER cycle
        INTEGER hop_index
        INTEGER ttl
        INTEGER timestamp_epoch_ms
        TEXT timestamp_local
        TEXT address
        TEXT hostname
        INTEGER sent
        INTEGER recv
        INTEGER drop
        REAL loss_percent
        REAL last_ms
        REAL best_ms
        REAL avg_ms
        REAL worst_ms
        REAL stdev_ms
        REAL gmean_ms
        REAL jitter_ms
        REAL javg_ms
        REAL jmax_ms
        REAL jint_ms
    }
    

SQLite Tables And Columns

Table	Column	Type	Purpose
sessions	session_id	TEXT PRIMARY KEY	Unique per-trace history identifier generated by HopHistoryStore.
sessions	created_at	TEXT NOT NULL	Local ISO timestamp for history creation.
hop_samples	id	INTEGER PRIMARY KEY AUTOINCREMENT	Local row identifier.
hop_samples	session_id	TEXT NOT NULL	Links rows to the owning trace session.
hop_samples	cycle	INTEGER NOT NULL	Trace cycle that produced the sample.
hop_samples	hop_index	INTEGER NOT NULL	Zero-based hop index matching HopStats.index.
hop_samples	ttl	INTEGER NOT NULL	One-based TTL/hop number shown to the user.
hop_samples	timestamp_epoch_ms	INTEGER NOT NULL	Sortable local timestamp used by charts and viewport bounds.
hop_samples	timestamp_local	TEXT NOT NULL	Local ISO timestamp used in chart tooltips and reports.
hop_samples	address	TEXT NOT NULL	Hop IP address captured in the visible snapshot.
hop_samples	hostname	TEXT NOT NULL	Hop hostname captured in the visible snapshot, or address/no-response label.
hop_samples	sent	INTEGER NOT NULL	Probes sent to this hop.
hop_samples	recv	INTEGER NOT NULL	Replies received from this hop.
hop_samples	drop	INTEGER NOT NULL	Completed probes without replies.
hop_samples	loss_percent	REAL NOT NULL	Packet loss percentage.
hop_samples	last_ms	REAL NOT NULL	Most recent RTT in milliseconds.
hop_samples	best_ms	REAL NOT NULL	Lowest observed RTT in milliseconds.
hop_samples	avg_ms	REAL NOT NULL	Mean observed RTT in milliseconds.
hop_samples	worst_ms	REAL NOT NULL	Highest observed RTT in milliseconds.
hop_samples	stdev_ms	REAL NOT NULL	Sample standard deviation of RTT values.
hop_samples	gmean_ms	REAL NOT NULL	Geometric mean RTT.
hop_samples	jitter_ms	REAL NOT NULL	Difference between latest and previous RTT.
hop_samples	javg_ms	REAL NOT NULL	Average jitter.
hop_samples	jmax_ms	REAL NOT NULL	Maximum jitter.
hop_samples	jint_ms	REAL NOT NULL	Smoothed interarrival jitter estimate.

The schema also creates idx_hop_samples_session_hop_time on (session_id, hop_index, timestamp_epoch_ms) for ordered per-hop chart and percentile reads.

PyMTR.ini Hierarchy

        flowchart TD
    Ini["PyMTR.ini at runtime_base_dir()"] --> Config["[Config]"]
    Ini --> Columns["[Columns]"]
    Ini --> Widths["[ColumnWidths]"]
    Ini --> Formatting["[ConditionalFormatting.{column_key}]"]
    Ini --> History["[History]"]

    Config --> IntervalMs["IntervalMs: milliseconds between cycles"]
    Config --> LegacyInterval["legacy Interval: seconds, read-only migration path"]
    Config --> PingSize["PingSize"]
    Config --> MaxLRU["MaxLRU"]
    Config --> UseDNS["UseDNS"]
    Config --> MaxHops["MaxHops"]
    Config --> TimeoutMs["TimeoutMs"]
    Config --> RouteMode["RouteMode: static or dynamic"]
    Config --> Debug["DebugEnabled and DebugLogPath"]
    Config --> Theme["Theme"]
    Config --> Language["Language"]
    Config --> ChartMetrics["ChartMetrics: persisted selected hop-chart metric keys"]

    Columns --> Order["Order: complete column catalog order"]
    Columns --> Visible["Visible: currently displayed/exported columns"]
    Widths --> PerColumn["{column_key}=pixel width, minimum 24"]
    Formatting --> Limits["NormalLimit and MediumLimit"]
    Formatting --> Colors["NormalColor, WarningColor and CriticalColor"]
    History --> Hosts["Hosts: pipe-delimited host LRU list"]
    

Screens, Dialogs, Menus And Settings Map

        flowchart TD
    Main["PyMTRWindow"] --> MenuBar["In-window menu bar"]
    Main --> HostPanel["Host combobox + Start/Stop button"]
    Main --> Grid["MetricGrid"]
    Main --> Footer["Footer status + app link"]

    MenuBar --> FileMenu["File menu"]
    MenuBar --> OptionsButton["Options button"]
    MenuBar --> AboutMenu["About menu"]
    MenuBar --> ExitButton["Exit button"]

    FileMenu --> CopyText["Copy text"]
    FileMenu --> CopyHtml["Copy HTML"]
    FileMenu --> ExportText["Export TXT"]
    FileMenu --> ExportCsv["Export CSV"]
    FileMenu --> ExportHtml["Export HTML"]
    FileMenu --> FullReport["Generate FullReport PDF"]
    FileMenu --> OpenLog["Open log folder"]
    FileMenu --> OpenTemp["Open temp folder"]

    OptionsButton --> OptionsDialog["OptionsDialog"]
    OptionsDialog --> RuntimeSettings["IntervalMs, PingSize, MaxLRU, UseDNS, DebugEnabled, DebugLogPath"]
    OptionsDialog --> UiSettings["Language, Theme, RouteMode"]
    OptionsDialog --> ColumnSettings["Visible column checkboxes"]

    Grid --> RowSelect["Row selection"]
    Grid --> RowDoubleClick["Double-click row"]
    Grid --> HeaderDrag["Header drag reorder"]
    Grid --> HeaderResize["Header resize"]
    Grid --> HeaderContext["Right-click numeric header"]

    RowDoubleClick --> PropertiesDialog["PropertiesDialog / Hop Details"]
    PropertiesDialog --> LiveRows["Live metric rows"]
    PropertiesDialog --> HistoryChart["HistoryChartFrame"]
    PropertiesDialog --> ChartExport["Chart export: PNG, JPG, HTML, PDF"]
    HistoryChart --> MetricGroups["Packet, latency and jitter metric groups"]
    HistoryChart --> ChartControls["Select all, Clear all, Reset zoom, Live"]
    HeaderContext --> ConditionalFormatDialog["ConditionalFormatDialog for that numeric column"]

    AboutMenu --> LicenseDialog["LicenseDialog"]
    AboutMenu --> HelpDialog["HelpDialog"]
    AboutMenu --> Docs["Open bundled documentation"]
    AboutMenu --> HotkeysDialog["HotkeysDialog"]
    AboutMenu --> Repository["Open repository URL"]
    

Historical Storage Decision

SQLite remains the active historical storage engine in HopHistoryStore. DuckDB was evaluated during earlier performance work, but it was not adopted for the active application because the improvement was limited to selected query paths and did not justify the extra dependency, packaging size, and operational complexity. The historical benchmark material is archived under legacy-docs; the public Sphinx site no longer publishes a Performance page.

The current SQLite optimization strategy is conservative:

• Use targeted column lists for chart reads instead of broad SELECT * queries.

• Cache full per-hop sample series until a new snapshot invalidates the cache.

• Calculate historical percentiles before applying a chart viewport filter.

• Downsample only the Canvas rendering path, never the stored history or report data.

• Keep Hop Details responsive by opening the window independently from heavy chart rendering paths.

Packaged Entry Points

The Windows self-contained release intentionally contains two executables:

• PyMTR.exe is windowed and starts the Tkinter GUI.

• PyMTR-CLI.exe is console-enabled and runs pymtr HOST, pymtr --report HOST, --help, --version, and export commands without losing terminal output.

From source or an installed Python package, the console command remains pymtr. This split avoids a Windows case-insensitive filesystem collision between PyMTR.exe and pymtr.exe while preserving Unix-style CLI documentation.

Operational Notes

• Runtime tracing uses PacketHelperBackend selected by select_backend(); FakeProbeBackend exists for deterministic tests only.

• Static route mode freezes the first complete route depth; dynamic route mode keeps the visible depth aligned with each cycle.

• HopHistoryStore records only visible snapshots, so stale hidden hops are not included in charts, percentiles or reports.

• Conditional formatting is configured from a numeric column header and is triggered per cell in that same column only; it does not evaluate or recolor other columns in the row.

• Percentile columns are derived from the temporary SQLite history and do not mutate HopStats incremental MTR-compatible metrics.

• TXT, CSV and HTML exports default to runtime_base_dir(); FullReport PDF uses the same default through default_report_export_dir().

• Temporary SQLite cleanup is guarded by the JSON sidecar’s pid and session_id to coexist with parallel PyMTR instances.