Add optional network loot tracking alongside OCR #21
Labels
No labels
status:done
status:in-progress
status:ready
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
rexlManu/garmith-app#21
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Research completed against Shoki697/BDOLootTracker b70403151f7ea93f89972dd68973786dbeed0568. Passive TCP parsing appears feasible but is not verified on this BDO/Proton build. Keep OCR working. Implement a Linux proof first, then Windows capture support and settings selection.
Acceptance: independently written parser; validated versioned region profiles; TCP reassembly/retransmit handling; ground-loot filtering that rejects storage/market/maid transfers; one loot source per session so OCR banners cannot double count; explicit capture prerequisite/status handling; filtered local capture without storing full packets by default; known item/quantity checks; compare a live session against inventory totals. Verify patch compatibility from samples, not a weekly timer. No automatic fallback that merges counts. Test stop, pause, reconnect, packet loss and malformed data.
Reference has no license file in the inspected checkout. Do not copy its implementation. docs/network-tracking.md records findings and scope. No live network capture or machine permission changes were made during research.
Research and proposed checks pushed in
5566751:5566751See docs/network-tracking.md. Current reference EU profile is EU-2026.09.10.3 with configurable framing, item/quantity offsets and ground-loot checks. Weekly patches require verification, but a weekly opcode change is not established. Linux offline parser and filtered live validation come first. Network mode must exclude both OCR feed and rare-banner counts. Implementation remains ready; no app behavior or system capture permissions changed.
Implemented and pushed:
4a42f8aSettings > Game capture now selects OCR or Network. Linux uses dumpcap/tcpdump; Windows uses Wireshark dumpcap with Npcap. The independent TypeScript parser handles classic pcap, IPv4/IPv6, TCP reassembly, duplicate/transfer exclusion and bounded input. Profiles can be imported as validated JSON. Only one source contributes loot; active/paused/recovered sessions lock source changes. Drain keeps TCP state so retransmitted paused drops do not count.
Checks: workspace typecheck and 283 behavior tests passed before final EOF handling; all 14 parser tests pass after that change. End-to-end synthetic packet helper through parser, IPC, session totals and persistence passes on the final build. Settings persistence/profile/sender/paused-session checks pass. Existing generated OCR pipeline and Stop checks pass. Design detector: zero findings, no suppressions.
No live traffic captured, no driver/helper permissions installed, no running app restarted. Current EU profile and real inventory totals remain unverified, as does actual Windows/Npcap gameplay. Keep this issue in progress. docs/network-tracking.md and WORKFLOW.md include setup and remaining limits. Automatic profile downloads, calibration and relay/VPN discovery remain future work.
Follow-up
b8c267b:b8c267bRead-only inspection showed a paused session and Full auto, so the source lock was expected. Found and fixed a second issue: OCR start/stop did not publish lock changes, leaving the open Settings panel disabled after watching stopped. Added lifecycle notifications and explicit Dashboard > Stop > Manual instructions. The test now starts/stops OCR watching without a session-store write and verifies the open panel locks/unlocks. Build, network Settings and Stop tests pass. Design detector has no findings or suppressions. No user session or preferences were changed.
Fixed empty adapter selection in commit
59711ab: tcpdump lists entries such as 1.enp18s0 without a space. Discovery now accepts both spaced and unspaced formats. Read-only discovery on this workstation lists the interfaces, including default-route adapter enp18s0. Checks passed: 11 capture helper tests, pnpm build, pnpm test:network-settings, and git diff --check. No live packets were captured or permissions changed. Restart pnpm dev, then use Refresh adapters. Live loot count validation remains open.Live empty-loot investigation: tcpdump is running on the default-route adapter and the game has an established TCP connection on profile port 8889. A bounded in-memory sample received 499 packets with no kernel drops, but zero decoded loot events. A second sample also had zero occurrences of the current profile signature. Raw packets were not saved. Investigating current message format; adding parser progress counters so receiving traffic and matching loot are distinct states. Network mode remains unverified.
Pushed
e8cb429: Settings now distinguishes received game packets, decoded loot events and ground-filter exclusions. Validation passed: 15 parser tests, 8 controller tests, pnpm typecheck, pnpm build, pnpm test:network-settings and pnpm test:network-tracking. Live samples received traffic with no kernel drops and no built-in signature matches. No raw captures saved; running app and profile unchanged. Live counting is still unresolved. Waiting for confirmed pickups and inventory totals before calibrating the profile; keep this issue in progress.User confirms BDO displays pickups while Garmoth stays empty. Follow-up bounded in-memory game capture received 2263 packets over two minutes with zero kernel drops and zero current-profile matches. A later sample also had zero matches. During inspection the real app capture helper was no longer running; only the diagnostic helper remained. Requested the current Settings status/error and expected trash item/inventory increase. Do not change the signature or offsets from incidental item-ID matches. Diagnostic helper stopped; temporary probe removed; no raw capture or app preferences saved. Issue remains unresolved.
Pushed
4c70938. Confirmed live Torn Buccaneer Lapel comparisons: 445 to 543 (+98) matched 17 observed quantity fields; 543 to 590 (+47) matched 9 decoder events. New EU-2026.09.22.1 profile uses the observed 258-byte single-item format, signature CF 18 01, item offset 33, quantity offset 37; all second-sample messages passed the new positive-sample byte checks. Startup upgrades only the unchanged initial profile and preserves adapter/custom profiles. Invalid-length signature collisions now resynchronize instead of stopping capture. Passed 17 parser tests, 10 controller tests, typecheck, build, network Settings and synthetic helper-to-session tests. The reported crystal pickup was not independently verified live; its item ID has a generated test. Keep experimental/in progress for non-loot transfer exclusion, other message formats, longer grinds and Windows/Npcap. No raw packets saved; running app and installed release unchanged. User should Stop, close the app and run pnpm dev to load the new profile.User confirmed the rebuilt network tracker works well. Saved the repeatable calibration workflow in docs/network-profile-calibration.md, commit
a922f49: capture diagnosis, bounded in-memory probes, TCP reassembly, known-item searches, candidate length/signature/offset checks, exact +98 and +47 count vectors, transfer-control limits, profile migration and validation commands. Linked from AGENTS.md, WORKFLOW.md and network-tracking.md. Documentation links and diff checks passed; no runtime changes or rebuild needed. Guide: https://git.it-lampe.de/rexlManu/garmith-app/src/branch/main/docs/network-profile-calibration.md . Keep issue open for transfer exclusion, broader drop/message coverage and Windows/Npcap.User reports horse deposits duplicating loot. Implementing the requested quantity safeguard: ignore each decoded message with 500 or more items; preserve smaller independent pickups even when coalesced in one TCP packet. This is a heuristic, not verified transfer identification. Keeping this issue in progress for live deposit checks and existing Windows/coverage limits.
Pushed 3020f0c. Default guard ignores each decoded message with 500 or more items, continues parsing later pickups, and reports ignored-message counts. Existing profiles inherit maximumPickupQuantity=499 unless explicitly overridden in a validated imported profile. Packet signatures/offsets and saved sessions are unchanged. Passed: 19 parser tests, 10 controller tests, typecheck, build, test:network-settings, and test:network-tracking including 500/3000-item exclusions. The running app was not restarted. This remains a quantity heuristic: smaller transfers can count and legitimate large pickups are excluded. Keep issue open for live horse-transfer verification and the existing coverage/Windows checks.
Pushed
e502405. The loot-source selector now links to Dashboard when locked. The recovered session or automatic watcher must be stopped and Manual selected before changing source; this prevents mixing OCR and network totals. Network pickup parsing and unavailable-adapter behavior remain open under this issue.2026-09-24 EU loot recalibration pushed in
5b4a3d9:5b4a3d9The old built-in profile received game traffic but matched no loot. Two bounded Lapel comparisons matched 0 to 104 (+104, 17 candidates) and 104 to 194 (+90, 17 candidates). A paired live sample used the actual NetworkLootDecoder and matched 240 to 287 (+47, eight events), with no capture gaps or parser errors. The new built-in profile EU-2026.09.24.1 uses the observed 248-byte format. Only exact saved built-in profiles migrate; edited/imported profiles stay unchanged. No raw capture was saved.
Checks passed: 20 parser tests, 10 controller tests, workspace typecheck/tests, synthetic helper-to-session test, Settings test and diff check. The active pnpm dev watcher rebuilt the main bundle; a separate production build was not run while that dev server was active. The installed release is unchanged. Keep this issue open for a short live session with the updated app, non-loot transfer exclusion, other items, longer grinds and Windows/Npcap.
Live development check complete: Emmanuel confirmed Settings shows EU-2026.09.24.1 and Live loot matches the Torn Buccaneer Lapel inventory increase after one pack. Recorded in
1d59a3e. Keep #21 open for transfer exclusion, other items, longer sessions and Windows/Npcap. The installed release remains unchanged.