Bernt
b934797e65
- Added four pilot phases (product research, not marketing): 1. Collection — what can actually be detected? 2. Analysis — are decisions understandable? 3. Verification — was recommendation correct? 4. Reflection — what needs to change? - Measurement principle: - Not: Did AI find a crack? - But: Did this become a decision a real person could act on? - Document 'non-decisions' — equally valuable as clear decisions - Film workflow, not just infrastructure: - How you find area, choose mission, document - What feels unclear, when you become uncertain - When system saves time - Observer mindset: document first, change model later - Build model from real workflows, not assumptions Rationale: First 20-50 real missions teach more than months of modeling.
4.8 KiB
FIELD TRIAL LOG
Observation Protocol for Decision Model Validation
| Version | 1.0 |
| Status | ACTIVE |
| Purpose | Collect raw data from real customer cases |
Log Entry Template
| Field | Example | Notes |
|---|---|---|
| Date | 2026-07-15 | |
| Customer Type | Municipality / Property Owner / Contractor | |
| Decision | Inspect Road A12 | What the system recommended |
| Decision Accepted | Yes / No | Did the user accept? |
| Action Executed | Yes / No / Pending | Was the action carried out? |
| Outcome | Crack confirmed | What actually happened |
| Time to Decision | 3 minutes | How long from login to decision |
| User Feedback | "We trusted the recommendation." | Verbatim or summary |
| Confidence | 87% | System confidence at decision time |
| Notes | Free text | Anything else observed |
Example Entries
Entry 1
| Field | Value |
|---|---|
| Date | 2026-07-15 |
| Customer Type | Municipality |
| Decision | Inspect Road A12 |
| Decision Accepted | Yes |
| Action Executed | Yes |
| Outcome | Crack confirmed, repaired within 14 days |
| Time to Decision | 3 minutes |
| User Feedback | "We trusted the recommendation." |
| Confidence | 87% |
| Notes | User clicked through Explainability chain |
Entry 2
| Field | Value |
|---|---|
| Date | 2026-07-16 |
| Customer Type | Property Owner |
| Decision | Repair facade immediately |
| Decision Accepted | No |
| Action Executed | No |
| Outcome | Deferred, user chose to wait |
| Time to Decision | 5 minutes |
| User Feedback | "Too expensive right now." |
| Confidence | 92% |
| Notes | Confidence was high but user rejected due to cost |
Questions This Log Answers
After collecting entries:
- Were recommendations accepted? (Decision Adoption Rate)
- Were actions executed? (Execution Rate)
- Did outcomes match predictions? (Decision Accuracy)
- Was confidence reasonable? (Calibration)
- Which recommendations were ignored? (Rejection Analysis)
- How long did decisions take? (Time to Decision)
Future KPIs
Decision Adoption Rate
Recommendations accepted
────────────────────────
Total recommendations
Example: 72 of 100 = 72%
Decision Accuracy
Decisions that proved correct
─────────────────────────────
Total decisions
Example: 65 of 72 executed = 90%
Note: These require months of data. Start collecting now, calculate later.
Pilot Structure
Four phases — product research, not marketing:
| Phase | What You Do | What You Learn |
|---|---|---|
| 1. Collection | Film, photograph, create observations | What can actually be detected? |
| 2. Analysis | Let pipeline create Findings and Decisions | Are decisions understandable and relevant? |
| 3. Verification | Compare with reality on site | Was the recommendation correct? |
| 4. Reflection | Document what worked and what didn't | What needs to change in the model? |
Full chain captured:
Reality → Observation → Evidence → Finding → Decision → Action → Outcome → Learning
What to Measure
Not: Did AI find a crack? But: Did this become a decision a real person could act on?
Example:
- AI finds 23 cracks
- LandveX says: "Inspect Road A12 within 30 days"
- The second is the product.
Document "Non-Decisions"
Also valuable:
- "No action needed"
- "More data required"
- "Cannot recommend anything yet"
If the system always tries to give advice even when evidence is weak, you risk building a system that feels confident when it shouldn't.
Film the Workflow
Since you are conducting the pilots, film more than just infrastructure. Also film:
- How you find an area
- Why you choose a mission
- How you document
- How long it takes
- What feels unclear
- When you become uncertain
- When the system saves time
This material becomes invaluable for product development, onboarding, training, and sales.
Observer Mindset
Principle: If data contradicts the model, the model should change.
Use the same approach during pilots. If you notice:
- A step feels unnecessary
- A decision becomes unclear
- A recommendation cannot be acted on
Document first, change model later. Build the model from real workflows instead of assumptions.
The first 20–50 real missions will give you more valuable product knowledge than months of additional modeling.
Status
ACTIVE — Awaiting first customer case
ändringshistoria
| Version | Datum | Beskrivning |
|---|---|---|
| 1.0 | 2026-07-02 | Initial field trial log template |
| 1.1 | 2026-07-02 | Added pilot phases, measurement principles, filming protocol |