Lab 9 — Diagnostic Challenge
Estimated time: 90 minutes Machines used: WAZUH-SRV, UB2604, WIN11 Prerequisites: Labs 0 through 8 complete; environment previously validated
Purpose
Every lab to this point handed you working commands. This one does not.
Your instructor has introduced a number of faults into the laboratory environment. Some are obvious. Some produce no error message at all and are visible only as an absence — a dashboard that is quieter than it should be.
Your task is to find them, prove what they are, fix them, and document your reasoning.
This is the closest exercise in the course to the actual work of security operations. The first two years of most analyst careers contain more "why is this not reporting?" than "here is a confirmed intrusion."
Why this matters: An investigation built on incomplete telemetry produces a confident wrong answer. Before you can trust any alert, you must be able to prove your instruments work.
Rules for This Lab
- Diagnose in pipeline order. Confirm collection before decoding, decoding before rules. Testing at random produces confusion, not answers.
- Prove each fault before you fix it. Record the evidence that identifies the root cause. "It works now" is not a diagnosis.
- Change one thing at a time. If you change three settings and the symptom clears, you have not learned which one mattered.
- Document as you go. Reconstructing your reasoning afterward from memory is unreliable, and the record is what you are graded on.
- You may use the Field Guide, the textbook, and official documentation. You may not ask another student for the answers.
The Reported Symptom
A colleague reports:
"The SOC dashboard looks fine, but something feels off. I ran an attack simulation on WIN11 an hour ago and I am not seeing what I expected. Some things are there, some are not. Can you check the environment?"
That is all the information you are given. It is also, realistically, all the information you would be given.
Part 1 — Establish What Is Actually True
Before forming any hypothesis, establish ground truth. Complete the table below with what you observe, not what you expect.
| Check | Command or location used | Observed result | Normal? |
|---|---|---|---|
| Wazuh Manager service state | |||
| Wazuh Indexer service state | |||
| Wazuh Dashboard service state | |||
| Indexer disk usage | |||
Agent u2604 status |
|||
Agent win11 status |
|||
ens192 interface flags |
|||
| Suricata service state | |||
| Suricata capture vs. drop counters | |||
| System time on WAZUH-SRV | |||
| System time on UB2604 | |||
| System time on WIN11 | |||
| Most recent event in Discover (each agent) |
Checkpoint 1. You have completed the table above using observed values. Do not proceed on assumptions — an unchecked row is an unknown, not a pass.
Part 2 — Identify the Faults
For each fault you find, complete one block. Use as many blocks as you need; blank blocks are acceptable if you find fewer faults than provided.
Fault A
Symptom observed:
Which pipeline stage is affected? (collection / transport / decoding / rule evaluation / storage / presentation)
Evidence that identifies the root cause (paste the specific output, field, or counter — not a description):
Root cause:
How you fixed it:
Evidence the fix worked:
Fault B
Symptom observed:
Which pipeline stage is affected?
Evidence that identifies the root cause:
Root cause:
How you fixed it:
Evidence the fix worked:
Fault C
Symptom observed:
Which pipeline stage is affected?
Evidence that identifies the root cause:
Root cause:
How you fixed it:
Evidence the fix worked:
Fault D
Symptom observed:
Which pipeline stage is affected?
Evidence that identifies the root cause:
Root cause:
How you fixed it:
Evidence the fix worked:
Fault E
Symptom observed:
Which pipeline stage is affected?
Evidence that identifies the root cause:
Root cause:
How you fixed it:
Evidence the fix worked:
Checkpoint 2. Every fault you claim is supported by specific evidence, not by the fact that the symptom disappeared after you changed something.
Part 3 — Verify End to End
Fixing individual faults is not the same as restoring the environment. Prove the whole path works.
- Generate activity on WIN11 that you know should produce telemetry.
- Confirm the event is recorded locally on the endpoint.
- Confirm the event reached the Manager.
- Confirm the event was decoded into populated fields.
- Confirm a rule evaluated it and produced an alert at the expected level.
- Confirm the alert is visible and searchable in the Dashboard.
Record the evidence for each stage:
| Stage | Evidence observed | Time |
|---|---|---|
| Generated on endpoint | ||
| Reached the Manager | ||
| Decoded into fields | ||
| Rule evaluated | ||
| Visible in Dashboard |
Checkpoint 3. You have traced one event end to end and can show evidence at every stage.
Part 4 — The Silent Fault
At least one fault in this environment produces no error message anywhere. It is visible only as missing data.
Which fault was it?
Why did it produce no error?
What routine check would have caught it before it mattered?
If this fault had gone unnoticed for a week, what would the operational consequence have been?
Part 5 — Operational Report
Write a brief report, as you would submit to a SOC lead. Use complete sentences.
Summary of findings (what was wrong, in plain language):
Impact assessment (what visibility was lost, and for how long, as far as you can determine):
What you could not determine (gaps are findings — state them):
Recommended preventive measures (which checks, at what interval, would have caught these earlier):
Checkpoint 4. Your report states impact and names at least one thing you could not determine.
Deliverables
- Completed ground-truth table from Part 1.
- One completed fault block for each fault identified.
- Completed end-to-end verification table from Part 3.
- Answers to Part 4.
- Operational report from Part 5.
Key Terms
Silent failure. A fault that degrades or stops a function without producing an error message. Detected only by monitoring the expected behavior, not by waiting for an alarm.
Ground truth. What is actually observed to be the case, as distinct from what is expected or assumed.
Pipeline-order diagnosis. Testing each stage of the telemetry path in sequence — collection, transport, decoding, evaluation, storage, presentation — rather than changing settings at random.