XDMA trained at x1? Compare the link you expected with the one captured.

1. Write down what x8 means in this system

“An x8 FPGA card in an x16 slot” does not establish an eight-lane electrical path. Record the card’s wired lanes, connector/riser route, slot’s electrical width, lane-sharing or bifurcation settings, generated PCIe IP width/speed and the host port’s capability. A mechanically long slot can provide fewer lanes. Read the platform manual rather than inferring wiring from its connector.

Also record the exact FPGA part, speed grade, PCIe hard block, IP release and generated design parameters. AMD PG195: supported device, speed and width combinations documents supported XDMA combinations; not every device and speed grade supports every configuration. Use the matching release for your build. A GUI screenshot from another board is not your configuration evidence.

If the endpoint does not appear at all, use the enumeration guide first. A queue-register investigation cannot explain a PCIe endpoint that the host has not discovered.

2. Capture current and maximum values on both ends

The default collector captures the selected function and discovers its upstream sysfs chain. Import the resulting JSON into the link investigation. It shows the nearest captured port separately from the endpoint; no network or device-memory access is used by the default collector.

# Review/download the collector from the Console first.
# Replace this example BDF with your enumerated FPGA function.
python3 header7f-linux-capture.py 0000:01:00.0 > before.json
lspci -D -t
lspci -D -PP -nn

# A focused manual sysfs view; absent files are missing evidence.
for field in current_link_speed current_link_width max_link_speed max_link_width; do
  printf "%s: " "$field"
  cat "/sys/bus/pci/devices/0000:01:00.0/$field"
done

The maximum fields belong to the function from which they were collected; they do not guarantee the operational capability of the entire path. Current attributes are snapshots. Their implementation is visible in the Linux source: current and maximum link attributes. If the endpoint says x8 and the immediate partner says x1, an x1 result is consistent with the lower captured partner capability. It does not prove a lane failure.

If both functions report x8 but current width is x1, you have a discrepancy worth investigating. You still need the physical wiring, platform policy and workload context. A capability comparison does not identify which lane, connector or component is responsible. Unknown maximum values must remain unknown; never substitute the card’s advertised headline for a captured attribute.

3. Repeat under a controlled condition

Keep an idle capture, then repeat the same capture during an existing vendor-supported workload that you already know how to run safely. Record the workload, transfer direction, duration, buffer/transfer size and whether the capture occurred during or after it. The collector does not generate DMA traffic or measure throughput.

# During the chosen controlled workload, in another terminal:
python3 header7f-linux-capture.py 0000:01:00.0 > during-load.json
# After that same workload has stopped:
python3 header7f-linux-capture.py 0000:01:00.0 > after.json

Keep host firmware, slot, bitstream and driver constant for the first comparison. A lower idle speed can reflect an operating policy rather than lost capacity. If a firmware target-speed limit or bandwidth-management policy applies, record it with the result. Avoid disabling power management globally merely because one reading is lower.

Separate a width/speed observation from an application throughput measurement. Payload efficiency, transfer size, host memory, DMA engine setup and software can limit throughput even when the link is at the expected width. Conversely, a clean sample transfer does not establish that the link never retrains or errors over a longer interval. Use the same workload and acceptance conditions for the working and failing run.

4. Look for counter changes rather than totals alone

The collector captures available AER text counters on the endpoint and upstream context. For a manual local view, use the verified immediate-port BDF as well as the endpoint:

# Example addresses only; identify the actual immediate partner first.
for device in 0000:01:00.0 0000:00:01.0; do
  for field in aer_dev_correctable aer_dev_nonfatal aer_dev_fatal; do
    printf "%s %s\n" "$device" "$field"
    cat "/sys/bus/pci/devices/$device/$field"
  done
done

Save before/after values around the same operation. Do not clear counters to make the display look clean. A nonzero total may predate the test; a missing attribute is not a zero. The Linux ABI: AER device and root-port counters explains that errors can be seen at the link partner while the endpoint reports zero, and root-port totals can cover a hierarchy. Do not sum category totals and individual rows as though they were independent events.

Correlate any increase with a time-bounded kernel excerpt. Linux: PCIe AER reporting and recovery distinguishes correctable from uncorrectable reporting. Correctable errors alone do not prove data loss. A PCIe Completion Timeout is also different from a driver waiting for a software completion entry. The Linux investigation keeps those observations separate and does not infer the responsible descriptor or payload.

5. Decode captured configuration when it is appropriate

When metadata and the expected configuration disagree, an extended config capture can add PCIe Link Capabilities/Status and AER register context. Review the vendor’s capture guidance first: pciutils: lspci tree, domain, path and configuration options notes that broader config reads can affect some devices. Use the exact function, and preserve the failing state before considering any configuration change.

# Optional configuration capture; use your actual endpoint BDF.
sudo lspci -D -s 0000:01:00.0 -xxxx > endpoint-config.txt
# A second capture of the known immediate partner, if appropriate:
sudo lspci -D -s 0000:00:01.0 -xxxx > port-config.txt

# Alternatively, include the endpoint bytes in collector JSON:
python3 header7f-linux-capture.py 0000:01:00.0 \
  --include-config > endpoint-with-config.json

-xxxx requests the extended 4 KiB configuration range; a 256-byte -xxx capture cannot contain extended capabilities starting at offset 0x100. Permissions or device behavior may still limit the actual bytes returned. Header7F preserves partial coverage and decodes only present bytes; missing AER bytes do not become zero status.

A supported DLL Active bit reported clear is an explicit inactive-link clue. An unsupported DLL Active reporting field is not the same condition. Sticky status bits need timing and baseline context. Configuration register access is separate from reading BAR/MMIO data; the Linux: PCI resources through sysfs documents those distinct interfaces. This workflow performs no register writes or link retraining commands.

6. Narrow the FPGA-side question with debug evidence

If the expected lane path and supported configuration are established, use a matching reference build as the next comparison. Save its build/tool identifiers and the same host capture. If the reference also negotiates x1, a custom AXI register change is a weaker lead than the shared board, host and PCIe path. If only your new build differs, inspect its generated IP parameters, lane placement, constraints, clock/reset connections and implementation results.

Where enabled by your IP release, collect negotiated width/speed, link-up and LTSSM debug evidence through the vendor’s documented facilities. AMD PG195: LTSSM, negotiated-width and speed debug options describes debug ports and transition capture options for XDMA. Use the release’s actual signal encoding; do not decode a numeric state with a table for a different core.

Choose one discriminating test: the known-good image in the same slot; the same image in another vendor-approved slot; or the supported connection without an optional riser. Change hardware only with the system powered down according to its service procedure. Record the result even when it does not change. A successful different-slot run narrows the experiment but does not, by itself, prove which part of the original route failed.

7. Move to DMA only after defining link acceptance

State the acceptance criterion explicitly: expected IDs and resources, the documented width/speed in the chosen operating state, and a controlled workload that completes with its expected data check. Include before/after error evidence and the original reproduction interval. “x8 once” is less useful than a repeatable startup and workload sequence.

If the link now meets that criterion but DMA still stalls, capture one operation end to end: submission, mapping, device progress, completion and interrupt handling. Host BAR allocations are not DMA buffer addresses; Linux: DMA addresses and mappings explains the address-space distinction. A stable link does not validate buffer permissions or a queue’s programming.

Export the investigation with its exact versions, physical route, current/max values and remaining unknowns. That gives a colleague or vendor a reproducible question: what changed between the working and failed state, and which single observation would distinguish the remaining explanations?