lif-viz — Data audit

Every job's data is checked automatically before anything is cleaned or plotted. The audit runs on the files you submitted, exactly as submitted, so problems are reported rather than quietly smoothed over.

Reading the results

Each check below produces one of three results. The badge next to "Audit Report" on your results page is the worst result of any single check.

PASS The check ran and found nothing.
WARN Something unusual that deserves a look. Your outputs are still produced, but confirm the flagged data is real before relying on it.
FAIL A problem serious enough that some outputs may be missing or wrong. Fix the input files and resubmit.

The results page lists only the WARN and FAIL lines; "View full report" opens the whole checklist, including every check that passed. It is also audit_report.txt in the "Download all results" zip file, which ends with a note of what the cleaning step below changed. Depths in the report are below ground surface, in centimeters, with feet in brackets.

The audit does not stop your job. Even with a FAIL, every output that can still be produced is produced. Deciding whether to use them is up to you.

What gets checked

Each line in the report starts with the check's name (e.g. depth_gap), followed by the boring(s) it applies to.

Check	What it looks for	Result
`boring_count`	The job has no borings, or two borings share the same Boring ID.	FAIL
`depth_coverage`	A boring with no readings at all (FAIL), or fewer than 5 readings (WARN).	FAIL / WARN
`depth_gap`	Two consecutive readings in one boring more than 15.0 cm (0.5 ft) apart, for example a rod change or a stretch where the probe stopped recording. The report gives the largest gap's depths.	WARN
`channel_consistency`	The set of fluorescence channels changes partway down a boring.	WARN
`pct_re_sanity`	Every %RE reading in a boring is exactly 0 (a possible sensor fault), or any reading is above 1000 %RE.	WARN
`duplicate_depth_record`	The same depth appears more than once in one boring.	WARN
`missing_channel_data`	Readings missing one or more fluorescence channel values.	WARN
`coordinate_completeness`	Any boring without coordinates. That boring still gets its own boring log, but is left out of the 3D and plan-view outputs.	WARN
`coincident_borings`	Two borings less than 0.5 m (1.6 ft) apart. The 3D interpolation may not be able to separate two borings at essentially the same spot. If it can't, the 3D and plan-view outputs are skipped, and you still get the boring logs and this report.	FAIL
`interpolation_coverage`	Fewer than two borings with coordinates. The 3D and plan-view outputs need at least two, so they aren't produced.	WARN
`collinear_borings`	Every boring lies within 1.0 m (3.3 ft) of one straight line (always true with only two borings). The 3D and plan-view outputs still render, but anything away from that line is extrapolated, not supported by data.	WARN
`site_image_overlap`	Only checked if you upload a site image. Some borings fall outside the image, or the image covers none of them. In that case the image is left out of the 3D and plan-view outputs, since it is probably not georeferenced, or the coordinates use a different CRS.	WARN

What happens to flagged data

After the audit, a small, fixed set of cleaning rules is applied before plotting:

Out-of-order depths: each boring's readings are sorted by depth.
Duplicate depths: the reading with the highest %RE is kept (with its own channel values) and the others are dropped. Keeping the highest is the conservative choice, so contamination is never under-reported.
Depth gaps: left as gaps. Missing intervals are never filled in. We never invent a reading that wasn't measured.
Missing channel values: left blank, never filled with zero.

Nothing else is changed. Flagged %RE values, for example, are reported but kept exactly as submitted.

The result is in each job's Cleaned Dataset: cleaned_logs.csv holds every reading the figures and volumes are built from, one row per reading, with depth in feet (depth_ft) and meters (depth_m), pct_re and each channel as reported, EC where your files had it, and each boring's coordinates and coordinate system on every row, so a GIS can load it as points. A blank cell is a value your file didn't have, never a zero. cleaning_changes.csv lists every change: each dropped duplicate, with the %RE dropped and the %RE kept, and each boring that was sorted. When nothing changed it holds only its header row. The job's readme.txt summarizes both.

Before the audit: the upload format check

Some problems are caught on the upload form, before a job starts or counts against your account: files that can't be read as CSV, missing or misspelled required columns, non-numeric values, negative depths, a missing, invalid, or mixed coordinate system, boring log files and coordinates rows that don't pair up by name, a boring ID listed twice, a boring with no coordinates, a boring so far from the rest of the site that its coordinates are almost certainly a typo, and coordinates that fall outside the region your coordinate system is defined for (usually swapped easting/northing, or feet entered as meters). See File input templates for the expected layout.

What the audit does not catch

A clean audit means your data passed the checks above, nothing more. In particular, it does not check:

Whether your coordinates are correct. Gross errors are caught at upload (see above), but a wrong coordinate system whose region overlaps the right one (such as a neighboring State Plane zone), or a typo that shifts one boring by tens of meters, will not be flagged. Check the plan view against your site map.
Whether depths are in the right unit. Depths entered in meters in a column meant for feet look perfectly plausible to the audit.
Whether %RE values are calibrated correctly. Only all-zero borings and values above the threshold are caught. Readings within that range that are wrong for another reason, such as calibration drift, will pass.
Whether your boring spacing suits 3D interpolation, beyond the straight-line and two-boring cases above. For example, a few tightly clustered borings with large unsampled areas between them.
Electrical conductivity (EC) data, if your files include it.
Ground surface elevations. The ground_surface_elev_ft column is optional; when given it is only checked to be numeric, and no output uses it yet: depths are taken from each boring's own ground surface, as if the site were flat.
Site conditions. The audit judges data quality, not contamination. A PASS doesn't mean a site is clean or contaminated, and it doesn't replace professional review of the results.