Instrument Data To Allotrope

# Instrument Data to Allotrope Converter

Convert instrument files into standardized Allotrope Simple Model (ASM) format for LIMS upload, data lakes, or handoff to data engineering teams.

> **Note: This is an Example Skill** > > This skill demonstrates how skills can support your data engineering tasks—automating schema transformations, parsing instrument outputs, and generating production-ready code. > > **To customize for your organization:** > - Modify the `references/` files to include your company's specific schemas or ontology mappings > - Use an MCP server to connect to systems that define your schemas (e.g., your LIMS, data catalog, or schema registry) > - Extend the `scripts/` to handle proprietary instrument formats or internal data standards > > This pattern can be adapted for any data transformation workflow where you need to convert between formats or validate against organizational standards.

## Workflow Overview

1. **Detect instrument type** from file contents (auto-detect or user-specified) 2. **Parse file** using allotropy library (native) or flexible fallback parser 3. **Generate outputs**: - ASM JSON (full semantic structure) - Flattened CSV (2D tabular format) - Python parser code (for data engineer handoff) 4. **Deliver** files with summary and usage instructions

> **When Uncertain:** If you're unsure how to map a field to ASM (e.g., is this raw data or calculated? device setting or environmental condition?), ask the user for clarification. Refer to `references/field_classification_guide.md` for guidance, but when ambiguity remains, confirm with the user rather than guessing.

## Quick Start

```python # Install requirements first pip install allotropy pandas openpyxl pdfplumber --break-system-packages

# Core conversion from allotropy.parser_factory import Vendor from allotropy.to_allotrope import allotrope_from_file

# Convert with allotropy asm = allotrope_from_file("instrument_data.csv", Vendor.BECKMAN_VI_CELL_BLU) ```

## Output Format Selection

**ASM JSON (default)** - Full semantic structure with ontology URIs - Best for: LIMS systems expecting ASM, data lakes, long-term archival - Validates against Allotrope schemas

**Flattened CSV** - 2D tabular representation - Best for: Quick analysis, Excel users, systems without JSON support - Each measurement becomes one row with metadata repeated

**Both** - Generate both formats for maximum flexibility

## Calculated Data Handling

**IMPORTANT:** Separate raw measurements from calculated/derived values.

- **Raw data** → `measurement-document` (direct instrument readings) - **Calculated data** → `calculated-data-aggregate-document` (derived values)

Calculated values MUST include traceability via `data-source-aggregate-document`:

```json "calculated-data-aggregate-document": { "calculated-data-document": [{ "calculated-data-identifier": "SAMPLE_B1_DIN_001", "calculated-data-name": "DNA integrity number", "calculated-result": {"value": 9.5, "unit": "(unitless)"}, "data-source-aggregate-document": { "data-source-document": [{ "data-source-identifier": "SAMPLE_B1_MEASUREMENT", "data-source-feature": "electrophoresis trace" }] } }] } ```

**Common calculated fields by instrument type:** | Instrument | Calculated Fields | |------------|-------------------| | Cell counter | Viability %, cell density dilution-adjusted values | | Spectrophotometer | Concentration (from absorbance), 260/280 ratio | | Plate reader | Concentrations from standard curve, %CV | | Electrophoresis | DIN/RIN, region concentrations, average sizes | | qPCR | Relative quantities, fold change |

See `references/field_classification_guide.md` for detailed guidance on raw vs. calculated classification.

## Validation

Always validate ASM output before delivering to the user:

```bash python scripts/validate_asm.py output.json python scripts/validate_asm.py output.json --reference known_good.json # Compare to reference python scripts/validate_asm.py output.json --strict # Treat warnings as errors ```

**Validation Rules:** - Based on Allotrope ASM specification (December 2024) - Last updated: 2026-01-07 - Source: https://gitlab.com/allotrope-public/asm

**Soft Validation Approach:** Unknown techniques, units, or sample roles generate **warnings** (not errors) to allow for forward compatibility. If Allotrope adds new values after December 2024, the validator won't block them—it will flag them for manual verification. Use `--strict` mode to treat warnings as errors if you need stricter validation.

**What it checks:** - Correct technique selection (e.g., multi-analyte profiling vs plate reader) - Field naming conventions (space-separated, not hyphenated) - Calculated data has traceability (`data-source-aggregate-document`) - Unique identifiers exist for measurements and calculated values - Required metadata present - Valid units and sample roles (with soft validation for unknown values)

## Supported Instrumen