Structured vs Unstructured
1. Purpose & Motivation
1.1 Why this distinction matters
- The type of data determines which models, tools, and preprocessing pipelines are appropriate
- Structured and unstructured data require fundamentally different handling strategies
- Most organizations have both types, and bridging them is a key ML challenge
1.2 The core problem it addresses
Understand the fundamental difference between organized, schema-defined data and free-form data without a predefined structure, and how this affects ML system design.
2. Definition & Core Idea
2.1 Formal definitions
- Structured data: Data organized in a predefined schema with fixed rows and columns, where each field has a known type and meaning. Stored in relational databases, spreadsheets, or CSV files.
- Unstructured data: Data without a predefined schema or organization. Includes images, audio, video, free text, and raw sensor streams.
2.2 Intuitive mental model
"Structured data is a neatly filled-out form; unstructured data is a handwritten letter -- both contain information, but extracting it requires very different approaches."
3. Comparison
| Aspect | Structured | Unstructured |
|---|---|---|
| Format | Tables (rows and columns) | Files (images, text, audio, video) |
| Schema | Predefined, rigid | None or loosely defined |
| Storage | SQL databases, data warehouses | Object storage, data lakes |
| Examples | Sales records, sensor readings | Photos, emails, medical scans |
| Volume (estimated) | ~20% of enterprise data | ~80% of enterprise data |
| Query ease | Easy (SQL queries) | Hard (requires specialized tools) |
4. Semi-Structured Data
4.1 Definition
Data that has some organizational structure (tags, keys, hierarchy) but does not conform to a rigid tabular schema.
4.2 Examples
- JSON and XML documents
- Log files with key-value pairs
- HTML pages
- Email (headers are structured, body is unstructured)
5. ML Approaches by Data Type
5.1 Structured data
| Approach | Details |
|---|---|
| Feature engineering | Create domain-specific features manually |
| Tree-based ensembles | XGBoost, LightGBM, Random Forest |
| Linear models | Logistic/linear regression as baseline |
| Preprocessing | Scaling, encoding, imputation |
5.2 Unstructured data
| Approach | Details |
|---|---|
| Deep learning | CNNs for images, Transformers for text |
| Learned representations | Model extracts features automatically |
| Transfer learning | Fine-tune pretrained models |
| Preprocessing | Tokenization, resizing, normalization |
5.3 Multimodal data
- Combines structured and unstructured data (e.g., patient records + X-ray images)
- Requires fusion strategies: early fusion (concatenate features), late fusion (combine predictions)
6. Practical Considerations
- Structured data is easier to clean, validate, and query but covers a smaller share of available information
- Unstructured data is abundant but requires significant preprocessing and compute
- Modern data platforms (data lakehouses) aim to handle both under one roof
- Converting unstructured data into structured features (e.g., extracting entities from text) is a common ML task
7. One-Sentence Summary
Structured data follows a fixed schema of rows and columns suited for tree-based models, while unstructured data (images, text, audio) requires deep learning to extract meaningful patterns.