Jason
506 lines
15 KiB
Markdown
506 lines
15 KiB
Markdown
# Encoding Table Generation System
|
|
|
|
## Problem Statement
|
|
|
|
Currently, encoding lookup tables require manual maintenance of both forward and reverse mappings:
|
|
|
|
```javascript
|
|
// Must maintain both of these manually
|
|
const PRODUCT_COLOR = {
|
|
'BLACK': 1,
|
|
'BRONZE': 2,
|
|
// ... add new color here
|
|
};
|
|
|
|
const PRODUCT_COLOR_REVERSE = {
|
|
1: 'BLACK',
|
|
2: 'BRONZE',
|
|
// ... and remember to add it here too!
|
|
};
|
|
```
|
|
|
|
**Issues:**
|
|
- ❌ Prone to human error (forgetting reverse mapping)
|
|
- ❌ Risk of mismatches between forward/reverse
|
|
- ❌ No validation for duplicate values
|
|
- ❌ No documentation of available slots
|
|
- ❌ Difficult for non-technical users to add values
|
|
|
|
## Proposed Solution: Python-Generated Tables
|
|
|
|
### Architecture
|
|
|
|
```
|
|
User edits simple source file
|
|
↓
|
|
Python validation script
|
|
↓
|
|
Generate JavaScript + Python constants
|
|
↓
|
|
Auto-served by Flask (or manual build step)
|
|
```
|
|
|
|
### File Structure
|
|
|
|
```
|
|
app/
|
|
data/
|
|
encoding_values.json (or .yaml) ← USER EDITS THIS
|
|
js/
|
|
encoding_tables.js ← AUTO-GENERATED
|
|
generate_encoding_tables.py ← GENERATOR SCRIPT
|
|
```
|
|
|
|
## Source File Format
|
|
|
|
### Option 1: JSON with Comments Support
|
|
|
|
**`app/data/encoding_values.json`**
|
|
```json
|
|
{
|
|
"metadata": {
|
|
"version": 0,
|
|
"description": "Product encoding value definitions"
|
|
},
|
|
"tables": {
|
|
"product_type": {
|
|
"max_value": 15,
|
|
"bit_size": 4,
|
|
"values": [
|
|
{"name": "PATIO_DOOR", "value": 1, "display": "Patio Door"},
|
|
{"name": "STORM_DOOR", "value": 3, "display": "Storm Door"},
|
|
{"name": "STORM_WINDOW", "value": 5, "display": "Storm Window"},
|
|
{"name": "PRIMARY_WINDOW", "value": 7, "display": "Primary Window"}
|
|
]
|
|
},
|
|
"product_color": {
|
|
"max_value": 15,
|
|
"bit_size": 4,
|
|
"values": [
|
|
{"name": "BLACK", "value": 1, "display": "Black"},
|
|
{"name": "BRONZE", "value": 2, "display": "Bronze"},
|
|
{"name": "SANDSTONE", "value": 3, "display": "Sandstone"},
|
|
{"name": "WHITE", "value": 4, "display": "White"},
|
|
{"name": "TAN", "value": 5, "display": "Tan"},
|
|
{"name": "MILL", "value": 6, "display": "Mill"}
|
|
]
|
|
},
|
|
"product_material": {
|
|
"max_value": 15,
|
|
"bit_size": 4,
|
|
"values": [
|
|
{"name": "ALUMINUM", "value": 1, "display": "Aluminum"},
|
|
{"name": "VINYL", "value": 2, "display": "Vinyl"}
|
|
]
|
|
},
|
|
"hardware_type": {
|
|
"max_value": 31,
|
|
"bit_size": 5,
|
|
"values": [
|
|
{"name": "NONE", "value": 0, "display": "None"},
|
|
{"name": "LEVER", "value": 1, "display": "Lever"},
|
|
{"name": "PULL", "value": 2, "display": "Pull"},
|
|
{"name": "PULL_HANDLE", "value": 3, "display": "Pull Handle"},
|
|
{"name": "DEADBOLT", "value": 4, "display": "Deadbolt"},
|
|
{"name": "HINGE", "value": 5, "display": "Hinge"}
|
|
]
|
|
},
|
|
"hardware_style": {
|
|
"max_value": 31,
|
|
"bit_size": 5,
|
|
"values": [
|
|
{"name": "NONE", "value": 0, "display": "None"},
|
|
{"name": "STANDARD", "value": 1, "display": "Standard"},
|
|
{"name": "PUSH", "value": 2, "display": "Push"},
|
|
{"name": "ALTERNATIVE", "value": 3, "display": "Alternative"},
|
|
{"name": "CONTEMPORARY", "value": 4, "display": "Contemporary"},
|
|
{"name": "TRADITIONAL", "value": 5, "display": "Traditional"}
|
|
]
|
|
},
|
|
"hardware_color": {
|
|
"max_value": 31,
|
|
"bit_size": 5,
|
|
"values": [
|
|
{"name": "NONE", "value": 0, "display": "None"},
|
|
{"name": "BRASS", "value": 1, "display": "Brass"},
|
|
{"name": "WHITE", "value": 2, "display": "White"},
|
|
{"name": "BLACK", "value": 3, "display": "Black"},
|
|
{"name": "SATIN", "value": 4, "display": "Satin"},
|
|
{"name": "NICKEL", "value": 5, "display": "Nickel"},
|
|
{"name": "BRONZE", "value": 6, "display": "Bronze"}
|
|
]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
### Option 2: YAML (Better for Comments)
|
|
|
|
**`app/data/encoding_values.yaml`**
|
|
```yaml
|
|
version: 0
|
|
description: Product encoding value definitions
|
|
|
|
tables:
|
|
product_type:
|
|
max_value: 15
|
|
bit_size: 4
|
|
values:
|
|
- name: PATIO_DOOR
|
|
value: 1
|
|
display: Patio Door
|
|
- name: STORM_DOOR
|
|
value: 3
|
|
display: Storm Door
|
|
# Reserved: 2, 4, 6, 8-15
|
|
|
|
product_color:
|
|
max_value: 15
|
|
bit_size: 4
|
|
values:
|
|
- name: BLACK
|
|
value: 1
|
|
display: Black
|
|
- name: BRONZE
|
|
value: 2
|
|
display: Bronze
|
|
# ... etc
|
|
# Reserved: 7-15 (9 slots available)
|
|
```
|
|
|
|
## User Workflow
|
|
|
|
### Adding a New Color
|
|
|
|
**Before (Manual):**
|
|
1. Open `encoding_helper.js`
|
|
2. Find `PRODUCT_COLOR` object
|
|
3. Add new entry with unused value
|
|
4. Find `PRODUCT_COLOR_REVERSE` object
|
|
5. Add matching reverse entry
|
|
6. Hope you didn't make a mistake
|
|
|
|
**After (Generated):**
|
|
1. Open `encoding_values.json`
|
|
2. Add one line: `{"name": "CHARCOAL", "value": 7, "display": "Charcoal"}`
|
|
3. Run `python generate_encoding_tables.py` (or auto-regenerated by Flask)
|
|
4. Done! JavaScript and Python constants updated
|
|
|
|
## Generator Script
|
|
|
|
**`app/generate_encoding_tables.py`**
|
|
|
|
```python
|
|
#!/usr/bin/env python3
|
|
"""
|
|
Generate JavaScript and Python encoding tables from source JSON/YAML.
|
|
|
|
Usage:
|
|
python generate_encoding_tables.py
|
|
|
|
Or add to Flask app for auto-generation on file change.
|
|
"""
|
|
|
|
import json
|
|
import os
|
|
from datetime import datetime
|
|
from pathlib import Path
|
|
|
|
def load_source_file(filepath):
|
|
"""Load encoding values from JSON or YAML."""
|
|
with open(filepath, 'r') as f:
|
|
if filepath.endswith('.yaml') or filepath.endswith('.yml'):
|
|
import yaml
|
|
return yaml.safe_load(f)
|
|
else:
|
|
return json.load(f)
|
|
|
|
def validate_table(table_name, table_data):
|
|
"""Validate a single table for errors."""
|
|
errors = []
|
|
max_value = table_data['max_value']
|
|
values = table_data['values']
|
|
|
|
used_values = set()
|
|
used_names = set()
|
|
|
|
for item in values:
|
|
name = item['name']
|
|
value = item['value']
|
|
|
|
# Check for duplicate values
|
|
if value in used_values:
|
|
errors.append(f"{table_name}: Duplicate value {value}")
|
|
used_values.add(value)
|
|
|
|
# Check for duplicate names
|
|
if name in used_names:
|
|
errors.append(f"{table_name}: Duplicate name '{name}'")
|
|
used_names.add(name)
|
|
|
|
# Check value range
|
|
if value < 0 or value > max_value:
|
|
errors.append(f"{table_name}: Value {value} out of range (0-{max_value})")
|
|
|
|
# Validate naming convention (uppercase with underscores)
|
|
if not name.isupper() or not name.replace('_', '').isalpha():
|
|
errors.append(f"{table_name}: Invalid name format '{name}' (use UPPERCASE_WITH_UNDERSCORES)")
|
|
|
|
# Report available slots
|
|
available = sorted(set(range(max_value + 1)) - used_values)
|
|
info = f"{table_name}: {len(used_values)}/{max_value + 1} slots used, {len(available)} available"
|
|
if available and len(available) <= 10:
|
|
info += f" {available}"
|
|
|
|
return errors, info
|
|
|
|
def generate_javascript(data, output_path):
|
|
"""Generate JavaScript constants file."""
|
|
lines = [
|
|
"// AUTO-GENERATED FILE - DO NOT EDIT MANUALLY",
|
|
"// Generated from: app/data/encoding_values.json",
|
|
f"// Generated on: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
|
|
"//",
|
|
"// To add new values:",
|
|
"// 1. Edit app/data/encoding_values.json",
|
|
"// 2. Run: python app/generate_encoding_tables.py",
|
|
"// 3. Restart Flask server (or it auto-reloads)",
|
|
"",
|
|
"// ============================================================================",
|
|
"// ENCODING CONSTANTS",
|
|
"// ============================================================================",
|
|
"",
|
|
f"const ENCODING_VERSION = {data['metadata']['version']};",
|
|
""
|
|
]
|
|
|
|
for table_name, table_data in data['tables'].items():
|
|
const_name = table_name.upper()
|
|
|
|
# Generate forward mapping
|
|
lines.append(f"// {const_name} (max: {table_data['max_value']}, bits: {table_data['bit_size']})")
|
|
lines.append(f"const {const_name} = {{")
|
|
for item in table_data['values']:
|
|
lines.append(f" '{item['name']}': {item['value']},")
|
|
lines.append("};")
|
|
lines.append("")
|
|
|
|
# Generate reverse mapping
|
|
lines.append(f"const {const_name}_REVERSE = {{")
|
|
for item in table_data['values']:
|
|
lines.append(f" {item['value']}: '{item['name']}',")
|
|
lines.append("};")
|
|
lines.append("")
|
|
|
|
# Generate display mapping
|
|
lines.append(f"const {const_name}_DISPLAY = {{")
|
|
for item in table_data['values']:
|
|
lines.append(f" '{item['name']}': '{item['display']}',")
|
|
lines.append("};")
|
|
lines.append("")
|
|
|
|
# Write file
|
|
with open(output_path, 'w') as f:
|
|
f.write('\n'.join(lines))
|
|
|
|
print(f"✓ Generated: {output_path}")
|
|
|
|
def generate_python(data, output_path):
|
|
"""Generate Python constants file (optional)."""
|
|
lines = [
|
|
'"""',
|
|
"AUTO-GENERATED FILE - DO NOT EDIT MANUALLY",
|
|
"Generated from: app/data/encoding_values.json",
|
|
f"Generated on: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}",
|
|
'"""',
|
|
"",
|
|
f"ENCODING_VERSION = {data['metadata']['version']}",
|
|
""
|
|
]
|
|
|
|
for table_name, table_data in data['tables'].items():
|
|
const_name = table_name.upper()
|
|
|
|
lines.append(f"# {const_name} (max: {table_data['max_value']}, bits: {table_data['bit_size']})")
|
|
lines.append(f"{const_name} = {{")
|
|
for item in table_data['values']:
|
|
lines.append(f" '{item['name']}': {item['value']},")
|
|
lines.append("}")
|
|
lines.append("")
|
|
|
|
lines.append(f"{const_name}_REVERSE = {{")
|
|
for item in table_data['values']:
|
|
lines.append(f" {item['value']}: '{item['name']}',")
|
|
lines.append("}")
|
|
lines.append("")
|
|
|
|
with open(output_path, 'w') as f:
|
|
f.write('\n'.join(lines))
|
|
|
|
print(f"✓ Generated: {output_path}")
|
|
|
|
def generate_documentation(data, output_path):
|
|
"""Generate markdown documentation."""
|
|
lines = [
|
|
"# Encoding Values Reference",
|
|
"",
|
|
f"**Version:** {data['metadata']['version']} ",
|
|
f"**Generated:** {datetime.now().strftime('%Y-%m-%d %H:%M:%S')} ",
|
|
"",
|
|
"## Overview",
|
|
"",
|
|
"This document lists all currently defined encoding values.",
|
|
"",
|
|
"## Quick Reference",
|
|
""
|
|
]
|
|
|
|
for table_name, table_data in data['tables'].items():
|
|
lines.append(f"### {table_name.replace('_', ' ').title()}")
|
|
lines.append("")
|
|
lines.append(f"**Capacity:** {len(table_data['values'])}/{table_data['max_value'] + 1} slots used ")
|
|
lines.append(f"**Bit Size:** {table_data['bit_size']} bits ")
|
|
lines.append("")
|
|
lines.append("| Value | Name | Display |")
|
|
lines.append("|-------|------|---------|")
|
|
|
|
for item in sorted(table_data['values'], key=lambda x: x['value']):
|
|
lines.append(f"| {item['value']} | `{item['name']}` | {item['display']} |")
|
|
|
|
lines.append("")
|
|
|
|
with open(output_path, 'w') as f:
|
|
f.write('\n'.join(lines))
|
|
|
|
print(f"✓ Generated: {output_path}")
|
|
|
|
def main():
|
|
"""Main generation function."""
|
|
base_dir = Path(__file__).parent
|
|
source_file = base_dir / 'data' / 'encoding_values.json'
|
|
|
|
# Load source data
|
|
print(f"Loading: {source_file}")
|
|
data = load_source_file(source_file)
|
|
|
|
# Validate all tables
|
|
print("\nValidating tables...")
|
|
all_errors = []
|
|
for table_name, table_data in data['tables'].items():
|
|
errors, info = validate_table(table_name, table_data)
|
|
print(f" {info}")
|
|
all_errors.extend(errors)
|
|
|
|
if all_errors:
|
|
print("\n❌ VALIDATION ERRORS:")
|
|
for error in all_errors:
|
|
print(f" - {error}")
|
|
return 1
|
|
|
|
print("\n✓ All tables valid")
|
|
|
|
# Generate files
|
|
print("\nGenerating files...")
|
|
generate_javascript(data, base_dir / 'js' / 'encoding_tables.js')
|
|
generate_python(data, base_dir / 'encoding_tables_constants.py')
|
|
generate_documentation(data, base_dir.parent / 'planning' / 'ENCODING_VALUES_REFERENCE.md')
|
|
|
|
print("\n✓ Generation complete!")
|
|
return 0
|
|
|
|
if __name__ == '__main__':
|
|
exit(main())
|
|
```
|
|
|
|
## Flask Integration (Auto-Generation)
|
|
|
|
Add to `app.py`:
|
|
|
|
```python
|
|
import os
|
|
from datetime import datetime
|
|
|
|
@app.before_request
|
|
def check_encoding_tables():
|
|
"""Auto-regenerate encoding tables if source has changed."""
|
|
source_file = 'data/encoding_values.json'
|
|
output_file = 'js/encoding_tables.js'
|
|
|
|
# Check if regeneration needed
|
|
if not os.path.exists(output_file):
|
|
print("Encoding tables not found, generating...")
|
|
os.system('python generate_encoding_tables.py')
|
|
else:
|
|
source_mtime = os.path.getmtime(source_file)
|
|
output_mtime = os.path.getmtime(output_file)
|
|
|
|
if source_mtime > output_mtime:
|
|
print("Encoding values changed, regenerating tables...")
|
|
os.system('python generate_encoding_tables.py')
|
|
```
|
|
|
|
## Benefits
|
|
|
|
### For Non-Technical Users
|
|
- ✅ Edit simple JSON file (no JavaScript knowledge needed)
|
|
- ✅ Clear structure with examples
|
|
- ✅ Automatic validation catches errors
|
|
- ✅ Can't create mismatches between forward/reverse
|
|
|
|
### For Developers
|
|
- ✅ Single source of truth
|
|
- ✅ Auto-generated documentation
|
|
- ✅ Validation prevents bugs
|
|
- ✅ Easy to add new tables
|
|
- ✅ Can generate for multiple targets (JS, Python, docs)
|
|
|
|
### For Maintenance
|
|
- ✅ Version control friendly (one file to track)
|
|
- ✅ Easy to review changes (just JSON diff)
|
|
- ✅ Reports available slots automatically
|
|
- ✅ Enforces naming conventions
|
|
|
|
## Migration Path
|
|
|
|
### Phase 1: Create Source File
|
|
1. Extract existing values to `encoding_values.json`
|
|
2. Test generator script
|
|
3. Verify output matches current `encoding_helper.js`
|
|
|
|
### Phase 2: Integrate Generator
|
|
1. Add generator script to project
|
|
2. Update build process / Flask app
|
|
3. Test auto-generation
|
|
|
|
### Phase 3: Switch to Generated Tables
|
|
1. Update `encoding_helper.js` to import generated file
|
|
2. Remove manual mappings
|
|
3. Update documentation
|
|
|
|
### Phase 4: User Documentation
|
|
1. Create guide for adding new values
|
|
2. Document validation rules
|
|
3. Add examples
|
|
|
|
## Future Enhancements
|
|
|
|
- Web UI for editing values (no need to edit JSON directly)
|
|
- Export to CSV for client review
|
|
- Import from Excel/CSV
|
|
- Visual capacity indicators
|
|
- Conflict detection across tables
|
|
- Automatic value assignment (find next available slot)
|
|
|
|
## Related Files
|
|
|
|
- [ENCODING_SYSTEM.md](ENCODING_SYSTEM.md) - Technical specification
|
|
- `app/js/encoding_helper.js` - Current manual implementation
|
|
- `app/js/encoding_tables.js` - Future generated file
|
|
|
|
---
|
|
|
|
**Status:** Planning
|
|
**Priority:** Medium
|
|
**Effort:** ~4-6 hours implementation + testing
|
|
**Dependencies:** None
|