Output Features

Split Outputs and Detail Level Control

Overview

The TYCHON PQC Scanner provides two powerful features for controlling output organization and size:

Split Outputs

Break up large scan reports into separate files per dataset type for easier parsing and processing.

  • One file per dataset (quantum, network, memory, etc.)
  • Maintains complete metadata in each file
  • Automatically skips empty datasets
  • Works with most output formats

Detail Levels

Control output verbosity to reduce file sizes while maintaining essential security information.

  • Full: All fields (current behavior)
  • Standard: ~30-40% size reduction
  • Minimal: ~60-70% size reduction
  • PQC-critical fields always included

Split Outputs Feature

Problem Statement

By default, the scanner outputs all datasets to a single file (e.g., scan_report.json), which can become very large (100MB+) when scanning systems with many certificates, network connections, and running processes. This makes the files:

  • Difficult to parse and process
  • Slow to load in text editors and analysis tools
  • Challenging to integrate with systems expecting specific data types
  • Inefficient for streaming analytics (must process entire file)

Solution: Split by Dataset Type

The -split-outputs flag breaks the report into separate files per dataset:

Dataset Name File Suffix Contains
quantum _quantum.json Quantum readiness assessment with 100-point scoring
network _network.json Host/port scan results with TLS certificates
memory _memory.json Process memory crypto library detection
filesystem _filesystem.json Certificates found on filesystem
keystore _keystore.json Keystore scan results (PKCS12, JKS, system stores)
outlook _outlook.json Outlook archive files (.pst, .ost)
vpn _vpn.json VPN client detection and PQC assessment
ipsec _ipsec.json IPSec tunnel configuration and security analysis

Usage Examples

# Basic split outputs
.\certscanner-windows-amd64.exe -local -split-outputs -output report.json

# Split outputs + keep consolidated file
.\certscanner-windows-amd64.exe -local -split-outputs -keep-consolidated -output report.json

Metadata in Split Files

Each split file is self-contained and includes:

  • Complete scanning_system_info: hostname, OS, IP addresses, BigFix client ID
  • Scan context: scan_type, target, timestamp, tags
  • New metadata fields:
    • dataset_type: Identifies which dataset (e.g., "quantum", "network")
    • record_count: Number of records in this file
    • split_mode: true when file is from split output

Empty Datasets

Files are NOT created for empty datasets. For example, if no VPN clients are detected, report_vpn.json will not be created. The scanner logs which files were created and which were skipped.

Detail Levels Feature

Problem Statement

The scanner currently includes ALL fields in output, regardless of their importance or use case. This results in:

  • Large file sizes (unnecessary for real-time monitoring)
  • Slower ingestion into SIEM and log aggregation systems
  • Higher storage costs
  • Verbose output for executive dashboards
  • Bandwidth constraints in remote scanning scenarios

Solution: Three Detail Levels

The -detail-level flag controls which fields are included in output:

Full Level (Default)

Includes: All fields

Size Reduction: 0% (baseline)

Use Cases:

  • Forensic analysis
  • Deep troubleshooting
  • Compliance audits
  • Detailed reporting

Command:

-detail-level full

Standard Level

Removes:

  • Raw binary data (PEM, DER, hex)
  • Verbose metadata (file permissions, ownership)
  • Internal tracking fields (Active, LastSeen)
  • Low-level details (memory addresses, kernel versions)
  • Statistical data (byte counts, latency)

Keeps: All security-critical fields, PQC flags, crypto parameters

Size Reduction: ~30-40%

Use Cases:

  • SIEM integration
  • Security monitoring
  • Dashboard ingestion
  • Regular scanning

Command:

-detail-level standard

Minimal Level

Includes ONLY:

  • System/asset identifiers
  • Security status indicators
  • PQC assessment flags
  • Key crypto parameters
  • Critical timestamps
  • Security scores and ratings
  • Top recommendations (limited)

Size Reduction: ~60-70%

Use Cases:

  • Real-time alerting
  • Executive dashboards
  • Bandwidth-constrained environments
  • Streaming analytics
  • High-frequency scanning

Command:

-detail-level minimal

Field Distribution Statistics

Across all dataset types in the documentation:

Pattern Field Count Visualization Meaning
✓ ✓ ✓ 83 fields
In ALL three levels (essential security fields)
✓ ✓ ✗ 124 fields
In Full + Standard (useful metadata, not essential)
✓ ✗ ✗ 29 fields
Full only (verbose/debug data)

Total: 236 fields documented across all dataset types

Documentation Reference

For complete field-by-field detail level classifications, see:

Size Reduction Benchmarks

Expected size reductions by dataset type when using Standard and Minimal detail levels:

Dataset Type Full Size Standard Size Minimal Size Standard % Minimal %
Quantum Readiness 8-12 KB 5-8 KB 2-3 KB -35% -70%
Network (per host) 50-200 KB 30-120 KB 15-50 KB -35% -70%
Process Memory 5-20 KB 3-14 KB 1-5 KB -30% -70%
Filesystem Certs 3-8 KB 2-5 KB 1-2 KB -35% -70%
Keystores 5-30 KB 3-20 KB 1-8 KB -35% -75%
Outlook Archives 1-3 KB 0.8-2 KB 0.4-0.8 KB -30% -65%
VPN Clients 4-15 KB 2-9 KB 1-3 KB -35% -75%
IPSec Tunnels 3-10 KB 2-6 KB 1-3 KB -35% -75%

Combined Usage: Split + Detail Levels

Both features can be used together for maximum flexibility and efficiency:

Example: Real-Time Security Monitoring

# Maximum efficiency: Split outputs + minimal detail
.\certscanner-windows-amd64.exe -local -split-outputs -detail-level minimal -output report.json

# Result: 8 small files instead of 1 large file
# - report_quantum.json    (2-3 KB instead of 12 KB)
# - report_network.json    (15-50 KB instead of 200 KB)
# - report_memory.json     (1-5 KB instead of 20 KB)
# - report_filesystem.json (1-2 KB per cert)
# - ... etc

Benefits:

  • Each dataset in its own file (easy parsing)
  • ~60-70% size reduction per file
  • Fast ingestion into SIEM systems
  • Reduced network bandwidth
  • Lower storage costs

Example: Comprehensive Audit with Organization

# Split outputs with full detail for forensic analysis
./certscanner -local -split-outputs -detail-level full -output audit_report.json

# Keep consolidated file for compliance
./certscanner -local -split-outputs -keep-consolidated -detail-level full -output audit_report.json

Benefits:

  • All fields retained (full detail)
  • Organized by dataset type (easy to find specific data)
  • Optional consolidated file for compliance requirements
  • Suitable for deep analysis and troubleshooting

Format and Integration Compatibility

Output Format Support

Format Split Outputs Detail Levels Notes
json ✓ Full ✓ Full 8 separate JSON files when split
flatndjson ✓ Full ✓ Full 8 separate NDJSON files when split
tychon ✓ Full ✓ Full 8 separate NDJSON files when split
cbom ✗ Not Compatible ✓ Full CBOM format incompatible with split outputs
html ✗ Not Compatible ✓ Full HTML format incompatible with split outputs
eventlog ✗ Not Compatible ✓ Full Writes to system logs, not files

Integration Platform Support

Platform Split Outputs Detail Levels Notes
Elasticsearch ✓ ✓ Posts to separate indices per dataset
Kafka ✓ ✓ Posts to topic-per-dataset pattern
Splunk ✓ ✓ Uses different sourcetypes per dataset
S3 Upload ✓ ✓ Uploads all split files maintaining folder structure

Use Case Recommendations

Real-Time Security Monitoring

-split-outputs -detail-level minimal

Maximum efficiency for continuous monitoring. Small files, fast processing, essential security data only.

SIEM Integration

-split-outputs -detail-level standard

Organized datasets with all security-relevant fields. Good balance of detail and size.

Compliance Audits

-split-outputs -keep-consolidated -detail-level full

Complete records organized by type. Consolidated file for archival requirements.

Executive Dashboards

-detail-level minimal

Single file with high-level security indicators, scores, and PQC readiness flags.

Forensic Investigation

-split-outputs -detail-level full

All fields retained, organized for analysis. Access raw certificates, memory addresses, full metadata.

Bandwidth-Constrained Remote Scanning

-detail-level minimal

Minimize data transfer. Essential fields only, ~60-70% smaller outputs.

Command-Line Reference

-split-outputs

Type: Boolean flag

Default: false

Description: Enable split output files (one per dataset type)

-keep-consolidated

Type: Boolean flag

Default: false

Requires: -split-outputs

Description: Keep consolidated file when using split outputs (creates both split files AND the main file)

-detail-level <level>

Type: String

Default: full

Valid Values: full, standard, minimal

Description: Control output verbosity (full = 0% reduction, standard = ~30-40% reduction, minimal = ~60-70% reduction)