Amazon OpenSearch Integration

Overview

Amazon OpenSearch Service provides a fully managed search and analytics solution compatible with Elasticsearch APIs. TYCHON Quantum Readiness leverages the same built-in Elasticsearch integration to seamlessly work with OpenSearch clusters.

AWS Managed Service: No infrastructure management required, automatic scaling, built-in security, and integrated AWS services connectivity.

OpenSearch Cluster Setup

1. Create OpenSearch Domain

AWS CLI Terraform CloudFormation

# Create OpenSearch domain for TYCHON Quantum Readiness
aws opensearch create-domain \
  --domain-name "tychon-crypto-assets" \
  --elasticsearch-version "OpenSearch_2.3" \
  --elasticsearch-cluster-config InstanceType=t3.small.search,InstanceCount=3 \
  --ebs-options EBSEnabled=true,VolumeType=gp2,VolumeSize=20 \
  --access-policies '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::ACCOUNT-ID:user/tychon-scanner"},
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:ACCOUNT-ID:domain/tychon-crypto-assets/*"
    }]
  }' \
  --domain-endpoint-options EnforceHTTPS=true,TLSSecurityPolicy=Policy-Min-TLS-1-2-2019-07 \
  --encryption-at-rest-options Enabled=true \
  --node-to-node-encryption-options Enabled=true

# terraform/opensearch.tf
resource "aws_opensearch_domain" "tychon_crypto_assets" {
  domain_name    = "tychon-crypto-assets"
  engine_version = "OpenSearch_2.3"

  cluster_config {
    instance_type            = "t3.small.search"
    instance_count           = 3
    dedicated_master_enabled = false
  }

  ebs_options {
    ebs_enabled = true
    volume_type = "gp2"
    volume_size = 20
  }

  encrypt_at_rest {
    enabled = true
  }

  node_to_node_encryption {
    enabled = true
  }

  domain_endpoint_options {
    enforce_https       = true
    tls_security_policy = "Policy-Min-TLS-1-2-2019-07"
  }

  access_policies = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:user/tychon-scanner"
        }
        Action   = "es:*"
        Resource = "arn:aws:es:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:domain/tychon-crypto-assets/*"
      }
    ]
  })

  tags = {
    Domain = "tychon-crypto-assets"
    Environment = "production"
  }
}

# cloudformation/opensearch-domain.yml
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Amazon OpenSearch domain for TYCHON Quantum Readiness'

Resources:
  TychonOpenSearchDomain:
    Type: AWS::OpenSearch::Domain
    Properties:
      DomainName: tychon-crypto-assets
      EngineVersion: OpenSearch_2.3
      ClusterConfig:
        InstanceType: t3.small.search
        InstanceCount: 3
        DedicatedMasterEnabled: false
      EBSOptions:
        EBSEnabled: true
        VolumeType: gp2
        VolumeSize: 20
      EncryptionAtRestOptions:
        Enabled: true
      NodeToNodeEncryptionOptions:
        Enabled: true
      DomainEndpointOptions:
        EnforceHTTPS: true
        TLSSecurityPolicy: Policy-Min-TLS-1-2-2019-07
      AccessPolicies:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub 'arn:aws:iam::${AWS::AccountId}:user/tychon-scanner'
            Action: 'es:*'
            Resource: !Sub 'arn:aws:es:${AWS::Region}:${AWS::AccountId}:domain/tychon-crypto-assets/*'

Outputs:
  DomainEndpoint:
    Value: !GetAtt TychonOpenSearchDomain.DomainEndpoint
    Export:
      Name: TychonOpenSearchEndpoint

Scanner Configuration

OpenSearch Integration Commands

Windows Linux macOS

# Direct OpenSearch integration
.\certscanner-windows-amd64.exe -host production-hosts.txt -cipherscan `
  -posttoelastic `
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" `
  -elasticapikey "$env:OPENSEARCH_API_KEY" `
  -elasticindex "crypto-assets-$(Get-Date -Format 'yyyy-MM')" `
  -tags "aws-opensearch,production"

# Memory and filesystem scanning with OpenSearch
.\certscanner-windows-amd64.exe -mode local -scanmemory -scanfilesystem `
  -posttoelastic `
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" `
  -elasticapikey "$env:OPENSEARCH_API_KEY" `
  -elasticindex "local-crypto-assets" `
  -tags "local-scan,aws-opensearch"

# Scheduled compliance scanning
.\certscanner-windows-amd64.exe -host critical-infrastructure.txt -cipherscan `
  -posttoelastic `
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" `
  -elasticapikey "$env:OPENSEARCH_API_KEY" `
  -elasticindex "compliance-$(Get-Date -Format 'yyyy-MM-dd')" `
  -tags "compliance,quarterly-audit"

# Direct OpenSearch integration
./certscanner-linux-x64 -host production-hosts.txt -cipherscan \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "crypto-assets-$(date +%Y-%m)" \
  -tags "aws-opensearch,production"

# Memory and filesystem scanning with OpenSearch
./certscanner-linux-x64 -mode local -scanmemory -scanfilesystem \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "local-crypto-assets" \
  -tags "local-scan,aws-opensearch"

# Scheduled compliance scanning  
./certscanner-linux-x64 -host critical-infrastructure.txt -cipherscan \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "compliance-$(date +%Y-%m-%d)" \
  -tags "compliance,quarterly-audit"

# Direct OpenSearch integration (Intel Macs)
./certscanner-darwin-amd64 -host production-hosts.txt -cipherscan \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "crypto-assets-$(date +%Y-%m)" \
  -tags "aws-opensearch,production"

# Direct OpenSearch integration (Apple Silicon)
./certscanner-darwin-arm64 -host production-hosts.txt -cipherscan \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "crypto-assets-$(date +%Y-%m)" \
  -tags "aws-opensearch,production"

# Scheduled compliance scanning
./certscanner-darwin-amd64 -host critical-infrastructure.txt -cipherscan \
  -posttoelastic \
  -elasticnode "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com" \
  -elasticapikey "$OPENSEARCH_API_KEY" \
  -elasticindex "compliance-$(date +%Y-%m-%d)" \
  -tags "compliance,quarterly-audit"

Authentication & Security

AWS IAM Authentication

Create IAM user and policy for TYCHON Quantum Readiness OpenSearch access

IAM Policy for OpenSearch Access

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "es:ESHttpPost",
        "es:ESHttpPut",
        "es:ESHttpGet"
      ],
      "Resource": "arn:aws:es:*:*:domain/tychon-crypto-assets/*"
    }
  ]
}

Create IAM User and Generate API Key

# Create IAM user for scanner
aws iam create-user --user-name tychon-scanner

# Attach policy to user
aws iam attach-user-policy \
  --user-name tychon-scanner \
  --policy-arn arn:aws:iam::ACCOUNT-ID:policy/TychonOpenSearchAccess

# Create access keys
aws iam create-access-key --user-name tychon-scanner

# Store credentials securely
export OPENSEARCH_API_KEY="your-access-key:your-secret-key"

VPC and Security Groups

# Configure security group for OpenSearch access
aws ec2 authorize-security-group-ingress \
  --group-id sg-opensearch-cluster \
  --protocol tcp \
  --port 443 \
  --source-group sg-tychon-scanners

# Allow scanner nodes access to OpenSearch domain
aws opensearch update-domain-config \
  --domain-name tychon-crypto-assets \
  --vpc-options SubnetIds=subnet-12345678,subnet-87654321,SecurityGroupIds=sg-opensearch-cluster

Index Templates & Mappings

Create Index Template for TYCHON Data

# Create index template for crypto assets
curl -X PUT "https://tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com/_index_template/tychon-crypto-assets" \
  -H "Authorization: AWS4-HMAC-SHA256 Credential=..." \
  -H "Content-Type: application/json" \
  -d '{
  "index_patterns": ["crypto-assets-*", "compliance-*"],
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1,
      "index.refresh_interval": "30s"
    },
    "mappings": {
      "properties": {
        "timestamp": { "type": "date" },
        "scan_id": { "type": "keyword" },
        "observer": {
          "properties": {
            "hostname": { "type": "keyword" },
            "ip": { "type": "ip" },
            "os": { "type": "keyword" }
          }
        },
        "target_host": {
          "properties": {
            "address": { "type": "ip" },
            "domain": { "type": "keyword" },
            "port": { "type": "integer" }
          }
        },
        "certificate": {
          "properties": {
            "subject": {
              "properties": {
                "common_name": { "type": "keyword" },
                "organization": { "type": "keyword" }
              }
            },
            "issuer": {
              "properties": {
                "common_name": { "type": "keyword" },
                "organization": { "type": "keyword" }
              }
            },
            "not_before": { "type": "date" },
            "not_after": { "type": "date" },
            "fingerprint_sha256": { "type": "keyword" }
          }
        },
        "cipher": {
          "properties": {
            "protocol": { "type": "keyword" },
            "name": { "type": "keyword" },
            "key_length_bits": { "type": "integer" },
            "intel": {
              "properties": {
                "security_level": { "type": "keyword" },
                "pqc_ready": { "type": "boolean" }
              }
            }
          }
        }
      }
    }
  }
}'

OpenSearch Dashboards

Dashboard Creation

• Certificate Inventory: Track certificate expiration and issuers
• Cipher Security: Monitor TLS cipher suite security levels
• PQC Readiness: Identify quantum-vulnerable cryptography
• Compliance Reports: Automated compliance dashboard generation

Visualization Types

• Time Series: Certificate discovery trends over time
• Heat Maps: Security level distribution across infrastructure
• Geospatial: Global certificate issuer distribution
• Data Tables: Searchable certificate inventories

Sample Dashboard Configuration

# Create dashboard for crypto asset monitoring
POST /_dashboards/api/saved_objects/_import
{
  "objects": [
    {
      "type": "dashboard",
      "id": "tychon-crypto-overview",
      "attributes": {
        "title": "TYCHON Crypto Asset Overview",
        "panels": [
          {
            "title": "Certificate Expiration Timeline",
            "type": "line",
            "query": {
              "bool": {
                "must": [
                  { "exists": { "field": "certificate.not_after" } }
                ]
              }
            }
          },
          {
            "title": "Cipher Security Distribution",
            "type": "pie",
            "query": {
              "bool": {
                "must": [
                  { "exists": { "field": "cipher.intel.security_level" } }
                ]
              }
            }
          }
        ]
      }
    }
  ]
}

Machine Learning & Analytics

Anomaly Detection

Use OpenSearch ML capabilities to detect unusual cryptographic patterns

# Create anomaly detector for certificate patterns
POST /_plugins/_anomaly_detector/detectors
{
  "name": "tychon-certificate-anomaly",
  "description": "Detect unusual certificate issuance patterns",
  "time_field": "timestamp",
  "indices": ["crypto-assets-*"],
  "feature_attributes": [
    {
      "feature_name": "certificate_count",
      "feature_enabled": true,
      "aggregation_query": {
        "certificate_count": {
          "value_count": {
            "field": "certificate.subject.common_name"
          }
        }
      }
    }
  ],
  "window_delay": {
    "period": {
      "interval": 1,
      "unit": "Minutes"
    }
  },
  "detection_interval": {
    "period": {
      "interval": 10,
      "unit": "Minutes"
    }
  }
}

Security Analytics

# Query for expiring certificates
GET /crypto-assets-*/_search
{
  "query": {
    "range": {
      "certificate.not_after": {
        "gte": "now",
        "lte": "now+30d"
      }
    }
  },
  "aggs": {
    "expiring_by_issuer": {
      "terms": {
        "field": "certificate.issuer.common_name"
      }
    }
  }
}

# Find quantum-vulnerable ciphers
GET /crypto-assets-*/_search
{
  "query": {
    "bool": {
      "must": [
        { "term": { "cipher.intel.pqc_ready": false } },
        { "term": { "cipher.intel.security_level": "medium" } }
      ]
    }
  }
}

AWS Service Integration

CloudWatch Integration

• Custom Metrics: Scanner execution metrics and success rates
• Log Groups: Centralized scanner logs with CloudWatch Logs
• Alarms: Alert on scan failures or security policy violations
• Dashboards: Combined OpenSearch and CloudWatch monitoring

Lambda Processing

• Data Transformation: Process scan results before indexing
• Alerting: Trigger notifications on security events
• Enrichment: Add threat intelligence to crypto assets
• Compliance: Automated compliance report generation

Sample Lambda Function for Data Processing

import json
import boto3
from opensearchpy import OpenSearch, RequestsHttpConnection
from aws_requests_auth.aws_auth import AWSRequestsAuth

def lambda_handler(event, context):
    # Initialize OpenSearch client with AWS authentication
    host = 'tychon-crypto-assets-abc123.us-west-2.es.amazonaws.com'
    region = 'us-west-2'
    service = 'es'
    credentials = boto3.Session().get_credentials()
    awsauth = AWSRequestsAuth(credentials, region, service)
    
    client = OpenSearch(
        hosts=[{'host': host, 'port': 443}],
        http_auth=awsauth,
        use_ssl=True,
        verify_certs=True,
        connection_class=RequestsHttpConnection
    )
    
    # Process TYCHON scan results
    for record in event['Records']:
        # Parse scan result
        scan_data = json.loads(record['body'])
        
        # Enrich with threat intelligence
        if 'cipher' in scan_data:
            scan_data['cipher']['threat_level'] = assess_cipher_risk(scan_data['cipher'])
        
        # Index to OpenSearch
        response = client.index(
            index=f"crypto-assets-{scan_data['timestamp'][:7]}",
            body=scan_data
        )
        
    return {'statusCode': 200, 'body': 'Processed successfully'}

def assess_cipher_risk(cipher_data):
    # Custom threat intelligence logic
    if not cipher_data.get('intel', {}).get('pqc_ready', True):
        return 'high'
    elif cipher_data.get('intel', {}).get('security_level') == 'low':
        return 'medium'
    return 'low'

Best Practices

Performance Optimization

• Index Lifecycle: Use ILM policies for automatic data management
• Sharding Strategy: Optimize shards based on data volume
• Batch Indexing: Use bulk operations for high-volume scanning
• Index Rollover: Monthly index rotation for performance

Security & Compliance

• Encryption: Enable encryption at rest and in transit
• Access Control: Use fine-grained access control (FGAC)
• Audit Logging: Enable OpenSearch audit logs
• VPC Deployment: Deploy in private subnets with VPC endpoints

Monitoring & Alerting

OpenSearch Alerting

# Create monitor for certificate expiration
POST /_plugins/_alerting/monitors
{
  "name": "Certificate Expiration Monitor",
  "type": "monitor",
  "monitor_type": "query_level_monitor",
  "enabled": true,
  "schedule": {
    "period": {
      "interval": 1,
      "unit": "DAYS"
    }
  },
  "inputs": [{
    "search": {
      "indices": ["crypto-assets-*"],
      "query": {
        "query": {
          "range": {
            "certificate.not_after": {
              "gte": "now",
              "lte": "now+30d"
            }
          }
        }
      }
    }
  }],
  "triggers": [{
    "name": "Certificates Expiring Soon",
    "severity": "2",
    "condition": {
      "script": {
        "source": "ctx.results[0].hits.total.value > 0"
      }
    },
    "actions": [{
      "name": "Send SNS Notification",
      "destination_id": "sns-crypto-alerts",
      "subject_template": {
        "source": "TYCHON: {{ctx.results.0.hits.total.value}} certificates expiring within 30 days"
      },
      "message_template": {
        "source": "Found {{ctx.results.0.hits.total.value}} certificates expiring in the next 30 days. Please review in OpenSearch Dashboards."
      }
    }]
  }]
}

CloudWatch Metrics

• Cluster health and performance
• Index size and document count
• Search and indexing latency
• Failed authentication attempts

SNS Integration

• Certificate expiration notifications
• Security policy violation alerts
• Scan failure notifications
• Compliance report delivery