From intuition to implementation
How Behavioral Sequence Modeling Works
仕組み
The idea is simple: treat security logs like language and let a transformer learn what normal looks like. The implementation takes that intuition through formalization, preprocessing, and training.
Selected implementation details are available in a private technical walkthrough.
A security engineering portfolio project by Vyshyvka Studio
The Shift
The question changed.
“Does this match a known pattern?”
Approach
LLM enrichment, IOC correlation, agentic triage
Output
847 findings/day
“Does this sequence make sense?”
Approach
Behavioral sequence modeling, perplexity detection
Output
5 stories/day
AI-Native Assessment
Kerberos auth. No IOC match.
In user's access scope. DLP clean.
1/5 threshold. No brute force pattern.
Post-failure re-auth. Normal retry.
Valid RBAC. UEBA score normal.
T1052.001. USB policy — verify.
T1048 possible. DLP — no trigger.
Device removal. No data in motion.
Read it as a story. See it now?
Every assessment is thorough. IOC lookups, MITRE mapping, DLP cross-references, UEBA scoring — genuinely good per-event analysis. But the sequence is a textbook exfiltration, and no amount of per-event enrichment can surface it.
The Language
Events become words.
Raw logs are messy, unbounded, infinite in variety. OCSF normalization extracts categorical fields. Categorical fields become template keys. Template keys map to token IDs. The result: a closed vocabulary.
{"EventID":4624,"LogonType":3,"AuthPkg":"Kerberos","Status":"0x0","TargetUser":"jsmith","WorkStation":"WS-4421","IpAddress":"10.0.1.47"}AUTH|Logon|Kerberos|Network|Success
Just as English has ~50,000 common words, behavioral language has 2000 words. Finite. Bounded. Learnable.
Not unbounded log patterns. A closed vocabulary. 2000 words. Enough to describe every security-relevant action a user can take.
Private technical disclosure
OCSF formalization and behavior language levels
The detailed schema formalization, vocabulary construction, and representation boundaries are available in a private technical walkthrough.
The Sequence
Words become stories.
Tokenized events are grouped into per-user session chains — one action after another, forming a behavioral sentence. The model never sees individual events. It reads complete user stories.
When the model is surprised — when the next event has very low probability — the story stopped making sense. We call this perplexity.
Process:Elevate has 0.1% probability given this user's history. The narrative just broke. Perplexity spiked. That is the signal.
Private technical disclosure
Sequence probability and mathematical foundations
The probability formulation, score construction, and worked examples are reserved for a focused technical discussion.
The Break
Same events. Different understanding.
Kerberos. No IOC correlation.
Config read. Standard admin.
Valid RBAC. UEBA: normal.
Temp write. No persistence IOC.
T1052.001. USB policy — verify.
T1048 candidate. DLP clean.
They classify events. We read stories.
The Threshold Trap
Why traditional detection fails
● All Metrics Normal
Every metric is below threshold. Traditional systems see nothing wrong.
● User C625 - Event Timeline
Consider User C625 from the LANL dataset. Over three weeks, they exfiltrated sensitive data while every metric stayed comfortably below threshold. Login frequency? Normal. Data access volume? Within bounds. Time-of-day patterns? Unremarkable.
The problem isn't the thresholds—it's the assumption that threats manifest as extreme values. Sophisticated insiders operate in the statistical middle, exploiting the blind spots created by univariate thinking.
The Narrative Hypothesis
"Insider threats manifest as narrative violations—sequences that are syntactically valid but semantically incoherent with a user's established behavioral story."
The Preprocessing Pipeline
From raw logs to semantic events
Three services transform chaotic log streams into structured behavioral sequences using OCSF categorical tokenization.
Gateway
Traffic Cop80% of logs are noise (heartbeats, health checks)
Multi-layer filtering + OCSF normalization + consistent hashing
Tokenizer
TranslatorInfinite log variations → unbounded vocabulary
OCSF categorical tokenization: template key = class_uid|activity_id|status_id|...
Profiler
Memory"Anomaly" is relative to role—what's normal for DevOps isn't for Finance
Template frequency distribution (L2-normalized) + K-Means clustering into peer groups
# Gateway Service: Log filtering and routingdef process_log(raw_log: str) -> Optional[GatewayOutput]: # Layer 1: Fast regex rejection (heartbeats, health checks) if NOISE_PATTERNS.match(raw_log): return None
# Layer 2: Parse and normalize fields parsed = parse_log_format(raw_log) # JSON, syslog, CEF, etc. normalized = { "timestamp": parse_timestamp(parsed.time), "user": normalize_user(parsed.user or parsed.src_user), "action": normalize_action(parsed.event_type), "resource": normalize_resource(parsed.target), "source_ip": parsed.src_ip, "metadata": extract_metadata(parsed) }
# Layer 3: Route via consistent hashing partition = consistent_hash(normalized.user) % NUM_PARTITIONS
return GatewayOutput( event=normalized, partition=partition, trace_id=generate_trace_id() )Gateway Filtering
Heartbeats, health checks, and routine polling are filtered out, leaving only security-relevant events.
Private technical walkthrough
The Transformer Architecture
The public overview stops at the architectural boundary. The model topology, contextual embedding strategy, sequence handling, and inference tradeoffs are reserved for a focused walkthrough.
Available for focused technical discussions and interviews.
Private technical walkthrough
Dual Training Objectives
The training strategy is discussed privately, including how the objectives interact, how negative examples are constructed, and how scoring behavior is evaluated.
Available for focused technical discussions and interviews.
The Output
Five stories. Not eight hundred findings.
Removable Media Data Staging (T1052.001)
USB device on WS-4421 by jsmith at 09:15 UTC. File transfer: customer_list.xlsx (2.3MB). MITRE: T1052.001. DLP: no policy trigger — file within RBAC scope. Threat intel: no active IOC. Recommend: verify business justification, check USB history...
Failed Auth → Success Sequence
jsmith Kerberos fail at 08:58 then success at 09:01 from 10.0.1.47. MITRE: T1110 — below threshold (1/5). No concurrent source IPs. Behavioral baseline: 3 similar patterns this month. Risk score: 12/100. Likely benign...
PII-Tagged File Access
jsmith accessed customer_list.xlsx (PII-SENSITIVE) at 09:12. Valid RBAC. DLP scan: clean. UEBA: normal for role. Logged for SOX compliance. No action required unless correlated with exfiltration event...
Auth:Logon → File:Read → Device:Mount → File:Copy → USB_EJECT
Auth:Logon → Auth:Logon → Auth:Logon (3 hosts)
File:Read → File:Read → File:Read (47 files, 12 min)
Auth:Logon → Process:Elevate → Registry:Write
Auth:Logon (02:47 UTC) → File:Read → Email:Send
The findings are thorough — MITRE mappings, IOC lookups, risk scores, UEBA baselines. Genuinely good per-event analysis. But an analyst still faces 847 of them every day. Better enrichment didn't solve the volume problem. Comprehension does.