Schema Reference

OpenEye uses Pydantic models for structured output. All models are defined in cli/openeye_ai/schema.py.

Normalized bounding box (0-1 range).

Field	Type	Description
model	str	Model name
task	str	Task type (detection, depth, segmentation, vla)
timestamp	str	ISO 8601 timestamp
image	ImageInfo	Input image info
objects	list[DetectedObject]	Detected objects
depth_map	str or null	Base64 depth map PNG
segmentation_masks	list[SegmentationMask] or null	Segmentation masks from SAM2
vla_action	list[float] or null	VLA action vector for robotic control
inference_ms	float	Inference time in milliseconds

A memory-worthy snapshot from one perception tick, used by the agentic pipeline.

Field	Type	Description
id	str	Auto-generated unique ID
tick	int	Tick number
timestamp	str	ISO 8601 timestamp
detections	list[DetectedObject]	Objects detected in this tick
scene_summary	str	Brief scene summary
change_description	str	What changed since last tick
significance	float	Significance score (0-1)
tags	list[str]	Searchable tags

One reasoning step produced by the LLM in the agentic loop.

Field	Type	Description
observation_summary	str	Summary of what was observed
memory_context	str	Relevant context recalled from memory
chain_of_thought	str	LLM reasoning chain
current_plan	list[str]	Current action plan steps
decided_action	str	Action decided by the agent
plan_changed	bool	Whether the plan was updated this tick

Complete tick output streamed to clients via SSE from /agent/stream.

Field	Type	Description
tick	int	Tick number
phase	str	Current phase: perceive, recall, reason, or act
prediction	PredictionResult or null	Raw prediction result
observation	Observation or null	Processed observation
reasoning	AgentReasoning or null	LLM reasoning output
action_taken	str	Action that was executed
memory_recalled	list[Observation]	Observations recalled from memory
current_plan	list[str]	Current plan steps
timestamp	str	ISO 8601 timestamp

Query for memory recall via POST /agent/recall.

Field	Type	Description
query	str	Free-text query
time_range	str or null	Time filter (e.g. "last_1h", "last_24h")
significance_min	float	Minimum significance threshold
limit	int	Max results to return (default 10)

Response from memory recall.