Add direct catalog entry lookup by identifier

adr/0010-direct-catalog-entry-lookup.md

@@ -0,0 +1,27 @@
+# ADR-0010: Direct Catalog Entry Lookup by Identifier
+
+## Status
+
+Proposed
+
+## Context
+
+ARD identifiers are stable, globally unique primary keys, but the Registry API previously offered only semantic Search and optional deterministic List operations. Search requires natural-language input and adds ranking metadata. List returns a paginated collection and does not define canonical exact-identifier semantics. Neither operation lets a client efficiently resolve an identifier retained from earlier discovery.
+
+The lookup operation also needs a clear federation boundary. Automatically forwarding a primary-key lookup would make latency and authority unpredictable, while returning a Search result would expose fields that are not part of the canonical Catalog Entry Object.
+
+## Decision
+
+Agent Registries expose mandatory `GET /agents/{identifier}` lookup.
+
+The complete identifier is UTF-8 percent-encoded as one path segment. A registry decodes it exactly once, validates the decoded ARD identifier, and performs an exact, case-sensitive lookup against its local index. The registry does not query upstream registries for this operation. Entries ingested from external publishers remain eligible because they are part of the local index.
+
+A successful lookup returns the canonical Catalog Entry Object without Search-only fields. Malformed identifiers return `400 INVALID_ARGUMENT`; valid identifiers absent from the local index return `404 NOT_FOUND`. The endpoint follows the deployment's existing Registry authentication policy and standard HTTP caching semantics.
+
+## Consequences
+
+- Clients can resolve a retained ARD identifier in one deterministic request.
+- OpenAPI clients receive a typed identifier parameter and Catalog Entry response.
+- Registries need an indexed exact-match route in addition to semantic Search.
+- Clients retaining federated Search results should retain `source` with the identifier and use it to resolve the entry at its originating registry.
+- Conformance testing verifies encoded success, malformed identifiers, and unknown identifiers.

conformance/README.md

@@ -53,7 +53,7 @@ Probes and validates a live running Agent Registry REST API server.
 ./bin/conformance-test registry http://localhost:9010/api
 ```
 
-For private or self-hosted registries that require request headers, pass one or more `--header` options. Headers are applied to all Registry API probes (`GET /agents`, `POST /search`, and `POST /explore`):
+For private or self-hosted registries that require request headers, pass one or more `--header` options. Headers are applied to all Registry API probes (`GET /agents`, `GET /agents/{identifier}`, `POST /search`, and `POST /explore`):
 
 ```bash
 ARD_REGISTRY_TOKEN=...
@@ -63,6 +63,13 @@ ARD_REGISTRY_TOKEN=...
 
 This is a conformance tooling option only; it does not require authentication for public registries or define a Registry API security model.
 
+The tester normally obtains an identifier for the successful exact-lookup probe from List or Search. If neither returns an entry, provide a known locally indexed identifier explicitly:
+
+```bash
+./bin/conformance-test registry https://registry.example.com/api/ard \
+  --lookup-identifier "urn:air:example.com:agent:assistant"
+```
+
 ---
 
 ## 🔍 What It Validates
@@ -89,6 +96,11 @@ When checking a live Agent Registry server, the tool executes the following prob
   * Contacts the GET endpoint to check if the registry supports deterministic browsing.
   * If supported (returns `200 OK`), it validates that the response contains the paginated `"items"` structure.
   * If not supported (returns `404` or `501`), it marks this as compliant since deterministic listing is optional.
+* **GET `/agents/{identifier}` (Mandatory Exact Lookup Probe)**:
+  * URL-encodes a known full URN as one path segment and verifies a `200 OK` canonical `CatalogEntry` response with the exact identifier.
+  * Verifies that Search-only fields such as `score` and `source` are absent.
+  * Verifies that an unknown valid URN returns the standard `404` / `NOT_FOUND` error envelope.
+  * Verifies that a malformed identifier returns the standard `400` / `INVALID_ARGUMENT` error envelope.
 * **POST `/search` (Mandated Search Probe)**:
   * Probes the search route which is required for dynamic semantic capability discovery.
   * Sends a mock natural-language query payload with required `query` string and optional `filter` / `limit` parameters.

conformance/bin/conformance-test

@@ -10,6 +10,7 @@ import re
 import json
 import urllib.request
 import urllib.error
+import urllib.parse
 
 # Strict URN Regex matching urn:air:<publisher>:<namespace>:<agent-name>
 URN_REGEX = re.compile(r"^urn:air:([a-zA-Z0-9.-]+)(?::([a-zA-Z0-9._:-]+))?:([a-zA-Z0-9._-]+)$")
@@ -75,6 +76,59 @@ class ConformanceTester:
         self.warnings.append(msg)
         print_warning(msg)
 
+    def validate_catalog_entry(self, entry, label):
+        if not isinstance(entry, dict):
+            self.add_error(f"[{label}] Catalog entry must be a JSON object.")
+            return False
+
+        valid = True
+        identifier = entry.get("identifier")
+        if not identifier:
+            self.add_error(f"[{label}] Catalog entry is missing 'identifier'.")
+            valid = False
+        elif not URN_REGEX.match(identifier):
+            self.add_error(f"[{label}] Catalog entry identifier '{identifier}' is not a valid ARD URN.")
+            valid = False
+
+        if not entry.get("displayName"):
+            self.add_error(f"[{label}] Catalog entry is missing 'displayName'.")
+            valid = False
+        if not entry.get("type"):
+            self.add_error(f"[{label}] Catalog entry is missing 'type'.")
+            valid = False
+
+        has_url = "url" in entry
+        has_data = "data" in entry
+        if has_url == has_data:
+            self.add_error(f"[{label}] Catalog entry must contain exactly one of 'url' or 'data'.")
+            valid = False
+
+        return valid
+
+    def validate_http_error(self, error, expected_status, expected_code, label):
+        valid = True
+        if error.code != expected_status:
+            self.add_error(f"{label} returned {error.code}; expected {expected_status}.")
+            valid = False
+
+        try:
+            data = json.loads(error.read().decode("utf-8"))
+        except (UnicodeDecodeError, json.JSONDecodeError) as exc:
+            self.add_error(f"{label} did not return a valid JSON error envelope: {exc}")
+            return False
+
+        if not isinstance(data, dict):
+            self.add_error(f"{label} error response must be a JSON object.")
+            return False
+        if data.get("errorCode") != expected_code:
+            self.add_error(f"{label} returned errorCode '{data.get('errorCode')}'; expected '{expected_code}'.")
+            valid = False
+        if not data.get("message"):
+            self.add_error(f"{label} error response is missing 'message'.")
+            valid = False
+
+        return valid
+
     def run_json_schema_validation(self, manifest_data):
         if not self.jsonschema_available:
             print_warning("Python 'jsonschema' package not installed. Skipping strict JSON Schema check.")
@@ -209,7 +263,7 @@ class ConformanceTester:
 
         return len(self.errors) == 0
 
-    def validate_registry(self, registry_base_url, request_headers=None):
+    def validate_registry(self, registry_base_url, request_headers=None, lookup_identifier=None):
         print_header(f"Validating Agent Registry: {registry_base_url}")
         base_url = registry_base_url.rstrip('/')
         request_headers = request_headers or {}
@@ -227,8 +281,20 @@ class ConformanceTester:
                     # Check schema conformance
                     if not isinstance(data, dict) or "items" not in data:
                         self.add_error("GET /agents response is not a valid paginated object. Missing 'items' array.")
+                    elif not isinstance(data["items"], list):
+                        self.add_error("GET /agents response 'items' is not a JSON array.")
                     else:
-                        print_success(f"GET /agents response contains valid 'items' list ({len(data.get('items', []))} items).")
+                        items = data["items"]
+                        print_success(f"GET /agents response contains valid 'items' list ({len(items)} items).")
+                        if lookup_identifier is None:
+                            lookup_identifier = next(
+                                (
+                                    item.get("identifier")
+                                    for item in items
+                                    if isinstance(item, dict) and URN_REGEX.match(item.get("identifier", ""))
+                                ),
+                                None
+                            )
                 else:
                     self.add_warning(f"GET /agents returned unexpected status: {response.status}")
         except urllib.error.HTTPError as e:
@@ -292,28 +358,91 @@ class ConformanceTester:
                             if not source:
                                 self.add_error(f"[{label}] Missing required SearchResultItem field 'source'.")
 
-                            ident = item.get("identifier")
-                            if not ident:
-                                self.add_error(f"[{label}] Nested catalog entry is missing 'identifier'.")
-
-                            disp_name = item.get("displayName")
-                            if not disp_name:
-                                self.add_error(f"[{label}] Nested catalog entry is missing 'displayName'.")
-
-                            mtype = item.get("type")
-                            if not mtype:
-                                self.add_error(f"[{label}] Nested catalog entry is missing 'type'.")
-
-                            if "url" not in item and "data" not in item:
-                                self.add_error(f"[{label}] Nested catalog entry must contain either 'url' or 'data'.")
+                            self.validate_catalog_entry(item, label)
+                            if lookup_identifier is None and URN_REGEX.match(item.get("identifier", "")):
+                                lookup_identifier = item["identifier"]
                 else:
                     self.add_error(f"POST /search returned unexpected HTTP status: {response.status}")
         except urllib.error.HTTPError as e:
             self.add_error(f"POST /search request failed: {e.code} {e.reason}")
         except Exception as e:
             self.add_error(f"Failed to execute search probe: {e}")
 
-        # 3. Test POST /explore (Optional Introspection Endpoint)
+        # 3. Test GET /agents/{identifier} (Mandatory Exact Lookup Endpoint)
+        if lookup_identifier is not None:
+            encoded_identifier = urllib.parse.quote(lookup_identifier, safe="")
+            lookup_url = f"{base_url}/agents/{encoded_identifier}"
+            print(f"\n  {COLOR_BOLD}Probing GET /agents/{{identifier}} with an encoded known URN...{COLOR_RESET}")
+            try:
+                req = urllib.request.Request(lookup_url, headers=request_headers, method="GET")
+                with urllib.request.urlopen(req, timeout=5) as response:
+                    body = response.read().decode("utf-8")
+                    data = json.loads(body)
+                    if response.status != 200:
+                        self.add_error(f"GET /agents/{{identifier}} returned unexpected HTTP status: {response.status}")
+                    else:
+                        valid_entry = self.validate_catalog_entry(data, "Lookup response")
+                        if valid_entry:
+                            if data["identifier"] != lookup_identifier:
+                                self.add_error(
+                                    "GET /agents/{identifier} did not return the exact requested identifier. "
+                                    f"Expected '{lookup_identifier}', got '{data['identifier']}'."
+                                )
+                                valid_entry = False
+                            for search_only_field in ("score", "source"):
+                                if search_only_field in data:
+                                    self.add_error(
+                                        f"GET /agents/{{identifier}} response contains Search-only field "
+                                        f"'{search_only_field}'."
+                                    )
+                                    valid_entry = False
+                        if valid_entry:
+                            print_success("GET /agents/{identifier} returned the exact catalog entry for an encoded URN.")
+            except urllib.error.HTTPError as e:
+                self.add_error(f"GET /agents/{{identifier}} request failed: {e.code} {e.reason}")
+            except (UnicodeDecodeError, json.JSONDecodeError) as e:
+                self.add_error(f"GET /agents/{{identifier}} returned invalid JSON: {e}")
+            except Exception as e:
+                self.add_error(f"Failed to execute exact lookup probe: {e}")
+        else:
+            self.add_warning(
+                "Could not infer a known catalog identifier from List or Search; "
+                "skipping the successful lookup probe. Pass --lookup-identifier to run it."
+            )
+
+        unknown_identifier = "urn:air:ard-conformance.invalid:probe:not-found"
+        unknown_url = f"{base_url}/agents/{urllib.parse.quote(unknown_identifier, safe='')}"
+        print(f"\n  {COLOR_BOLD}Probing GET /agents/{{identifier}} with an unknown URN...{COLOR_RESET}")
+        try:
+            req = urllib.request.Request(unknown_url, headers=request_headers, method="GET")
+            with urllib.request.urlopen(req, timeout=5) as response:
+                self.add_error(
+                    "GET /agents/{identifier} returned "
+                    f"{response.status} for an unknown identifier; expected 404."
+                )
+        except urllib.error.HTTPError as e:
+            if self.validate_http_error(e, 404, "NOT_FOUND", "Unknown identifier lookup"):
+                print_success("Unknown identifier returned 404 NOT_FOUND.")
+        except Exception as e:
+            self.add_error(f"Failed to execute unknown identifier probe: {e}")
+
+        malformed_identifier = "not-a-valid-ard-urn"
+        malformed_url = f"{base_url}/agents/{urllib.parse.quote(malformed_identifier, safe='')}"
+        print(f"\n  {COLOR_BOLD}Probing GET /agents/{{identifier}} with a malformed identifier...{COLOR_RESET}")
+        try:
+            req = urllib.request.Request(malformed_url, headers=request_headers, method="GET")
+            with urllib.request.urlopen(req, timeout=5) as response:
+                self.add_error(
+                    "GET /agents/{identifier} returned "
+                    f"{response.status} for a malformed identifier; expected 400."
+                )
+        except urllib.error.HTTPError as e:
+            if self.validate_http_error(e, 400, "INVALID_ARGUMENT", "Malformed identifier lookup"):
+                print_success("Malformed identifier returned 400 INVALID_ARGUMENT.")
+        except Exception as e:
+            self.add_error(f"Failed to execute malformed identifier probe: {e}")
+
+        # 4. Test POST /explore (Optional Introspection Endpoint)
         explore_url = f"{base_url}/explore"
         print(f"\n  {COLOR_BOLD}Probing POST /explore (Optional Introspection endpoint)...{COLOR_RESET}")
 
@@ -375,7 +504,7 @@ def main():
         print(f"{COLOR_BOLD}Agentic Resource Discovery Conformance CLI v0.5.0{COLOR_RESET}")
         print("Usage:")
         print("  conformance-test manifest <local_file_path_or_url>")
-        print("  conformance-test registry <registry_base_url> [--header 'Name: value']...")
+        print("  conformance-test registry <registry_base_url> [--header 'Name: value']... [--lookup-identifier <urn>]")
         sys.exit(1)
 
     command = sys.argv[1].lower()
@@ -410,16 +539,26 @@ def main():
                 sys.exit(1)
     elif command == "registry":
         header_args = []
+        lookup_identifier = None
         idx = 0
         while idx < len(args):
             arg = args[idx]
-            if arg != "--header":
-                print_failure(f"Unknown registry option '{arg}'. Supported option: --header 'Name: value'")
+            if arg not in ["--header", "--lookup-identifier"]:
+                print_failure(
+                    f"Unknown registry option '{arg}'. Supported options: "
+                    "--header 'Name: value', --lookup-identifier <urn>"
+                )
                 sys.exit(1)
             if idx + 1 >= len(args):
-                print_failure("--header requires a value in 'Name: value' format.")
+                print_failure(f"{arg} requires a value.")
                 sys.exit(1)
-            header_args.append(args[idx + 1])
+            if arg == "--header":
+                header_args.append(args[idx + 1])
+            else:
+                if lookup_identifier is not None:
+                    print_failure("--lookup-identifier may be specified only once.")
+                    sys.exit(1)
+                lookup_identifier = args[idx + 1]
             idx += 2
 
         try:
@@ -428,7 +567,15 @@ def main():
             print_failure(str(e))
             sys.exit(1)
 
-        success = tester.validate_registry(target, request_headers=request_headers)
+        if lookup_identifier is not None and not URN_REGEX.match(lookup_identifier):
+            print_failure("--lookup-identifier must be a valid urn:air: identifier.")
+            sys.exit(1)
+
+        success = tester.validate_registry(
+            target,
+            request_headers=request_headers,
+            lookup_identifier=lookup_identifier
+        )
     else:
         print_failure(f"Unknown test suite command '{command}'. Must be 'manifest' or 'registry'.")
         sys.exit(1)

conformance/bin/run-conformance-demo

@@ -52,7 +52,8 @@ sleep 1.5
 
 # 3. Validate live running registry server
 echo -e "\n${BOLD}[Step 3/3] Probing and validating live example registry API...${RESET}"
-"$PROJECT_ROOT/conformance/bin/conformance-test" registry "http://127.0.0.1:9010"
+"$PROJECT_ROOT/conformance/bin/conformance-test" registry "http://127.0.0.1:9010" \
+    --lookup-identifier "urn:air:acme.com:agent:assistant"
 REGISTRY_STATUS=$?
 
 if [ $REGISTRY_STATUS -eq 0 ]; then