Keyword for skipping context for AI

[dmitromikh]

It could be useful for some parts of the semantic layer to exist in YAML but be excluded from the AI context. For example, functional columns or when experimenting with different context configurations. Sometimes, extra data in the context can actually harm SQL generation, so having this option provides an additional level of control.

This could be implemented as an optional key:

ai_context:
  skip: true

[PioMen]

I agree that experimenting with context is key, but shouldn't this be handled by the AI application rather than the semantic layer?

​I can see some reasons why handling it in the YAML might be too restrictive:

• Multiple tools: If you have different AI tools pulling from the same model, a hardcoded skip is a bit of a "one size fits all" solution. One tool might need a specific field while another doesn't.
• ​Easier testing: Controlling what AI can "see" in the AI tool makes it faster to test how different configurations affect SQL generation without having to redeploy the semantic layer.

​My take is: while extra data can definitely harm generation, it feels to me like that logic belongs in the app layer where the actual experimentation happens.

[QMalcolm]

This might be a hot take, but I think having AI generate the sql is long term the wrong approach. I believe the plan is for OSI to have a query engine of reference It should be able to generate SQL idempotently. We've run some expirements at dbt Labs of agents generating SQL to answer a question vs agents looking at a semantic layer catalog, and then using a query engine for the semantic layer to answer the question. That latter produced much more predictable and accurate results. The former meant getting slightly different SQL each invocation and getting potentially very wrong SQL 😅

That is to say, I don't think agents should generate the SQL for semantic layer definitions, but instead use semantic layer definitions to figure out what metric(s) to query with an engine.

[PioMen]

You hit the nail on the head with this one. I've been thinking about this a lot and the scenario you mention is ideal. I can think there might be some edge cases though. Let me give you an example. It's pretty obvious that if you define "churn" as a metric and you want to analyze it through different dimensions - querying the semantic layer is the best option.

But what if your request is less standard? Like you need to find users who churned, then started using services again, then churned again. So we are talking about complex patterns not predefined within the layer.

I've seen similar scenario with traditional self-service. You share the model with the business and they can use it to address most common questions. But there is still quite a large chunk that has to be custom built by analyst.

I'm wondering what are your thoughts on this :)