Hacker Newsnew | past | comments | ask | show | jobs | submitloginShow HN: Airbyte Agents – context for agents across multiple data sources156 points by mtricot 39 days ago | hide | past | favorite | 48 commentsI’m Michel, co-founder and CEO of Airbyte (https://airbyte.com/). We’ve spent the last six years building data connectors. Today we're launching Airbyte Agents (https://docs.airbyte.com/ai-agents/), a unified data layer for agents to discover information and take action across operational systems.Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fgAs agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building! swyx 39 days ago | next [–] (former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!)hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K...i think my most sad/interesting observation about ai engineers is that many ai apps are super data hungry, but many dont have the necessary data engineering background to even know they need an airbyte or what tradeoffs to make in an etl pipeline. would love a "data engineering for ai engineers" type braindump session from someone from airbyte at AIE (https://ai.engineer/cfp )aaronsteers 39 days ago | parent | next [–] Hey, swyx! Great seeing you here.> airbyte agents could serve as a form of MCP gatewayExactly! And a single set of tools for agents to access both realtime (direct reads/writes) as well as cached (Context Store), bringing hopefully the best access path for each different use case.> would love a "data engineering for ai engineers" type braindump ... at AIEGreat idea - we have a booth at AIE, and we'll submit there for a talk. Mario will reach out to you about this. :)jeanlaf 39 days ago | parent | prev | next [–] Thanks swyx! We'd love to do that session "data engineering for ai engineers", will make you an intro to the right person in the team.swyx 38 days ago | root | parent | next [–] saw your email, will get back!sails 38 days ago | parent | prev | next [–] I think this is right ( a big gap ) but I don’t think data companies even now what the right shape is for AI.It’s definitely not old school ETL + dbt + BI tool, it might be something like this, but it’s very earlyaaronsteers 37 days ago | root | parent | next [–] Agreed 100% - we're still super early in this journey, gathering data from our own usage and from our customers' feedback.slurpyb 39 days ago | prev | next [–] Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human.It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience isdavinchia 38 days ago | parent | next [–] Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!mtricot 39 days ago | parent | prev | next [–] Let me see what's up and fix that!jscheel 39 days ago | prev | next [–] I feel like we've been working in parallel here :) We are using PyAirbyte (hi aaronsteers) for our users to connect their data sources to our agents. We originally wanted to use the airbyte white-label platform, but the team said that it was being deprecated. I think this really drives home just how crucial it is to have a clear model for accessing your data, and Airbyte has been great at that for quite a while.aaronsteers 39 days ago | parent | next [–] Hello, Jared! Small world! Yes, we did deprecate our old PbA (Powered by Airbyte) offering, but in many ways our new Agents and Embedded offering is a more robust and agent-friendly successor to that older offering.I am happy to hear you are still getting value out of PyAirbyte! If you do try out Airbyte Agents, please let us know how it goes! We are always listening to feedback and would love to hear from you as you explore the new tools and capabilities.jessewmc 39 days ago | prev | next [–] Looks interesting!If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?Have you done any testing on guided indexing, or metadata layers on top of the data? My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time. I.e. _what_ you index, and how you signpost for agents, becomes really important (unless your data is super clean I guess). This does look like a good foundation for that kind of tooling though!aaronsteers 39 days ago | parent | next [–] Hi, @jessewmc. Thanks for your reply. Regarding your points:> If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?While we haven't yet published details on the backend implementation, I can say that our implementation performs very well without needing to prioritize specific fields for indexing. We aim for large text fields to perform decently and retrieval based on small/compressible fields like ints to be fast. (More to come on this in the coming months.)> Have you done any testing on guided indexing, or metadata layers on top of the data?We've been testing with different data scales and shapes. Nothing detailed to share yet, but performance has (so far) never itself become the bottleneck in our agent testing. (The LLM thinking itself is often the bottleneck.)> My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time.Airbyte has rich metadata on our upstream connector's data models, which I think helps us a lot to deliver helpful context to the agent. Another option, when optimizing for specific use cases, is to build your own agent tools on top of our Agent SDK. This allows you to make the calls organic and build the tools in a way that makes natural sense to the agent, regardless of source shape or which system(s) that data is coming from.> This does look like a good foundation for that kind of tooling though!We agree! Thanks again for sharing your thoughts here.juancs 36 days ago | root | parent | next [–] Great launch btw! I have some questions if you don't mindyou mentioned that performance was never an issue, I am really intrigued how this is achieved.I have 3 General questions:1. How big (estimate in bytes) and complex were the test datasources? I couldn't find this in the benchmark repo.2. how is the business context managed? In the blog "Airbyte Agents: A New Era for Airbyte" it was mentioned handling the business context but in the context layer docs it only talks about schema discovery (I got a bit confused)3. When you said performance was never an issue, do you mean the user always got the answer it was looking for?dennispi 38 days ago | prev | next [–] We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compareWe likewise measured token savings, (wall clock) time, # tool calls, and # turns.nerdright 39 days ago | prev | next [–] This is such a great direction airbyte is taking and congrats to the lunch! I think you're very well-positioned for this opportunity than most people realize, given your reputable brand and your uncanny expertise in etl. It's honestly a natural progression of airbyte as far as the current AI landscape goes. Kudos to you and the team!(We use airbyte at my company, although we self-host it.)aaronsteers 39 days ago | parent | next [–] Thanks! Really appreciate the kind words. Looking forward to seeing what our amazing community builds with these new tools.andai 38 days ago | prev | next [–] The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query?(I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)sho 38 days ago | parent | next [–] That's actually the approach we took with https://gentility.ai/ - we either provide almost-raw SQL query access to the DBs themselves or we synthesize from API into DuckDB via parquet and make that available to the agent to just directly query. It works well - my philosophy is to give agents the sharpest tools you can, and SQL is the best tool there is.I understand the instinct to try to make a proprietary moat around it all but I think the pattern is useful and obvious enough that all big orgs will be doing something very similar within 5 years or so.aaronsteers 38 days ago | parent | prev | next [–] Helpful feedback, thank you! And your instincts are spot on. As of now, we have API based search, with filter predicates and field selection in JSON. While we haven't published anything on the backend implementation, I can say it does use a cloud-native storage medium where the filters are indeed pushed down as SQL. We want to be careful about if/when we offer direct SQL access, specifically because SQL dialects can differ drastically and we wouldn't want to break consumers if/when we change which dialect(s) are supported.That said, please stay tuned - and thank you again for this valuable feedback.thecopy 38 days ago | prev | next [–] Super interesting idea! Congrats on the launch. Context is definitely something that is lacking in my experience. Im always frustrated when an agent cannot answer business-related questions, and i compare them to coding agents which seem to be able to answer everything. The difference is that coding agent has the context right there at the fingertips, while for business its gated behind a bunch of services and custom data models. Context is king :)How do you handle encryption and confidentiality? Im building in this space too (MCP gateway https://www.gatana.ai/) which already have semantic search for tool outputs, and ensuring encryption and confidentiality is not trivial.ck_one 38 days ago | prev | next [–] More and more SaaS companies like ServiceNow or Hubspot are creating new tollgates for agent api calls. How do you think will this impact Airbyte Agents? I guess that replicating data locally will be harder since the platforms will try to protect it or charge for it.aaronsteers 37 days ago | parent | next [–] It's a good question and I won't pretend to predict the future on this one. I will say, I think Airbyte Agents is in a good position because our core Data Replication product has always had to mitigate the impacts of rate limiting and cumbersome upstream APIs. The new Agents toolset gives you the ability to query the upstream APIs directly (read: as a passthrough) while also letting you bypass them entirely when your agent can answer its question via the Context Store directly. Time and feedback from our users will confirm, but I do think this gives our customers a good balance of control - when to query upstream directly and when to utilize the Context Store to work around API limitations - whether inherent or artificially enforced by the vendor.mtricot 39 days ago | prev | next [–] Just want to call out a couple of nuances in our methodology. In general, we tried our best to do apples-to-apples comparisons where we could, and gave ourselves a discount where we couldn’t. Unsurprisingly, it’s a challenge to find MCPs for various vendors (which is another reason we are trying to solve this). Here’s a video walkthrough of the benchmark harness:https://www.loom.com/share/9d96c8c64c1a4b7fad0356774fc54accWhere the comparison wasn't valid or not apples-to-apples:Gong and Zendesk: no official native MCP exists, so we used the most popular community implementations we could find. We were only able to benchmark Gong Search as the Gong MCP does not have a Get tool call.While our Search testing yielded the same number of records on either path, vendor-specific search implementations means results aren’t identical. Contents are similar in general, so the ratios remain directionally correct.The general test set:2 scenarios (Retrieval and Search) across 4 connectors isn’t a huge test set. While we hope to extend this over time, we’ve made the harness public so anyone can contribute in the meantime. Let us know if you find any MCP with better results!Where the vendor MCP wins or ties:Salesforce showed the smallest win at 16%. This is primarily because Salesforce, unlike many vendors, uniquely provides great search support out of the box with their SOQL.We see identical records for Get. As noted, Search returns different sets of identical counts. Airbyte uses fewer tokens because the Salesforce records contain mandatory metadata (type and url).Where the vendor MCP is costly to context:Zendesk is a great example of this. The extreme gap is because the Zendesk MCP (reminder - a community alternative) returns the entire API response in search results. This averages to 9KB per record against our production Zendesk account!Airbyte’s implementation provides filtering, which allows agents to retrieve the minimal data needed to achieve the outcome, explaining the drastic gap.ecares 39 days ago | prev | next [–] Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?aaronsteers 39 days ago | parent | next [–] AJ here, from Airbyte.Yes, we've definitely found that some API data models are easier for models to navigate than others.The largest factors of Agent inefficiency we've identified so far are: 1. Many APIs lack robust-enough search, forcing agents to page through hundreds or thousands of paginated responses until they find the record they are looking for (our Context Store addresses this). 2. Many APIs have HUGE response sets. Our MCP helps handle this by letting the agent decide exactly what fields they can return. 3. With our SDK, you can literally build your own MCP on top of any source we support (50+ right now and will grow). This is super powerful, and allows you to build more ergonomic MCP servers and tools - even if the models themselves are not intuitive or easy for the LLM to leverage directly.Combining all three of these together, we see the vast majority of challenges can be addressed via a strong system prompt for guidance. Fine tuning could get you further but anyway, you'd still want your fine tuned model to build on this same foundation, since the efficiences will transfer across use cases and models.@ecares - Does this answer your question? What do you think?woeirua 39 days ago | root | parent | next [–] Your point about search being a bottleneck is spot on. IMO, search APIs should return guidance to agents to help them winnow down the results faster. For example, if your query returns 1000 results, then it should tell the agent, "too many results, we recommend you filter on column X because of Y to improve your search. Here are the possible values in column X: ..."carefulfungi 39 days ago | root | parent | next [–] There are a lot of APIs like this that I really wish would expose downloading a parquet file instead of trying to implement server-side filtering and reporting query features.aaronsteers 39 days ago | root | parent | next [–] +1Working with APIs is often frustrating and the worst ones are terribly ineficient and frustrating. Our Agent SDK and Agent Context Store insulates you and your agent from this headache, allowing you to query from those synced datasets directly.The feedback about wanting to download a parquet file is super interesting...aaronsteers 39 days ago | root | parent | prev | next [–] Glad to hear this resonates with you also. We're aiming to give agents more control over their context, and easier access paths regardless of the source system.ritonlajoie 39 days ago | prev | next [–] Hi Michel, congrats and I have nice memories of working with you in lafayette street !! Keep up the good work on airbyte ! :)mtricot 39 days ago | parent | next [–] Great to see you here!xcf_seetan 39 days ago | prev | next [–] Shameless plug: I have written a paper about using the MCP server architecture to enable agents to overcome the knowledge cutoff, to work with software released after the training stop.[https://zenodo.org/records/19925469]zitoshi 31 days ago | prev | next [–] interesting, i've been building in the "connector space" - but more for personal life context vs business.are you focused on just connecting the data? or the specific cross understanding between the different types of information?afxuh 38 days ago | prev | next [–] Congrats, you built an ETL pipeline and called it an agent. The industry has come full circle.davinchia 38 days ago | parent | next [–] Haha indeed!On a more serious note, just as swyx mentioned in a comment further up, we do believe a lot of the challenges of reliably operationalising agents boil down to data. All of which is non-obvious to AI engineers (besides Frontier Labs gathering/generating data for model training).What the right shape is - we are all figuring it out. Happy to trade notes.smadam9 38 days ago | prev | next [–] What are the main differences to Glean? My company is evaluating Glean and I feel like Airbyte is a strong alternative (at least for some use cases).How does Airbyte handle data authorization?pjm331 39 days ago | prev | next [–] sounds very familiar to what I ended up doing on my internal system - especially anything to do with search - much better to just sync everything to a DB and give the agent access to the DBaaronsteers 39 days ago | parent | next [–] That's great to hear - great minds think alike!> give the agent access to the DBThis is where Airbyte really can shine, I think, and the total can be more the sum of the parts. Because Airbyte excels at data replication already, we can populate your the Agent Context Store without users or agents ever needing to think about the words "ELT" or "ETL".We're listening carefully to feedback so we hope you will give it a try and let us know how it goes! Thanks!pjm331 38 days ago | root | parent | next [–] yeah this is one of the few AI-related products that I have seen that make sense to mebut i also wonder to what extent this needs to be its own thing or if this is just something that it looks like we need but really people just need to shovel more stuff into their data warehouse / data lake that you never had reason to before, because now that's all fodder for agentic searchaaronsteers 37 days ago | root | parent | next [–] Great point. Many of Airbyte's customers are doing just that - adding new sources to their warehouses - like Google Drive, Gong, and a ton of sources that weren't as interesting previously for data analytics. But this creates a ton of work for the data engineering teams - to not only load all that extra data, but to deal with rate limits and then to conform the schemas into a usable format after loading.For now, I think its 100% appropriate to think of the Context Store complementing the Warehouse and not replacing it per se. We're evaluating future integration options between the new Context Store and the traditional data warehouse, but nothing we have publicly announced as of now. I think both approaches have their strengths and killer use cases.Tsarp 39 days ago | prev | next [–] Doesn't Skills solve all of this?OpenClaw, Hermes and other agents have already made skill adoption mainstream?Are you guys still seeing a future where people are dumping entire MCP tool defs into context?aaronsteers 39 days ago | parent | next [–] Great question, @Tsarp - Skill and tools work great together. What we've found is that agents generally need both to achieve great results. We're actually not trying to replace skills, but to give them new super powers.Are there any examples you've run into where skills were missing tools (or data) that they needed for a specific task?Tsarp 38 days ago | root | parent | next [–] Hmm, hoping this isn't a generic LLM generated response.Skills have the scripts folder and you can precisely describe when and when not to use a script. This can end up directly wrapping API(s), CLIs, generic scripts or even other MCP servers.CC and codex both have the skill creator and you can have them build the skill for you.Havent run into any scenarios where skills were missing tools. 1-2 iterations and its usually taken care off quite quickly.aaronsteers 38 days ago | root | parent | next [–] Hey, fair enough. (100% human here, btw.) I think I misread your original question to be asking "why do we need a service (whether accessed via API/SDK/MCP/etc.)" vs just having skills (markdown + scripts)".If you are already leveraging skills as scripts and APIs in your skills, then you understand the distinction. I'll attempt to re-answer your question with now hopefully a better understanding:I think Airbyte Agents helps your agent by giving access to data across any and all of the systems it may need to get data from, or write data to. While you could hit the service APIs directly (via REST/CLI/etc.), in practice we find that not all use cases are amenable to this. Airbyte Agents does have REST APIs as well as SDKs and of course the MCP interface - so it's not really about MCP tools specifically, more about how you can access the data. The Airbyte Agents interface also reduces the number of creds that the agent needs to handle, giving a single portal (with logging and audit capabilities) for all the actions your agent is taking.Sorry for the red herring of skills-v-tools. Let me know if you have any additional questions!SachitRafa 35 days ago | prev | next [–] How would you know if the agent is not querying stale data here ?tomrod 39 days ago | prev | next [–] What actions does agents enable that weren't already available from Airbyte?aaronsteers 39 days ago | parent | next [–] The new Airbyte Agents offering brings a ton of new capabilities actually.1. Programmatic Interfaces: Including a new REST API, SDK, and MCP Server. 2. New action verbs: Not just replication anymore. We have get/set/list/update/upload, and more! 3. New credentials passthrough: For all the above, you OAuth to Airbyte and we OAuth on your behalf to the systems your agent needs. No need to provide your agents dozens of different secrets in order to access the systems it needs. 4. Context Store. Like your agents' own data warehouse, but completely automatic and hands-free. For those use cases that just aren't possible when calling the REST API directly.Again - thanks for your comment and sorry for the longwinded response. More info here: https://docs.airbyte.com/ai-agents/ Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Hacker Newsnew | past | comments | ask | show | jobs | submitloginShow HN: Airbyte Agents – context for agents across multiple data sources156 points by mtricot 39 days ago | hide | past | favorite | 48 commentsI’m Michel, co-founder and CEO of Airbyte (https://airbyte.com/). We’ve spent the last six years building data connectors. Today we're launching Airbyte Agents (https://docs.airbyte.com/ai-agents/), a unified data layer for agents to discover information and take action across operational systems.Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fgAs agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building! swyx 39 days ago | next [–] (former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!)hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K...i think my most sad/interesting observation about ai engineers is that many ai apps are super data hungry, but many dont have the necessary data engineering background to even know they need an airbyte or what tradeoffs to make in an etl pipeline. would love a "data engineering for ai engineers" type braindump session from someone from airbyte at AIE (https://ai.engineer/cfp )aaronsteers 39 days ago | parent | next [–] Hey, swyx! Great seeing you here.> airbyte agents could serve as a form of MCP gatewayExactly! And a single set of tools for agents to access both realtime (direct reads/writes) as well as cached (Context Store), bringing hopefully the best access path for each different use case.> would love a "data engineering for ai engineers" type braindump ... at AIEGreat idea - we have a booth at AIE, and we'll submit there for a talk. Mario will reach out to you about this. :)jeanlaf 39 days ago | parent | prev | next [–] Thanks swyx! We'd love to do that session "data engineering for ai engineers", will make you an intro to the right person in the team.swyx 38 days ago | root | parent | next [–] saw your email, will get back!sails 38 days ago | parent | prev | next [–] I think this is right ( a big gap ) but I don’t think data companies even now what the right shape is for AI.It’s definitely not old school ETL + dbt + BI tool, it might be something like this, but it’s very earlyaaronsteers 37 days ago | root | parent | next [–] Agreed 100% - we're still super early in this journey, gathering data from our own usage and from our customers' feedback.slurpyb 39 days ago | prev | next [–] Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human.It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience isdavinchia 38 days ago | parent | next [–] Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!mtricot 39 days ago | parent | prev | next [–] Let me see what's up and fix that!jscheel 39 days ago | prev | next [–] I feel like we've been working in parallel here :) We are using PyAirbyte (hi aaronsteers) for our users to connect their data sources to our agents. We originally wanted to use the airbyte white-label platform, but the team said that it was being deprecated. I think this really drives home just how crucial it is to have a clear model for accessing your data, and Airbyte has been great at that for quite a while.aaronsteers 39 days ago | parent | next [–] Hello, Jared! Small world! Yes, we did deprecate our old PbA (Powered by Airbyte) offering, but in many ways our new Agents and Embedded offering is a more robust and agent-friendly successor to that older offering.I am happy to hear you are still getting value out of PyAirbyte! If you do try out Airbyte Agents, please let us know how it goes! We are always listening to feedback and would love to hear from you as you explore the new tools and capabilities.jessewmc 39 days ago | prev | next [–] Looks interesting!If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?Have you done any testing on guided indexing, or metadata layers on top of the data? My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time. I.e. _what_ you index, and how you signpost for agents, becomes really important (unless your data is super clean I guess). This does look like a good foundation for that kind of tooling though!aaronsteers 39 days ago | parent | next [–] Hi, @jessewmc. Thanks for your reply. Regarding your points:> If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?While we haven't yet published details on the backend implementation, I can say that our implementation performs very well without needing to prioritize specific fields for indexing. We aim for large text fields to perform decently and retrieval based on small/compressible fields like ints to be fast. (More to come on this in the coming months.)> Have you done any testing on guided indexing, or metadata layers on top of the data?We've been testing with different data scales and shapes. Nothing detailed to share yet, but performance has (so far) never itself become the bottleneck in our agent testing. (The LLM thinking itself is often the bottleneck.)> My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time.Airbyte has rich metadata on our upstream connector's data models, which I think helps us a lot to deliver helpful context to the agent. Another option, when optimizing for specific use cases, is to build your own agent tools on top of our Agent SDK. This allows you to make the calls organic and build the tools in a way that makes natural sense to the agent, regardless of source shape or which system(s) that data is coming from.> This does look like a good foundation for that kind of tooling though!We agree! Thanks again for sharing your thoughts here.juancs 36 days ago | root | parent | next [–] Great launch btw! I have some questions if you don't mindyou mentioned that performance was never an issue, I am really intrigued how this is achieved.I have 3 General questions:1. How big (estimate in bytes) and complex were the test datasources? I couldn't find this in the benchmark repo.2. how is the business context managed? In the blog "Airbyte Agents: A New Era for Airbyte" it was mentioned handling the business context but in the context layer docs it only talks about schema discovery (I got a bit confused)3. When you said performance was never an issue, do you mean the user always got the answer it was looking for?dennispi 38 days ago | prev | next [–] We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compareWe likewise measured token savings, (wall clock) time, # tool calls, and # turns.nerdright 39 days ago | prev | next [–] This is such a great direction airbyte is taking and congrats to the lunch! I think you're very well-positioned for this opportunity than most people realize, given your reputable brand and your uncanny expertise in etl. It's honestly a natural progression of airbyte as far as the current AI landscape goes. Kudos to you and the team!(We use airbyte at my company, although we self-host it.)aaronsteers 39 days ago | parent | next [–] Thanks! Really appreciate the kind words. Looking forward to seeing what our amazing community builds with these new tools.andai 38 days ago | prev | next [–] The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query?(I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)sho 38 days ago | parent | next [–] That's actually the approach we took with https://gentility.ai/ - we either provide almost-raw SQL query access to the DBs themselves or we synthesize from API into DuckDB via parquet and make that available to the agent to just directly query. It works well - my philosophy is to give agents the sharpest tools you can, and SQL is the best tool there is.I understand the instinct to try to make a proprietary moat around it all but I think the pattern is useful and obvious enough that all big orgs will be doing something very similar within 5 years or so.aaronsteers 38 days ago | parent | prev | next [–] Helpful feedback, thank you! And your instincts are spot on. As of now, we have API based search, with filter predicates and field selection in JSON. While we haven't published anything on the backend implementation, I can say it does use a cloud-native storage medium where the filters are indeed pushed down as SQL. We want to be careful about if/when we offer direct SQL access, specifically because SQL dialects can differ drastically and we wouldn't want to break consumers if/when we change which dialect(s) are supported.That said, please stay tuned - and thank you again for this valuable feedback.thecopy 38 days ago | prev | next [–] Super interesting idea! Congrats on the launch. Context is definitely something that is lacking in my experience. Im always frustrated when an agent cannot answer business-related questions, and i compare them to coding agents which seem to be able to answer everything. The difference is that coding agent has the context right there at the fingertips, while for business its gated behind a bunch of services and custom data models. Context is king :)How do you handle encryption and confidentiality? Im building in this space too (MCP gateway https://www.gatana.ai/) which already have semantic search for tool outputs, and ensuring encryption and confidentiality is not trivial.ck_one 38 days ago | prev | next [–] More and more SaaS companies like ServiceNow or Hubspot are creating new tollgates for agent api calls. How do you think will this impact Airbyte Agents? I guess that replicating data locally will be harder since the platforms will try to protect it or charge for it.aaronsteers 37 days ago | parent | next [–] It's a good question and I won't pretend to predict the future on this one. I will say, I think Airbyte Agents is in a good position because our core Data Replication product has always had to mitigate the impacts of rate limiting and cumbersome upstream APIs. The new Agents toolset gives you the ability to query the upstream APIs directly (read: as a passthrough) while also letting you bypass them entirely when your agent can answer its question via the Context Store directly. Time and feedback from our users will confirm, but I do think this gives our customers a good balance of control - when to query upstream directly and when to utilize the Context Store to work around API limitations - whether inherent or artificially enforced by the vendor.mtricot 39 days ago | prev | next [–] Just want to call out a couple of nuances in our methodology. In general, we tried our best to do apples-to-apples comparisons where we could, and gave ourselves a discount where we couldn’t. Unsurprisingly, it’s a challenge to find MCPs for various vendors (which is another reason we are trying to solve this). Here’s a video walkthrough of the benchmark harness:https://www.loom.com/share/9d96c8c64c1a4b7fad0356774fc54accWhere the comparison wasn't valid or not apples-to-apples:Gong and Zendesk: no official native MCP exists, so we used the most popular community implementations we could find. We were only able to benchmark Gong Search as the Gong MCP does not have a Get tool call.While our Search testing yielded the same number of records on either path, vendor-specific search implementations means results aren’t identical. Contents are similar in general, so the ratios remain directionally correct.The general test set:2 scenarios (Retrieval and Search) across 4 connectors isn’t a huge test set. While we hope to extend this over time, we’ve made the harness public so anyone can contribute in the meantime. Let us know if you find any MCP with better results!Where the vendor MCP wins or ties:Salesforce showed the smallest win at 16%. This is primarily because Salesforce, unlike many vendors, uniquely provides great search support out of the box with their SOQL.We see identical records for Get. As noted, Search returns different sets of identical counts. Airbyte uses fewer tokens because the Salesforce records contain mandatory metadata (type and url).Where the vendor MCP is costly to context:Zendesk is a great example of this. The extreme gap is because the Zendesk MCP (reminder - a community alternative) returns the entire API response in search results. This averages to 9KB per record against our production Zendesk account!Airbyte’s implementation provides filtering, which allows agents to retrieve the minimal data needed to achieve the outcome, explaining the drastic gap.ecares 39 days ago | prev | next [–] Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?aaronsteers 39 days ago | parent | next [–] AJ here, from Airbyte.Yes, we've definitely found that some API data models are easier for models to navigate than others.The largest factors of Agent inefficiency we've identified so far are: 1. Many APIs lack robust-enough search, forcing agents to page through hundreds or thousands of paginated responses until they find the record they are looking for (our Context Store addresses this). 2. Many APIs have HUGE response sets. Our MCP helps handle this by letting the agent decide exactly what fields they can return. 3. With our SDK, you can literally build your own MCP on top of any source we support (50+ right now and will grow). This is super powerful, and allows you to build more ergonomic MCP servers and tools - even if the models themselves are not intuitive or easy for the LLM to leverage directly.Combining all three of these together, we see the vast majority of challenges can be addressed via a strong system prompt for guidance. Fine tuning could get you further but anyway, you'd still want your fine tuned model to build on this same foundation, since the efficiences will transfer across use cases and models.@ecares - Does this answer your question? What do you think?woeirua 39 days ago | root | parent | next [–] Your point about search being a bottleneck is spot on. IMO, search APIs should return guidance to agents to help them winnow down the results faster. For example, if your query returns 1000 results, then it should tell the agent, "too many results, we recommend you filter on column X because of Y to improve your search. Here are the possible values in column X: ..."carefulfungi 39 days ago | root | parent | next [–] There are a lot of APIs like this that I really wish would expose downloading a parquet file instead of trying to implement server-side filtering and reporting query features.aaronsteers 39 days ago | root | parent | next [–] +1Working with APIs is often frustrating and the worst ones are terribly ineficient and frustrating. Our Agent SDK and Agent Context Store insulates you and your agent from this headache, allowing you to query from those synced datasets directly.The feedback about wanting to download a parquet file is super interesting...aaronsteers 39 days ago | root | parent | prev | next [–] Glad to hear this resonates with you also. We're aiming to give agents more control over their context, and easier access paths regardless of the source system.ritonlajoie 39 days ago | prev | next [–] Hi Michel, congrats and I have nice memories of working with you in lafayette street !! Keep up the good work on airbyte ! :)mtricot 39 days ago | parent | next [–] Great to see you here!xcf_seetan 39 days ago | prev | next [–] Shameless plug: I have written a paper about using the MCP server architecture to enable agents to overcome the knowledge cutoff, to work with software released after the training stop.[https://zenodo.org/records/19925469]zitoshi 31 days ago | prev | next [–] interesting, i've been building in the "connector space" - but more for personal life context vs business.are you focused on just connecting the data? or the specific cross understanding between the different types of information?afxuh 38 days ago | prev | next [–] Congrats, you built an ETL pipeline and called it an agent. The industry has come full circle.davinchia 38 days ago | parent | next [–] Haha indeed!On a more serious note, just as swyx mentioned in a comment further up, we do believe a lot of the challenges of reliably operationalising agents boil down to data. All of which is non-obvious to AI engineers (besides Frontier Labs gathering/generating data for model training).What the right shape is - we are all figuring it out. Happy to trade notes.smadam9 38 days ago | prev | next [–] What are the main differences to Glean? My company is evaluating Glean and I feel like Airbyte is a strong alternative (at least for some use cases).How does Airbyte handle data authorization?pjm331 39 days ago | prev | next [–] sounds very familiar to what I ended up doing on my internal system - especially anything to do with search - much better to just sync everything to a DB and give the agent access to the DBaaronsteers 39 days ago | parent | next [–] That's great to hear - great minds think alike!> give the agent access to the DBThis is where Airbyte really can shine, I think, and the total can be more the sum of the parts. Because Airbyte excels at data replication already, we can populate your the Agent Context Store without users or agents ever needing to think about the words "ELT" or "ETL".We're listening carefully to feedback so we hope you will give it a try and let us know how it goes! Thanks!pjm331 38 days ago | root | parent | next [–] yeah this is one of the few AI-related products that I have seen that make sense to mebut i also wonder to what extent this needs to be its own thing or if this is just something that it looks like we need but really people just need to shovel more stuff into their data warehouse / data lake that you never had reason to before, because now that's all fodder for agentic searchaaronsteers 37 days ago | root | parent | next [–] Great point. Many of Airbyte's customers are doing just that - adding new sources to their warehouses - like Google Drive, Gong, and a ton of sources that weren't as interesting previously for data analytics. But this creates a ton of work for the data engineering teams - to not only load all that extra data, but to deal with rate limits and then to conform the schemas into a usable format after loading.For now, I think its 100% appropriate to think of the Context Store complementing the Warehouse and not replacing it per se. We're evaluating future integration options between the new Context Store and the traditional data warehouse, but nothing we have publicly announced as of now. I think both approaches have their strengths and killer use cases.Tsarp 39 days ago | prev | next [–] Doesn't Skills solve all of this?OpenClaw, Hermes and other agents have already made skill adoption mainstream?Are you guys still seeing a future where people are dumping entire MCP tool defs into context?aaronsteers 39 days ago | parent | next [–] Great question, @Tsarp - Skill and tools work great together. What we've found is that agents generally need both to achieve great results. We're actually not trying to replace skills, but to give them new super powers.Are there any examples you've run into where skills were missing tools (or data) that they needed for a specific task?Tsarp 38 days ago | root | parent | next [–] Hmm, hoping this isn't a generic LLM generated response.Skills have the scripts folder and you can precisely describe when and when not to use a script. This can end up directly wrapping API(s), CLIs, generic scripts or even other MCP servers.CC and codex both have the skill creator and you can have them build the skill for you.Havent run into any scenarios where skills were missing tools. 1-2 iterations and its usually taken care off quite quickly.aaronsteers 38 days ago | root | parent | next [–] Hey, fair enough. (100% human here, btw.) I think I misread your original question to be asking "why do we need a service (whether accessed via API/SDK/MCP/etc.)" vs just having skills (markdown + scripts)".If you are already leveraging skills as scripts and APIs in your skills, then you understand the distinction. I'll attempt to re-answer your question with now hopefully a better understanding:I think Airbyte Agents helps your agent by giving access to data across any and all of the systems it may need to get data from, or write data to. While you could hit the service APIs directly (via REST/CLI/etc.), in practice we find that not all use cases are amenable to this. Airbyte Agents does have REST APIs as well as SDKs and of course the MCP interface - so it's not really about MCP tools specifically, more about how you can access the data. The Airbyte Agents interface also reduces the number of creds that the agent needs to handle, giving a single portal (with logging and audit capabilities) for all the actions your agent is taking.Sorry for the red herring of skills-v-tools. Let me know if you have any additional questions!SachitRafa 35 days ago | prev | next [–] How would you know if the agent is not querying stale data here ?tomrod 39 days ago | prev | next [–] What actions does agents enable that weren't already available from Airbyte?aaronsteers 39 days ago | parent | next [–] The new Airbyte Agents offering brings a ton of new capabilities actually.1. Programmatic Interfaces: Including a new REST API, SDK, and MCP Server. 2. New action verbs: Not just replication anymore. We have get/set/list/update/upload, and more! 3. New credentials passthrough: For all the above, you OAuth to Airbyte and we OAuth on your behalf to the systems your agent needs. No need to provide your agents dozens of different secrets in order to access the systems it needs. 4. Context Store. Like your agents' own data warehouse, but completely automatic and hands-free. For those use cases that just aren't possible when calling the REST API directly.Again - thanks for your comment and sorry for the longwinded response. More info here: https://docs.airbyte.com/ai-agents/ Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact
Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Airbyte Agents – context for agents across multiple data sources156 points by mtricot 39 days ago | hide | past | favorite | 48 commentsI’m Michel, co-founder and CEO of Airbyte (https://airbyte.com/). We’ve spent the last six years building data connectors. Today we're launching Airbyte Agents (https://docs.airbyte.com/ai-agents/), a unified data layer for agents to discover information and take action across operational systems.Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fgAs agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building!
Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fg
As agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.
Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.
An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.
So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!
This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.
What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.
That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”
Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.
Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.
For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.
Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).
Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!
It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?
Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building!
swyx 39 days ago | next [–] (former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!)hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K...i think my most sad/interesting observation about ai engineers is that many ai apps are super data hungry, but many dont have the necessary data engineering background to even know they need an airbyte or what tradeoffs to make in an etl pipeline. would love a "data engineering for ai engineers" type braindump session from someone from airbyte at AIE (https://ai.engineer/cfp )aaronsteers 39 days ago | parent | next [–] Hey, swyx! Great seeing you here.> airbyte agents could serve as a form of MCP gatewayExactly! And a single set of tools for agents to access both realtime (direct reads/writes) as well as cached (Context Store), bringing hopefully the best access path for each different use case.> would love a "data engineering for ai engineers" type braindump ... at AIEGreat idea - we have a booth at AIE, and we'll submit there for a talk. Mario will reach out to you about this. :)jeanlaf 39 days ago | parent | prev | next [–] Thanks swyx! We'd love to do that session "data engineering for ai engineers", will make you an intro to the right person in the team.swyx 38 days ago | root | parent | next [–] saw your email, will get back!sails 38 days ago | parent | prev | next [–] I think this is right ( a big gap ) but I don’t think data companies even now what the right shape is for AI.It’s definitely not old school ETL + dbt + BI tool, it might be something like this, but it’s very earlyaaronsteers 37 days ago | root | parent | next [–] Agreed 100% - we're still super early in this journey, gathering data from our own usage and from our customers' feedback.slurpyb 39 days ago | prev | next [–] Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human.It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience isdavinchia 38 days ago | parent | next [–] Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!mtricot 39 days ago | parent | prev | next [–] Let me see what's up and fix that!jscheel 39 days ago | prev | next [–] I feel like we've been working in parallel here :) We are using PyAirbyte (hi aaronsteers) for our users to connect their data sources to our agents. We originally wanted to use the airbyte white-label platform, but the team said that it was being deprecated. I think this really drives home just how crucial it is to have a clear model for accessing your data, and Airbyte has been great at that for quite a while.aaronsteers 39 days ago | parent | next [–] Hello, Jared! Small world! Yes, we did deprecate our old PbA (Powered by Airbyte) offering, but in many ways our new Agents and Embedded offering is a more robust and agent-friendly successor to that older offering.I am happy to hear you are still getting value out of PyAirbyte! If you do try out Airbyte Agents, please let us know how it goes! We are always listening to feedback and would love to hear from you as you explore the new tools and capabilities.jessewmc 39 days ago | prev | next [–] Looks interesting!If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?Have you done any testing on guided indexing, or metadata layers on top of the data? My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time. I.e. _what_ you index, and how you signpost for agents, becomes really important (unless your data is super clean I guess). This does look like a good foundation for that kind of tooling though!aaronsteers 39 days ago | parent | next [–] Hi, @jessewmc. Thanks for your reply. Regarding your points:> If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?While we haven't yet published details on the backend implementation, I can say that our implementation performs very well without needing to prioritize specific fields for indexing. We aim for large text fields to perform decently and retrieval based on small/compressible fields like ints to be fast. (More to come on this in the coming months.)> Have you done any testing on guided indexing, or metadata layers on top of the data?We've been testing with different data scales and shapes. Nothing detailed to share yet, but performance has (so far) never itself become the bottleneck in our agent testing. (The LLM thinking itself is often the bottleneck.)> My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time.Airbyte has rich metadata on our upstream connector's data models, which I think helps us a lot to deliver helpful context to the agent. Another option, when optimizing for specific use cases, is to build your own agent tools on top of our Agent SDK. This allows you to make the calls organic and build the tools in a way that makes natural sense to the agent, regardless of source shape or which system(s) that data is coming from.> This does look like a good foundation for that kind of tooling though!We agree! Thanks again for sharing your thoughts here.juancs 36 days ago | root | parent | next [–] Great launch btw! I have some questions if you don't mindyou mentioned that performance was never an issue, I am really intrigued how this is achieved.I have 3 General questions:1. How big (estimate in bytes) and complex were the test datasources? I couldn't find this in the benchmark repo.2. how is the business context managed? In the blog "Airbyte Agents: A New Era for Airbyte" it was mentioned handling the business context but in the context layer docs it only talks about schema discovery (I got a bit confused)3. When you said performance was never an issue, do you mean the user always got the answer it was looking for?dennispi 38 days ago | prev | next [–] We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compareWe likewise measured token savings, (wall clock) time, # tool calls, and # turns.nerdright 39 days ago | prev | next [–] This is such a great direction airbyte is taking and congrats to the lunch! I think you're very well-positioned for this opportunity than most people realize, given your reputable brand and your uncanny expertise in etl. It's honestly a natural progression of airbyte as far as the current AI landscape goes. Kudos to you and the team!(We use airbyte at my company, although we self-host it.)aaronsteers 39 days ago | parent | next [–] Thanks! Really appreciate the kind words. Looking forward to seeing what our amazing community builds with these new tools.andai 38 days ago | prev | next [–] The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query?(I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)sho 38 days ago | parent | next [–] That's actually the approach we took with https://gentility.ai/ - we either provide almost-raw SQL query access to the DBs themselves or we synthesize from API into DuckDB via parquet and make that available to the agent to just directly query. It works well - my philosophy is to give agents the sharpest tools you can, and SQL is the best tool there is.I understand the instinct to try to make a proprietary moat around it all but I think the pattern is useful and obvious enough that all big orgs will be doing something very similar within 5 years or so.aaronsteers 38 days ago | parent | prev | next [–] Helpful feedback, thank you! And your instincts are spot on. As of now, we have API based search, with filter predicates and field selection in JSON. While we haven't published anything on the backend implementation, I can say it does use a cloud-native storage medium where the filters are indeed pushed down as SQL. We want to be careful about if/when we offer direct SQL access, specifically because SQL dialects can differ drastically and we wouldn't want to break consumers if/when we change which dialect(s) are supported.That said, please stay tuned - and thank you again for this valuable feedback.thecopy 38 days ago | prev | next [–] Super interesting idea! Congrats on the launch. Context is definitely something that is lacking in my experience. Im always frustrated when an agent cannot answer business-related questions, and i compare them to coding agents which seem to be able to answer everything. The difference is that coding agent has the context right there at the fingertips, while for business its gated behind a bunch of services and custom data models. Context is king :)How do you handle encryption and confidentiality? Im building in this space too (MCP gateway https://www.gatana.ai/) which already have semantic search for tool outputs, and ensuring encryption and confidentiality is not trivial.ck_one 38 days ago | prev | next [–] More and more SaaS companies like ServiceNow or Hubspot are creating new tollgates for agent api calls. How do you think will this impact Airbyte Agents? I guess that replicating data locally will be harder since the platforms will try to protect it or charge for it.aaronsteers 37 days ago | parent | next [–] It's a good question and I won't pretend to predict the future on this one. I will say, I think Airbyte Agents is in a good position because our core Data Replication product has always had to mitigate the impacts of rate limiting and cumbersome upstream APIs. The new Agents toolset gives you the ability to query the upstream APIs directly (read: as a passthrough) while also letting you bypass them entirely when your agent can answer its question via the Context Store directly. Time and feedback from our users will confirm, but I do think this gives our customers a good balance of control - when to query upstream directly and when to utilize the Context Store to work around API limitations - whether inherent or artificially enforced by the vendor.mtricot 39 days ago | prev | next [–] Just want to call out a couple of nuances in our methodology. In general, we tried our best to do apples-to-apples comparisons where we could, and gave ourselves a discount where we couldn’t. Unsurprisingly, it’s a challenge to find MCPs for various vendors (which is another reason we are trying to solve this). Here’s a video walkthrough of the benchmark harness:https://www.loom.com/share/9d96c8c64c1a4b7fad0356774fc54accWhere the comparison wasn't valid or not apples-to-apples:Gong and Zendesk: no official native MCP exists, so we used the most popular community implementations we could find. We were only able to benchmark Gong Search as the Gong MCP does not have a Get tool call.While our Search testing yielded the same number of records on either path, vendor-specific search implementations means results aren’t identical. Contents are similar in general, so the ratios remain directionally correct.The general test set:2 scenarios (Retrieval and Search) across 4 connectors isn’t a huge test set. While we hope to extend this over time, we’ve made the harness public so anyone can contribute in the meantime. Let us know if you find any MCP with better results!Where the vendor MCP wins or ties:Salesforce showed the smallest win at 16%. This is primarily because Salesforce, unlike many vendors, uniquely provides great search support out of the box with their SOQL.We see identical records for Get. As noted, Search returns different sets of identical counts. Airbyte uses fewer tokens because the Salesforce records contain mandatory metadata (type and url).Where the vendor MCP is costly to context:Zendesk is a great example of this. The extreme gap is because the Zendesk MCP (reminder - a community alternative) returns the entire API response in search results. This averages to 9KB per record against our production Zendesk account!Airbyte’s implementation provides filtering, which allows agents to retrieve the minimal data needed to achieve the outcome, explaining the drastic gap.ecares 39 days ago | prev | next [–] Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?aaronsteers 39 days ago | parent | next [–] AJ here, from Airbyte.Yes, we've definitely found that some API data models are easier for models to navigate than others.The largest factors of Agent inefficiency we've identified so far are: 1. Many APIs lack robust-enough search, forcing agents to page through hundreds or thousands of paginated responses until they find the record they are looking for (our Context Store addresses this). 2. Many APIs have HUGE response sets. Our MCP helps handle this by letting the agent decide exactly what fields they can return. 3. With our SDK, you can literally build your own MCP on top of any source we support (50+ right now and will grow). This is super powerful, and allows you to build more ergonomic MCP servers and tools - even if the models themselves are not intuitive or easy for the LLM to leverage directly.Combining all three of these together, we see the vast majority of challenges can be addressed via a strong system prompt for guidance. Fine tuning could get you further but anyway, you'd still want your fine tuned model to build on this same foundation, since the efficiences will transfer across use cases and models.@ecares - Does this answer your question? What do you think?woeirua 39 days ago | root | parent | next [–] Your point about search being a bottleneck is spot on. IMO, search APIs should return guidance to agents to help them winnow down the results faster. For example, if your query returns 1000 results, then it should tell the agent, "too many results, we recommend you filter on column X because of Y to improve your search. Here are the possible values in column X: ..."carefulfungi 39 days ago | root | parent | next [–] There are a lot of APIs like this that I really wish would expose downloading a parquet file instead of trying to implement server-side filtering and reporting query features.aaronsteers 39 days ago | root | parent | next [–] +1Working with APIs is often frustrating and the worst ones are terribly ineficient and frustrating. Our Agent SDK and Agent Context Store insulates you and your agent from this headache, allowing you to query from those synced datasets directly.The feedback about wanting to download a parquet file is super interesting...aaronsteers 39 days ago | root | parent | prev | next [–] Glad to hear this resonates with you also. We're aiming to give agents more control over their context, and easier access paths regardless of the source system.ritonlajoie 39 days ago | prev | next [–] Hi Michel, congrats and I ha
正文较长，站内已展示前半部分，完整内容请阅读原文。
Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Show HN: Airbyte Agents – context for agents across multiple data sources156 points by mtricot 39 days ago | hide | past | favorite | 48 commentsI’m Michel, co-founder and CEO of Airbyte (https://airbyte.com/). We’ve spent the last six years building data connectors. Today we're launching Airbyte Agents (https://docs.airbyte.com/ai-agents/), a unified data layer for agents to discover information and take action across operational systems.Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fgAs agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building!
Here’s a quick walkthrough: https://www.youtube.com/watch?v=ZosDytyf1fg
As agents move into real workflows, they need access to more tools (e.g. Slack, Salesforce, Linear). That means a ton of API plumbing: authentication, pagination, filters, handling schema, and matching entities across systems.
Most MCPs don’t fix this. They’re thin wrappers over APIs, so agents inherit their weak primitives and still get it wrong most of the time, especially when working across tools.
An even deeper issue is that APIs assume you already know what to query (think endpoints, Object IDs, fields), whereas agents usually start one step earlier: they need first to discover what matters before they can even start reasoning.
So we built Airbyte Agents to be a context layer between your Agents and all of your data. The core of this is something we call Context Store: a data index optimized for agentic search, populated by our replication connectors. All that work on data connectors the last six years comes in handy here!
This gives agents a structured way to discover data, while still allowing them to read and write directly to the upstream system when needed.
What got us working on this was an insane trace from an agent we were migrating to our new SDK. It was supposed to answer "which customers are at risk of leaving this quarter?" The trace had 47 steps. Most were API calls. The agent first had to find a bunch of accounts, then map them to the right customers, then look for tickets, bla bla... and when the Agent finally responded, the answer sounded ok, but was wrong. Not only that, it was excruciatingly slow. So we had to do something about it.
That 47-step agent is one example of a question where Airbyte Agents does particularly well. Other examples: - “Show me all enterprise deals closing this month with open support tickets." - “Find every support ticket that doesn’t have a Github issue opened”
Some of these might sound simple, but the quality of the answer changes dramatically when the agent doesn’t have to assemble all that context at runtime.
Once we had an early version of the product, I spent a weekend building a benchmark harness to see if it worked. Also for fun, I like writing benchmarks :). I compared calling the Airbyte Agent MCP vs calling a bunch of vendor MCPs directly. I tested retrieval, and search.
For the sake of simplicity, I used token consumption as a unit of measure. I think that’s a good proxy for how well agents are working. A failing agent (like the one that took 47 steps), will churn through lots of tokens while getting nowhere, while a successful one will get straight to the point.
Here's what I found when measuring: for Gong, it used up to 80% fewer tokens than their own MCP, for Zendesk up to 90% fewer, for Linear up to 75%, and for Salesforce up to 16% (Salesforce’s own SOQL does a good job here).
Of course there is the usual obvious bias: we are the builders of what we are benchmarking. So we made the test harness public: https://github.com/airbytehq/airbyte-agents-benchmarks. Feel free to poke at it, and please tell us what you find if you do!
It's still early and some parts are rough, but we wanted to share this with the community asap. We'd love to hear from people building agents: - Are you indexing data ahead of time, or letting the agent call APIs live? - How are you matching entities across systems?
Would also love to hear any thoughts, comments, or ideas of how we could make this better, and if there are obvious things we’re missing. For now, we’re excited to keep building!
swyx 39 days ago | next [–] (former employee here) congrats Michel! so glad to see you guys adapting to the AI age so well (and using the crap out of Devin!)hmm so airbyte agents could serve as a form of MCP gateway, or a key building block of an MCP gateway, which btw is how anthropic uses mcp themselves for all their internal apps https://www.youtube.com/watch?v=CD6R4Wf3jnY&t=1s&pp=0gcJCd4K...i think my most sad/interesting observation about ai engineers is that many ai apps are super data hungry, but many dont have the necessary data engineering background to even know they need an airbyte or what tradeoffs to make in an etl pipeline. would love a "data engineering for ai engineers" type braindump session from someone from airbyte at AIE (https://ai.engineer/cfp )aaronsteers 39 days ago | parent | next [–] Hey, swyx! Great seeing you here.> airbyte agents could serve as a form of MCP gatewayExactly! And a single set of tools for agents to access both realtime (direct reads/writes) as well as cached (Context Store), bringing hopefully the best access path for each different use case.> would love a "data engineering for ai engineers" type braindump ... at AIEGreat idea - we have a booth at AIE, and we'll submit there for a talk. Mario will reach out to you about this. :)jeanlaf 39 days ago | parent | prev | next [–] Thanks swyx! We'd love to do that session "data engineering for ai engineers", will make you an intro to the right person in the team.swyx 38 days ago | root | parent | next [–] saw your email, will get back!sails 38 days ago | parent | prev | next [–] I think this is right ( a big gap ) but I don’t think data companies even now what the right shape is for AI.It’s definitely not old school ETL + dbt + BI tool, it might be something like this, but it’s very earlyaaronsteers 37 days ago | root | parent | next [–] Agreed 100% - we're still super early in this journey, gathering data from our own usage and from our customers' feedback.slurpyb 39 days ago | prev | next [–] Your billing support email forwards to a google group which rejects the email entirely. So i embedded my question inside the websites sales enquiry form and received multiple rounds of emails that couldn’t be further from human.It’s not why we started using posthog but it definitely sealed the deal when you see how simple and reliable that experience isdavinchia 38 days ago | parent | next [–] Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!mtricot 39 days ago | parent | prev | next [–] Let me see what's up and fix that!jscheel 39 days ago | prev | next [–] I feel like we've been working in parallel here :) We are using PyAirbyte (hi aaronsteers) for our users to connect their data sources to our agents. We originally wanted to use the airbyte white-label platform, but the team said that it was being deprecated. I think this really drives home just how crucial it is to have a clear model for accessing your data, and Airbyte has been great at that for quite a while.aaronsteers 39 days ago | parent | next [–] Hello, Jared! Small world! Yes, we did deprecate our old PbA (Powered by Airbyte) offering, but in many ways our new Agents and Embedded offering is a more robust and agent-friendly successor to that older offering.I am happy to hear you are still getting value out of PyAirbyte! If you do try out Airbyte Agents, please let us know how it goes! We are always listening to feedback and would love to hear from you as you explore the new tools and capabilities.jessewmc 39 days ago | prev | next [–] Looks interesting!If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?Have you done any testing on guided indexing, or metadata layers on top of the data? My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time. I.e. _what_ you index, and how you signpost for agents, becomes really important (unless your data is super clean I guess). This does look like a good foundation for that kind of tooling though!aaronsteers 39 days ago | parent | next [–] Hi, @jessewmc. Thanks for your reply. Regarding your points:> If I'm reading correctly, the indexing (Context Store) is neutral/unopinionated? How does it select fields for indexing?While we haven't yet published details on the backend implementation, I can say that our implementation performs very well without needing to prioritize specific fields for indexing. We aim for large text fields to perform decently and retrieval based on small/compressible fields like ints to be fast. (More to come on this in the coming months.)> Have you done any testing on guided indexing, or metadata layers on top of the data?We've been testing with different data scales and shapes. Nothing detailed to share yet, but performance has (so far) never itself become the bottleneck in our agent testing. (The LLM thinking itself is often the bottleneck.)> My experience so far on similar work is that getting data in front of an agent isn't enough context to get useful/reliable answers enough of the time.Airbyte has rich metadata on our upstream connector's data models, which I think helps us a lot to deliver helpful context to the agent. Another option, when optimizing for specific use cases, is to build your own agent tools on top of our Agent SDK. This allows you to make the calls organic and build the tools in a way that makes natural sense to the agent, regardless of source shape or which system(s) that data is coming from.> This does look like a good foundation for that kind of tooling though!We agree! Thanks again for sharing your thoughts here.juancs 36 days ago | root | parent | next [–] Great launch btw! I have some questions if you don't mindyou mentioned that performance was never an issue, I am really intrigued how this is achieved.I have 3 General questions:1. How big (estimate in bytes) and complex were the test datasources? I couldn't find this in the benchmark repo.2. how is the business context managed? In the blog "Airbyte Agents: A New Era for Airbyte" it was mentioned handling the business context but in the context layer docs it only talks about schema discovery (I got a bit confused)3. When you said performance was never an issue, do you mean the user always got the answer it was looking for?dennispi 38 days ago | prev | next [–] We built something similar an A/B testing framework that measures Unblocked's impact on real AI coding agents.It spawns agent CLIs (Claude Code, Codex, Cursor, GitHub Copilot) with and without Unblocked's MCP server attached, then statistically compares the results: https://github.com/unblocked/unblocked-harness-compareWe likewise measured token savings, (wall clock) time, # tool calls, and # turns.nerdright 39 days ago | prev | next [–] This is such a great direction airbyte is taking and congrats to the lunch! I think you're very well-positioned for this opportunity than most people realize, given your reputable brand and your uncanny expertise in etl. It's honestly a natural progression of airbyte as far as the current AI landscape goes. Kudos to you and the team!(We use airbyte at my company, although we self-host it.)aaronsteers 39 days ago | parent | next [–] Thanks! Really appreciate the kind words. Looking forward to seeing what our amazing community builds with these new tools.andai 38 days ago | prev | next [–] The prompts you mentioned here sound like SQL. Is there any way to run actual SQL on these systems? Is "agents need to poke around endlessly" a symptom of the fact that there isn't a way to run an actual query?(I'd guess there is actually SQL at the bottom layer, but there's no way to talk to it?)sho 38 days ago | parent | next [–] That's actually the approach we took with https://gentility.ai/ - we either provide almost-raw SQL query access to the DBs themselves or we synthesize from API into DuckDB via parquet and make that available to the agent to just directly query. It works well - my philosophy is to give agents the sharpest tools you can, and SQL is the best tool there is.I understand the instinct to try to make a proprietary moat around it all but I think the pattern is useful and obvious enough that all big orgs will be doing something very similar within 5 years or so.aaronsteers 38 days ago | parent | prev | next [–] Helpful feedback, thank you! And your instincts are spot on. As of now, we have API based search, with filter predicates and field selection in JSON. While we haven't published anything on the backend implementation, I can say it does use a cloud-native storage medium where the filters are indeed pushed down as SQL. We want to be careful about if/when we offer direct SQL access, specifically because SQL dialects can differ drastically and we wouldn't want to break consumers if/when we change which dialect(s) are supported.That said, please stay tuned - and thank you again for this valuable feedback.thecopy 38 days ago | prev | next [–] Super interesting idea! Congrats on the launch. Context is definitely something that is lacking in my experience. Im always frustrated when an agent cannot answer business-related questions, and i compare them to coding agents which seem to be able to answer everything. The difference is that coding agent has the context right there at the fingertips, while for business its gated behind a bunch of services and custom data models. Context is king :)How do you handle encryption and confidentiality? Im building in this space too (MCP gateway https://www.gatana.ai/) which already have semantic search for tool outputs, and ensuring encryption and confidentiality is not trivial.ck_one 38 days ago | prev | next [–] More and more SaaS companies like ServiceNow or Hubspot are creating new tollgates for agent api calls. How do you think will this impact Airbyte Agents? I guess that replicating data locally will be harder since the platforms will try to protect it or charge for it.aaronsteers 37 days ago | parent | next [–] It's a good question and I won't pretend to predict the future on this one. I will say, I think Airbyte Agents is in a good position because our core Data Replication product has always had to mitigate the impacts of rate limiting and cumbersome upstream APIs. The new Agents toolset gives you the ability to query the upstream APIs directly (read: as a passthrough) while also letting you bypass them entirely when your agent can answer its question via the Context Store directly. Time and feedback from our users will confirm, but I do think this gives our customers a good balance of control - when to query upstream directly and when to utilize the Context Store to work around API limitations - whether inherent or artificially enforced by the vendor.mtricot 39 days ago | prev | next [–] Just want to call out a couple of nuances in our methodology. In general, we tried our best to do apples-to-apples comparisons where we could, and gave ourselves a discount where we couldn’t. Unsurprisingly, it’s a challenge to find MCPs for various vendors (which is another reason we are trying to solve this). Here’s a video walkthrough of the benchmark harness:https://www.loom.com/share/9d96c8c64c1a4b7fad0356774fc54accWhere the comparison wasn't valid or not apples-to-apples:Gong and Zendesk: no official native MCP exists, so we used the most popular community implementations we could find. We were only able to benchmark Gong Search as the Gong MCP does not have a Get tool call.While our Search testing yielded the same number of records on either path, vendor-specific search implementations means results aren’t identical. Contents are similar in general, so the ratios remain directionally correct.The general test set:2 scenarios (Retrieval and Search) across 4 connectors isn’t a huge test set. While we hope to extend this over time, we’ve made the harness public so anyone can contribute in the meantime. Let us know if you find any MCP with better results!Where the vendor MCP wins or ties:Salesforce showed the smallest win at 16%. This is primarily because Salesforce, unlike many vendors, uniquely provides great search support out of the box with their SOQL.We see identical records for Get. As noted, Search returns different sets of identical counts. Airbyte uses fewer tokens because the Salesforce records contain mandatory metadata (type and url).Where the vendor MCP is costly to context:Zendesk is a great example of this. The extreme gap is because the Zendesk MCP (reminder - a community alternative) returns the entire API response in search results. This averages to 9KB per record against our production Zendesk account!Airbyte’s implementation provides filtering, which allows agents to retrieve the minimal data needed to achieve the outcome, explaining the drastic gap.ecares 39 days ago | prev | next [–] Did you find that some data model patterns were easier to detect for some LLM ? I am curious on how training might have made some agents better at graph navigation for instance?aaronsteers 39 days ago | parent | next [–] AJ here, from Airbyte.Yes, we've definitely found that some API data models are easier for models to navigate than others.The largest factors of Agent inefficiency we've identified so far are: 1. Many APIs lack robust-enough search, forcing agents to page through hundreds or thousands of paginated responses until they find the record they are looking for (our Context Store addresses this). 2. Many APIs have HUGE response sets. Our MCP helps handle this by letting the agent decide exactly what fields they can return. 3. With our SDK, you can literally build your own MCP on top of any source we support (50+ right now and will grow). This is super powerful, and allows you to build more ergonomic MCP servers and tools - even if the models themselves are not intuitive or easy for the LLM to leverage directly.Combining all three of these together, we see the vast majority of challenges can be addressed via a strong system prompt for guidance. Fine tuning could get you further but anyway, you'd still want your fine tuned model to build on this same foundation, since the efficiences will transfer across use cases and models.@ecares - Does this answer your question? What do you think?woeirua 39 days ago | root | parent | next [–] Your point about search being a bottleneck is spot on. IMO, search APIs should return guidance to agents to help them winnow down the results faster. For example, if your query returns 1000 results, then it should tell the agent, "too many results, we recommend you filter on column X because of Y to improve your search. Here are the possible values in column X: ..."carefulfungi 39 days ago | root | parent | next [–] There are a lot of APIs like this that I really wish would expose downloading a parquet file instead of trying to implement server-side filtering and reporting query features.aaronsteers 39 days ago | root | parent | next [–] +1Working with APIs is often frustrating and the worst ones are terribly ineficient and frustrating. Our Agent SDK and Agent Context Store insulates you and your agent from this headache, allowing you to query from those synced datasets directly.The feedback about wanting to download a parquet file is super interesting...aaronsteers 39 days ago | root | parent | prev | next [–] Glad to hear this resonates with you also. We're aiming to give agents more control over their context, and easier access paths regardless of the source system.ritonlajoie 39 days ago | prev | next [–] Hi Michel, congrats and I ha