#137foundry

A perspective post, written from the inside of how we actually build these systems for clients, on what makes the difference between a replay tool we feel good about shipping and a "run this script and hope" approach that the client will inherit and fear.

The short version: we treat replay capability as a first-class feature of any integration we build, not a recovery afterthought. The reasoning has been the same across enough client engagements that we now bake it into the integration template from day one.

Photo by Sergei Starostin on Pexels

The pattern we keep landing on

Across roughly twenty data integration projects in the last few years, the replay design we trust is always the same shape:

A durable per-event log on our side of the integration, indexed by upstream event ID, stamped with ingest time and processing outcome.

A check-then-act block in the processing code that looks up the existing downstream record by event ID before deciding to create or update.

A separate writer layer that the processing code calls into for all side-effecting work, so the same code path can run in dry-run mode or live mode.

A replay command that takes mandatory time-window arguments, defaults to a conservative rate limit, and supports a dry-run mode out of the box.

A dashboard view that shows the event log's recent ingest counts, processing outcomes, and any backlog, so on-call can see at a glance whether the integration is healthy.

The reason we keep landing on this shape is that every client engagement we have done has eventually faced the same situation: an upstream outage, a few hours of missed events, and a need to recover without producing duplicates. The teams that had this pattern in place recovered cleanly. The teams that did not had a follow-up incident that was usually worse than the original outage. The pattern is not novel; it is just the answer to the question, and we have stopped pretending each client's situation requires a fresh design.

What we have learned to push back on

A few specific client requests come up often, and we have learned to push back on them politely but firmly.

"Can we skip the per-event log and just rely on the upstream system's buffer?" We almost always say no. The upstream system's retention is theirs; they will change it without telling us; the first time we find out is during an outage. The per-event log is ours and we control its retention. The cost is modest. The value during recovery is the entire point.

"Can we make the replay default to running everything since the outage started?" Also no. The right replay scope is "the specific events that did not succeed," not "everything in the time window." The two are different sets, and replaying the second produces duplicates from events that did succeed during partial-availability periods. We bake the right scoping into the replay tool's interface so the wrong scoping is not an easy mistake.

"Can we skip the dry-run mode for now and add it later?" This is the one we feel most strongly about. A replay tool without a dry-run mode is operationally dangerous, and the version of "later" where the team adds the dry-run mode is almost always "after the first incident where we wished we had it." The marginal cost of building the writer layer with both modes from the start is small. The cost of retrofitting it later is large.

"Replay tooling is one of those features that looks like overkill until the first time you need it, and then it looks like the cheapest thing you ever built. The pattern is well understood; the discipline is in actually building it before you need it instead of after." - Dennis Traina, founder of 137Foundry

The retrofitting cost is what motivates us to push for the full pattern from day one, even on integrations that are "simple enough not to need it." The integrations that turned out not to need a replay tool cost us a small amount of work to build one. The integrations that turned out to need it would have cost the client a multi-day incident if we had not.

Photo by Atlantic Ambience on Pexels

What "good" looks like from the operator's seat

We measure the replay capability against a specific scenario: an on-call engineer, alone, at 2 AM, after a multi-hour upstream outage. Can they:

See on the dashboard that the integration was degraded between time A and time B.

Query the event log for the specific events in that window that did not succeed.

Run a dry-run replay scoped to those events and read the expected outcome counts.

Run a live replay scoped to the same events with the default rate limit.

Confirm via the dashboard that the gap closed.

If any of those steps requires a Slack thread with a senior engineer, ad-hoc SQL against the production database, or hand-editing a script during the incident, the capability is not done. We test against this scenario explicitly before we consider a replay tool shipped. The "drill" is a real exercise we run in staging, with the on-call engineer who will actually be carrying the pager.

The drill is uncomfortable the first time. The second time it is just a habit. By the third one, the engineer has the muscle memory to do it cold at 2 AM, and that is the entire point.

The architectural decisions that compound

A few specific design choices pay off across every integration we have built. They are not novel either; we are not the first people to land on them. But they compound across the lifetime of the integration in ways that are easy to underweight at design time.

The first is the writer layer. Pulling all side-effecting calls behind a writer interface costs maybe a day of design work at the start of the project. Across the lifetime of the integration, it makes dry-run mode possible, makes testing radically easier, makes the next migration to a different downstream system tractable, and prevents an entire class of "we forgot to honor the dry-run flag in this one path" bugs.

The second is the per-event log. Adding a Postgres table to the integration from day one costs an hour. Across the lifetime, it makes replay scoping correct, makes the "where did this event go" debugging question answerable in five minutes instead of an afternoon, and makes the integration legible to operations engineers who did not write it.

The third is the rate-limited replay default. Setting a conservative default rate on the replay tool costs zero effort. Across the lifetime, it prevents the "the replay finished, but now the downstream system fell over from the burst" follow-up incident exactly once for each integration that runs into it.

None of these are large decisions individually. Together they are the difference between an integration that the team feels comfortable operating and one that the team is afraid of. The discipline of making all three decisions on day one, not after the first incident, is what we have ended up codifying.

A perspective post from inside the actual mistakes, not the polished after-the-fact framing. Each item below is something we got wrong on a real project, what happened as a result, and the design change that prevents it on every project since. The pattern is that none of the mistakes were exotic; all of them were the kind of "we'll handle that later" decision that compounded into a problem that was harder to fix later than it would have been to address up front.

We share these because we know other teams have made the same calls. The lessons are not unique to 137Foundry, and the fixes are not proprietary; they are the ordinary design discipline that any team can adopt once they have decided to take replay seriously.

Photo by Liam Briese on Unsplash

Mistake 1: trusting the upstream system's event buffer for recovery

Early on, we built an integration that did not have its own event log. The reasoning was: the upstream system buffers events for seven days, so if we go down for anything less than a week, we can just ask the upstream to replay.

This worked until the upstream system, without warning, reduced their buffer retention from seven days to three days. We found out during an incident that exceeded three days. The replayable events were the ones from the last three days; everything older was lost.

The fix: every integration we build now has its own per-event log on our infrastructure, indexed by upstream event ID, with our own retention policy that we control. The upstream system's buffer is a nice-to-have for cross-checking; it is not the source of truth.

Mistake 2: idempotency keys computed over the entire payload

A second early project used a SHA-256 hash of the full event payload as the idempotency key. The reasoning was: the same event should produce the same hash, and the hash detects duplicates.

This worked until the upstream system added a new optional field to the payload. The hash changed for every event that included the field. Our duplicate-detection check returned false. We reprocessed thousands of events that had already been processed during the partial-availability window of an unrelated outage, producing duplicate downstream records that took two weeks to clean up.

The fix: idempotency keys are now computed over a deliberately-chosen subset of fields, with the algorithm and the field set treated as a contract. New optional fields in the upstream payload are ignored at the hash step. Schema-version prefixes (v1:hash(...), v2:hash(...)) handle the case where the contract has to change. The general pattern is the same one that the Stripe API documentation describes for client-provided idempotency keys, applied to the integration's internal key design.

Mistake 3: building a replay tool without a dry-run mode

A third project shipped a replay tool that did not have a dry-run mode. The reasoning was: the team can read the SQL query that drives the replay and confirm what it will do.

The team could indeed read the query. The query was correct. The replay was correct on the first run. The replay was also correct on the second run, where the operator typoed a single character in the time-window argument and replayed the wrong day's events. The downstream impact was a few hundred duplicate records.

The fix: every replay tool now requires a dry-run mode out of the box, and the dry-run runs by default unless the operator explicitly passes a --live flag. The output of the dry-run is the structured summary the operator reads to confirm intent before running for real. The wrong-day typo would have been caught by the dry-run output showing zero unprocessed events in the window, which is a different number than the operator expected.

Mistake 4: defaulting to the integration's normal throughput on replay

A fourth project's replay tool defaulted to the integration's normal steady-state throughput, on the reasoning that "if the integration can handle it at steady state, it can handle it during recovery."

This was wrong because the downstream system had been handling steady-state traffic during the outage, but the burst of replay traffic on top of that pushed it over its connection limit. The replay finished, the downstream fell over, and we now had a second outage to recover from.

The fix: every replay tool now defaults to a rate limit that is conservatively below the integration's normal steady-state rate, often three to five events per second when the integration normally runs at ten. The operator can opt into higher rates if the situation requires it. The default protects against the case where the operator does not have time to think carefully.

Photo by Diego F. Parra on Pexels

Mistake 5: no dashboard for the event log

A fifth project shipped the per-event log and the replay tool but no dashboard. The reasoning was: operations can SQL the log when they need to, and we will build a dashboard if it becomes a problem.

It became a problem during the first incident. The on-call engineer, at 2 AM, did not know the schema of the log table, did not have a saved query for "events with outcome != success in the last six hours," and ended up writing the query inline and getting it wrong twice before the third version returned the right count.

The fix: every integration ships with a small dashboard that surfaces the event log's recent ingest counts, processing outcomes, and any backlog. The dashboard has a saved "find unprocessed events in this time window" query. The on-call engineer does not have to remember the schema; they click a button. The dashboard takes a day to build the first time and most of that day's effort is reusable across projects.

What these five mistakes have in common

Every one of these was a "we'll handle that later" decision at design time. Every one of them turned into a real production incident that cost more to recover from than the original design work would have cost to do correctly.

The pattern is the same across the five: the design decision looked optional at the time, looked like overkill in the absence of an incident, and looked like the obvious right call in the presence of an incident. The discipline we have ended up codifying is to make all five decisions on day one, regardless of how simple the integration appears. The marginal cost is small. The downside protection is large.

The deeper framing here is that reliability work is bought, not earned. Every reliability feature has to be built explicitly; none of them appear automatically because the team "cares about reliability." The teams we work with that consistently have clean recoveries are the teams that built the recovery tooling before they needed it. The teams we work with that have incident-driven cleanups are the teams that built the recovery tooling after they needed it, which is to say after the first incident has already happened.

Quick post from inside 137Foundry. We run on a stack of about twenty SaaS vendors and we help clients evaluate vendor stacks regularly. The tools below are the ones that earned a permanent spot in how we manage that, listed in order of how much value they deliver per hour we spend with them. None of these are sponsored. None are affiliate links. Some are free, some cost money, some compete with each other. Pick whatever fits.

1. A shared vendor registry in Notion (or Airtable, or a spreadsheet)

This is the foundational one. A single source of truth that lists every SaaS vendor we use, with for each one: contract start date, renewal date, annual cost, primary owner, escalation contact, contract notice period, last evaluation date.

We use Notion because the team already lived in it for other reasons. Airtable works equivalently. A Google Sheet works for a small team. The specific tool matters less than the discipline of keeping it current.

What this catches: renewals coming up that nobody flagged, contracts where the auto-renewal notice period is shorter than we remember, contract owners who have left the company. It is the most boring item on the list and probably the most useful. The Notion templates gallery has reasonable starting templates if you do not want to build from scratch.

2. A status page aggregator (homegrown, ~200 lines of Python)

We wrote this ourselves because the commercial options cost more than the engineering time we saved building it. It polls every vendor status page every two minutes and surfaces degraded status into a Slack channel. The longer guide on building one of these covers the technical details.

What this catches: vendor outages we would otherwise not notice until customers complained, or until an engineer spent an hour debugging the wrong layer. The pattern across the team is consistent: "wait, is X down?" gets answered in five seconds instead of fifteen minutes of debugging.

3. A vendor scorecard template in Google Sheets

The scorecard template walks through eight dimensions of vendor risk: security, data handling, reliability, financial health, integration depth, switching cost, roadmap alignment, support quality. The longer writeup on how 137Foundry approaches procurement covers what each dimension contains and how to weight them. Every vendor evaluation worth more than $25K annual gets one of these filled in.

What this catches: feature-driven decisions that ignore financial-health risk, security-driven decisions that ignore switching cost, all the asymmetric blind spots an evaluation falls into when it is not structured.

4. A renewal calendar with 90-day lead times

Most SaaS contracts auto-renew. Most auto-renewals require 30 to 60 days notice to cancel. The renewal calendar fires alerts 90 days before each renewal date, which is enough lead time to run a structured re-evaluation, negotiate price, or migrate.

We use Notion calendar views with alert rules. Google Calendar works. The discipline matters more than the tooling. A renewal that catches us by surprise gives us no leverage and forces us into a defensive contract negotiation. A renewal we prepared for three months out lets us walk away credibly, which usually leads to better pricing even if we end up renewing.

5. A "vendor health" rotating agenda item in monthly engineering leadership

Once a month, in our engineering leadership meeting, one person walks through the vendor registry highlighting: vendors with upcoming renewals, vendors that had incidents this month, vendors that had pricing or product changes, vendors flagged for re-evaluation. The discussion runs 15 minutes and surfaces issues that would otherwise sit in someone's mental queue.

This is a process rather than a tool, but the discipline is the same as the others. Vendor health goes from "something we should pay attention to" to "something we actually pay attention to" when it has a recurring slot on someone's calendar.

6. Independent review platforms as a sanity check

When evaluating a new vendor, G2, Gartner Peer Insights, and TrustRadius get checked for recent reviews. Filter by company size similar to ours (or to the client's). Read the negative reviews carefully. Look for patterns in what reviewers complain about.

What this catches: the gap between vendor marketing and lived experience. Vendor pricing pages do not mention the support team's average response time; vendor case studies do not mention the migration pain; vendor demo videos do not show the failed integrations. Reviews do.

What did not make the list

A few categories of tool that we have tried and de-emphasized:

Spend management platforms (Vendr, Tropic, etc). These are excellent for companies with serious SaaS spend (typically $1M+ annual across the stack). For a 137Foundry-sized operation, the platforms cost more in license and onboarding than they save in negotiation outcomes. They become valuable around the $2M annual stack threshold.

SaaS management platforms (Zylo, BetterCloud, Torii). Same story: useful at scale, overkill at our size. We get most of the benefit from the homegrown registry plus the renewal calendar.

Vendor due diligence platforms. A few startups in this space try to automate vendor security questionnaire responses. The output is often shallow and the time saved is small. We have not adopted any of them yet.

How this connects to our procurement framework

The six tools together are the operational layer underneath a procurement framework. The framework itself (eight-dimension scorecard, structured re-evaluation cadence, exit plans in every memo) is what 137Foundry uses to make decisions. The tools are what make the framework actually happen instead of being aspirational.

The reverse is also true: without the framework, the tools are just busywork. A renewal calendar that fires alerts nobody acts on is worse than no calendar at all, because it teaches the team to ignore vendor signals. A scorecard nobody fills in is worse than no scorecard. The tools and the framework reinforce each other; either alone is incomplete.

For broader context on how we structure the actual evaluation work, the full vendor scorecard guide covers the framework side in detail.

A few useful external references

NIST cybersecurity framework for the security risk vocabulary

Cloud Security Alliance CCM for cloud-vendor-specific dimensions

G2 Crowd for independent reviews

Gartner Peer Insights for enterprise-focused reviews

What we are still working on

A few things we have not yet figured out:

How to integrate vendor SLA compliance tracking with our own observability so we can argue convincingly when a vendor misses their commitments. The current approach is manual; we want it automated.

How to share scorecards across client engagements without leaking confidential vendor evaluations between clients.

How to factor in newer vendor categories (LLM API providers, vector databases, AI infrastructure) where the evaluation criteria are still evolving and our scorecard dimensions feel slightly off.

If anyone has solved these problems, we are interested. The space is moving fast and the playbook from three years ago does not cover today's vendor categories cleanly.

Setup time, in case you are wondering

The total setup time for the six tools above, starting from a blank slate, was roughly one engineering week distributed over a quarter. The vendor registry was a half day. The status aggregator was two days. The scorecard template was a few hours. The renewal calendar was less than an hour. The monthly meeting slot was a calendar invite. The review-platform checks are not really a tool, just a habit we built.

Quick post from inside 137Foundry. We have been on the recommending side of a lot of SaaS procurement decisions over the years, both for our own stack and for client engagements. The last quarter had three decisions worth writing about, because each one taught us something we did not fully appreciate before.

These are the unvarnished notes. Vendor names redacted because we are still mid-relationship with two of them, and the third does not need a public callout. The patterns are what matter.

Decision 1: replacing the data integration tool we had used for four years

The incumbent vendor in this category had been steadily eroding in three ways: pricing was up about 40 percent over the four years we used them, the API was incrementally degrading (deprecations without long notice periods), and the support response times had drifted from 4 hours to 36 hours over the same period.

We evaluated three competitors. The winner was 15 percent more expensive than the incumbent on the per-month line item, but the engineering hours we were spending working around the incumbent's quirks were burning more than the cost differential each month. Total cost of ownership flipped immediately.

What we learned: we should have run the structured evaluation 18 months earlier. By the time we ran it, we had already absorbed about $40K of unbilled engineering time keeping the old integration alive. Earlier evaluation cycles would have caught the deterioration in time to switch without that overhang.

This is a pattern: SaaS vendors degrade gradually after they reach product maturity, and the degradation is invisible from inside an existing relationship until it has been priced in. We are now scheduling a structured re-evaluation of every vendor in our top ten spend categories every 18 months instead of every 36. The longer guide at 137foundry.com walks through the scorecard we now use for these evaluations.

Decision 2: choosing an identity provider for a client engagement

The client was a 150-person SaaS company that had been getting by on a homegrown auth system. They needed to move to a proper identity provider for SOC 2 reasons and because their enterprise customers were asking for SAML SSO.

We ran a comparison between two of the major identity vendors and one smaller, less well-known competitor. The big vendors were strong on integrations, brand recognition, and feature completeness. The smaller competitor was strong on pricing, simplicity, and a more focused product.

The client picked one of the big vendors. The decision came down to one dimension: the smaller competitor had been founded only four years ago and was post-Series-B but pre-profitability. The big vendor had been around for over a decade with a strong financial position. For a five-year identity infrastructure commitment, the financial-health weighting moved the decision decisively.

What we learned: the financial-health dimension of vendor evaluation gets undervalued by engineers (who tend to weight feature completeness and developer experience) and weighted appropriately by procurement (who tend to weight contract terms and pricing). Both perspectives are partial. The structured scorecard approach forces both to be part of the conversation, which is what we wanted but had not always achieved when running evaluations more loosely.

Decision 3: replacing a marketing automation tool

This one was for our own internal use. The incumbent had been acquired six months earlier by a larger company in an adjacent category. The roadmap had quietly shifted from "deep marketing automation" to "marketing module in a larger suite." Two of the features we used were marked as deprecated with an 18-month sunset.

We did the evaluation. Picked a competitor, migrated, finished the move within the deprecation window. Total time to migrate was about 100 engineering hours, plus 40 hours of content rebuild on the marketing side. The migration was painful but the alternative (waiting for the deprecation to force the issue, then migrating under time pressure with no leverage) would have been worse.

What we learned: vendor acquisition is one of the underrated risks in SaaS procurement. The financial-health dimension catches it partially (acquirers often surface in public news months before the deal closes), but the deeper question is "what is the acquirer's incentive to maintain this product?" When a larger company acquires a smaller SaaS, the smaller product is often a stepping stone toward something else, and the parts of it that do not fit the acquirer's strategy get sunset.

Our current heuristic: any vendor that has been acquired within the last 12 months goes onto an annual re-evaluation cycle, regardless of how well the product is otherwise serving us. We are not panicking; we are just making sure we have an exit plan that does not require a 90-day fire drill.

Three patterns from these three decisions

A few things became clearer after writing this up.

Pattern 1: re-evaluate more often than you think you need to. SaaS vendor quality is a moving target. Pricing changes, support degrades, roadmaps shift, acquisitions happen. The version of a vendor you signed with three years ago is rarely the version you have today. An 18-month re-evaluation cadence on top-spend vendors costs a few hours per quarter and surfaces problems while you still have leverage.

Pattern 2: weight financial health more than your engineering team wants to. Engineers are biased toward picking the most exciting product. Procurement is biased toward picking the cheapest one. Financial health (which favors the boring, well-established option) is the dimension that both biases tend to underweight, and it is also the dimension that catches the most expensive procurement mistakes.

Pattern 3: write the exit plan before you sign. Every SaaS vendor relationship ends eventually. The vendor goes out of business, gets acquired, raises prices past your willingness to pay, or simply stops serving your needs. The migration plan you write before signing is far better than the migration plan you write under time pressure when the relationship has gone bad. We now require an "exit plan" section in our vendor evaluation memos. The section is usually three paragraphs and takes 20 minutes to write. It saves orders of magnitude more time later.

What we are doing differently now

Concrete changes we have made in the last six months:

Quarterly review of every vendor in our top-ten spend categories

Structured 8-dimension scorecard for every renewal worth more than $25K annual

Exit plan paragraph required in every initial vendor memo

Status page aggregator that surfaces vendor reliability into Slack

Rotating "vendor health" agenda item in our monthly engineering leadership meeting

None of these are sophisticated. They are mostly forcing functions to ensure we actually do the things we already knew we should be doing.

A few external references we lean on

AICPA SOC 2 framework for the security dimension

NIST cybersecurity framework for general risk-management vocabulary

Cloud Security Alliance CCM for cloud-vendor-specific risk

Gartner Peer Insights for independent vendor reviews

What this is and is not

This is three data points, not a study. Different industries have different risk profiles. Different company sizes mean different procurement constraints. Some of what we do at 137Foundry would be too heavy for a five-person startup and not nearly thorough enough for a Fortune 500.

A behind-the-scenes look at the decision we make most often on data integration projects: log-based change data capture or trigger-based. We have done this conversation enough times that the pattern is worth writing down.

The short answer is that log-based wins for most teams once they understand what they are buying, but trigger-based is the right answer often enough that the framing "log-based is always better" is misleading. Worth walking through what we actually think about when a client team asks which one fits.

Photo by Brett Sayles on Pexels

The default position

When we walk into a new client engagement and the question of CDC comes up, our starting position is log-based. Not because it is universally better, but because the failure modes of log-based are operational (which we can mitigate) and the failure modes of trigger-based are structural (which we cannot).

Log-based fails when: - The replication slot fills the WAL and the database runs out of disk. - The CDC tool falls behind for long enough that the source database reclaims the log space. - A schema evolution is not handled correctly by the consumer.

All three of these failure modes are observable, monitorable, and recoverable with the right operational practice. We can set alerts on replication slot lag. We can configure WAL retention to survive realistic outages. We can use a schema registry to handle evolution. The work is real, but it is bounded.

Trigger-based fails when: - The trigger logic has a bug that corrupts the source data. - The trigger overhead degrades source-database write performance under load. - The audit-table cleanup process loses track of which rows have shipped.

These failure modes are harder to recover from because they affect the source database directly. A trigger bug that has been silently producing wrong audit rows for a week is a deeper problem than a log-based pipeline that has been silently lagging. The blast radius is larger.

So we start with log-based and only deviate when something specific about the client situation makes log-based impractical.

When trigger-based is the right answer

Specific client scenarios where we recommend trigger-based over log-based:

The source database is a managed service that does not expose replication-level access. Some managed databases on smaller cloud providers do not give you the ability to consume the log directly. If the database is on a provider that does not offer logical replication, log-based CDC is just not available. Trigger-based works on any database that supports triggers, which is essentially all of them.

The team does not have the operational headroom to run Kafka or a managed CDC service. Log-based CDC is operationally cheap once you have Kafka and Debezium running, and operationally expensive if you have to set them up just for CDC. For a five-person engineering team with no existing Kafka infrastructure, the trigger-based pattern (write to an audit table, read it on a schedule, ship the changes) is often the better answer simply because the team can run it without becoming distributed-systems engineers.

The throughput is low enough that the trigger overhead is invisible. A source database doing tens of writes per second can take the trigger overhead with no measurable performance impact. At thousands of writes per second, the same trigger overhead becomes measurable and starts to matter. We profile the actual write rate before recommending one or the other.

The team needs a queryable change history for non-streaming use cases. A trigger-based audit table is a normal SQL table. Analysts can query it directly. Reports can run against it. The log-based pattern requires the data to flow through Kafka before it is queryable, which is fine for streaming consumers but inconvenient for ad-hoc analysis.

In each of these cases, trigger-based is not a compromise; it is the better answer for the specific situation.

When log-based is the obvious choice

The flip side: situations where we do not even discuss trigger-based.

Sub-second latency requirements. Trigger-based pipelines have an inherent lag between the trigger fire and the audit-table reader catching the new row. Log-based pipelines can deliver sub-second end-to-end latency. For real-time fraud detection, live dashboards, or operational data syncing, log-based is the only credible answer.

High-throughput sources where trigger overhead matters. A database doing thousands of writes per second under steady load will feel the trigger overhead. We measure it with a load test before going live, and if the overhead exceeds the team's tolerance, log-based is the path.

Existing Kafka infrastructure. If the client is already running Kafka for other reasons, adding Debezium for CDC is a small marginal cost. The Kafka cluster pays for itself across multiple use cases, and CDC is just one more producer.

Complex schemas with frequent evolution. Log-based tools handle schema evolution more cleanly than hand-rolled trigger pipelines. For a source database that changes schema every couple of weeks, the schema-evolution cost of trigger-based piles up.

What we do not recommend

Two patterns we actively recommend against:

Building log-based CDC from scratch. Parsing the PostgreSQL WAL or the MySQL binlog directly is possible. The libraries exist. Teams who do this end up rebuilding Debezium badly. The standard tools have solved the operational problems already; rolling your own gives you control at the cost of two engineers full-time for a year. Almost never the right call.

Mixing log-based and trigger-based on the same database without clear boundaries. We have seen teams run log-based CDC for some tables and trigger-based for others on the same source database, with no documentation of which is which. The next engineer who joins has to spend a week figuring out the layout before they can change anything. Pick one pattern per database when possible. If two patterns are required, document the boundary explicitly.

The conversation with the client

When we sit down with a client engineering team and walk through the choice, the conversation usually goes like this:

We start by asking three questions: 1. What is your latency tolerance? (Sub-second, seconds, minutes, hours?) 2. What is your peak write throughput on the source tables you want to capture? 3. Do you have schema-write access and replication access to the source database?

The answers point us at one of the two patterns most of the time. The edge cases (where the answers do not converge on a clear winner) are usually situations where the team should pilot both and pick based on measured behavior.

A pilot is cheaper than a wrong choice. We typically build a small log-based pipeline on one table and a trigger-based pipeline on another table, measure latency and overhead over a few weeks, and let the data settle the question. The cost of the pilot is two engineer-weeks. The cost of choosing wrong and discovering it six months in is twenty engineer-weeks.

The bigger picture

The CDC pattern conversation is part of a larger 137Foundry data integration engagement, where we look at the broader architecture: source databases, downstream consumers, latency requirements, operational headroom, build-vs-buy on the tooling. CDC is one of the questions; the full picture is several.

The longer reference we point clients to is the guide How to Implement Change Data Capture Without Polling Your Database, which lays out all three CDC patterns (log-based, trigger-based, timestamp polling) with a decision rule for picking between them. The internal decision-making we do on client projects is more nuanced than the article (because we have specific client context), but the framework is the same.

For broader background on the underlying ideas, the Wikipedia entry on change data capture is a reasonable starting point.

The 137Foundry view

The CDC pattern is the first technical choice on most data integration projects, and the choice carries through the rest of the build. Get it right and the rest of the architecture follows naturally. Get it wrong and you fight against the pattern for the life of the system.

We tend to recommend log-based when the operational headroom exists, trigger-based when it does not, and timestamp polling only for low-stakes pipelines where the failure modes are acceptable. Most clients land somewhere in the first two categories. Few land in the third.

The right answer is the one that fits your specific situation. The wrong answer is the one that fits someone else's situation but feels safer because they have written about it. We aim to make the choice deliberately, with the client team in the room, on every project.

We're 137Foundry — a small studio that builds web applications, AI integrations, and the kind of internal automation that makes a business actually run.

This Tumblr is the team blog. The main site at 137foundry.com covers our services and case studies in their finished form; this is the looser space — half-shaped ideas, working notes, problems we hit in client work and how we ended up solving them. The kind of thing that doesn't quite fit on the main blog yet but is too interesting not to write down.

A rough idea of what to expect here:

Engineering notes — odd bugs, unexpected platform behaviors, things we learned from a postmortem

AI and automation perspectives — what we're actually shipping in client work, what's overhyped, what's genuinely useful

Hot takes — opinions on web dev tooling, framework drama, the state of the developer ecosystem

Behind-the-scenes — process posts about how we scope projects, write proposals, structure engagements

We post when we have something to share, not on a schedule. If you're a founder or operator thinking about a build, the front door is the main site or our about page — happy to talk.