Murat Can Ümit

Solutions architect and founder. I build production AI systems alone, and I know what they should refuse to claim.

Twelve years across banking, travel distribution, telecoms and consulting. For the last two years I have designed, built and operated two AI products end to end: architecture, inference, entitlement, release and on-call. Currently in Istanbul and relocating to the United Kingdom.

01 — The system

Tactiq

Tactiq is a live football match-analysis platform with over twenty thousand users across 175 markets. What follows is not a feature list. It is the three parts of the architecture where the decisions were difficult, and where getting them wrong would have failed quietly rather than loudly.

2a — Platform
Clients iOS · Android · web · desktop API Gateway latency-based routing health-check failover us-east-1 80 functions primary · batch, calibration, tournament eu-central-1 44 functions user path ap-northeast-1 52 functions user path Single-region authority entitlements · refresh tokens one writer atomic conditional write reconciled every minute active-active · 176 functions · 41 DynamoDB tables 16 Route 53 health checks 36 tables replicate globally, five are held outside replication deliberately
Clients iOS · Android · web · desktop API Gateway latency-based routing health-check failover us-east-1 80 functions primary · batch, calibration, tournament eu-central-1 44 functions user path ap-northeast-1 52 functions user path Single-region authority entitlements · refresh tokens one writer · atomic conditional write reconciled every minute active-active · 176 functions 41 DynamoDB tables · 16 Route 53 checks 36 tables replicate globally, five are held outside replication

Three regions, one exception

One hundred and seventy-six Lambda functions sit behind API Gateway across three AWS regions, deployed active-active with latency-based routing and sixteen Route 53 health checks. The distribution is deliberate rather than symmetric: us-east-1 carries eighty functions and runs the batch, calibration and tournament workloads, while eu-central-1 and ap-northeast-1 carry forty-four and fifty-two and serve the user path. A user in Tokyo and a user in Frankfurt are served by different regions and see the same product.

Forty-one DynamoDB tables hold the state. Thirty-six of them replicate across all three regions. Five do not, and that exception is the design decision: paid entitlement and refresh tokens live in a single-region authority with one writer and atomic conditional writes, reconciled every minute, and every region reads its verdict from there rather than replicating it. A replication canary runs every five minutes and alarms on cross-region latency, because a replication lag you cannot see is the same as one that is not happening.

Multi-region last-writer-wins is fine until it silently drops a tier change, and a customer losing paid access is a failure you find out about from an angry message rather than an alert.

2b — Guardrails
Request App attestation HMAC request signature Signed short-lived token fail-closed fail-closed fail-closed Generation Grounding check Schema validation Language guard server-authoritative shape enforced requested locale only Cache decision Response Discarded rejected, never repaired flow control and authority rejected HMAC, JWT and batch secrets held per region language mismatch raises a named alarm
Request App attestationfail-closed HMAC signaturefail-closed Signed tokenfail-closed Generation Grounding checkserver-authoritative Schema validationshape enforced Language guardrequested locale Cache decision Response Discarded rejected, never repaired flow control and authority rejected HMAC, JWT and batch secrets held per region language mismatch raises a named alarm

Structural constraint, not instruction

Three fail-closed checks run before a request reaches a model: app attestation, HMAC request signing and a signed short-lived token. If any of them cannot be verified, the request stops there.

After generation the output passes a server-authoritative grounding layer, schema validation and a language guard before any caching decision is made. The model cannot emit a number the server did not supply, cannot return a shape the schema does not accept, and cannot answer in a language the request did not ask for. Output that fails is discarded rather than patched.

Prompt-level guardrails fail silently and you find out from a screenshot on social media.

2c — Calibration
Predictions issued Real outcomes match ends, number is fixed Graded Hit rate perconfidence band Calibration errorvs live baseline not an absolute target Drift alarm threshold breach Automatic rollback weekly cycle weekly refit Sunday 22:00 UTC · incremental tracking every six hours · counterfactual replay Mondays thin samples are held, not fitted rollback target is the last model with a clean band
Predictions issued Real outcomesmatch ends Graded Hit rate per confidence band Calibration error vs baseline Drift alarmthreshold Automatic rollback ↺ weekly cycle weekly refit Sunday 22:00 UTC incremental tracking every six hours thin samples are held, not fitted rollback target is the last clean band

Measured against a live baseline

Every week the system scores its own predictions against real results, grades them, and computes hit rate per confidence band. The distance between stated confidence and observed accuracy is the calibration error. Beyond threshold it raises a drift alarm and rolls back automatically.

The comparison is against a live baseline rather than an absolute target, because an absolute target is a number you get to choose. Football supplies ground truth on a fixed schedule and does not negotiate.

I could not construct an evaluation that flattered me even if I wanted to.

2d — The whole system

The shape of the whole thing

The user path is the smallest part of this. Most of the system is what runs when nobody is looking: data synchronisation from six providers on schedules ranging from every minute to once a month, prediction engines that recompute team ratings and playing-style profiles weekly, calibration that grades its own past work, and a notification layer that has to know a user's timezone before it is allowed to wake them.

The World Cup hub was built as an isolated fleet with its own analysis, sync, live and accuracy-tracking functions. A seasonal event with unpredictable traffic should not be able to take down a product that runs all year, and when the tournament ends the fleet can be removed without touching anything else.

One person wrote all of this, and that is exactly why the boundaries are where they are.

CLIENTS
iOS · iPadOS
macOS · Android
32 locales
EDGE
API GATEWAY
3 REST APIs
10 HTTP APIs
~50 routes per region
CLOUDFRONT
global distribution
ROUTE 53
16 health checks
COMPUTE
USER PATH
auth · authorizer · me · history
analyze · analyze-v2 · analyze-live
simulate · momentum · stats · search
PREDICTION
compute-v5-elo
style-profile · match-style-forecast
player-impact · counterfactual
CALIBRATION
auto-calibration
calibration-tracker
weekly-monitor
bedrock-daily-report
DATA SYNC
fixtures · teams · players
leagues · standings · injuries
xg-rolling · shot-profile
search-sync
NOTIFICATIONS
push-worker · match-reminder
daily-reminder · favorite-reminder
SQS queue + DLQ
TOURNAMENT
wc-26-analyze · wc-26-live
wc-26-sync · wc-26-accuracy-tracker
wc-26-push-scheduler
isolated fleet
PLATFORM
warmer · health · replication-canary
log-export · entitlement-sync
export-user-data · user-hard-delete
STATE
DYNAMODB
41 tables
36 global · 5 single-region
LARGEST
search-index 332k
football-data-v2 55k
player-impact 39k
analyses 20k
S3
11 buckets
SECRETS MANAGER
per-region rotation
80 scheduled jobs from every-minute to monthly · 65 CloudWatch alarms across three regions
staging runs as a parallel fleet with its own authorizer
20,000+ users32 languages175 markets4 operating systems
3 regions176 functions41 tables80 scheduled jobs65 alarms
Full technical document (PDF) See the product at mcvibeapps.com
02 — Inventory

What it takes to keep it running

Architecture diagrams flatter a system. They show the parts that were designed and hide the parts that accumulated. This is the accumulation, taken from the live account rather than from memory.

176
Lambda functions across three regions
41
DynamoDB tables, 36 replicated globally
80
scheduled jobs, from every minute to monthly
65
CloudWatch alarms with named thresholds
13
API Gateway deployments, REST and HTTP
16
Route 53 health checks
11
S3 buckets
6
external data providers reconciled
332,000
rows in the search index
55,000
fixture records with rolling statistics
39,000
player impact records
9,900
calibration records with reliability bands
7,900
audit log entries
4
payment rails with full lifecycle handling
2
hard budget caps with alarms attached
1
person

Some of these numbers are the result of decisions and some are the cost of them. Eighty scheduled jobs is not a boast, it is what six data providers on different refresh cadences actually requires. Sixty-five alarms is what it takes to sleep when there is nobody on the other shift.

03 — Decisions

Five calls I would make again

01

I removed features that worked

Calibration showed those outputs were overconfident. Nobody was complaining. They were among the most used features in the product. I removed them anyway, lost revenue in the month that followed, and watched retention go up.

The product does less now and retention improved.

02

I ignored an alarm

A drift alarm fired off-season on a thin sample of roughly eighty events. The high-confidence bucket held ten of them. Refitting on ten events would have taught the model the shape of a fortnight rather than the shape of the game, so I left it alone. Two weeks later, at four hundred events, calibration was healthy.

Knowing when your evaluation is not yet telling you anything matters as much as knowing what it says.

03

I gave one table its own architecture

All state is multi-region except one table. Paid entitlement is held in a single-region authority with one writer and atomic conditional writes, and the other regions read from it rather than replicate it. It costs a little latency on a rare path and removes an entire class of silent failure.

I stopped treating consistency as a uniform property of the system.

04

I did not use a machine learning framework

The learning layer is plain Python. Serverless rewards a small footprint and a fast cold start, and a transparent model I can correct beats an opaque one that is hard to debug when a single feed shifts. When calibration moves, I can read the reason in the code rather than infer it from a loss curve.

Choosing the cheaper, smaller, inspectable option over the more impressive one is a habit rather than a compromise.

05

I put a hard ceiling on my own inference spend

Two budget caps sit on the account with alarms attached: three hundred dollars a month for Bedrock inference, five hundred for everything else. A generative product without a ceiling discovers its unit economics in a billing email. Setting the ceiling first forced the two-tier model routing, the caching rules and the decision about which requests justify a stronger model. There is a daily report on token spend for the same reason.

They are not forecasts. They are alarms with a name on them.

04 — Second product

Naryu

Naryu turns one birth record into daily personal guidance across astrology, numerology, Human Design, Vedic astrology, BaZi and feng shui, currently thirty-six distinct reading types. It is in pre-launch, with the computation layer, content pipeline and inference path complete.

The personalisation layer is deterministic, not generative. A Python engine derives planetary positions from JPL ephemeris data and produces the chart, alongside house systems, progressions, transits, dasha periods and BaZi pillars, each validated against published reference charts. The generative layer computes nothing at all; it interprets a result that has already been established, and it is not permitted to introduce a value the engine did not produce.

An interpretation built on a wrong degree is wrong however well it reads.

21 languages36 reading types2 AWS regions~2,000 automated checks
05 — Before

Twelve years before this

The enterprise years. It started at Doğan Dağıtım, building backend systems for national press distribution, where a wrong assumption did not produce a bug ticket but a vehicle in the wrong place. Then Garanti, one of Turkey's largest private banks, working on systems holding thirty-eight million customer records under formal change control, with mainframe at the centre of them. Then Amadeus, in travel distribution, where correctness at volume is the entire product.

Then telecoms. Orion Innovation, on telecom platforms and customer onboarding. Then Vodafone, leading solution design on initiatives that spanned more than twenty engineering and business teams, none of which reported to me, and acting as technical architect for an internal enterprise AI product including the identity and access management model covering the entire employee base.

Leadership and consulting. Three and a half years at BlockchainIST leading seven engineers, with enterprise clients in banking, insurance and the public sector. The recurring situation was an organisation arriving with its technology already chosen and a problem it had not yet named. Most of the work was finding the problem underneath the request, and then saying so.

The startup. Tosia Tech, as co-founder. Funding was withdrawn. The team and the technology moved, the business found profitability and later raised. The technical part was the easier half.

06 — Contact

I am open to conversations about architecture, principal engineering, and teams building AI systems that have to survive contact with real users.

Currently in Istanbul. Relocating to the UK, open to Skilled Worker sponsorship. No notice period.

LinkedIn