Our homegrown file uploader was three years old, battle-tested, and quietly eating an engineer’s summer every year. I noticed because the annual planning meeting had a familiar line on it. Upload hardening, third year running, sized at two sprints.
Someone newer than the rest of us asked why it kept coming back.. “We spent 14 engineer-weeks in one year maintaining our homegrown file uploader: retry logic, a scanning queue, storage lifecycle bugs, and two mobile-specific incidents. We stopped because uploads were nobody’s product at our company, and everything downstream of them was.”.
The question was never whether we could build an uploader. We had, and it worked. The question was whether uploads were our product..
Key takeaways. Upload maintenance hides in the backlog as small tickets, so nobody adds up what it costs across a year.. Fourteen engineer-weeks went to retries, virus scanning, storage lifecycle bugs, and two mobile incidents..
Compare loaded engineer cost against subscription cost before assuming that in-house is the cheaper option.. Migrating deleted roughly 4,000 lines of custom code and kept only validation rules and the metadata model.. Building in-house still wins for exotic protocols, air-gapped deployments, and companies whose product is uploading..
What we built and why it felt right. In 2021 the requirement fit in one sentence. Let customers attach files to a record.
One endpoint, a size check, a write to object storage. Two days of work, and it was correct.. Then it grew, one reasonable feature at a time.
Progress bars, because a 40 MB PDF looked frozen without one. Chunking, because a customer started sending 300 MB videos. A scanning queue after a security review.
Resume, because chunking without resume is a tease. Thumbnails. An expiry policy.
Every one of those was justified, small, and shipped by someone competent. Nobody ever proposed building upload infrastructure. We just never stopped..
Pride was part of it, and I would rather say that plainly than pretend the decision was purely technical. The uploader was ours. We understood it.
Understanding your own code is a real advantage, right up until you are the only people on earth who can fix it at two in the morning.. The maintenance ledger. The question in that planning meeting deserved a real answer, so I went back through a year of tickets and incident reviews and added the hours up.
I expected the number to be uncomfortable. It was worse than that, because it was boring.. A year of upload maintenance, itemized.
Nothing on the list shipped a feature.. Retry and resume logic took four weeks, most of it in the ugly middle cases: a part that arrived twice, a resume that picked up from the wrong offset, a transfer that reported success while writing nothing. The scanning queue took three, and almost none of that was scanning.
It was the queue, the dead-letter handling, and the runbook for when a worker wedged. Storage lifecycle bugs took another three, courtesy of cleanup jobs that raced each other and orphaned objects nobody could account for. Two mobile incidents took two more, one on iOS background suspension and one on an Android memory regression..
Fourteen weeks. The question I should have asked three years earlier is the plainest one available, which is what the best way is to handle file uploads without building your own infrastructure. What the ledger really shows is total cost of ownership applied to a component nobody files as a purchase, because the cost arrives as small tickets rather than an invoice..
The spreadsheet moment. Then came the part everyone eventually does on the back of an envelope. We put a loaded cost on an engineer-week, deliberately conservative, and multiplied.
Fourteen weeks came to roughly fifty-three thousand dollars a year in salary alone, before any of the second-order costs: the on-call load, the review time, the context switching for whoever caught the incident.. The managed plan we were quoted came in at about a fifth of that. I want to be careful here, because a subscription cost is easy to compare and an engineering cost is easy to ignore, and that asymmetry is exactly why teams keep building this themselves.
The comparison only looks close when you leave the maintenance line off the sheet.. The ROI question answers itself once the ledger exists. What the spreadsheet could not hold was the opportunity cost, which is the thing that actually decided it: fourteen weeks of roadmap we did not get.
Martin Fowler’s split between utility and strategic software is the cleanest framing I know for this. Strategic software is where you win, so you build it. Utility software has to work, and building it yourself buys you nothing a customer will ever notice..
The migration sprint. We migrated to a managed file uploader over a single sprint. I should be honest about why one sprint was enough, which is that most of the work was deletion and the integration itself took a few days..
The checklist we evaluated against came from asking what features to look for in a reliable file upload service, and it stayed short. Chunked transfer with resume. Virus detection that runs inside the upload pipeline.
Direct-to-storage delivery. And a metadata model that mapped cleanly onto the one we already had.. Chunked transfer replaced the retry code, which is the single largest block on the ledger and the one we were proudest of.
Virus detection removed a whole service, not a file: the queue, the worker, the dead-letter handling and the runbook all went at once. Direct-to-storage transfer took upload bytes off our application servers entirely, and we watched that land in the load graphs the same week.. What we kept was small and specific.
Validation rules, because what our product accepts is a product decision. The metadata model, because how a file relates to everything else we store is the part that was ever really ours. About three hundred and twenty lines, all told..
Roughly 4,000 lines deleted. The 320 lines we kept were the ones about our product.. Where DIY still wins.
This does not generalize to everyone, and the piece would be dishonest if I let it read that way. There are teams for whom the ledger points the other direction, and the reasons are specific rather than sentimental.. Build it yourself when uploading is the product.
If transfer itself is your differentiation, that is strategic software by definition and you should own every line of it. Build it yourself for exotic transports as well: satellite links, industrial equipment, medical imaging gear that speaks its own dialect. And build it yourself for air-gapped deployments, where nothing leaves the network by design, which is the normal state of affairs in defense and parts of healthcare..
Those are real conditions and they belong in the decision. They were simply not our conditions. We had a product shaped like a CRM, where files were attachments to the records that mattered, and uploading was a step on the way to the thing customers came for..
Uploads were nobody’s product here. The ledger showed fourteen weeks. The spreadsheet showed a subscription at a fraction of it.
The sprint deleted about four thousand lines. And in the quarter that followed, on-call pages attributable to uploads went to zero, which is the number I did not think to predict and the one I would lead with now.. The point is not that our uploader was bad.
It was decent, and the people who built it were good at their jobs. The point is where those fourteen weeks went this year instead. They went into the part of the product customers actually ask us about, which is what they should have been doing the entire time..
Frequently asked questions. How much does maintaining a homegrown file uploader actually cost?. For us, fourteen engineer-weeks in one year, itemized as four weeks of retry and resume logic, three on the virus scanning queue, three on storage lifecycle bugs, two on mobile incidents, and two on routine upkeep..
What replaced the code that was deleted?. Chunked transfer replaced the retry and resume logic, virus detection inside the managed pipeline removed the scanning worker entirely, and direct-to-storage delivery took upload bytes off our application servers.. When does building in-house still make sense?.
When uploading is your product, when you need an exotic transport, or when you deploy air-gapped and nothing may leave the network. We had none of those conditions.. What would you do differently?.
Measure sooner. The ledger took an afternoon to put together, and we could have run it in year one instead of waiting until upload hardening reached a planning meeting for the third time.. Tags: trends
