When I first scoped RSS support for Loreo, the plan was simple: paste in a feed URL, see what's currently available, and let the user pick which items go into their library
Structurally, that looked like CSV import: bring in a batch of known content, decide once, and finish
The assumption was reasonable because imports were already a solved problem in Loreo. But it was still the wrong model for what the feature was becoming. A CSV file is a fixed collection. An RSS feed is a subscription to content that will keep arriving
The feature eventually became a review surface grouped by source. Users choose their own feeds, and Loreo keeps collecting new items from them. Pocket's old Explore feed gave me a useful mental model for an ongoing stream, but Loreo's version isn't app-curated. It's built entirely from sources the user selected
Once that became clear, I had to revisit nearly every assumption in the one-shot plan
What actually changed
A CSV import is bounded. You upload a file, Loreo reads every row, and when the last row is processed, the import is over. There's a clear "done" state. The content is fixed when you upload it
The original RSS plan inherited that shape without me deciding to inherit it: paste a URL, see a fixed list of what's currently there, choose, and finish. It quietly treats a feed like a file
A feed has no last row. After the user adds a URL, Loreo has to keep checking it while the subscription remains active. The publisher might write once a month or publish fifty items in an hour. Loreo doesn't control the schedule or know the future contents in advance
Imports are bounded batches of known content. Feeds are open-ended streams of content arriving on someone else's schedule
The difference isn't just conceptual. It changes the jobs, storage, and user experience around ingestion
Why the distinction matters mechanically
There's no finished state. An import job completes. A feed subscription is polled while it remains active. That means recurring scheduling, manual refreshes, failure handling, and backoff. The system also has to behave sensibly when the server restarts or a feed stays quiet for months before publishing again
Arrival rate and volume are unknown. A file has a finite number of rows. A feed can publish one item a month or fifty in an hour, and Loreo finds out only as it polls. Deduplication prevents repeated entries, backoff limits work against failing feeds, and retention keeps stored feed history bounded. Loreo's current boundary is 90 days and the latest 500 items per feed
The user can't preview the future. When someone imports a file, they've chosen a fixed input. A feed exposes them to content that doesn't exist yet. Loreo stages the newest 50 items when a subscription is created, giving the user enough context to decide what they want without dumping the feed's entire history into Review
These behaviors become necessary once the source keeps producing content
The design choices that follow
Once I treated feeds as ongoing subscriptions, several PR decisions stopped looking like arbitrary UI choices
Review is a staging area separate from the library. New feed items land in Review, separate from Saved. Choosing an exported collection already expresses the user's intent to migrate those links. Subscribing to a source doesn't express the same intent for every article it publishes. A feed item arrived because the user subscribed to its source, but that doesn't necessarily mean the individual article belongs in the library
The default path is:
feed poll → staged item → user saves or dismisses it
Auto-save applies only to future items. Turning on Auto-save for a feed leaves the items already waiting in Review alone. Auto-save is a decision about content that hasn't arrived yet. Applying it to the existing backlog would treat those items as if the user had already made that decision
Initial staging and retention impose boundaries. A new subscription stages the newest 50 items. After that, Loreo keeps only the latest 500 items from a feed and removes items older than 90 days. Those limits prevent a prolific or long-running feed from flooding Review or growing without limit in the database
Dismissed identities stay dismissed. This matters because the feed keeps polling. Loreo needs to remember that the user dismissed an item so entries returned by later polls don't keep reappearing in Review. An import ends, so it doesn't need the same long-lived memory of past decisions
The operational work around the stream
The recurring model also changes how Loreo talks to feeds
Users provide feed URLs, so Loreo applies its public-URL safety rules when fetching them. RSS and Atom feeds also need to be normalized into the same internal feed-item shape. Poll failures can't stop the whole system, so subscriptions back off after errors and become eligible for another attempt later
Polling runs through a queue and worker rather than inside the request that adds a feed. That keeps adding a subscription responsive and lets scheduled refreshes, manual refreshes, retries, and graceful shutdown share one ingestion path
These details are less visible than the Review UI, but they're part of the same design problem. A recurring source needs a system that can keep returning to it safely
The place the two models still meet
Here's the part I think is the actual payoff: despite their different input models, saving a feed item and saving through CSV import use the same link-save layer:
ingestion → duplicate-aware link save → saved link → extraction when needed
A feed item can reuse an article that already exists in the reader. A new link can enter the extraction queue without creating a duplicate saved link
The source lifecycles remain different:
- CSV is bounded and one-shot
- Feeds are recurring and open-ended
- CSV content is selected as a batch
- Feed items arrive individually and usually need review
The save path doesn't need to be different. One path can serve several ingestion sources without forcing those sources into the same lifecycle
That's the architectural boundary that mattered here: share the behavior that should be shared, and keep the recurring and one-time parts separate
The general lesson
This is a "recurring vs one-time" problem, not only an RSS problem. It appears in RSS subscriptions, API polling, and sync jobs against third-party services. Webhooks have different mechanics, but they raise a similar question about whether incoming data continues after the initial setup
The question to ask early is:
Am I handling a bounded batch, or subscribing to an open-ended source that will keep producing content?
Naming that distinction before writing the ingestion code makes decisions about staging, retention, retries, and deduplication much easier. It's cheaper than building a one-shot importer and retrofitting the system after discovering that the source never stops