A typed failure model
Every fallible operation returns kotlin.Result. HttpFailure carries TransportFailure (nothing usable came back), UpstreamFailure (RateLimited, UpstreamError), BudgetExhausted, DecodeFailure and Closed.
Kotlin / JVM · Apache-2.0 · Maven Central
A small Kotlin/JVM collection of internet I/O helpers, published as independent, single-concern artifacts:
depend only on the one you need. Every fallible operation returns kotlin.Result rather than
throwing, so callers decide how to recover.
FeedFetcher.untrusted("my-app/1.0 (+https://example.com/contact)").use { fetcher ->
val stored: FeedValidators = loadValidators(feedId) // FeedValidators.None on the first poll
fetcher.fetch(feedUrl, stored).fold(
onSuccess = { fetch ->
saveValidators(feedId, fetch.validators) // after every response
fetch.permanentlyMovedTo?.let { moved -> considerMoving(feedId, moved) }
when (fetch) {
is FeedFetch.Fetched -> store(fetch.feed.entries) // new items: entries whose id you have not seen, never by date
is FeedFetch.NotModified -> Unit
}
},
onFailure = { failure -> report(failure) },
)
}
5xx are each returned onceEach artifact adds one concern, so you can take the parts you want and leave the rest.
Every fallible operation returns kotlin.Result. HttpFailure carries TransportFailure (nothing usable came back), UpstreamFailure (RateLimited, UpstreamError), BudgetExhausted, DecodeFailure and Closed.
UpstreamClient checks each call against the outbound-URL policy, then the response cache, then the call budget, then the transport. A hit costs nothing, and N concurrent callers of one key cost one permit and one request.
Trusted for an upstream you chose, Untrusted for a URL a third party supplied: lenient host forms, reserved names, denied addresses, other ports and sensitive headers are refused.
CaffeineResponseCache: one load per key at a time, per-entry lifetimes, a weight bound, a per-call maxAge, failures never stored. ResponseCache.Disabled stores and shares nothing.
WindowedCallBudget: epoch-aligned rate windows with a boundary guard, per-lane floors, first-come-first-served waits that fail fast or are refunded and re-planned, pauses from Retry-After, and a hard window that survives a restart through a BudgetStore.
RSS 0.90 to 2.0, RSS 1.0 and Atom 0.3 and 1.0 behind FeedParser. RomeFeedParser refuses a DOCTYPE before any parser runs and bounds depth, attributes, elements and namespaces; FeedFetcher does a stateless conditional GET and returns a 304 as a value.
MultiConnectionUDPServer multiplexes every client over one socket after a NAT-traversal handshake. UdpChannel gives each connection an UNRELIABLE or a RELIABLE_ORDERED delivery mode.
webtools-http's test fixtures (Gradle testFixtures(...)): FakeHttpTransport, VirtualClock, .http fixtures, a recorder, LoopbackFetchers and FakeBudgetStore.
Five artifacts, one concern each. Only webtools-feeds depends on another.
| Artifact | Responsibility |
|---|---|
webtools-http | The suspend HTTP client stack: request and response types, a typed failure model, the transport, cache, budget and outbound-URL policy ports with their supplied implementations, and the UpstreamClient and UrlFetcher facades. |
webtools-feeds | RSS and Atom behind a hardened parser port, the feed model with a frozen entry id, and FeedFetcher, a stateless conditional GET over UrlFetcher. Exposes webtools-http as an api dependency. |
webtools-udp | A multi-client UDP connection layer: one common port, a NAT-traversal handshake, framed datagrams, keepalive, a link-quality probe and a reliable-ordered channel. Depends on slf4j-api only. |
webtools-scraping | Connector: open a URL and read it line by line, plus one-shot get, skrape and image download helpers. Archived. |
webtools-browser | WebViewer: headless-browser screenshots through Selenium Manager. Archived. |
Read how each is built: the HTTP stack and feeds, the UDP components, threads and lifecycles and the UDP wire protocol.
Depend on the one artifact you need.
| Artifact | Coordinates | Version | Status | API |
|---|---|---|---|---|
webtools-udp | io.github.spartanlaboratories:webtools-udp | 2.0.1 | Stable Core | API |
webtools-http | io.github.spartanlaboratories:webtools-http | 1.0.0 | Stable Core | API |
webtools-feeds | io.github.spartanlaboratories:webtools-feeds | 1.0.0 | Stable Core | API |
webtools-scraping | io.github.spartanlaboratories:webtools-scraping | 1.0.0 | archived | API |
webtools-browser | io.github.spartanlaboratories:webtools-browser | 1.0.0 | archived | API |
Everything a consumer calls, passes, configures or matches is Stable Core. Implementing a seam of webtools-http or webtools-feeds (a HttpTransport, ResponseCache, CallBudget, FeedParser, ...) is Experimental and needs @OptIn(ExperimentalForInheritanceWebToolsApi::class).
Published to Maven Central. Requires JDK 11 or newer; built with Kotlin 2.2.
dependencies {
implementation("io.github.spartanlaboratories:webtools-udp:2.0.1")
// and/or
implementation("io.github.spartanlaboratories:webtools-http:1.0.0")
implementation("io.github.spartanlaboratories:webtools-feeds:1.0.0")
// tests only (Gradle): fakes, a virtual clock, .http fixtures and a recorder
testImplementation(testFixtures("io.github.spartanlaboratories:webtools-http:1.0.0"))
}
dependencies {
implementation 'io.github.spartanlaboratories:webtools-udp:2.0.1'
implementation 'io.github.spartanlaboratories:webtools-http:1.0.0'
implementation 'io.github.spartanlaboratories:webtools-feeds:1.0.0'
testImplementation testFixtures('io.github.spartanlaboratories:webtools-http:1.0.0')
}
<dependency>
<groupId>io.github.spartanlaboratories</groupId>
<artifactId>webtools-udp</artifactId>
<version>2.0.1</version>
</dependency>
<dependency>
<groupId>io.github.spartanlaboratories</groupId>
<artifactId>webtools-http-jvm</artifactId>
<version>1.0.0</version>
</dependency>
<dependency>
<groupId>io.github.spartanlaboratories</groupId>
<artifactId>webtools-feeds-jvm</artifactId>
<version>1.0.0</version>
</dependency>
webtools-http and webtools-feeds are Kotlin Multiplatform modules that publish webtools-http and webtools-http-jvm (likewise for feeds). A Gradle build resolves the right one itself; a Maven build depends on the -jvm artifact. testFixtures(...) is Gradle-only.
webtools-http's okhttp-jvm 5.5.0 dependency requires kotlin-stdlib 2.2.21, so a consumer's Kotlin standard library resolves to 2.2.21 or newer.
webtools-scraping and webtools-browser are archived. Their install lines and the Java 11 floor notes for their older releases are in the README.
A call to an upstream, a UDP session, and a test without a network.
val client = UpstreamClient(Url.parse("https://api.example.com/v2/").getOrThrow(), "my-app/1.0 (+https://example.com/contact)") {
headers = HttpHeaders { addSensitive("Authorization", "Bearer $token") } // default headers; credentials are never printed
budget(listOf(RateWindow("minute", 60, 1.minutes) { margin = 10 })) // built and owned by the client
}
// Declare each call once: a stable id, a decoder, how long to keep the answer, which status means "nothing there".
val player = Endpoint("player", ::decodePlayer) {
ttl = 10.minutes
misses(404)
}
val result: Result<Lookup<Player>> = client.call(player) {
path("players", id) // raw segments; each is encoded, so "a/b" stays one segment
query("region", "eu")
maxAge = 1.minutes // accept a cached answer no older than this
}
result.fold(
onSuccess = { lookup -> when (lookup) { is Lookup.Found -> show(lookup.value); Lookup.Missing -> showNotFound() } },
onFailure = { failure -> when (failure) {
is UpstreamFailure.RateLimited -> backOff(failure.retryAfter) // wait this long, then call
is HttpFailure.BudgetExhausted -> backOff(failure.wait)
else -> report(failure) // keep an else: new failure types may be added
} },
)
client.flush() // the exact final write of a budget with a persistent store
client.close() // closes what the client built (cache, budget, transport), never what it was handed
val client = MultiConnectionUDPClient(serverAddress)
client.handshake("alice").getOrThrow()
val unreliable = client.channel(DeliveryMode.UNRELIABLE)
unreliable.actuate { message -> /* handle inbound data; runs on client's dispatch thread */ }
unreliable.send("hello")
// client.start { ... } / client.send("hello") - the pre-2.0-channel spelling, deprecated but
// still fully functional; channel(UNRELIABLE) is the recommended replacement (see Migrating to 2.0)
client.startKeepAlive() // library times it; no caller timer needed
// client.sendKeepAlive() is still available for callers who want to drive the cadence themselves
// or, for a binary protocol:
unreliable.actuateBytes { bytes -> /* exact datagram body, no decode, no trim */ }
unreliable.send(byteArrayOf(0x01, 0x02, 0x03)) // any bytes, no reserved first byte needed
// ...
client.stop()
runTest {
val clock = virtualClock()
val fake = FakeHttpTransport(clock.timeSource).on(RequestMatch.get("/v2/players/7"), Step.respond(200, body = """{"name":"Ada"}"""))
UpstreamClient(fake, baseUrl) { timeSource = clock.timeSource; wallClock = clock.wallClock }.use { client ->
assertEquals(Lookup.Found(Player("Ada")), client.call(player) { path("players", "7") }.getOrThrow())
assertEquals(1, fake.calls.size)
}
}
Seven level tasks, each independently runnable so CI can gate them separately. Building needs a JDK 17 to 26; every JVM module compiles to Java 11 bytecode against the Java 11 API.
| Gradle task | Level | Scope |
|---|---|---|
gatingTest | 1 - Gating | Fast pre-commit checks |
componentTest | 2 - Component | Isolated component behaviour |
integrationTest | 3 - Integration | Real sockets and external interfaces |
deterministicTest | 4a - Deterministic | Pure input-to-output mappings |
e2eTest | 4b - End-to-end | Full-stack flows |
nonfunctionalTest | 4c - Non-functional | Robustness and security properties |
uatTest | 5 - Acceptance | Manual acceptance (mostly @Disabled) |
./gradlew build # compile + full test suite, every module
./gradlew :webtools-udp:build # one module only
./gradlew componentTest # just the isolated component tests
./gradlew verifyJavaFloor # the Java 11 floor of every published module
./gradlew :build-logic:test # build-logic's pre-commit gate
./gradlew dokkaGeneratePublicationHtml # generate this site's API reference
The full Dokka documentation for every published module, regenerated on each push to master.
Source, issues and pull requests.
Every release, in Keep a Changelog form.
The stack as built: components, call path, failure model, threads, stability tiers and known limitations.
Components, threads and lifecycles.
Every datagram's byte layout and the handling of malformed input.
Every published artifact and version.