rest.getAsStream()

rest.getAsStream()

Description

The getAsStream(String url) method allows you to execute an HTTP GET request and retrieve the response body as a stream. This is useful when working with large responses that should not be fully loaded into memory, such as large JSON payloads, text files, or streamed content.

The method returns a response object that provides access to the response content as an InputStream.

info

Always close both the InputStream and the response object to avoid resource leaks.

Parameters

Parameter	Type	Explanation
url	String	The full URL of the resource to retrieve.

Returns

A response object containing the HTTP response. Use getContent() to access the InputStream.

Example

The following example demonstrates how to use getAsStream to process a large response line by line.

url = "http://localhost:3000/d"

try {
    http = option.rest
    res  = http.getAsStream(url)

    // Stream the HTTP response to avoid loading the full payload into memory
    // Optional: provide a root node name to start streaming from (e.g. "results")

    stream = parser.asStream(res.getContent())

    // Prime the iterator before entering the loop
    resultMap = stream.next()
    i = 0

    // Stop when stream is exhausted or optional maxItems limit is reached
    while (resultMap && (!maxItems || i < maxItems)) {
        i++

        // Periodic progress logging for long-running imports
        if (i % 500 == 0) log.debug("${i} messages processed")

        try {
            // Process single record (business logic)
        } finally {
            // Ensure per-record cleanup even if processing fails
        }

        // Fetch next record only after current one has been fully processed
        resultMap = stream.next()
    }

    // No data returned by endpoint
    if (i == 0) {
        log.debug("No records found in response from ${url}")
    }
} catch (Exception e) {
    log.error("Exception processing records from ${url}: ${e.message}")
} finally {
    // Ensure HTTP stream is closed even on error
    try { res.close() } catch (Exception e) {}
}

When to Use getAsStream

Use getAsStream when:

• Downloading large responses
• Processing streaming data
• Reading large text or JSON files
• You want to avoid loading the entire response into memory

For smaller responses, consider using standard REST methods that return the response body directly.

Best Practices

• Always use a try-with-resources block for the InputStream.
• Always close the response object using res.close().
• Handle IOException properly.
• Avoid storing the entire stream content in memory unless necessary.