HackDay: Bridging the Gap - Building Synchronous IoT APIs with AWS Durable Lambda

At base2Services, we live and breathe cloud infrastructure. While our daily operations focus on stability and scalability, our HackDays are the perfect excuse to push boundaries and get our hands dirty with the very latest AWS releases. For our most recent session, we set our sights on a challenge that has frustrated developers for years: building a truly synchronous API on top of an asynchronous, event-driven architecture.

Our tool of choice? The recently released AWS Durable Lambda functions.

The Challenge: Sync vs. Async

In the world of IoT, communication is almost exclusively asynchronous. You send a command to a device via MQTT, and eventually, the device reports back. However, modern web applications often demand a synchronous RESTful experience -  the user clicks a button, waits a moment, and expects an immediate confirmation that the action was completed on the device.

Historically, bridging this gap involved complex workarounds:

• Polling: The client keeps asking, "Is it done yet?" (Inefficient and slow).
• WebSockets: Great for real-time, but heavy to implement for simple request-response cycles.
• Step Functions: Powerful, but can introduce latency and cost overhead for high-volume, low-latency transactions.

We wanted to see if the new AWS Durable Lambda  feature could solve this architecture problem without the polling overhead.

The Experiment: "Pause and Resume"

Our goal was to create a solution where an Amazon API Gateway request could trigger a process that communicates with a physical device and returns a response in a single HTTP transaction - without the Lambda function spinning idly or polling a database.

We built a Proof of Concept (PoC) leveraging AWS IoT Core and the new AWS Durable Lambda capabilities. Here is the flow we designed:

1. The Trigger: A user invokes an Amazon API Gateway HTTP endpoint.
2. The Pause: Amazon API Gateway triggers an AWS Durable Lambda. This function publishes a message to an IoT device via MQTT using AWS IoT Core. Crucially, instead of finishing, the Lambda pauses execution and creates a callback token.
3. The Action: The IoT device (located in Melbourne) receives the message, runs a local query, and publishes the result back to a response topic in AWS IoT Core.
4. The Resume: AWS IoT Core triggers a secondary "Callback Lambda." This function takes the device response and uses the token to resume the original AWS Durable Lambda.
5. The Response: The original AWS Durable Lambda wakes up exactly where it left off and returns the device's data as a standard HTTP response to the Amazon API Gateway.

The Results: Sub-Second Latency

The theory was sound, but we were sceptical about the performance. Could a request travel from a user to the cloud, to a physical device, and back through a paused function fast enough for a web API?

The results were impressive.

Testing a round trip from a user in Melbourne, connecting to our API in Sydney, which communicated with a device back in Melbourne, we achieved latency of less than 1 second.

Key Takeaway: We achieved a fully event-driven architecture with zero polling. The Lambda only runs when it is processing or resuming, making it highly cost-effective, while the user gets the instant feedback of a synchronous web request.

What We Learned

This HackDay proved that AWS Durable Lambdas are a game-changer for "Device-in-the-loop" or "Human-in-the-loop" workflows.

By abstracting away the state management that usually requires Amazon DynamoDB or Step Functions, we were able to write code that looks synchronous but behaves asynchronously. This significantly reduces the complexity of IoT APIs and opens up new possibilities for real-time device interaction.

We validated that you can indeed have the best of both worlds: the reliability of event-driven architecture and the simplicity of RESTful APIs.

Contact us today to discuss modernising your cloud infrastructure with the latest AWS serverless capabilities.