Azure service bus queue vs storage queue vs Event Hub vs IoT Hub

Service Bus vs Storage Queue

• Service Bus is an enterprise scale message queuing system.  – up to 1 MB message size ; Storage Queue – 64KB message size
• Duplicate Detection – Only Service Bus – not Storage Queue

IoT Hub (also see Azure Stream Analytics)

2 way Iot Hub; One way – Event Hub (lower cost)

• Device Identity
• Cloud to Device messaging
• Telemetry
• Connections
• File Uploads
• Message routing

Data can be real time analyzed using stream analytics (also on IoT Edge)

Event Hubs –

Event Hubs represents the “front door” for an event pipeline, often called an event ingestor in solution architectures. An event ingestor is a component or service that sits between event publishers and event consumers to decouple the production of an event stream from the consumption of those events. Event Hubs provides a unified streaming platform with time retention buffer, decoupling event producers from event consumers.

1. Dedicated Model – Single Tenant deployment
2. Flex Model – Multi Tenant

• Ordering is maintained and throughput can be scaled (by scaling azure functions)
• Standard Namespace vs Basic vs dedicated
• Standard allows 1000 brokered connections (1 day retention)
• Standard allows 7 day retention, Dedicated 90 days 
• Big data pipeline
• Event streaming (series)
• Telemetry and distributed data streaming
• low latency
• capable of receiving and processing millions of events per second
• at least once delivery

Queues (Service Bus)

• Don’t care about ordering (session enabled queues do)
• Order processing and financial transactions
• reliable asynchronous message delivery (enterprise messaging as a service) that requires polling
• advanced messaging features like FIFO, batching/sessions, transactions, dead-lettering, temporal control, routing and filtering, and duplicate detection
• at least once delivery
• optional in-order delivery
• 7 day retention
• Messages are sent to and received from queues. Queues enable you to store messages until the receiving application is available to receive and process them.

Queues vs Topics

• You can also use topics to send and receive messages.
• While a queue is often used for point-to-point communication, topics are useful in publish/subscribe scenarios.

Event Grid

• The only other note I’ll make in regards to Event Grid is it is meant for processing events, not messages
• Reactive programming
• Event distribution (discrete) –  Azure Event Grid is the distribution fabric for discrete “business logic activity” events that stand alone and are valuable outside of a stream context. Because those events are not as strongly correlated and also don’t require processing in batches, the model for how those events are being dispatched for processing is very different.v
• React to status changes

 

Azure Time Series Insights – Spot anomalies in real time

Time Series Insights has four key jobs:

• It’s fully integrated with cloud gateways like Azure IoT Hub and Azure Event Hubs. It easily connects to these event sources and parses JSON from messages and structures that have data in clean rows and columns. It joins metadata with telemetry and indexes your data in a columnar store.
• Time Series Insights manages the storage of your data. To make sure that data is always easily accessible, it stores your data in memory and SSDs for up to 400 days. You can interactively query billions of events in seconds–on demand.
• Time Series Insights provides out-of-the-box visualization through the Time Series Insights explorer.
• Time Series Insights provides a query service, both in the Time Series Insights explorer and by using APIs that are easy to integrate to embed your time series data into custom applications.

Azure Logic App

Connectors and triggers

Connector – Dynamics CRM Conn, Twitter Connector, Biztalk connector

Triggers – e.g. Twitter Trigger, Http Trigger , Biztalk Server workflow trigger

Azure Queue (diff from Storage queue and Service Bus Queue) – Single Role Instance – Multithreaded Request Processing. Can connect multiple worker roles and web roles