About Hotrod
Hotrod is a general-purpose, distributed runtime for data engineering workloads. As a platform, Hotrod handles the ingestion, transformation and movement of data from sources to destinations.
Features
The key features of Hotrod is all about making data engineering easier and faster.
A Simple Mental Model
Data processing workloads and jobs are defined as Hotrod Pipes, which expresses inputs, transformations and outputs.
A Language For Data Engeering
Hotrod’s powerful Pipe Language provides a consistent, easy to learn method of defining data engineering workloads in a standardized manner, with advanced features like templating, contextual variables and central configuration.
Performant
Built with Rust, Hotrod let’s you get more done with less, especially when your data flows are measured in megabits per second.
General Purpose
With a variety of inputs & outputs, Hotrod supports reading and writing data to and from files, network sockets, API’s, databases, queues, and more.
Use Cases
Hotrod can be leveraged in a variety of data processing scenarios.
Ingest Data
Hotrod can act as an ingestion layer for incoming data streams to perform filtering, sampling, pre-processing and data volume reduction, and then egress data into your existing systems, storage (on-disk, AWS S3, Azure Blob Storage), or your favourite analytics solution like Elastic Search, Scuba or Splunk.
Connect & Bridge Data Flows
Hotrod can be deployed as a relay or a bridge that connects different systems, protocols and technologies. Use Hotrod as a bridge between HTTP services and Queues (AMQP, Kafka, NSQ), or turn static data sources (e.g. SQL sources) into data streams. Hotrod can even consume UDP and TCP data streams directly.
Collect Data
By co-locating Hotrod agents with other applications running on servers, Hotrod can collect and route metrics, logs and other data to different locations. Use Hotrod for log shipping, or to instrument systems that lack observability and telemetry mechanisms.
Perform Monitoring
Hotrod can compliment your existing monitoring systems and infrastructure. Use Hotrod to generate, collect and process data and monitoring events centrally, or at the edge.
Reduce Data Volume
Hotrod can be leveraged to reduce the volume of data flows. Hotrod’s runtime can perform sampling, aggregation, filtering and down-sizing on and within data streams, to enable data volume reduction.
Learn More
Hotrod’s Architecture
The Pipe Language
Download Hotrod