A data pipeline is a series of automated steps that moves data from one or more sources to a destination while applying the processing needed along the way. It can collect, clean, transform, validate, and deliver data between systems, allowing information to move reliably without requiring each step to be performed manually.
For example, an e-commerce pipeline may capture orders from a website, validate and transform the records, then load them into a warehouse where dashboards and analytics tools can use them. Pipelines may run continuously or on scheduled intervals, depending on the application. Related concepts include ETL, ELT, data ingestion, data transformation, and workflow orchestration.
