> For the complete documentation index, see [llms.txt](https://docs.datasuite.vn/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.datasuite.vn/data-platform/index.md).

# ETL Developer Guide

Developer guide for getting started with DataSuite ETL workflows. This comprehensive documentation will guide you through understanding, setting up, and implementing ETL data pipelines using the DataSuite platform.

## What You'll Learn

* **System Architecture**: Understanding the complete DataSuite ETL ecosystem
* **Environment Setup**: Getting your local development environment running
* **Pipeline Development**: Building ETL workflows with LogStash and DBT
* **Testing & Validation**: Ensuring data quality and pipeline reliability

## Prerequisites

* Docker and Docker Compose installed
* Basic understanding of SQL and data concepts
* Familiarity with command line operations
* 8GB+ RAM recommended for full stack

## Navigation Guide

### 🏗️ Architecture & Design

Start here to understand the system before implementation.

* [**System Overview**](/data-platform/index/architecture/overview.md) - High-level architecture and data flow
* [**Component Details**](/data-platform/index/architecture/components.md) - Deep dive into each system component
* [**Data Flow**](/data-platform/index/architecture/data-flow.md) - How data moves through the pipeline

### ⚙️ Environment Setup

Get your development environment up and running.

* [**Prerequisites**](/data-platform/index/setup/prerequisites.md) - System requirements and dependencies
* [**Quick Start (Docker Compose)**](/data-platform/index/setup/docker-compose-setup.md) - Recommended for beginners
* [**Advanced Setup (Individual Containers)**](/data-platform/index/setup/individual-containers.md) - For custom configurations
* [**Service Verification**](/data-platform/index/setup/verification.md) - Confirm your installation is working

### 🛠️ Development Workflow

Build and customize your ETL pipelines.

* [**LogStash Configuration**](/data-platform/index/development/logstash-configuration.md) - Data ingestion setup
* [**DBT Getting Started**](/data-platform/index/development/dbt-getting-started.md) - Environment and project setup
* [**DBT Modeling Guide**](/data-platform/index/development/dbt-modeling-guide.md) - Creating dimensional models
* [**Testing & Validation**](/data-platform/index/development/testing-validation.md) - Data quality assurance

### 🚨 Troubleshooting

Resolve common issues and debug problems.

* [**Common Issues**](/data-platform/index/troubleshooting/common-issues.md) - Frequently encountered problems
* [**Debugging Guide**](/data-platform/index/troubleshooting/debugging-guide.md) - Systematic problem resolution

## Quick Start Path

**New to DataSuite?** Follow this recommended learning path:

1. 📖 Read [System Overview](/data-platform/index/architecture/overview.md) to understand the big picture
2. 🚀 Complete [Quick Start Setup](/data-platform/index/setup/docker-compose-setup.md) to get running fast
3. ✅ Run [Service Verification](/data-platform/index/setup/verification.md) to confirm everything works
4. 🔧 Follow [DBT Getting Started](/data-platform/index/development/dbt-getting-started.md) for your first pipeline
5. 📊 Build your first model with [DBT Modeling Guide](/data-platform/index/development/dbt-modeling-guide.md)

## Expected Outcomes

After completing this guide, you will be able to:

* ✅ Set up a complete DataSuite ETL development environment
* ✅ Configure LogStash for data ingestion from various sources
* ✅ Build dimensional data models using DBT
* ✅ Test and validate your data transformations
* ✅ Troubleshoot common pipeline issues

## Getting Help

* 🐛 **Issues**: Check [troubleshooting guides](https://github.com/FPT-Data-Suite/documents/blob/master/docs/troubleshooting/README.md) first
* 📚 **Documentation**: All guides include detailed examples and explanations
* 🔍 **Search**: Use your browser's search function to find specific topics

***

**Ready to get started?** Begin with the [System Overview](/data-platform/index/architecture/overview.md) to understand how all the pieces fit together.