playground
Learn the Git flow a data team ships with
A simulated terminal with a GitHub remote and two environments behind it: dev deploys to staging, main to production. Twelve levels take a dbt project from clone to hotfix: staging only what belongs, branching, local and remote branches, pull requests that must pass CI, conflicts, releases and rollbacks. Type real git commands and watch your laptop, GitHub and both environments change.
Trainee
0/36
3 more ★ to Junior data engineer
Level 1 of 12 · Your laptop
Clone the analytics repo
First day on the data team. The dbt project and the Airflow DAGs live on GitHub at git@github.com:acme/analytics.git. Get your own copy onto your laptop and have a look around.
Goals
- Clone the repo from GitHub (to do)
- Step into it with cd analytics (to do)
- See which branch you're on with git status (to do)
- Read the history with git log --oneline (to do)
The idea
git clone copies the whole history, not just the latest files, and names the GitHub copy origin. You end up with three things: the working tree (files you edit), your local repository (every commit, on your laptop), and remote-tracking branches like origin/main, which are your laptop's last-known snapshot of GitHub.
Using a hint gives up this level's no-hint star.
0 moves · par 1
Moves count git and gh commands that change something. Looking around is free: status, log, diff, show, branch.
zsh · ~
↑↓ history · Tab completes · Ctrl+L clears
Your laptop
Nothing here yet. git clone copies the repository from GitHub: every file, the full history and its branches.
GitHub
origin · acme/analyticsBranches on GitHub
- cd87e18devstagingAdd stg_payments
- 5b96c83mainv1.3.0prodAdd daily_orders DAG
- 2dae9d9Initial dbt project
Pull requests
No open pull requests. Push a branch, then open one here or with gh pr create.
Environments
Stagingruns dev · cd87e18
Healthy
Productionruns main · 5b96c83
Healthy
A simulation of recent Git, the GitHub CLI and a dbt CI pipeline. Output is abridged and the repository is fictional. Nothing you type leaves your browser; your stars are saved in this browser only.
the_flow
The whole flow, command by command
The daily loop first, then the release loop. It's the same sequence the levels teach, in the order you'd run it.
- 1
Start from the latest dev
git switch dev git pull
Branch from a stale dev and your pull request is out of date before anyone reads it.
- 2
One branch per change
git switch -c feature/daily-revenue
One model or one fix per branch keeps review short and a revert clean.
- 3
Commit on purpose
git status git diff git add models/marts/fct_daily_revenue.sql git commit -m "Add fct_daily_revenue"
Stage by path. git add . is how a profiles.yml with a password ends up on GitHub.
- 4
Run what CI runs, first
dbt build --select fct_daily_revenue+
The model you changed and everything downstream of it, on your own schema.
- 5
Push, then open a pull request
git push -u origin feature/daily-revenue gh pr create --base dev
Without --base, gh targets the repository's default branch, which is usually main.
- 6
Keep the branch current
git fetch git rebase origin/dev git push --force-with-lease
--force-with-lease refuses when a teammate has pushed to your branch; plain --force doesn't.
- 7
Squash-merge into dev, then look at staging
gh pr merge --squash git switch dev git pull --prune git branch -D feature/daily-revenue
dev deploys to staging. Check the numbers there, not only the checks.
- 8
Release dev to main with a merge commit
gh pr create --base main --head dev gh pr merge --merge git fetch git tag v1.4.0 origin/main git push origin v1.4.0
main gets exactly the commits staging ran, and the tag names what shipped.
- 9
Hotfix from main, then back into dev
git switch -c hotfix/revenue-fanout origin/main gh pr create --base main gh pr create --base dev --head main
Branching from main ships the fix without unreleased work. The back-merge stops the next release from undoing it.
branches_and_environments
Which branch goes where
Environments follow branches. dev and main are protected: nothing lands on them except through a pull request whose checks are green.
| Branch | What it is | Deploys to | Changes arrive as | Leaves as |
|---|---|---|---|---|
| feature/*, fix/* | One change, one owner | None. CI builds it in a throwaway schema | Your commits | Squash into dev |
| dev | Integration | Staging | Squash-merged pull requests | Merge commit into main, as a release |
| main | What production runs | Production | Release PRs from dev, hotfix PRs | Back-merged into dev after a hotfix |
| hotfix/* | An urgent production fix | None until merged | Branched from main | Squash into main |
| v1.4.0 (tag) | A name for a shipped commit | The one you roll back to | git tag, then git push origin v1.4.0 | Never moves |
data_rules
Git rules that are different for data work
The flow is the same as for any code. What goes into the repository isn't.
Credentials never enter Git
profiles.yml, .env files and service-account JSON go in .gitignore. Read secrets with env_var() in profiles.yml. If one was ever pushed, deleting the file isn't enough, because old commits still hold it. Rotate it.
Neither does data
A CSV you exported to debug usually carries customer emails, and Git keeps every version forever. Keep extracts in the warehouse or object storage, and ignore the whole data/ folder.
Generated files stay out
dbt's target/, dbt_packages/ and logs/, virtualenvs and notebook outputs change on every run, so they only ever produce conflicts. Strip notebook outputs before committing with nbstripout.
Reverting code doesn't restore data
git revert puts a dropped column back in the model, but tables dbt already rebuilt stay wrong until they're rebuilt again: a --full-refresh for incremental models, or a backfill.
Big files belong in object storage
GitHub rejects files over 100 MB and warns from 50 MB. Keep seeds to small lookup tables and load anything bigger from S3 or GCS.
Two different stagings
The staging area is the next commit you're assembling with git add. The staging environment is where dev deploys. Same word, no connection.
faq
Questions & answers
- What's the difference between git fetch and git pull?
- git fetch downloads new commits from GitHub and moves your remote-tracking branches, such as origin/dev. It never changes your own branches or files, so it's always safe to run. git pull is a fetch followed by a merge or rebase of your current branch onto its upstream. When the two have diverged, recent Git refuses to guess: pass --rebase or --no-rebase, or set pull.rebase once.
- What's the difference between a local branch and a remote branch?
- A local branch such as feature/daily-revenue lives on your laptop and moves when you commit. origin/feature/daily-revenue is a remote-tracking branch: your laptop's record of where that branch was on GitHub the last time you fetched, pulled or pushed. git push -u links the two, so git status can say you're ahead or behind, as of that last fetch.
- What is the staging area in Git?
- The staging area, or index, is the next commit being assembled. git add copies a file's current content into it, and git commit saves exactly what's staged. That's how you commit one fix while a scratch query and your credentials stay out. It has nothing to do with a staging environment, which is a place you deploy to.
- Should a data team squash or merge pull requests?
- Squash feature pull requests into dev, so each change is one commit that's easy to read and to revert. Merge release pull requests from dev into main with a merge commit, so main holds exactly the commits staging ran and the two branches keep a shared history. Squash a release and every later release pull request lists old changes again.
- What branching strategy works for a dbt project?
- A small Gitflow: short-lived feature branches off dev, dev deploying to a staging schema or warehouse, main deploying to production, releases as pull requests from dev into main, and hotfixes branched from main and merged back into dev. CI runs dbt build on every pull request, ideally only on the modified models and their children (state:modified+).
- How do I undo a commit that's already been pushed?
- On a shared branch like dev or main, use git revert with the commit. It adds a new commit that undoes the change, so nobody's history breaks. git reset --hard rewrites history, which is only safe for commits nobody else has, and a protected branch rejects the push anyway. For data, reverting a model doesn't rebuild the tables it already changed: rebuild or backfill them.
- Why does git branch -d say my branch isn't fully merged when the pull request was merged?
- Because it was squash-merged. dev received one new commit, not your branch's commits, so once the remote branch is deleted and pruned, Git can't prove your commits were merged and refuses. Check the pull request really did merge, then delete the branch with git branch -D.
- When should I use git push --force-with-lease?
- After you rebase or amend commits that are already on GitHub, a normal push is rejected because the history changed. --force-with-lease overwrites the remote branch only if it's still where you last saw it, so it won't silently throw away a commit a teammate pushed in the meantime. Plain --force would. Neither works on a protected branch.
- Is this real Git?
- No. It's a simulation that runs in your browser, built to match the behaviour and wording of recent Git and the GitHub CLI, with a fictional repository, CI and deploy pipeline. Output is abridged. Nothing you type leaves the page, and your stars are stored only in this browser.