Neon

Neon is a serverless open-source alternative to AWS Aurora Postgres. It separates storage and compute, substituting the PostgreSQL storage layer by redistributing data across a cluster of nodes.

Quick Start

Try the Neon Free Tier to create a serverless Postgres instance. Connect to it with your preferred Postgres client (psql, DBeaver, etc.) or use the online SQL Editor. See Connect from any application for connection instructions.

Alternatively, compile and run the project locally.

Architecture Overview

A Neon installation consists of compute nodes and the Neon storage engine. Compute nodes are stateless PostgreSQL nodes backed by the Neon storage engine.

Key Components

  • Pageserver: A scalable storage backend for the compute nodes.
  • Safekeepers: A redundant WAL service that receives WAL from the compute node and stores it durably until it has been processed by the pageserver and uploaded to cloud storage.

For more details, see the developer documentation in SUMMARY.md.

Running Local Installation

Installing Dependencies on Linux

Ubuntu or Debian

apt install build-essential libtool libreadline-dev zlib1g-dev flex bison libseccomp-dev \
libssl-dev clang pkg-config libpq-dev cmake postgresql-client protobuf-compiler \
libcurl4-openssl-dev openssl python3-poetry lsof libicu-dev

Fedora

dnf install flex bison readline-devel zlib-devel openssl-devel \
  libseccomp-devel perl clang cmake postgresql postgresql-contrib protobuf-compiler \
  protobuf-devel libcurl-devel openssl poetry lsof libicu-devel libpq-devel python3-devel \
  libffi-devel

Arch Based Systems

pacman -S base-devel readline zlib libseccomp openssl clang \
postgresql-libs cmake postgresql protobuf curl lsof

Building Neon requires version 3.15+ of protoc (protobuf-compiler). If your distribution provides an older version, you can install a newer version from here.

Installing Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Installing Dependencies on macOS

xcode-select --install
brew install protobuf openssl flex bison icu4c pkg-config
echo 'export PATH="$(brew --prefix openssl)/bin:$PATH"' >> ~/.zshrc

Install PostgreSQL Client

brew install libpq
brew link --force libpq

Building Neon

Linux

git clone --recursive https://github.com/neondatabase/neon.git
cd neon
make -j`nproc` -s

macOS

git clone --recursive https://github.com/neondatabase/neon.git
cd neon
make -j`sysctl -n hw.logicalcpu` -s

Running Neon Database

  1. Initialize and start the database:
cargo neon init
cargo neon start
  1. Create a tenant and set it as default:
cargo neon tenant create --set-default
  1. Create and start a PostgreSQL compute node:
cargo neon endpoint create main
cargo neon endpoint start main
  1. Connect to PostgreSQL and run some queries:
psql -p 55432 -h 127.0.0.1 -U cloud_admin postgres
  1. Create branches and run PostgreSQL on them:
cargo neon timeline branch --branch-name migration_check
cargo neon endpoint create migration_check --branch-name migration_check
cargo neon endpoint start migration_check

Stop all running instances:

cargo neon stop

For more advanced usages, refer to the Control Plane and Neon Local.

Running Tests

Rust Unit Tests

Install cargo-nextest:

cargo install cargo-nextest

Run tests:

cargo nextest run

Integration Tests

Ensure your dependencies are installed. Clone the repo and build with the testing feature:

git clone --recursive https://github.com/neondatabase/neon.git
CARGO_BUILD_FLAGS="--features=testing" make
./scripts/pytest

Run a specific set of tests:

DEFAULT_PG_VERSION=15 BUILD_TYPE=release ./scripts/pytest

Flamegraphs

Generate flamegraphs using flamegraph-rs or the original flamegraph.pl. Note that if you're using lld or mold, you need the --no-rosegment linker argument.

Cleanup

Clean the source tree from build artifacts:

make clean

Remove all artifacts from build and configure steps:

make distclean

Documentation

The docs directory contains an overview of all available markdown documentation. Use cargo doc --no-deps --open to view rustdoc documentation in a browser.

Other resources:

Postgres-Specific Terms

Neon's close relation with PostgreSQL internals necessitates using specific terms. For more details, refer to:

Join the Development

  • Read CONTRIBUTING.md for project code style and practices.
  • Use sourcetree.md to get familiar with the source tree layout.
  • Learn more about PostgreSQL internals here.

Similar Projects

Rustfinity.com

Links

  1. Home
  2. Learn Rust
  3. Get Started
  4. Practice Rust
  5. Challenges
  6. Tutorials
  7. Blog
  8. Open source
  9. Learn Gleam

Socials

  1. GitHub
  2. X

Legal

  1. Privacy Policy
  2. Terms of Service