Skip to main content
Support Online
MirrorLooking Glass
RabisuRabisu
Client Panel
TutorialsKnowledgebase

BTCPay Server Setup and WHMCS Integration - Bitcoin and Lightning Network Payment Guide

This guide explains, step by step, how to install BTCPay Server on your own server, configure Bitcoin + Lightning Network, and integrate with WHMCS 8.x. It is a complete resource for building a zero-commission, non-custodial (your own wallet), fully open-source crypto payment infrastructure.

BTCPay Server is a self-hosted crypto payment processor. It has no third-party dependency, and payments go directly to your own wallet. It is used by large organizations such as Mozilla and DG Lab.

Source: GitHub - btcpayserver/btcpayserver-docker WHMCS Plugin: GitHub - btcpayserver/whmcs-plugin Official Documentation: docs.btcpayserver.org

  • CPU: 2 vCPU (minimum)
  • RAM: 2 GB (minimum), 4 GB recommended
  • Disk: 80 GB SSD minimum (pruned mode), 1TB+ for full node
  • IP: Static public IP
  • Domain: A subdomain such as pay.rabisu.com
  • Ports: 80 (HTTP), 443 (HTTPS), 9735 (Lightning Network), 22 (SSH)
  • WHMCS: v8.9+ (plugin tested with v8.13)
  • PHP: 8.1+ with curl, gd, intl, json, gmp (or bcmath), mbstring extensions

Step 1: VPS Preparation​

To do:

  • Provision a VPS (Rabisu VPS or your own infrastructure)
  • Set VPS hostname: sudo hostnamectl set-hostname pay.rabisu.com
  • Add to /etc/hosts: 127.0.0.1 pay.rabisu.com (for local hostname resolution)
  • Point the pay.rabisu.com A record to the VPS IP in DNS. IMPORTANT: If Cloudflare is used, the orange cloud (proxy) must be OFF and gray cloud (DNS only) must be selected. If proxy is enabled, Let's Encrypt SSL, WebSocket, and Lightning connections may fail.
  • Ensure SSH access to the server (root or sudo-enabled user)

Step 2: BTCPay Server Installation (Docker)​

BTCPay's official installation script is Docker Compose-based. Installation is done with one command:

# Required packages (run as root)
apt update && apt install -y git

# Clone BTCPay repository
cd ~
git clone https://github.com/btcpayserver/btcpayserver-docker
cd btcpayserver-docker

# Set environment variables FIRST, then run the script (must be in same shell)
export BTCPAY_HOST="pay.rabisu.com"
export NBITCOIN_NETWORK="mainnet"
export BTCPAYGEN_CRYPTO1="btc"
export BTCPAYGEN_LIGHTNING="clightning"
export BTCPAYGEN_REVERSEPROXY="nginx"
export BTCPAYGEN_ADDITIONAL_FRAGMENTS="opt-save-storage-s"

# Start installation (must start with dot, to source)
. ./btcpay-setup.sh -i

Important parameters:

  • opt-save-storage-s - Bitcoin pruned mode, keeps about 3 months of block data (enough for Lightning). Requires at least 80 GB disk.
  • SSL certificate is obtained automatically via Let's Encrypt
  • The script automatically installs Docker and Docker Compose, and configures auto-start after reboot

Initial sync: Bitcoin blockchain synchronization can take up to 72 hours even in pruned mode (official docs state 12-48 hours, but in practice we observed 2-3 days). The system is not usable during this period and you must wait. You can monitor progress from BTCPay Dashboard or with docker logs btcpaygen_bitcoind -f --tail 100.

Step 3: BTCPay Server Configuration​

After installation is complete, access https://pay.rabisu.com:

  1. Create admin account - The first registered user becomes admin, register immediately
  2. Enable 2FA - After login, enable it from Account in the left menu
  3. Create store - Create a store named "Rabisu"
  4. Preferred Price Source - Choose Kraken (reliable, supports many fiat currencies)
  5. Create wallet (On-chain):
  • Select Set up a wallet > Create a new wallet > Hot Wallet
  • Address type: Segwit (Recommended) - bc1q... addresses, lower fees, Lightning-compatible
  • PayJoin: Keep disabled (remove tick) - may create unnecessary complexity and customer compatibility issues
  • Advanced settings:
    • Passphrase (BIP39): Leave empty - if forgotten, wallet access can be permanently lost
    • Import keys to RPC: Enable (optional, for CLI balance checks)
  • Seed phrase: Write the generated 24 words on paper and store securely. Do not store digitally, do not take screenshots. These words are the only wallet recovery method.
  1. Lightning Node connection:
  • c-lightning is installed automatically during setup
  • Select Use internal node and click Save
  • Lightning Settings:
    • Display amounts in Satoshis: Off (customers should see BTC units)
    • Add hop hints for private channels: Off
    • Unify on-chain and lightning URL/QR code: Enable - A single QR supports both on-chain and Lightning, no forced user choice
    • Description template: Keep default (Paid to {StoreName} (Order ID: {OrderId}))
    • LNURL: Keep enabled (better wallet compatibility)
    • LNURL Classic Mode: Keep enabled
    • Allow payee to pass a comment: Keep enabled
  • Ride The Lightning (RTL): Lightning management panel installed automatically. Used for opening/closing channels and monitoring balances. Do not remove it; useful later.
  • NOTE: The red dot next to Lightning in the left menu means blockchain sync is not complete. It turns green automatically once sync finishes.
  1. Checkout Experience settings (Settings > Checkout Appearance):
  • Default payment method: BTC-CHAIN (since Unify is enabled, both methods appear together)
  • Enable payment methods only when amount is...: Leave BTC-LN and BTC-CHAIN rows empty (no minimum amount limit)
  • Timer before expiration: 5 minutes (default)
  • Celebrate payment with confetti: On (nice touch)
  • Enable sounds on checkout page: On (sound plays when payment arrives)
  • Show the store header: On
  • Show "Pay in wallet" button: On
  • Unify on-chain and lightning URL/QR code: On (already enabled in Lightning settings)
  • Display Lightning amounts in Satoshis: Off
  • Auto-detect language on checkout: On (Rabisu is multilingual, checkout auto-selects by browser language)
  • Default language: English
  • Support URL: https://www.rabisu.com/en/contact ("Contact Us" button appears on checkout page)
  • Only enable payment method after user chooses: Off
  • Redirect invoice to redirect url after paid: Off (WHMCS handles its own redirection)
  • Receipt settings: Keep all enabled (public receipt, payment list, QR code)

Step 4: WHMCS Module Installation​

Official BTCPay WHMCS module: btcpayserver/whmcs-plugin

Requirements:

  • WHMCS v8.9+ (plugin tested with v8.13)
  • PHP 8.1+
  • PHP extensions: curl, gd, intl, json, gmp (or bcmath), mbstring
  • Valid SSL certificate (on WHMCS server)

Installation steps:

  1. Download and extract the module:
  • Download latest version from GitHub Releases: https://github.com/btcpayserver/whmcs-plugin/releases
  • Extract ZIP, it will contain modules/gateways/btcpay/
  • Copy these files into WHMCS root, or move only btcpay directory under modules/gateways/
  • Verify: files should exist under PATH_TO_WHMCS/modules/gateways/btcpay/
  1. Generate Legacy API Key in BTCPay:
  • Log in to BTCPay Server
  • Select the relevant store
  • Go to Settings > Access Tokens from the left menu
  • Click Generate under "Legacy API Keys"
  • Copy the generated API Key ID string (example: 43rp4rpa24d6Bz4BR44j8zL44PrU4npVv4DtJA4Kb8)
  1. Activate and configure in WHMCS:
  • WHMCS Admin > Settings > Apps & Integrations
  • Search for "BTCPay" at top right, select "BTCPay Server (legacy API)"
  • Click Activate
  • Settings:
    • Show on Order Form: Must be enabled
    • Display Name: "Bitcoin / Lightning Network" (or preferred label)
    • Legacy API Key: Paste key copied in Step 2
    • BTCPay Server URL: https://pay.rabisu.com
    • Transaction Speed: "Medium" (1 block confirmation, ~10 min - recommended)
      • High: 0-conf (instant but unsafe, do not use)
      • Medium: 1 block (~10 min, recommended)
      • Low: 6 blocks (~1 hour)
  • Click Save Changes

Note: Webhook setup is not required - plugin manages it automatically.

Step 5: Testing​

IMPORTANT: You cannot receive payments before Bitcoin blockchain sync is complete. Sync in pruned mode can take up to 72 hours (2-3 days). You can monitor progress from BTCPay Dashboard. While sync is running, keep the payment method disabled in WHMCS (uncheck Show on Order Form); otherwise customers may try to pay and transactions will fail. Re-enable it after sync completes.

  1. Create test order - Create a test order in WHMCS
  2. Check payment page - Verify BTCPay checkout page opens
  3. Lightning QR - Verify Lightning invoice is displayed correctly
  4. Small real payment - Perform a real test with a small amount on mainnet
  5. Webhook validation - Confirm WHMCS invoice is automatically marked "Paid" after payment
  6. Service activation - Confirm WHMCS automatically activates the service
  7. Altcoin payment test - If Exolix is installed, select an altcoin from Altcoins button and test (minimum $10)

Optional: Altcoin Support with Exolix Plugin​

By default, BTCPay Server supports only Bitcoin (on-chain + Lightning). To let customers pay with altcoins like USDT, ETH, SOL, you can install the Exolix plugin.

What is Exolix?​

Exolix is a crypto swap service. The customer pays with an altcoin, Exolix automatically converts it to BTC and sends it to the BTCPay wallet. The merchant (you) always receives BTC.

Pricing​

Exolix does not charge a direct commission. It earns from exchange rate spread in swap transactions. This spread varies by coin and network congestion, typically around 1-2%. This cost is reflected to the customer; there is no extra fee for the merchant.

Minimum amount: Exolix requires around ~$10 minimum per swap. For payments below this amount, customers can use Bitcoin (on-chain or Lightning).

Installation​

  1. BTCPay Admin > Server Settings > Plugins
  2. Search for "Exolix", click Install
  3. Restart BTCPay Server (btcpay-restart.sh)
  4. Open Exolix page in store settings
  5. Enable Enabled
  6. Select coins you want to accept (Ctrl+click for multi-select):
  • Altcoins: Ethereum, Litecoin, Solana, Binance Coin, Tron, Polygon
  • Stablecoins USD: USDT Tether (Tron), USDT Tether (Binance), USDT Tether (Ethereum), USDC Circle (Solana), USDC Circle (Ethereum)
  • Stablecoins EUR: EURI Eurite (Ethereum), EURI Eurite (Binance)
  1. Click Save

No API key is required; plugin works out of the box.

Checkout Flow​

On checkout page, customer should first select Bitcoin payment method, then click Altcoins button. They select a coin from the dropdown and receive payment details. Once Exolix swap is completed, BTCPay invoice closes automatically.

Swap my BTC Balance (Optional)​

Exolix plugin also lets you convert accumulated BTC balance into altcoins (USDT, EUR stablecoins, etc.). From Exolix plugin page, enter balance percentage, target coin, and destination address to start swap.

Important Notes​

  • Invoice expiration should be at least 15 minutes. Altcoin transactions require confirmation time; short windows like 5 minutes may cause invoice timeout.
  • Exolix is a third-party service, not part of BTCPay itself. If Exolix is down, you can still receive payments via Bitcoin and Lightning.