Go Build Optimization: A Guide to Progress and Elapsed Time Statistics for Large Projects

When building large Go projects,go build Default silent output often makes people uneasy—is it working properly, or is it stuck? How long will the build take? How can you ensure that the project's local dependencies are used?

This article will provide you with a complete solution covering progress tracking, time-consumption statistics, Vendor management, and reusable automation scripts.

• Why should we care about the build process?
• 1. Basics: Check build progress
  • 1.1 Show the package currently being compiled
  • 1.2 Display detailed execution commands
  • 1.3 Combined Use
• II. Core: Using the Vendor Directory
  • 2.1 Generate/Update Vendor
  • 2.2 Mandatory Use of Vendor Build
  • 2.3 Verify Whether the Vendor Is Effective
  • 2.4 Advantages of the Vendor Model
• III. Advanced: Add build duration statistics
  • 3.1 Usage time Command (simplest)
  • 3.2 One-line command version
  • 3.3 Script for Packet Count Statistics
• IV. Ultimate Solution: Makefile Encapsulation
• V. Incremental Build Mechanism of Go Compilation
  • 5.1 Why is the first compilation slow?
  • 5.2 Reasons for slow first compilation
  • 5.3 How Incremental Compilation Works
  • 5.4 Practical Demonstration
  • 5.5 Accelerating Builds with Caching
  • 5.6 Force rebuild all packages
  • 5.7 Compilation Time Optimization Suggestions
  • 5.8 Important Notes
  • 5.9 Verify Incremental Compilation Effects
• VI. Hands-on: Complete Build Command
  • 6.1 Recommended for daily use
  • 6.2 With Complete Statistical Information
  • 6.3 Force rebuild all packages
  • 6.4 Practical Project Example
• VII. Troubleshooting Common Issues
  • Q1: Message "vendor directory not found"
  • Q2: Dependencies are still downloaded during the build
  • Q3: Slow build speed
  • Q4: How to view the dependency graph
  • Q5: Out of memory during compilation
• VIII. Summary of Best Practices
  • Key points
  • Quick reference card
• 9. CI/CD Integration Example
  • GitHub Actions
  • GitLab CI
  • Docker build optimization
• 10. Summary

Why should we care about the build process?

pain point	Solution
I don't know the build progress.	Use -v Show the package being compiled
Not sure about the build time.	Add a time statistics script
Downloading is slow due to network dependency.	Use -mod=vendor Fixed dependency
Cannot determine whether it is stuck.	实时输出编译详情
Repetitive work is inefficient.	Package as a Makefile or script
The first compilation is slow; subsequent compilations are fast.	Understand the incremental compilation mechanism

1. Basics: Check build progress

1.1 Show the package currently being compiled

go build -v -o myapp main.go

Output example:

cmd/internal/objfile
cmd/internal/src
cmd/internal/sys
runtime
encoding/binary
...
myapp

1.2 Display detailed execution commands

go build -x -o myapp main.go

-x It will output all executed compilation and linking commands, suitable for in-depth debugging.

1.3 Combined Use

go build -v -x -o myapp main.go

II. Core: Using the Vendor Directory

The vendor directory pins dependencies within the project, ensuring builds do not rely on the network and improving stability and build speed.

2.1 Generate/Update Vendor

go mod vendor

2.2 Mandatory Use of Vendor Build

go build -mod=vendor -v -o myapp main.go

2.3 Verify Whether the Vendor Is Effective

# View current module mode
go list -m

# Check whether the vendor directory exists
ls -la vendor/

2.4 Advantages of the Vendor Model

Characteristics	Explanation
Offline build	No network connection required
Dependency pinning	Dependencies will not be upgraded unexpectedly
CI/CD friendly	Predictable build results
Accelerate the building	Save time on downloading dependencies

III. Advanced: Add build duration statistics

3.1 Usage time Command (simplest)

time go build -mod=vendor -v -o myapp main.go

Output:

real    0m5.234s
user    0m4.123s
sys     0m0.456s

3.2 One-line command version

echo "Start: $(date)"; time go build -mod=vendor -v -o myapp main.go; echo "End: $(date)"

3.3 Script for Packet Count Statistics

Save as build.sh:

#!/bin/bash

START_TIME=$(date +"%Y-%m-%d %H:%M:%S")
START_TS=$(date +%s)

echo "========================================="
echo "🚀 Starting build: $START_TIME"
echo "========================================="

TEMP_FILE=$(mktemp)

go build -mod=vendor -v -o myapp main.go 2>&1 | tee $TEMP_FILE
BUILD_EXIT=${PIPESTATUS[0]}

END_TIME=$(date +"%Y-%m-%d %H:%M:%S")
END_TS=$(date +%s)
DURATION=$((END_TS - START_TS))

PACKAGES=$(grep -c "^[a-z]" $TEMP_FILE 2>/dev/null || echo "0")

echo ""
echo "========================================="
if [ $BUILD_EXIT -eq 0 ]; then
    echo "✅ Build successful!"
    echo "📦 Packages compiled: $PACKAGES"
    echo "📁 File size: $(ls -lh myapp 2>/dev/null | awk '{print $5}')"
else
    echo "❌ Build failed!"
fi
echo "⏱️  Start time: $START_TIME"
echo "⏱️  End time: $END_TIME"
echo "⏱️  Total time: $((DURATION / 60)) min $((DURATION % 60)) sec ($DURATION sec)"
echo "========================================="

rm -f $TEMP_FILE
exit $BUILD_EXIT

IV. Ultimate Solution: Makefile Encapsulation

Create Makefile:

# Variable definition
BINARY_NAME := myapp
BUILD_TIME := $(shell date +%Y%m%d-%H%M)
BINARY := $(BINARY_NAME)-$(BUILD_TIME)

# Color output
RED := \033[0;31m
GREEN := \033[0;32m
YELLOW := \033[0;33m
NC := \033[0m # No Color

.PHONY: build
build: vendor
	@echo "========================================="
	@echo "$(GREEN)🚀 Starting build: $(shell date '+%Y-%m-%d %H:%M:%S')$(NC)"
	@echo "========================================="
	@time go build -mod=vendor -v -o $(BINARY) main.go
	@echo "========================================="
	@echo "$(GREEN)✅ Build complete: $(BINARY)$(NC)"
	@echo "📁 File size: $$(ls -lh $(BINARY) | awk '{print $$5}')"
	@echo "========================================="

.PHONY: vendor
vendor:
	@echo "$(YELLOW)📦 Check/Update vendor...$(NC)"
	@go mod vendor -v

.PHONY: clean
clean:
	@echo "$(YELLOW)🧹 Cleaning build artifacts...$(NC)"
	rm -f $(BINARY_NAME)-*

.PHONY: test
test:
	@echo "$(YELLOW)🧪 Running tests...$(NC)"
	go test -v ./...

.PHONY: help
help:
	@echo "Available commands:"
	@echo "  make build  - Build the project (automatically update vendor)"
	@echo "  make vendor - Update vendor directory"
	@echo "  make clean  - Clean build artifacts"
	@echo "  make test   - Run tests"

Usage:

make build   # Build the project
make clean   # Clean artifacts
make help    # View help

V. Incremental Build Mechanism of Go Compilation

5.1 Why is the first compilation slow?

The Go compiler has Intelligent incremental build Ability:

Scene	Behavior	Time-consuming
First compilation	Compile all dependency packages	较长（可能数分钟）
Modify a single file	Only recompile this package and the packages that depend on it.	Extremely fast (seconds-level)
Modify main.go	Recompile only the main package	almost instantly
Modify the underlying common package	Recompile all packages that depend on this package.	Relatively long
Modify vendor dependencies	Recompile the related packages	Medium

5.2 Reasons for slow first compilation

• Compile all dependencies: On the first run, all packages that the project depends on need to be compiled (including the standard library and third-party libraries).

• No cache available:$GOPATH/pkg/ 下没有预编译的 .a File

• CGO compilation: If CGO is enabled, the first build may require compiling C code.

5.3 How Incremental Compilation Works

5.4 Practical Demonstration

# First compilation - slower
$ time go build -mod=vendor -v -o myapp main.go
...
real    0m12.345s  # It may take 10+ seconds the first time.

# Compile again after modifying main.go - extremely fast
$ time go build -mod=vendor -v -o myapp main.go
command-line-arguments  # Compile only the main package.
real    0m0.856s      # Less than 1 second!

# Compile after modifying the internal package - Medium
$ time go build -mod=vendor -v -o myapp main.go
yanrong.com/krypton-api/internal/api/hosts  # Compile only the modified package
yanrong.com/krypton-api/internal/router     # as well as the packages that depend on it
command-line-arguments
real    0m1.234s

5.5 Accelerating Builds with Caching

Go will cache compilation results to $GOPATH/pkg/ Table of Contents:

# View cache location
go env GOCACHE

# View cache statistics
go clean -cache -n

# Clear cache (if needed)
go clean -cache

5.6 Force rebuild all packages

When a complete rebuild is required (for example, if cache corruption is suspected):

# Force rebuild all packages
go build -a -mod=vendor -v -o myapp main.go

5.7 Compilation Time Optimization Suggestions

Suggestion	Description
Keep the development environment stable.	Avoid frequent cache clearing
Use -mod=vendor	Reduce dependency download time
Rational package division	Put frequently modified code in a separate package.
Use incremental compilation	In daily development, frequent builds will get faster and faster.
Use go install	Install to $GOPATH/bin, the next build will be faster

5.8 Important Notes

💡Key insight:

• The first compilation takes a long time, which is Normal phenomenon No need to worry

• The subsequent compilation will significantly accelerate, because Go only recompiles changed code

• If you feel the build is slowing down, you can use go build -v Check which packages were specifically compiled.

• In a CI/CD environment, it is recommended Preserve build cache to speed up subsequent builds

• If you clone the project from Git and build it immediately, it is equivalent to a "first build" and will be relatively slow.

5.9 Verify Incremental Compilation Effects

# 1. First build (record time)
time go build -mod=vendor -v -o myapp main.go

# 2. Make no changes and build again (it should be faster)
time go build -mod=vendor -v -o myapp main.go
# At this point there should be almost no output, because all packages are already cached.

# 3. Build after modifying a file
echo "// comment" >> main.go
time go build -mod=vendor -v -o myapp main.go
# Compile only the modified package

VI. Hands-on: Complete Build Command

6.1 Recommended for daily use

go build -mod=vendor -v -o myapp main.go

6.2 With Complete Statistical Information

START=$(date +%s); echo "Start: $(date)"; time go build -mod=vendor -v -o myapp main.go; END=$(date +%s); echo "End: $(date)"; echo "Total elapsed: $((END-START)) seconds"

6.3 Force rebuild all packages

go build -a -mod=vendor -v -o myapp main.go

6.4 Practical Project Example

# Take the krypton-api project as an example.
TIMESTAMP=$(date +%Y%m%d-%H%M)
echo "Starting build: $(date '+%Y-%m-%d %H:%M:%S')"
time go build -mod=vendor -v -o krypton-api-${TIMESTAMP} main.go
echo "Build complete: $(date '+%Y-%m-%d %H:%M:%S')"

Output example:

yanrong.com/krypton-api/internal/api/hosts
yanrong.com/krypton-api/internal/router
command-line-arguments

real    0m2.345s
user    0m1.876s
sys     0m0.234s

VII. Troubleshooting Common Issues

Q1: Message "vendor directory not found"

go mod vendor

Q2: Dependencies are still downloaded during the build

Confirm whether to use -mod=vendor, and check GOFLAGS Environment variables:

go env GOFLAGS
# Ensure that no other mode is forcibly used.

Q3: Slow build speed

cause	Solution
CGO compilation is slow	Settings CGO_ENABLED=0
First build	This is normal; it will speed up later.
Heavy reliance	Use vendor Reduce network overhead
Disk I/O is slow	Use -buildvcs=false Skip version control
Cache has been cleared	Avoid frequent execution go clean -cache

Q4: How to view the dependency graph

go mod graph | head -20

Q5: Out of memory during compilation

# Limit the number of parallel compilations
go build -p 4 -mod=vendor -v -o myapp main.go

VIII. Summary of Best Practices

Key points

• Always use -mod=vendor: Ensure a consistent build environment

• Add -v Sign: Real-time visibility into build progress

• Measure build time: Facilitates performance analysis and optimization

• Encapsulate as a script/Makefile: Avoid duplicate input

• Regularly update vendor:go mod vendor -v

• Understanding Incremental Compilation: It's normal to be slow the first time; it will be fast afterward.

• Keep build cache Do not clean up arbitrarily.$GOPATH/pkg/

Quick reference card

Requirement	Command
Daily development	go build -mod=vendor -v
View elapsed time	time go build -mod=vendor -v
Force rebuild	go build -a -mod=vendor -v
Generate vendor	go mod vendor
Clear cache	go clean -cache
View cache location	go env GOCACHE

9. CI/CD Integration Example

GitHub Actions

name: Build

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Go
        uses: actions/setup-go@v4
        with:
          go-version: '1.21'
          cache: true  # Enable cache acceleration

      - name: Vendor dependencies
        run: go mod vendor

      - name: Build
        run: |
          echo "Starting build: $(date)"
          time go build -mod=vendor -v -o myapp main.go
          echo "Build complete: $(date)"

      - name: Upload artifact
        uses: actions/upload-artifact@v3
        with:
          name: myapp
          path: myapp

GitLab CI

image: golang:1.21

variables:
  GOCACHE: $CI_PROJECT_DIR/.cache/go-build

cache:
  paths:
    - .cache/go-build/  # Cache compilation artifacts to accelerate builds
    - vendor/

before_script:
  - go mod vendor

build:
  stage: build
  script:
    - echo "Starting build: $(date)"
    - time go build -mod=vendor -v -o myapp main.go
    - echo "Build complete: $(date)"
  artifacts:
    paths:
      - myapp

Docker build optimization

# Use Docker layer caching to speed up builds
FROM golang:1.21 AS builder

WORKDIR /app

# First copy the dependency files (using cache)
COPY go.mod go.sum ./
RUN go mod download

# Copy vendor (if used)
COPY vendor/ vendor/

# Copy the source code and build
COPY . .
RUN time go build -mod=vendor -v -o myapp main.go

# final image
FROM alpine:latest
COPY --from=builder /app/myapp /usr/local/bin/
CMD ["myapp"]

10. Summary

With the above approach, you can fully control the build process of large Go projects:

• ✅Progress visualization: Use -v Track compilation progress in real time

• ✅Time consumption is quantifiable: Through time and precise statistics for the script

• ✅Dependency controllability: Use -mod=vendor Solidified dependency

• ✅Build optimizable: Understand the incremental compilation mechanism and make full use of caching

• ✅Process automation: Encapsulate as a Makefile or script, and integrate with CI/CD

No longer worry about "whether it's stuck" and "how long it takes"—make the Go build process clearly visible! 🚀

Related Resources:

• Go Build Official Documentation

• Go Modules Official Documentation

• Go Build Cache Description