build: pre-merge audit fixes for reproducibility work
Bugs / inconsistencies found while reviewing the branch for merge: - dependenciesInfo comment falsely claimed Play "still derives this data server-side, nothing is lost." Not true: includeInBundle=false means the .aab carries no dependency metadata, so Play Console's dependency-insights / SDK-vulnerability alerts go unpopulated (uploads still succeed). Corrected the comment and the BUILDING.md framing (it called the blob "the one remaining blocker" when the Arti .so was the bigger one). - Version-bump workflow was broken: the README told you to run `build-arti.sh --clean` to refresh Cargo.lock, but the build is now --locked (fails on a stale lock) and the clone moved to the canonical /tmp path. Added a dedicated `--regen-lock` mode (clone + cargo generate-lockfile, no NDK needed) and pointed the docs at it. Verified it reproduces the committed lock byte-for-byte. - verify-reproducible.sh: new helper that builds twice and diffs to prove byte-for-byte reproducibility; uses portable sha256 (sha256sum/shasum) and plain `sort` so it runs on macOS too. - README verify recipe referenced paths that only resolved from the repo root while telling you to cd into tools/arti-build — replaced with the helper. - rust-toolchain.toml listed four Android targets but only two ABIs ship a .so; trimmed to match (check_prerequisites adds any other on the fly). - BUILDING.md: documented that the bundled Arti .so is reproducible-from-source and that secp256k1/webrtc are version-pinned Maven prebuilts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JtjUcSjjpu4auFndw1QKeU
This commit is contained in:
Claude
+7
-1
@@ -184,7 +184,13 @@ else. What makes that hold:
|
||||
- **No dependency-metadata blob.** `dependenciesInfo { includeInApk = false;
|
||||
includeInBundle = false }` in `amethyst/build.gradle.kts` stops AGP from
|
||||
embedding the Google-encrypted dependency protobuf in the signing block — that
|
||||
ciphertext is non-deterministic and was the one remaining blocker.
|
||||
ciphertext is non-deterministic.
|
||||
- **Reproducible native library.** The bundled Tor (Arti) `.so` is the one
|
||||
binary we compile ourselves; it is built reproducibly from source (pinned Rust
|
||||
toolchain, locked deps, canonical build path). See
|
||||
[`tools/arti-build/README.md`](tools/arti-build/README.md) → "Reproducible
|
||||
builds". All other native libs (`secp256k1`, `webrtc`) are version-pinned Maven
|
||||
prebuilts and so are byte-identical by download.
|
||||
|
||||
### Verify a release APK reproduces
|
||||
|
||||
|
||||
@@ -268,9 +268,14 @@ android {
|
||||
// in the APK/AAB. That blob is a protobuf of the resolved dependency tree
|
||||
// encrypted with a Google public key; the ciphertext is non-deterministic,
|
||||
// so its presence makes every release artifact impossible to reproduce
|
||||
// bit-for-bit. Dropping it lets F-Droid / Zapstore independently rebuild and
|
||||
// verify our developer-signed APKs. (Play still derives this data server-side
|
||||
// from the upload, so nothing is lost for the Play channel.)
|
||||
// bit-for-bit. Dropping it (F-Droid's documented recommendation) lets
|
||||
// F-Droid / Zapstore independently rebuild and verify our developer-signed
|
||||
// APKs.
|
||||
//
|
||||
// Play-channel trade-off: with includeInBundle = false the uploaded .aab no
|
||||
// longer carries this metadata, so Play Console's app-dependency insights /
|
||||
// known-vulnerability SDK alerts go unpopulated. Uploads still succeed; only
|
||||
// that advisory feature is lost.
|
||||
dependenciesInfo {
|
||||
includeInApk = false
|
||||
includeInBundle = false
|
||||
|
||||
+10
-11
@@ -49,18 +49,17 @@ release profile in `Cargo.toml` (`lto`, `codegen-units = 1`, `strip`,
|
||||
|
||||
### Verify the committed binary reproduces
|
||||
|
||||
From `tools/arti-build/`, the helper builds twice from clean and diffs the output:
|
||||
|
||||
```bash
|
||||
# Build, record the hash, then do a clean rebuild and confirm it matches.
|
||||
# Both runs compile in the canonical /tmp/amethyst-arti-build, so the bytes match
|
||||
# regardless of where this repo is checked out.
|
||||
./build-arti.sh --release
|
||||
sha256sum amethyst/src/main/jniLibs/arm64-v8a/libarti_android.so
|
||||
./build-arti.sh --clean --release
|
||||
sha256sum amethyst/src/main/jniLibs/arm64-v8a/libarti_android.so
|
||||
./verify-reproducible.sh # both ABIs (arm64-v8a + x86_64)
|
||||
./verify-reproducible.sh --release # arm64-v8a only (faster)
|
||||
```
|
||||
|
||||
Both hashes match each other and the committed
|
||||
`amethyst/src/main/jniLibs/arm64-v8a/libarti_android.so`.
|
||||
It prints `✅ REPRODUCIBLE` when two clean builds produce identical bytes, then
|
||||
reports whether that matches the committed `.so`. Both builds compile in the
|
||||
canonical `/tmp/amethyst-arti-build`, so the result is independent of where the
|
||||
repo is checked out.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -145,6 +144,7 @@ tools/arti-build/
|
||||
├── repro-env.sh # Deterministic build env (path remapping, epoch) — sourced by both scripts
|
||||
├── build-arti.sh # Build script (Android targets, shipped in APK)
|
||||
├── build-arti-host.sh # Build script (host target, for JVM integration tests)
|
||||
├── verify-reproducible.sh # Builds twice + diffs to prove byte-for-byte reproducibility
|
||||
└── src/
|
||||
└── lib.rs # JNI bridge (Rust → Kotlin)
|
||||
|
||||
@@ -174,8 +174,7 @@ tools/arti-build/
|
||||
4. Regenerate the committed lockfile so the new versions are pinned (builds run
|
||||
`--locked` and will fail until this is refreshed):
|
||||
```bash
|
||||
./build-arti.sh --clean # clones the new tag + sets up the wrapper
|
||||
cp .arti-source/arti-android-wrapper/Cargo.lock ./Cargo.lock
|
||||
./build-arti.sh --regen-lock # re-resolves + rewrites ./Cargo.lock, no compile
|
||||
```
|
||||
If you also bump the Rust toolchain, edit `channel` in `rust-toolchain.toml`.
|
||||
|
||||
|
||||
@@ -43,13 +43,19 @@ MIN_SDK_VERSION=26
|
||||
TARGETS=("aarch64-linux-android" "x86_64-linux-android")
|
||||
RELEASE_ONLY=false
|
||||
CLEAN=false
|
||||
REGEN_LOCK=false
|
||||
|
||||
# Parse arguments
|
||||
for arg in "$@"; do
|
||||
case $arg in
|
||||
--release) RELEASE_ONLY=true; TARGETS=("aarch64-linux-android") ;;
|
||||
--clean) CLEAN=true ;;
|
||||
--help) echo "Usage: $0 [--release] [--clean] [--help]"; exit 0 ;;
|
||||
# Refresh the committed Cargo.lock from the pinned Arti tag, then exit
|
||||
# (no compile — needs only git + cargo, not the NDK). Use after bumping
|
||||
# ARTI_VERSION / Cargo.toml; the normal build is --locked and will fail
|
||||
# until the lock is regenerated and committed.
|
||||
--regen-lock) REGEN_LOCK=true; CLEAN=true ;;
|
||||
--help) echo "Usage: $0 [--release] [--clean] [--regen-lock] [--help]"; exit 0 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
@@ -144,14 +150,14 @@ setup_wrapper() {
|
||||
cp "$SCRIPT_DIR/src/lib.rs" "$wrapper_dir/src/lib.rs"
|
||||
|
||||
# Reproducibility: build against the committed lockfile so transitive
|
||||
# dependency versions are identical for everyone. `cargo --locked` (below)
|
||||
# fails if this lock is missing or stale rather than silently re-resolving.
|
||||
# dependency versions are identical for everyone. `cargo --locked` (in
|
||||
# build_for_target) fails loudly if this lock is missing or stale rather than
|
||||
# silently re-resolving. (Missing is only expected during --regen-lock.)
|
||||
if [ -f "$SCRIPT_DIR/Cargo.lock" ]; then
|
||||
cp "$SCRIPT_DIR/Cargo.lock" "$wrapper_dir/Cargo.lock"
|
||||
print_success "Pinned dependencies from committed Cargo.lock"
|
||||
else
|
||||
print_error "tools/arti-build/Cargo.lock missing — generate it with: cargo generate-lockfile (see README)"
|
||||
exit 1
|
||||
print_info "No committed Cargo.lock yet — run with --regen-lock to create it"
|
||||
fi
|
||||
|
||||
# Patch Cargo.toml to use local arti-client from the source tree
|
||||
@@ -248,10 +254,20 @@ verify_jni_symbols() {
|
||||
main() {
|
||||
echo -e "${BLUE}Arti Android Build — version $ARTI_VERSION${NC}"
|
||||
|
||||
check_prerequisites
|
||||
# --regen-lock only needs git + cargo, not the NDK/cargo-ndk toolchain.
|
||||
[ "$REGEN_LOCK" = true ] || check_prerequisites
|
||||
clone_or_update_arti
|
||||
setup_wrapper
|
||||
|
||||
if [ "$REGEN_LOCK" = true ]; then
|
||||
print_header "Regenerating Cargo.lock"
|
||||
local manifest="$ARTI_SOURCE_DIR/arti-android-wrapper/Cargo.toml"
|
||||
cargo generate-lockfile --manifest-path "$manifest"
|
||||
cp "$ARTI_SOURCE_DIR/arti-android-wrapper/Cargo.lock" "$SCRIPT_DIR/Cargo.lock"
|
||||
print_success "Updated $SCRIPT_DIR/Cargo.lock — commit it, then re-run the build."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Deterministic build env (needs ARTI_SOURCE_DIR cloned above for SOURCE_DATE_EPOCH).
|
||||
# shellcheck source=repro-env.sh
|
||||
source "$SCRIPT_DIR/repro-env.sh"
|
||||
|
||||
@@ -10,9 +10,10 @@
|
||||
channel = "1.94.1"
|
||||
profile = "minimal"
|
||||
components = ["rustc", "cargo", "rust-std"]
|
||||
# Only the two ABIs we actually ship libarti_android.so for (see jniLibs/). If
|
||||
# build-arti.sh is extended to armv7/i686, add them here too — check_prerequisites
|
||||
# also adds any missing target on the fly.
|
||||
targets = [
|
||||
"aarch64-linux-android",
|
||||
"x86_64-linux-android",
|
||||
"armv7-linux-androideabi",
|
||||
"i686-linux-android",
|
||||
]
|
||||
|
||||
Executable
+69
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# Verify that libarti_android.so builds reproducibly.
|
||||
#
|
||||
# Builds the Arti native library twice from a clean state and confirms the two
|
||||
# outputs are byte-for-byte identical. Both builds compile in the canonical path
|
||||
# (/tmp/amethyst-arti-build), so a match here means any checkout — ours,
|
||||
# F-Droid's, an auditor's — produces the same bytes. See README.md →
|
||||
# "Reproducible builds".
|
||||
#
|
||||
# Usage:
|
||||
# ./verify-reproducible.sh # both ABIs (arm64-v8a + x86_64)
|
||||
# ./verify-reproducible.sh --release # arm64-v8a only (faster)
|
||||
#
|
||||
# Prerequisites are the same as build-arti.sh (rustup, cargo-ndk, Android NDK).
|
||||
# Exit 0 = reproducible, exit 1 = builds differ.
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PROJECT_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
|
||||
JNILIBS="$PROJECT_ROOT/amethyst/src/main/jniLibs"
|
||||
PASSTHRU=("$@")
|
||||
|
||||
# Portable sha256 (coreutils sha256sum on Linux, shasum on macOS).
|
||||
sha256() {
|
||||
if command -v sha256sum >/dev/null 2>&1; then sha256sum "$@"; else shasum -a 256 "$@"; fi
|
||||
}
|
||||
|
||||
# sha256 of every built .so, keyed by ABI dir (relative paths → stable keys).
|
||||
# `find | sort` (plain text sort) is portable across GNU and BSD userlands.
|
||||
hashes() {
|
||||
( cd "$JNILIBS" && find . -name libarti_android.so | sort | while IFS= read -r f; do
|
||||
sha256 "$f"
|
||||
done )
|
||||
}
|
||||
|
||||
echo "### Reproducibility check for libarti_android.so"
|
||||
echo "### Canonical build path: ${ARTI_REPRO_DIR:-/tmp/amethyst-arti-build}"
|
||||
echo
|
||||
|
||||
echo "### Build 1 of 2 (clean)…"
|
||||
"$SCRIPT_DIR/build-arti.sh" --clean ${PASSTHRU[@]+"${PASSTHRU[@]}"}
|
||||
H1="$(hashes)"
|
||||
echo "--- build 1 hashes ---"; echo "$H1"; echo
|
||||
|
||||
echo "### Build 2 of 2 (clean)…"
|
||||
"$SCRIPT_DIR/build-arti.sh" --clean ${PASSTHRU[@]+"${PASSTHRU[@]}"}
|
||||
H2="$(hashes)"
|
||||
echo "--- build 2 hashes ---"; echo "$H2"; echo
|
||||
|
||||
if [ "$H1" = "$H2" ]; then
|
||||
echo "✅ REPRODUCIBLE — both clean builds produced identical .so bytes."
|
||||
else
|
||||
echo "❌ NOT REPRODUCIBLE — the two builds differ:"
|
||||
diff <(echo "$H1") <(echo "$H2") || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Informational: is the binary committed in git already the reproducible one?
|
||||
echo
|
||||
echo "### vs. the committed binaries:"
|
||||
if git -C "$PROJECT_ROOT" diff --quiet -- amethyst/src/main/jniLibs/; then
|
||||
echo "✓ The reproducible build matches what's committed — the shipped .so is verifiable as-is."
|
||||
else
|
||||
echo "⚠ The reproducible build differs from the committed .so (e.g. the committed one"
|
||||
echo " predates this toolchain). Commit the rebuilt binaries so the shipped artifact"
|
||||
echo " is itself a reproducible build:"
|
||||
echo " git -C \"$PROJECT_ROOT\" add amethyst/src/main/jniLibs && git commit"
|
||||
fi
|
||||
Reference in New Issue
Block a user