Skip to content

Changelog

0.4.0 (2026-09-14)

Debuggability, backend unification, memory, and a reproducible writeup. tangent.explain now returns a dead_inputs list and a data-flow graph, so the Tangent 2 paper notebook (which uses it) needs this release.

Added in 0.4.0

  • Public custom-gradient API. tangent.register_adjoint / tangent.register_tangent register a reverse-/forward-mode template rule for a library op you do not own (e.g. numpy.hypot), alongside the existing @tangent.custom_vjp for your own functions and black boxes. Documented in docs/custom_gradients.md.
  • Typed public API (PEP 561). Ships a tangent/py.typed marker and inline types, so df = tangent.grad(f) is a typed callable to mypy/pyright; for a single-argument function the gradient carries the input's type.
  • explain data-flow graph + static dead-input detection. For straight-line functions tangent.explain prints the primal's data-flow graph and returns dead_inputs — arguments that provably never affect the output — plus a dataflow entry. Conservative: control flow yields no graph rather than a wrong claim.
  • Opt-in tape-liveness optimization (optimizations={'tape_liveness': True}). Stores only the shape of primals the adjoint reads for shape (a TapedShape carrier), cutting peak gradient memory ~90% on array loops with an identical gradient, proven safe by reaching-definition analysis.

Changed in 0.4.0

  • Unified backend op catalog (tangent/op_catalog.py). The six array backends register their shared elementwise, binary and reduction rules through one register / register_reductions entry, so no backend can silently drift.
  • Coarsening → backend kernel handoff. Straight-line coarsening now emits the symbolic VJP in the primal's own backend for JAX and PyTorch, so pairing it with compile='jax' / torch.compile fuses the segment into one kernel (TensorFlow / Keras still fall back).
  • insert_grad_of is now documented as a first-class gradient-surgery API in the API reference, with a gradient-surgery notebook.
  • Honest benchmarks. A cross-framework benchmark harness (benchmarks/vs_frameworks.py, jax / torch.func / autograd / finite differences on MLP / conv / scalar) and a single canonical building-simulation benchmark page; the reconciled optimization impact is ~2.6× from tape-aware DCE (the symbolic passes add nothing on that array-dominated workload).

Fixed in 0.4.0

  • Generated primal/tangent functions no longer carry the source method's decorators, so differentiating a @staticmethod works on Python 3.9 (a module-level staticmethod object is not callable before 3.10).

Docs and reproducibility

  • A reproducible Tangent 2 writeup in paper/ (runnable notebook + paper/reproduce.py), a Colab notebook gallery landing page, and case studies differentiating messy NumPy simulators (LIBOR Market Model greeks, projectile-with-drag ODE, SIR epidemic calibration).
  • Developer tooling: make check / pre-commit hooks for the ruff CI gates.

0.3.0 (2026-09-11)

Everything below shipped since the 0.2.0 release: differentiable ODEs and shape checking, backend-compiled adjoints, a persistent disk cache, custom gradient rules, while-loop and annotated checkpointing, SciPy support, a notebook/REPL front door, and IR-invariant verification.

Added in 0.3.0

  • IR invariant verification (tangent/verify.py): the frontend is a sequence of source-to-source passes, each assuming the previous ones' output shape - "works until two passes disagree." Each pass's output contract is now a checkable invariant (ANF form after anf, resolved func annotations after resolve_calls, no nested defs after class_desugar, no lambdas after lambda_desugar). With TANGENT_VERIFY_IR=1 (or run_passes(..., verify=True)) the pass manager checks the relevant invariant after each pass and raises IRInvariantError naming the pass and offending node, so malformed IR fails at its source rather than three passes later. Off by default; the whole corpus is verified to satisfy its invariants (tests/test_ir_invariants.py, and the full 76k suite passes with verification on). This is the safe, shippable realization of "one explicit IR with a documented invariant"; a wholesale single-rewrite core and interprocedural activity analysis remain future work.
  • SciPy gradients (tangent/scipy_extensions.py, optional): adjoints and forward-mode tangents for scipy.special - the error-function family (erf, erfc), the gamma family (gammaln, gamma, digamma/psi), the logistic pair (expit, logit), xlogy, and the logsumexp reduction (adjoint routes through the softmax) - plus scipy.linalg.solve (vector and matrix RHS) and scipy.linalg.inv. Scientific and statistical code written against SciPy now differentiates like NumPy code. All FD-verified; loaded silently when SciPy is present.
  • Works in notebooks, the REPL, and dynamic code (@tangent.function): source-to-source AD needs the function's source, which inspect.getsource cannot retrieve in the plain REPL, in exec-defined functions, or after the source file moves. @tangent.function captures the source at definition time; tangent.grad(f, source=...) and @tangent.function(source=...) accept it explicitly. Captured source survives even deletion of the defining file. The "source unavailable" error now points at these fixes.
  • Compile-time shape checking (tangent.check_shapes): abstractly evaluates a function on ShapedArray inputs (shape + dtype, no data), applying NumPy's real broadcasting and linear-algebra rules, and raises a ShapeError pointing at the offending source line on a rank/broadcast/ matmul/reshape mismatch - catching the class of bug finite differences only reveal at run time. Rides NumPy's dispatch protocols; an unmodeled op yields an unknown shape rather than a guess, so there are no false positives.
  • Differentiable ODE integration (tangent.odeint): integrate dy/dt = func(y, t, *args) and differentiate the solution w.r.t. the initial state and parameters via the continuous adjoint method (Chen et al. 2018) - memory constant in the number of integration steps. The adjoint dynamics are vector-Jacobian products of the user's own func, generated by tangent.vjp (the "define the dynamics in NumPy, get the adjoint solve for free" story). Fixed-step RK4, reverse-mode first order; forward mode and higher order raise cleanly. See examples/neural_ode.py.
  • Checkpointing for while-loops and data-dependent iteration: online (Stumm-Walther) checkpointing handles loops whose trip count is unknown at compile time (convergence loops, fixed-point solvers). Loop-carried-state snapshots are kept under a fixed budget and geometrically thinned as the loop runs, so peak tape memory stays bounded by 2 * budget snapshots for any number of iterations (97%+ reduction measured on a 3500-iteration solver), with gradients identical to the fully-taped path. Engaged by with tangent.checkpoint(): or grad(..., checkpoint=True); break inside the loop is supported. A latent bug was fixed en route: FixGrad could zero-initialize the adjoint's seed parameter when its only use was buried in nested loops, which now never happens (a parameter is always defined).
  • Compile the adjoint, don't just print it: tangent.grad(f, compile='jax'|'torch'|'tinygrad') lowers the generated gradient into a backend compiler while keeping the readable source attached (df.__tangent_source__). Measured: the jax-lowered Tangent adjoint of an MLP loss matches jax.grad+jit eval time exactly (0.075 vs 0.077 ms). The runtime type-dispatch registries now resolve subclasses (jit tracers, FakeTensors) by isinstance with caching, which is what makes lowering possible.
  • Persistent disk cache: generated gradient source is cached across processes (~/.cache/tangent-ad, TANGENT_CACHE_DIR/TANGENT_DISK_CACHE to override), keyed by function source + options + a fingerprint of Tangent's own sources; ~13x faster cold grad() measured. Load verifies every referenced global resolves and falls back to full recompilation on any problem.
  • tangent.explain(f, x) prints the primal, the generated adjoint, the gradient, a finite-difference cross-check, and flags arguments that do not affect the output; tangent.source_map(df) maps generated-gradient lines back to the primal statements (and user-file line numbers) they differentiate.
  • Public custom-gradient API: @tangent.custom_vjp with func.defvjp(bwd) / func.defjvp(jvp) registers user reverse/forward rules (JAX-shaped; Tangent re-provides primals so there is no residual plumbing), and tangent.stop_gradient(x) freezes a value in both modes.
  • with tangent.checkpoint(): annotates specific loops for segment checkpointing - no global flag, no length threshold, and runtime loop bounds are allowed (96%+ peak-tape reduction measured, gradients bit-identical). Implemented as an early desugar pass because the dataflow analyses treat with as opaque.
  • Dynamic np.concatenate/np.stack: lists built by .append() in loops now differentiate through concat/stack (the adjoint returns a list of per-element gradients that flows through the list machinery). Also fixes a latent inliner bug where a dynamically built list could be replaced by its initial [] literal.
  • python -m tangent doctor diagnoses the classic installation traps: the abandoned 2017 tangent package colliding with tangent-ad, broken optional backends, and a smoke gradient.
  • Framework benchmarks (benchmarks/vs_frameworks.py): tangent vs jax.grad vs torch.autograd on a tensor-heavy MLP and a 1000-step data-dependent scalar recurrence, compile and eval time reported separately; results in the Performance guide (tangent is 6x faster than eager torch on the recurrence at 1% of JAX's unroll-compile cost).

0.2.0 (2026-09-11)

First version of this fork to be numbered ahead of Google's final PyPI release (0.1.9, December 2017), so that installs from this repository upgrade cleanly over the abandoned upstream package.

Highlights since upstream 0.1.9

  • New backends: JAX, TensorFlow 2.x (eager), PyTorch, Keras 3 (backend-agnostic keras.ops), and tinygrad (method-based Tensor.* API, including conv2d/max_pool2d/avg_pool2d/layernorm/batchnorm).
  • Higher-order differentiation: second and third derivatives, Hessian-vector products, forward-over-reverse.
  • Language coverage: classes and inheritance, closures, lambdas, comprehensions, enumerate/zip, f-strings, chained and augmented assignment (including subscript targets), early returns, tuple unpacking, the @ operator, pytree (container) arguments — tracked in docs/features/PYTHON_FEATURE_SUPPORT.md and enforced by tests/test_feature_matrix.py.
  • Gradient checkpointing: grad(f, checkpoint=True) performs segment (√n) recomputation on eligible counted loops — O(√n) peak tape memory with identical gradients (96.8% peak reduction measured). See docs/checkpointing_user_guide.md and the "Real gradient checkpointing" entry below.
  • Optimizations: dead-code elimination, common-subexpression elimination, algebraic simplification (SymPy), and a straight-line coarsening prototype.
  • Tooling: gradient-flow and computation-graph visualization, gradient-function caching, clear rejection errors for non-differentiable syntax, a finite-difference oracle for backend adjoints.
  • Modernization: Python 3.9–3.13, gast 0.6/0.7, NumPy 2.x, pyproject-based packaging, GitHub Actions CI, 75k+ parameterized tests.

Added in 0.2.0

  • Differentiable list building: xs.append(v) and v = xs.pop() are now differentiated (previously .append() was silently treated as non-differentiable, returning zero gradients). They are desugared into rebindings through three new primitives (tangent.list_append / list_last / list_init) whose adjoints are written in terms of each other, so forward mode and second/third derivatives work, including appends inside dynamic-length loops. In-place mutations that cannot be expressed as a rebinding (extend/insert/remove/sort/reverse, or append through an attribute/subscript) are rejected with a clear error instead of silently dropping gradients.
  • Distribution, docs site, and release pipeline: the package is now distributed on PyPI as tangent-ad (the import name stays tangent; PyPI's tangent is the abandoned 2017 upstream release - uninstall it before installing this). A documentation site built with Material for MkDocs (landing page, getting-started guides, user guide, generated API reference, benchmarks) deploys to GitHub Pages on every push, and pushing a vX.Y.Z tag builds, verifies (twine + clean-venv wheel smoke test + tag/version match), publishes to PyPI via Trusted Publishing, and creates a GitHub Release. See RELEASING.md for the one-time PyPI/Pages setup.
  • Derivative coverage tail: registered adjoints (and forward-mode rules where mechanical) for np.sort (permutation-routing, closed under higher order), np.argsort (non-differentiable, zero tangent), np.cumprod, np.pad (constant mode), np.take (flat and 1-D-indices-with-axis forms, with repeated-index accumulation), two-operand explicit-output np.einsum (the gradient equation is rearranged at run time; ellipsis, implicit output, and diagonal/trace forms raise a clean NotImplementedError), np.linalg.cholesky and np.linalg.eigvalsh, plus a forward-mode rule for TF's rsqrt (previously pinned as forward-unimplemented). All verified against the finite-difference oracle.
  • Compile-time scalability: functions a few hundred statements long used to crash with RecursionError before they could be differentiated (the CFG dataflow walk recursed once per statement); the walk is now an iterative worklist, and a 400-statement chain compiles fine. The optimizer's fixpoints report changes directly instead of serializing the entire AST with gast.dump twice per iteration, dead-code elimination peels cascading dead chains in a single dataflow analysis instead of one analysis per layer, and the post-advanced-DCE cleanup round only runs when advanced DCE changed something. Compilation of a 150-statement function dropped ~33%, and the full test suite runs ~25% faster.
  • Real gradient checkpointing: tangent.grad(f, checkpoint=True) now implements segment (√n) recomputation — eligible loops run untaped with a snapshot of the loop-carried state every ceil(√n) iterations, and the backward pass replays one segment at a time. Peak tape memory drops from O(n) to O(√n) with gradients identical to the fully-taped path (measured: 49.2 MB → 1.6 MB, 96.8%, on a 2000-iteration loop with 1000-float state; benchmarks/checkpointing_memory.py). The previous implementation stored only the loop target selectively (~0% measured reduction on state-carrying loops) and was restricted to zero-based range(n); any constant-bound range(start, stop, step) is now eligible. First-order reverse mode only.
  • Mode consistency: every corpus construct now behaves the same in every differentiation mode. The last "first-order-only" gaps are closed - multi- output functions (return 2*a, a; polar transforms) and subscript-scatter loops now pass forward mode and reverse-over-reverse too - and the audited harness exclusion list contains only by-design entries. Builtin casts and comparisons gained symmetric rules in both modes: float() (identity), int() (zero derivative a.e.), and forward-mode twins for abs/min/max. Forward mode also seeds loop-counter tangents, fixing a NameError on any x * i arithmetic inside loops.
  • Clean errors restored for unregistered ops: NumPy 2.x wraps most public functions in _ArrayFunctionDispatcher, which the unimplemented-op scan did not recognize - the unimplemented sets were nearly empty, so calls like np.sort(x) recursed into NumPy's own source and crashed with an opaque AttributeError instead of raising ReverseNotImplementedError / ForwardNotImplementedError. The scan now recognizes dispatchers, and forward mode's no-source fallback raises the standard clean error instead of a bare ValueError.
  • break and continue: both now differentiate exactly (historically break miscomputed gradients via tape replay, then both were rejected). A new loop_exit_desugar pass lowers them into guard flags before differentiation: continue becomes a per-iteration skip flag guarding the rest of the body; break adds a loop-level flag that a while folds into its condition and a for uses to skip remaining iterations. Works with data-dependent exits, nested loops, forward mode, both motions, and second order. Loops with an else clause remain rejected (while/else now rejected explicitly, like for/else).
  • return inside loops: lowered into an assignment plus a returning flag plus break (which the break lowering then handles), with the exit propagated past each enclosing loop and the final return lifted by the single-exit transform. Previously rejected.
  • Iteration over computed sequences: for v in x * 2.0, for v in np.flip(x), and for v in [x, x * 2] (a literal of active values) now differentiate - the iterable expression is hoisted into a named intermediate and indexed, the same rewrite long applied to named sequences. Previously these loops were neither rewritten nor rejected and gradients through the loop variable were silently dropped. Tuple-unpacking targets (for a, b in pairs) work; set literals of active values are rejected (no stable order).
  • List comprehensions over runtime iterables: [f(v) for v in xs] with a runtime xs is now lowered into an indexed loop built on tangent.list_append (previously rejected; before that, silently wrong). Runtime if filters and nested comprehensions are supported; multiple generators and tuple targets are still rejected cleanly.
  • Container (pytree) return values in reverse mode: functions returning dicts, lists, or nested containers of arrays can now be differentiated (previously a KeyError crash). The default seed expands to a matching pytree of ones (the gradient of the sum of all leaves), a caller-supplied seed of the same structure is used as the cotangent, and second derivatives through container arguments work. A structurally mismatched seed raises a clear ValueError instead of being silently replaced.
  • New NumPy derivatives (reverse and forward mode, validated against a finite-difference oracle): the ufunc spellings np.add, np.subtract, np.divide, np.negative, np.power, np.float_power; np.arctan2, np.hypot, np.logaddexp, np.arcsinh, np.arccosh, np.arctanh, np.exp2, np.cbrt, np.fmax, np.fmin; the shape ops np.cumsum, np.flip, np.ravel, np.swapaxes, np.moveaxis, np.tile, np.repeat, np.roll; and np.linalg.solve plus np.linalg.norm (default 2-norm/Frobenius).
  • Backend parity: the @ matmul operator now differentiates for PyTorch (and Keras); softmax/log_softmax adjoints for PyTorch, Keras, JAX and TensorFlow; concatenate/stack differentiate across NumPy, JAX, TensorFlow, PyTorch and Keras (previously JAX-only); elementwise gaps closed across backends (e.g. torch expm1/rsqrt, jax reciprocal/log1p/expm1/exp2, keras log/exp family, tf.math.reciprocal — whose old TF-1-only spelling registered nothing on TF 2.x). Unary elementwise rules are now generated from one shared backend-neutral table, so every table op has both reverse- and forward-mode rules on every backend.
  • Forward mode catches up: the extended NumPy and TensorFlow modules previously registered zero forward-mode tangents; they now cover their op catalogs (abs, clip, where, min/max/prod, trig, linalg.inv, trace, softmax, concat/stack, ...).
  • tangent.backend_status() reports each optional backend as 'available', 'not installed', or 'broken: <error>'.
  • tangent.passes: a documented pass registry for the desugaring pipeline with register_pass(..., before=/after=) and unregister_pass as an extension hook for inserting custom frontend passes.
  • Property-based tests (hypothesis) drive a corpus sample against the finite-difference oracle; CI now also runs the tinygrad-backed tests, ruff lint + format checks, and a coverage floor.

Changed in 0.2.0

  • import tangent is silent. Missing optional backends (jax, torch, tensorflow, keras, tinygrad) no longer emit a wall of UserWarnings and stdout banners; they are logged at DEBUG level on the 'tangent' logger. A UserWarning is kept only when a backend package is installed but its extensions fail to load (a broken install is actionable). Use tangent.backend_status() to see what loaded.
  • Gradient-cache keys now include compilation settings. grad(f) followed by grad(f, optimizations={...}) (or a different checkpoint_config/grad_config) previously returned the same cached function; such calls now compile and cache separately, so previously colliding call sites will each trigger their own (correct) compilation. Cache hits are also much cheaper: source hashes are memoized instead of re-reading the source file on every lookup.
  • Checkpointing consolidated to what demonstrably works: grad(f, checkpoint=True) (zero-based constant range(n) loops) and the manual checkpointed_loop forward-pass helper. grad_with_checkpointing now raises NotImplementedError at wrapper-creation time with an actionable message. checkpoint=True now composes with optimized=True (optimization is no longer force-disabled).
  • List/set/dict comprehensions over dynamic iterables are rejected with a clear TangentParseError at transform time. The old .append()-loop fallback crashed in return position and silently produced zero gradients in assignment position. Comprehensions over compile-time-constant iterables are still unrolled and differentiate correctly.
  • numpy/tf concatenate/stack over a dynamically built list raise NotImplementedError instead of silently returning zero gradients.
  • The codebase is uniformly formatted with ruff format (4-space indent, 100 columns), enforced in CI; the reformat commit is listed in .git-blame-ignore-revs.
  • Docs pruned: ~15 stale point-in-time progress reports (phase "COMPLETE" writeups, outdated test-status snapshots) were deleted or folded into the living feature guides; remaining status claims match measured reality.

Fixed in 0.2.0

  • tangent.autodiff(f, optimizations=...), checkpoint_config=... and grad_config=... raised TypeError: the caching wrapper re-declared a narrower parameter list than the real autodiff. The full signature is restored.
  • Wrong or crashing higher-order derivatives with optimized=True: dead-code elimination trusted per-op-id tape annotations that are not unique in differentiated gradient code, so it could delete the wrong tape push (second derivatives read stale primal values, or tripped the stack op-id assertion). DCE now computes its own balanced push/pop pairing and never removes an ambiguous pair; the two long-standing expected failures in reverse-over-reverse are eliminated.
  • Multi-output functions with check_dims=False crashed (or could get a wrong default seed): the output arity was inferred by scanning generated code for the shape-check assert. It is now recorded as an annotation during the reverse-mode transformation.
  • grad(..., checkpoint=True) could produce wrong gradients for for i in range(start, stop) loops with start != 0: the checkpointed adjoint reconstructs the loop target as the zero-based iteration index. Such loops now fall back to the standard (non-checkpointed) templates.
  • Broadcasting operands of np.multiply and np.maximum received wrongly-shaped gradients (missing unbroadcast reduction).
  • ValueError for mismatched forward-mode derivative arguments was raised incorrectly (a TypeError masked the real message).
  • The stack variable in the subscript-assignment adjoint template was bound via a typo'd keyword (_stack_), working only because the default name matched.
  • Package metadata now identifies this fork (maintainer, repository URLs) and exposes tangent.__version__.

Removed in 0.2.0

  • An unreachable, unfinished automatic-checkpointing pipeline (tangent/checkpointing/, tangent/analysis/, tangent/preprocessing/), the placeholder checkpointed_backward (its default path returned the incoming gradient unchanged), and checkpointed_grad (its "simplified backward" returned the single-step gradient at the final state, not the loop gradient).
  • The vendored Python 2 funcsigs backport, in favor of inspect.signature.