Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/network-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,40 @@ No automatic migration is provided; separate operator resolution is required.
Funding uses signed `EscrowLock` via `POST /v2/runs/{run_id}/escrow/lock`, not a
CLI `--bond` flag. Test genesis has pinned nonfinancial origins; production mode
has separate reward authority/issuance bounds and no test-genesis conversion.

Production funding is one coordinator-signed `ProductionBootstrap`, posted as an
`ht/2` envelope to `POST /v2/runs/{run_id}/admin/bootstrap` (admin bearer). The
run's `EconomicsPolicyV2.bootstrap_allocation_hash` pins its owner/units/maturity
list (`genesis_allocation_hash` over `[owner, units, mature_at]`); only
`ledger_mode="production"` may set it, and an unset field is omitted, so earlier
policy digests and run IDs are unchanged. Origins are
`ht-origin-bootstrap-v1|run|allocation_hash|index`. The ledger accepts it once per
run: the same `operation_id` and body replay the stored receipt, the same
`operation_id` with other content answers `OPERATION_CONFLICT`, and any second
bootstrap answers `PRODUCTION_BOOTSTRAP_FORBIDDEN`. It is refused when the inner
authority signature is not `reward_authority` (pinned to `coord_pubkey`), when
the envelope signer is not the coordinator, when the allocation differs from the
pin, when bootstrap plus issued and reserved shadow units would exceed
`max_total_issuance` (`TOTAL_ISSUANCE_BUDGET`; later rewards share the same cap),
when the run has no CUDA qualification (production refuses the CPU backend) and
when the latest beacon round is not BLS-verified (`UNVERIFIED_BEACON`).
Bootstrap units mature on the beacon loop and can be locked or transferred;
like genesis units they are collateral and never pay through `get_weights`.

Production eligibility also needs the run's `economic-bound` record. `POST
/v2/runs/{run_id}/admin/economic-bound` (admin bearer, no body) writes it once,
coordinator-signed, only when the pinned policy satisfies
`q_floor*s_lower*beta*(R_collectible+S_min) >= G_max` in exact ppm integers
(`challenge/trust_v2.py:economic_bound`); otherwise it answers 409 and writes
nothing. The record binds the economics policy hash; later calls return it
unchanged, and a stored record for another policy answers 409.

A test-ledger run published on the production challenge steers real emission:
as soon as its first live round finalizes and its reward matures, those units
enter `get_weights` like any production reward
(`tests/challenge/test_v2_weights_bridge.py::test_v2_matured_reward_reaches_get_weights`).
`ledger_mode="test"` does not keep a run out of on-chain weights; do not host a
test-ledger run on the production challenge unless that emission is intended.
Rewards are issued only through accepted finality, uniquely tracked by origin,
then mature after vesting/dispute conditions. Transfers preserve origin units;
fraud burns collectible exposure rather than redistributing it. Shadow work does
Expand Down
2 changes: 1 addition & 1 deletion docs/operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ uv run --frozen python -m hypertrain.ops.manifest_v2 build \
--exp-round 1000000 --state-dir /tmp/state --out /tmp/out
```

The example's `coord_pubkey`, two auditors, referees, `reward_authority` and relay public key are **placeholder SS58 public identities, not operational keys**. Replace them before a real run; `coord_pubkey` must match the service's K_coord. The example uses a tiny decoder, `ledger_mode="test"`, a pinned genesis allocation hash and one `.test` relay. It creates no dataset, genesis funding or training round. Custom CPU/GPU runs use the same three JSON shapes; provide valid model dimensions and parameter count, an audit budget of `2H`, and the H/step-bound-derived audit lease. Build computes `init_state_hash`; an existing value must match. It also rejects economics whose `reward_authority` differs from `training.coord_pubkey` (`PINNED_AUTHORITIES`) or whose `round_reward_units` differs from `training.budget.epochs_per_round * 1_000_000` (`MANIFEST_REWARD_BUDGET`). The service's configured auditor set must match `training.auditors`; the offline CLI cannot validate that configuration. Building needs the existing `trainer` extra (`uv sync --frozen --extra trainer`); help and example do not.
The example's `coord_pubkey`, two auditors, referees, `reward_authority` and relay public key are **placeholder SS58 public identities, not operational keys**. Replace them before a real run; `coord_pubkey` must match the service's K_coord. The example uses a tiny decoder, `ledger_mode="test"`, a pinned genesis allocation hash and one `.test` relay. It creates no dataset, genesis funding or training round. `example --ledger-mode production` instead writes a production economics policy pinning a placeholder bootstrap allocation (`bootstrap_allocation_hash`, no genesis); funding then uses `POST /v2/runs/{run_id}/admin/bootstrap` and `POST /v2/runs/{run_id}/admin/economic-bound` (see [network-v2.md](network-v2.md)). A test-ledger run hosted on the production challenge steers real emission once its first live reward matures; keep test-mode runs off the production challenge unless that is intended. Custom CPU/GPU runs use the same three JSON shapes; provide valid model dimensions and parameter count, an audit budget of `2H`, and the H/step-bound-derived audit lease. Build computes `init_state_hash`; an existing value must match. It also rejects economics whose `reward_authority` differs from `training.coord_pubkey` (`PINNED_AUTHORITIES`) or whose `round_reward_units` differs from `training.budget.epochs_per_round * 1_000_000` (`MANIFEST_REWARD_BUDGET`). The service's configured auditor set must match `training.auditors`; the offline CLI cannot validate that configuration. Building needs the existing `trainer` extra (`uv sync --frozen --extra trainer`); help and example do not.

The owner seed is an existing private, current-user-owned 0600 file containing 32 raw bytes or 64 hex characters; it is read through the miner's checked loader, never printed or copied. The shared loader strips leading/trailing ASCII whitespace bytes: provide hex for any raw seed whose first or last byte is ASCII whitespace. Choose an expiry drand round later than submission. Both commands require a new or empty output directory. Build writes `manifest.json`, `manifest-envelope.json`, five `policy-<name>.json` files, `relay-registry.json` and `index.json`. The index records `run_id`, `owner_ss58`, six canonical object hashes and hashes of the eight other files; it excludes its own hash.

Expand Down
11 changes: 11 additions & 0 deletions src/hypertrain/challenge/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -570,6 +570,17 @@ async def genesis_v2(run_id: str, request: Request, authorization: Auth = None)
admin(authorization)
return await run(store.for_run(run_id).genesis_v2, run_id, await v2_raw(request))

@app.post("/v2/runs/{run_id}/admin/bootstrap")
async def bootstrap_v2(run_id: str, request: Request, authorization: Auth = None) -> Any:
admin(authorization)
hosted = store.for_run(run_id)
return await run(hosted.production_bootstrap_v2, run_id, await v2_raw(request))

@app.post("/v2/runs/{run_id}/admin/economic-bound")
async def economic_bound_v2(run_id: str, authorization: Auth = None) -> Any:
admin(authorization)
return await run(store.for_run(run_id).economic_bound_v2, run_id)

@app.post("/v2/runs/{run_id}/admin/rounds")
async def open_v2(run_id: str, request: Request, authorization: Auth = None) -> Any:
admin(authorization)
Expand Down
48 changes: 48 additions & 0 deletions src/hypertrain/challenge/store.py
Original file line number Diff line number Diff line change
Expand Up @@ -4145,6 +4145,54 @@ def genesis_v2(self, run_id: str, raw: bytes) -> dict[str, JsonValue]:
)
return receipt.body()

def production_bootstrap_v2(self, run_id: str, raw: bytes) -> dict[str, JsonValue]:
"""Pinned production issuance; units mature on the beacon loop, never pay out."""
from hypertrain.protocol.messages_v2 import ProductionBootstrap

escrow, admission, _ = self._services(run_id) # production refuses CPU here
with self._tx():
admission._beacon(self._now(self._db)) # production requires a BLS-verified round
model = self._authenticated_v2(run_id, raw, "ProductionBootstrap", self._coord().ss58)
assert isinstance(model, ProductionBootstrap)
return escrow.production_bootstrap(model).body()

def economic_bound_v2(self, run_id: str) -> dict[str, JsonValue]:
"""Write the immutable coordinator-signed record only when trust_v2 holds the bound."""
from hypertrain.challenge.trust_v2 import economic_bound
from hypertrain.protocol.messages import Receipt

escrow, _, _ = self._services(run_id)
with self._tx():
row = self._db.execute(
"SELECT data FROM records_v2 WHERE run_id=? AND kind='economic-bound' AND id='run'",
(run_id,),
).fetchone()
if row is not None:
stored: dict[str, JsonValue] = json.loads(row[0])
if stored["economics_policy_hash"] != escrow.manifest.network.economics_policy_hash:
raise ChallengeError(409, "immutable economic-bound record conflicts")
return stored
policy = escrow.policy
if policy.ledger_mode != "production":
raise ChallengeError(409, "economic-bound record is production only")
if not economic_bound(policy):
raise ChallengeError(409, "economic bound q*s*beta*(R+S)>=G fails")
evidence: dict[str, JsonValue] = {
"run_id": run_id,
"economics_policy_hash": escrow.manifest.network.economics_policy_hash,
"bound": "q*s*beta*(R+S)>=G",
}
now = self._now(self._db)
receipt = Receipt(
w=0, commit_hash=sha256_hex(canonicalize(evidence)), received_round=max(now, 1)
)
record = {
**evidence,
"receipt": envelope_v2.seal(self._coord(), "Receipt", run_id, receipt, NEVER),
}
self._put_record_v2(run_id, "economic-bound", "run", record)
return record

def _service_profile_v2(self, run_id: str) -> _ServiceCapacityV1:
"""Derive the sole admitted tiny shape; never allocate tensors."""
manifest = self._run_v2(run_id)
Expand Down
11 changes: 11 additions & 0 deletions src/hypertrain/challenge/trust_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,17 @@ def audit_floor(
return AuditFloor(min(q, 1_000_000), required, allowed)


def economic_bound(policy: EconomicsPolicyV2) -> bool:
"""Run-level q*s*beta*(R+S)>=G at the pinned audit floor; exact ppm integers."""
return (
policy.q_floor
* policy.s_lower_ppm
* policy.beta_ppm
* (policy.R_collectible_units + policy.S_min_units)
>= policy.G_max_units * 10**18
)


@dataclass(frozen=True, slots=True)
class AuditSelection:
selected: bool
Expand Down
63 changes: 59 additions & 4 deletions src/hypertrain/ledger/escrow_v2.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Conserved origin units. SQLite event journal and receipts commit with bucket changes.

No legacy entitlement import, external deposit, client mint or client slash exists.
Production issuance outside rewards is one coordinator-signed, policy-pinned bootstrap.
The service supplies authenticated finality/replay and settlement from its own records.
"""

Expand Down Expand Up @@ -35,6 +36,7 @@
EscrowTransfer,
Hex64,
OriginAllocation,
ProductionBootstrap,
RewardFinalize,
RunManifestV2,
ShadowReservationRequestV1,
Expand All @@ -55,7 +57,9 @@ def digest(model: BaseModel) -> str:
return body_digest(model.model_dump(mode="json"))


def authority_message(model: TestGenesis | RewardFinalize | ShadowRewardFinalizeV1) -> bytes:
def authority_message(
model: TestGenesis | ProductionBootstrap | RewardFinalize | ShadowRewardFinalizeV1,
) -> bytes:
"""Origin authority signature is independent of the enclosing envelope signer."""
return (
b"hypertrain/origin/2|"
Expand All @@ -76,6 +80,10 @@ def genesis_origin_id(run_id: str, allocation_hash: str, index: int) -> str:
return sha256_hex(f"ht-origin-genesis-v2|{run_id}|{allocation_hash}|{index}".encode())


def bootstrap_origin_id(run_id: str, allocation_hash: str, index: int) -> str:
return sha256_hex(f"ht-origin-bootstrap-v1|{run_id}|{allocation_hash}|{index}".encode())


def reward_origin_id(run_id: str, w: int, hotkey: str, finalize_hash: str) -> str:
return sha256_hex(f"ht-origin-reward-v2|{run_id}|{w}|{hotkey}|{finalize_hash}".encode())

Expand Down Expand Up @@ -306,7 +314,9 @@ def _event(self, operation_id: str, kind: str, request: str, owner: str) -> Escr
self.db.execute("INSERT INTO escrow_snapshots VALUES(?,?)", (operation_id, state))
return receipt

def _authority(self, model: TestGenesis | RewardFinalize | ShadowRewardFinalizeV1) -> None:
def _authority(
self, model: TestGenesis | ProductionBootstrap | RewardFinalize | ShadowRewardFinalizeV1
) -> None:
if model.run_id != self.run_id or not verify(
decode_hotkey(self.policy.reward_authority),
authority_message(model),
Expand Down Expand Up @@ -351,6 +361,51 @@ def test_genesis(self, genesis: TestGenesis) -> EscrowReceipt:
self._issue(o, None, None)
return self._event(operation, "TEST_GENESIS", request, genesis.origins[0].owner)

def production_bootstrap(self, bootstrap: ProductionBootstrap) -> EscrowReceipt:
"""One pinned production issuance per run; collateral like genesis, never emission."""
self._authority(bootstrap)
with self.tx():
request = bootstrap.model_dump_json(exclude={"authority_sig"})
old = self._existing(bootstrap.operation_id, request)
if old is not None:
return old
if (
self.policy.ledger_mode != "production"
or self.policy.bootstrap_allocation_hash is None
or self.db.execute(
"SELECT 1 FROM escrow_events WHERE kind='PRODUCTION_BOOTSTRAP'"
).fetchone()
):
raise EscrowError("PRODUCTION_BOOTSTRAP_FORBIDDEN")
alloc = [(o.owner, o.units, o.mature_at) for o in bootstrap.origins]
if (
bootstrap.allocation_hash != self.policy.bootstrap_allocation_hash
or genesis_allocation_hash(alloc) != bootstrap.allocation_hash
):
raise EscrowError("BOOTSTRAP_ALLOCATION")
pending = self.db.execute(
"SELECT COUNT(*) FROM records_v2 WHERE run_id=? AND kind='shadow-reservation' "
"AND json_extract(data,'$.accepted') IS NULL",
(self.run_id,),
).fetchone()[0]
if (
self.balances().issued
+ pending * self.policy.round_reward_units
+ bootstrap.total_units
> self.policy.max_total_issuance
):
raise EscrowError("TOTAL_ISSUANCE_BUDGET")
for i, o in enumerate(bootstrap.origins):
if o.origin_id != bootstrap_origin_id(self.run_id, bootstrap.allocation_hash, i):
raise EscrowError("BOOTSTRAP_ORIGIN_DOMAIN")
self._issue(o, None, None) # hotkey NULL: payable() never emits these units
return self._event(
bootstrap.operation_id,
"PRODUCTION_BOOTSTRAP",
request,
bootstrap.origins[0].owner,
)

def reward_allocations(
self, reward: RewardFinalize | ShadowRewardFinalizeV1, evidence: FinalityEvidence
) -> tuple[OriginAllocation, ...]:
Expand Down Expand Up @@ -973,8 +1028,8 @@ def pay(self, op: EscrowOperation, *, signer: str) -> EscrowReceipt:
def payable(self, now_beacon: int) -> list[sqlite3.Row]:
"""Free units of live reward origins matured by now_beacon, oldest maturity first.

Shadow (bootstrap) and genesis origins are collateral, never emission. Only units still
held by the origin's issued owner pay (its emptied reward_pending row marks it): units
Shadow, production-bootstrap and genesis origins are collateral, never emission. Only units
still held by the origin's issued owner pay (its emptied reward_pending row marks it): units
transferred to another owner or a referee are not the earning hotkey's to emit.
"""
return self.db.execute(
Expand Down
12 changes: 8 additions & 4 deletions src/hypertrain/ops/manifest_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ def _build(args: argparse.Namespace) -> dict[str, JsonValue]:
return index


def _example() -> dict[str, bytes]:
def _example(ledger_mode: str = "test") -> dict[str, bytes]:
coord, auditor0, auditor1, referee, relay, cold, other = (
encode_hotkey(bytes([i]) * 32) for i in range(1, 8)
)
Expand Down Expand Up @@ -197,20 +197,23 @@ def _example() -> dict[str, bytes]:
max_object_bytes=1_000_000, max_body_bytes=65536, max_chunk_manifest_bytes=1048576
),
)
allocation = genesis_allocation_hash([(cold, 20000, 1), (other, 10000, 1)])
production = ledger_mode == "production"
economics = EconomicsPolicyV2(
ledger_mode="test",
ledger_mode="production" if production else "test",
G_max_units=100,
R_collectible_units=100,
S_min_units=1000,
beta_ppm=1_000_000,
gammaV_units=0,
s_lower_ppm=1_000_000,
q_floor=1_000_000,
genesis_allocation_hash=genesis_allocation_hash([(cold, 20000, 1), (other, 10000, 1)]),
genesis_allocation_hash=None if production else allocation,
reward_authority=coord,
round_reward_units=1_000_000,
max_total_issuance=20_000_000,
shadow_bootstrap_rounds=12,
bootstrap_allocation_hash=allocation if production else None,
)
aggregation = AggregationPolicyV2(
arithmetic="flat-cclip-cap-v1",
Expand Down Expand Up @@ -295,13 +298,14 @@ def main(argv: list[str] | None = None) -> int:
build.add_argument("--state-dir", type=Path)
example = sub.add_parser("example", help="write tiny CPU inputs with public placeholders")
example.add_argument("--out", type=Path, required=True)
example.add_argument("--ledger-mode", choices=("test", "production"), default="test")
try:
args = parser.parse_args(argv)
if args.cmd == "build":
result = _build(args)
else:
_check_out(args.out)
files = _example()
files = _example(args.ledger_mode)
_write(args.out, files)
result = {"files": {name: sha256_hex(raw) for name, raw in files.items()}}
except ValidationError as error:
Expand Down
Loading
Loading