Scoring Mechanism

At its core, the Sportstensor scoring mechanism operates on the principle that true predictive edge manifests through consistent outperformance across diverse market conditions. The scoring function evaluates miner contributions through a rigorous set of metrics including prediction accuracy, market impact, and cross-sectional performance relative to the network's collective intelligence. This creates a natural selection pressure where only strategies delivering genuine value survive and thrive.

Critical to the mechanism's integrity is Bittensor's validation architecture. Every prediction is independently verified by our network of validators, with performance metrics permanently recorded on-chain. This distributed consensus approach makes manipulation statistically infeasible while creating an immutable track record of model performance.

Core Components

1. Statistical Significance (ρ)

Purpose Statistical significance measures how consistently a miner participates in the prediction network relative to required thresholds. This component ensures that miners maintain a steady stream of predictions rather than making sporadic contributions.

$$\rho = \frac{1}{1 + e^{-\alpha(x-\text{threshold})}}$$

Parameters

• x: Number of miner predictions in the evaluation period

• threshold: Required prediction threshold for the league

• α: Sensitivity parameter (typically between 0.1 and 0.5)

Example scenario:

miner_predictions = 45
league_threshold = 40
alpha = 0.2

Calculate ρ:

difference = miner_predictions - league_threshold  # 5
exponent = -alpha * difference  # -1
denominator = 1 + math.exp(exponent)  # 1.368
rho = 1 / denominator  # 0.731

Impact Analysis

How ρ changes with different prediction counts:

Predictions	Threshold	ρ Value
20	40	0.018
30	40	0.122
40	40	0.500
50	40	0.878
60	40	0.982

This sigmoid curve ensures:

• Scores below threshold are suppressed.

• Scores near threshold receive moderate weights.

• Diminishing returns for far exceeding threshold.

2. Incentive Score (v)

Purpose The incentive score combines timing and market value components to reward predictions that are capturing market inefficiencies.

Components

A. Time Component

$$\text{time\_component} = \exp(-\gamma \cdot \Delta t)$$

Parameters:

• γ: Time decay parameter (typically 0.001 to 0.005)

• Δt: Minutes between prediction and match start

While all predictions have value, the time component algorithm places emphasis on predictions closest to match start.

Example:

# Early prediction (24 hours before)
gamma = 0.002
delta_t = 24 * 60  # 1440 minutes
time_score_early = math.exp(-gamma * delta_t)  # 0.056

# Late prediction (1 hour before)
delta_t_late = 60  # 60 minutes
time_score_late = math.exp(-gamma * delta_t_late)  # 0.886

B. CLV Component

$$\text{clv\_component} = \frac{1 - (2\beta)}{1 + \exp(\kappa \cdot \text{clv})} + \beta$$

Parameters:

• clv: Closing Line Value (difference between prediction odds and closing odds)

• κ: Transition parameter (typically 1-5)

• β: Extremis parameter (typically 0.1-0.3)

Example:

# Favorable CLV scenario
clv = 0.15  # 15% better than closing odds
kappa = 2
beta = 0.2
favorable_clv = (1 - (2*0.2)) / (1 + math.exp(2 * 0.15)) + 0.2  # 0.723

# Unfavorable CLV scenario
clv = -0.10  # 10% worse than closing odds
unfavorable_clv = (1 - (2*0.2)) / (1 + math.exp(2 * -0.10)) + 0.2  # 0.412

Combined Incentive Score

$$v = \text{time\_component} + (1 - \text{time\_component}) \cdot \text{clv\_component}$$

Example Full Calculation:

# Early prediction with good CLV
time_comp = 0.056  # from earlier example
clv_comp = 0.723   # from earlier example
v_early_good = time_comp + (1 - time_comp) * clv_comp  # 0.737

# Late prediction with poor CLV
time_comp_late = 0.886
clv_comp_poor = 0.412
v_late_poor = time_comp_late + (1 - time_comp_late) * clv_comp_poor  # 0.459

3. Closing Line Value (CLV)

Purpose CLV measures the value captured between prediction odds and closing odds, indicating a miner's ability to identify market inefficiencies.

$$\text{clv} = \text{closing\_odds} - \text{prediction\_odds}$$

Detailed Example:

# Scenario 1: Value Captured
prediction = {
    'team': 'TeamA',
    'prediction_odds': 2.00,  # Implied probability 40%
    'closing_odds': 2.50,     # Implied probability 50%
    'actual_winner': 'TeamA'
}
clv = 2.50 - 2.00  # 0.50 (positive value captured)

# Scenario 2: Value Lost
prediction = {
    'team': 'TeamB',
    'prediction_odds': 2.20,  # Implied probability 55.6%
    'closing_odds': 1.80,     # Implied probability 45.5%
    'actual_winner': 'TeamB'
}
clv = 1.80 - 2.20  # -0.40 (negative value lost)

4. Gaussian Filter

Purpose The Gaussian filter prevents gaming of the system by suppressing scores for predictions that deviate significantly from market consensus. The code can be found here.

Sigma (σ):

σ represents the standard deviation in the Gaussian filter and is used to adjust the sensitivity of the scoring system. This ensures that the influence of predictions aligns with the market consensus, suppressing extreme deviations.

$$\sigma = \log\left(\frac{1}{\text{closing\_odds}^2}\right)$$

Weight (w):

w is a threshold parameter that defines the acceptable range of deviations from market consensus. It depends on the closing_odds and their logarithmic value. The term (closing_odds - 1.0) scales the sensitivity, while the logarithmic factor provides a diminishing impact as odds grow, ensuring predictions are not overly penalized for slight deviations.

$$w = (\text{closing\_odds} - 1.0) \cdot \frac{\log(\text{closing\_odds})}{2}$$

Absolute Difference (diff)

diff represents the absolute difference between the closing_odds (market consensus) and the implied probability derived from the prediction (1 / prediction_probability). It quantifies how far the prediction deviates from the market’s expected outcome. A smaller diff value indicates closer alignment with market expectations, while larger values highlight significant deviations.

$$\text{diff} = \left|\text{closing\_odds} - \frac{1}{\text{prediction\_probability}}\right|$$

Filter:

The filter applies a scoring adjustment based on diff. If the diff is within the threshold w, no penalty is applied (filter = 1.0). For larger deviations, an exponential penalty proportional to -diff² / (4σ²) is applied, which sharply reduces the score as the deviation increases. This discourages predictions that significantly diverge from market consensus.

$$\text{filter} = \begin{cases} 1.0 & \text{if } \text{diff} \leq w \\ \exp\left(-\frac{\text{diff}^2}{4\sigma^2}\right) & \text{otherwise} \end{cases}$$

Example Calculations:

# Conservative prediction close to market
closing_odds = 1.90
prediction_prob = 0.54  # implied odds ≈ 1.85
sigma = math.log(1/1.90**2)
w = (1.90 - 1.0) * math.log(1.90)/2
diff = abs(1.90 - 1/0.54)
filter_conservative = 1.0  # Within acceptable range

# Aggressive prediction far from market
closing_odds = 1.90
prediction_prob = 0.80  # implied odds = 1.25
diff_aggressive = abs(1.90 - 1/0.80)
filter_aggressive = math.exp(-diff_aggressive**2/(4*sigma**2))  # ≈ 0.342

5. Return On Income (ROI)

Purpose Calculating ROI sheds light on an additional aspect of predictions and is a primary metric most investors and participants will look at.

$$roi = \frac{\text{earned\_payouts}}{\text{predictions} \cdot \text{bet\_amount}}$$

Score Calculation Process

1. Individual Prediction Edge Score

Each prediction's edge score is calculated by combining all components:

def calculate_prediction_score(prediction, match_data):
    # Calculate components
    v = calculate_incentive_score(
        delta_t=get_time_delta(prediction.time, match_data.start_time),
        clv=calculate_clv(prediction, match_data)
    )
    sigma = calculate_closing_edge(prediction, match_data)
    gfilter = apply_gaussian_filter(prediction, match_data)
    
    # Combine for final prediction score
    return v * sigma * gfilter

Example Calculation:

prediction_data = {
    'time': '2024-01-01 12:00:00',
    'odds': 2.50,
    'probability': 0.40
}
match_data = {
    'start_time': '2024-01-02 15:00:00',
    'closing_odds': 2.00,
    'actual_winner': 'TeamA'
}

edge_score = calculate_prediction_score(prediction_data, match_data)

2. Individual ROI Score

Each prediction has its ROI calculated and aggregated for the miner.

Example Calculations:

# Calculate ROI for the prediction
ROI_BET_AMOUNT = 1.0
league_roi_counts[league][index] += 1

# If prediction was correct, update aggregate with positive earned payout
if pwmd.prediction.get_predicted_team() == pwmd.get_actual_winner():
    league_roi_payouts[league][index] += ROI_BET_AMOUNT * \
                                         (pwmd.get_actual_winner_odds()-1)

# If prediction was incorrect, update aggregate with negative bet amount
else:
    league_roi_payouts[league][index] -= ROI_BET_AMOUNT

3. League Score Calculation

Purpose League scores are the sum of the total aggregated prediction edge score and the total roi score while considering league-specific requirements and weightings as well as the weightings for edge and roi scores, respectfully. Both the edge and roi scores have rho (ρ), or the statistical significance, applied.

Mathematical Definition

$$\text{edge\_score} = \rho \cdot \sum(\text{prediction\_edge\_scores})$$

$$\text{roi\_score} = \rho \cdot \sum(\text{prediction\_roi\_scores})$$

Detailed Edge Score Example:

# Sample prediction scores for a miner in Premier League
predictions = {
    'match_1': {
        'score': 0.85,
        'timestamp': '2024-01-01 15:00:00',
        'correct': True
    },
    'match_2': {
        'score': -0.32,
        'timestamp': '2024-01-02 20:00:00',
        'correct': False
    },
    'match_3': {
        'score': 0.64,
        'timestamp': '2024-01-03 19:30:00',
        'correct': True
    }
}

# Calculate components
prediction_sum = sum(p['score'] for p in predictions.values())  # 1.17
num_predictions = len(predictions)  # 3
threshold = ROLLING_PREDICTION_THRESHOLD_BY_LEAGUE['PREMIER_LEAGUE']  # e.g., 5
rho = compute_significance_score(num_predictions, threshold, alpha=0.2)  # 0.731

edge_score = rho * prediction_sum  # 0.855

Detailed ROI Score Example:

# Sample prediction scores for a miner in Premier League
predictions = {
    'match_1': {
        'odds': 2.85,
        'timestamp': '2024-01-01 15:00:00',
        'correct': True
    },
    'match_2': {
        'odds': 3.32,
        'timestamp': '2024-01-02 20:00:00',
        'correct': False
    },
    'match_3': {
        'odds': 1.64,
        'timestamp': '2024-01-03 19:30:00',
        'correct': True
    }
}
ROI_BET_AMOUNT = 1.0

# Calculate components
for p in predictions.values():
    if p['correct'] == True:
        roi_sum += ROI_BET_AMOUNT * (p['odds'] - 1)
    else:
        roi_sum -= ROI_BET_AMOUNT
# roi_sum == (1.0 * 2.85-1) - (1.0) + (1.0 * 1.64-1) == 1.49

num_predictions = len(predictions)  # 3
rho = compute_significance_score(num_predictions, threshold, alpha=0.2)  # 0.731

roi_score = rho * (roi_sum / (num_predictions * ROI_BET_AMOUNT)) # 0.363

4. Score Normalization and Score Weighting

Purpose Calculates edge and roi scores to the same scale and applies scoring weights. We apply rho again after normalizing and combining scores.

Normalization Calculation Example:

# Normalize edge scores
min_edge, max_edge = min(league_scores[league]), max(league_scores[league])
normalized_edge = [(score - min_edge) / (max_edge - min_edge) \
                  if score > 0 else 0 for score in league_scores[league]]

# Normalize ROI scores
min_roi, max_roi = min(league_roi_scores[league]), max(league_roi_scores[league])
normalized_roi = [(score - min_roi) / (max_roi - min_roi) \
                 if (max_roi - min_roi) > 0 else 0 for score in league_roi_scores[league]]

Score Weighting Example:

# Apply weights, combine, and apply rho to set final league scores
ROI_SCORING_WEIGHT = 0.5 # calculates 50/50 weighting to edge and roi scores
league_scores[league] = [
    ((1-ROI_SCORING_WEIGHT) * e + ROI_SCORING_WEIGHT * r) * rho
    if r > 0 else 0
    for e, r, rho in zip(normalized_edge, normalized_roi, league_rhos[league])
]

5. Overall Score Aggregation

Purpose Combines scores across different leagues while respecting league weights.

Mathematical Definition

$$\text{overall\_score} = \sum(\text{league\_score} \cdot \text{league\_weight})$$

Implementation

1. Calculate total positive scores for each league

league_totals = {league: 0.0 for league in ACTIVE_LEAGUES}
for league in ACTIVE_LEAGUES:
    league_totals[league] = sum(score for score in league_scores[league] if score > 0)

1. Scale scores within each league to match allocation percentage

scaled_scores_per_league = {league: [0.0] * len(all_uids) for league in ACTIVE_LEAGUES}
for league in ACTIVE_LEAGUES:
    total_league_score = league_totals[league]
    allocation = LEAGUE_SCORING_PERCENTAGES[league] * 100  # Convert to percentage
    
    if total_league_score > 0:
        scaling_factor = allocation / total_league_score  # Factor to scale league scores
        scaled_scores_per_league[league] = [
            (score * scaling_factor if score > 0 else 0) for score in league_scores[league]
        ]

1. Aggregate scaled scores across all leagues

for i in range(len(all_uids)):
    all_scores[i] = sum(scaled_scores_per_league[league][i] for league in ACTIVE_LEAGUES)

Penalty System

1. League Commitment Penalties

Purpose Ensures miners maintain active participation across leagues by penalizing those without a league commitment.

Implementation

• LeagueCommitmentRequests are sent to miners every 15 minutes. For every consecutive request that is missed, an accumulating penalty of -0.1 is calculated and applied to a miner's final score.

• As soon as a miner properly responds with a league commitment, the penalty resets to 0.

• Additionally, if a miner has failed to respond to a LeagueCommitmentRequest for 24 hours, their final score will be set to 0 until they properly commit to a league again or their UID is deregistered.

2. No-Response Penalties

Purpose Ensures miners respond to prediction requests in a timely manner.

Implementation

• A validator performs a scoring step every 30 minutes. For each 30 minute window leading up to a scoring step, if a miner fails to respond to a MatchPredictionRequest, they will accrue a penalty of -0.1.

• For example, if a miner fails to respond to 6 MatchPredictionRequests from a validator, that miner will have a total of -0.6 points applied to their final score in the next scoring step.

• After the scoring step has completed, the no-response penalties are reset to 0.

A miner has 15 seconds to respond to a validator request before it is considered a no-response.

Final Score Distribution

1. Pareto Distribution Application

Purpose Transforms final scores to maintain competitive differentiation while preventing extreme outliers.

Implementation

def apply_pareto(scores: List[float], mu: float, alpha: int) -> List[float]:
    """
    Apply Pareto distribution to scores.
    
    Parameters:
        scores: Raw scores
        mu: Minimum value parameter
        alpha: Shape parameter
    """
    scores_array = np.array(scores)
    positive_mask = scores_array > 0
    positive_scores = scores_array[positive_mask]
    
    transformed_scores = np.zeros_like(scores_array)
    
    if len(positive_scores) > 0:
        # Transform positive scores
        min_score = np.min(positive_scores)
        range_transformed = (positive_scores - min_score) + 1
        transformed_positive = mu * np.power(range_transformed, alpha)
        transformed_scores[positive_mask] = transformed_positive
    
    return transformed_scores

Example Scenario:

raw_scores = [0.705, 0.432, 0.891, 0.156, 0.543]
mu = 0.1
alpha = 2

transformed_scores = apply_pareto(raw_scores, mu, alpha)

# Example output analysis:
# Raw scores: [0.705, 0.432, 0.891, 0.156, 0.543]
# Transformed: [1.247, 0.583, 2.103, 0.112, 0.891]