Machine Learning Architecture

Advanced forecasting and prediction capabilities using ML.NET for financial data analysis and stock market predictions.

ML ARCHITECTURE

rendering diagram…

sequenceDiagram
    participant App as Report Generator
    participant Svc as MLForecastingService
    participant ML as ML.NET MLContext
    participant Out as Excel - JSON Output
    App->>Svc: ForecastRevenueAsync - 4 quarters
    Svc->>ML: SSA pipeline - ForecastBySsa
    ML-->>Svc: ForecastedValues - LowerBound - UpperBound
    App->>Svc: ForecastDebtAsync - ForecastCashFlowAsync
    Svc->>ML: SSA with simple trend fallback
    ML-->>Svc: Quarterly predictions
    App->>Svc: DetectAnomaliesAsync
    Svc->>ML: DetectIidSpike pipeline
    ML-->>Svc: Volume spikes - ratio outliers
    App->>Svc: CalculateHealthScoresAsync
    Svc->>ML: Score components 0-25 each
    ML-->>Svc: Health score 0-100
    App->>Svc: ForecastSentimentAsync - 4 weeks
    Svc->>ML: Multi-class classification
    ML-->>Svc: Bullish - Neutral - Bearish
    Svc-->>Out: Embed forecasts in report

MLForecastingService orchestrates seven async ML operations — SSA forecasting, anomaly detection, health scoring, and sentiment classification — before embedding results in Excel and JSON reports.

ML.NET Integration Overview

ML.NET Integration: Grade My Investments incorporates Microsoft's ML.NET framework to provide sophisticated machine learning capabilities for stock market analysis, forecasting, and anomaly detection directly within our .NET ecosystem.

Core ML Components

Time Series Forecasting: Predicts future stock prices and financial metrics
Anomaly Detection: Identifies unusual patterns in trading data
Regression Analysis: Revenue, debt, and equity forecasting
Health Scoring: Calculates financial health scores for companies
Sentiment Analysis: Predicts market sentiment and investor confidence trends

ML.NET Architecture

Historical Data Input

ML.NET Processing Engine

Forecasted Results

MLForecastingService Implementation

The MLForecastingService class (implementing the IMLForecastingService interface) serves as the core engine for all machine learning operations. It is integrated into both the Report Generator Alpha and PyEasy console applications:

Key Forecasting Capabilities

Financial Metrics Forecasting

Revenue growth prediction
Debt trajectory analysis
Cash flow forecasting
Equity value predictions

Market Analysis

Stock price forecasting
Volume trend analysis
Market anomaly detection
Risk assessment scoring
Sentiment trend forecasting

                        // IMLForecastingService interface methods

                        var revenue = await mlService.ForecastRevenueAsync(symbolsData, horizonQuarters: 4);

                        var debt = await mlService.ForecastDebtAsync(symbolsData, horizonQuarters: 4);

                        var cashFlow = await mlService.ForecastCashFlowAsync(symbolsData, horizonQuarters: 4);

                        var equity = await mlService.ForecastEquityAsync(symbolsData, horizonQuarters: 4);

                        var anomalies = await mlService.DetectAnomaliesAsync(symbolsData);

                        var healthScores = await mlService.CalculateHealthScoresAsync(symbolsData);

                        var sentiment = await mlService.ForecastSentimentAsync(symbolsData, horizonWeeks: 4);

                        // Results include ForecastedValues, LowerBoundValues, UpperBoundValues

ML Data Flow

Data Preparation Historical financial data is cleaned and normalized

Feature Engineering Time series data is transformed into ML-ready features

Model Training ML.NET trains forecasting models on historical patterns

Prediction Models generate future predictions with confidence intervals

Report Integration Forecasts are embedded into Excel and JSON reports

Forecasting Algorithms

Time Series Forecasting Methods

Singular Spectrum Analysis (SSA)

Primary forecasting algorithm for revenue, debt, cash flow, and equity. Uses ML.NET's ForecastBySsa with 95% confidence level. Automatically adjusts window size and train size based on available data.

Simple Trend Forecasting

Fallback method when SSA fails due to data constraints. Uses last-value projection with decay factor for confidence intervals.

IID Spike Detection

Detects anomalies in debt-to-equity ratios using ML.NET's spike detection pipeline. Identifies sudden changes that may indicate financial distress.

Multi-class Classification

Classifies sentiment into Bullish, Neutral, and Bearish categories using technical indicators (RSI, MACD, price momentum, volume patterns).

Anomaly Detection

Anomaly Detection Capabilities

Volume Spikes Real-time
Price Volatility Critical
Financial Ratio Outliers Analysis
Unusual Trading Patterns Detection

Health Score Algorithm

Starts at a neutral base of 50, then adds up to 25 points per factor based on financial metrics:

Revenue growth rate (0-25 points)
Debt-to-equity ratio (0-25 points)
Cash flow status (0-25 points)
Profitability metrics (0-25 points)

Dashboard grades: A (80+), B (70-79), C (60-69), D (50-59), F (<50)

Stock Sentiment Forecasting

Advanced Sentiment Analysis: Grade My Investments now incorporates sentiment forecasting to predict market sentiment trends, investor confidence levels, and sentiment-driven price movements using ML.NET's classification and regression capabilities.

Sentiment Data Sources

Historical Price Movements: Analyzes price action patterns to infer sentiment
Volume Patterns: Trading volume spikes and patterns indicate sentiment shifts
Volatility Analysis: Market volatility as a sentiment indicator
Technical Indicators: RSI, MACD, and momentum indicators reflect sentiment
Time-based Patterns: Seasonal and cyclical sentiment trends

Sentiment Forecasting Features

Bullish Sentiment Prediction Forecasts periods of positive investor sentiment

Bearish Sentiment Prediction Identifies potential negative sentiment periods

Neutral Sentiment Detection Recognizes periods of market indecision

Sentiment Intensity Scoring Measures the strength of sentiment on a 0-100 scale

Sentiment Forecasting Algorithms

Binary Classification

Predicts bullish/bearish sentiment using historical price and volume patterns

Accuracy: 78-85%

Multi-class Classification

Categorizes sentiment into Bullish, Bearish, Neutral, and Volatile states

Accuracy: 72-80%

Regression Analysis

Predicts sentiment intensity scores and confidence levels

R² Score: 0.65-0.75

Sentiment ML Pipeline

                        > Initializing sentiment forecasting pipeline...

                        > Loading historical price and volume data for AAPL

                        > Building SentimentData features (PriceChange, Volume, Volatility, RSI, MACD)

                        > Training multi-class classification model (Bearish=0, Neutral=1, Bullish=2)...

                        > Generating SentimentForecastResult for 4 weeks ahead...

                        > Week 1: Sentiment=Bullish, IntensityScore=74, Confidence=0.78, RiskLevel=Low

                        > Week 2: Sentiment=Bullish, IntensityScore=68, Confidence=0.72, RiskLevel=Low

                        > Week 3: Sentiment=Neutral, IntensityScore=45, Confidence=0.65, RiskLevel=Medium

                        > Week 4: Sentiment=Bearish, IntensityScore=25, Confidence=0.60, RiskLevel=High

                        > Adding sentiment predictions to Excel and JSON reports...

Sentiment Features Used

Price Momentum: 5, 10, 20-day price changes
Volume Ratios: Current vs average volume
Volatility Metrics: ATR, Bollinger Band width
Technical Oscillators: RSI, Stochastic, Williams %R
Moving Average Convergence: Price vs MA relationships
Support/Resistance: Distance from key levels

Sentiment Output Metrics

Sentiment Direction Bullish, Bearish, Neutral

Intensity Score 0-100 strength rating

Confidence Level Model prediction certainty

Trend Duration Expected sentiment period

Risk Assessment Sentiment-based risk level

Technical Implementation

ML.NET Framework Integration

Grade My Investments leverages ML.NET's comprehensive machine learning capabilities:

MLContext: Central hub for all ML operations (seed: 0 for reproducibility)
Data Loading: LoadFromEnumerable for in-memory financial datasets
SSA Pipelines: ForecastBySsa with dynamic window/train sizing
Spike Detection: DetectIidSpike for anomaly identification
Classification: Multi-class classification for sentiment analysis
Minimum data: 4 quarters required for forecasting per symbol

Performance Characteristics

Processing Speed: ~1000 stocks/minute

Memory Usage: Optimized for large datasets

Accuracy: 85-95% confidence intervals

Scalability: Multi-threaded processing