Market Data Integration

A resilient, rate-limited, and cached integration layer over Financial Modeling Prep (FMP) that feeds the Grade My Investments report engine with real-time and fundamental market data across 13+ endpoints.

REQUEST FLOW

rendering diagram…

flowchart LR
    A([Report Engine<br/>per-symbol fan-out]):::app
    B[Rate Limiter<br/>token bucket ~3000-min<br/>SemaphoreSlim guarded]:::svc
    C[Pre-flight acquire<br/>~13 tokens]:::svc
    D[GetWithRetryAsync<br/>backoff 1s-2s-4s]:::svc
    E{Cache hit}:::svc
    F[(FMP Proxy<br/>5-min cache<br/>ConcurrentDictionary)]:::data
    G[(Symbol lookup cache<br/>30-min TTL)]:::data
    H[FMP API<br/>13+ endpoints]:::ext
    A --> C
    C --> B
    B --> D
    D --> F
    F --> E
    E -->|hit| A
    E -->|miss| H
    H -->|429 or 5xx retry| D
    H -->|response cached| F
    B -.batch validation.-> G
    G -.miss.-> H
classDef app fill:#1f2630,stroke:#C0894E,color:#F3EFE8;
classDef svc fill:#241c14,stroke:#E2B583,color:#F3EFE8;
classDef data fill:#22201a,stroke:#C0894E,color:#EBD3AE;
classDef ext fill:#1b1f27,stroke:#5A616B,color:#C4C0B8;
classDef sec fill:#1d2733,stroke:#5cc8e0,color:#eafaff;

The report engine routes every FMP call through a token-bucket rate limiter and a 5-minute caching proxy, with exponential backoff retry on throttling and server errors.

Financial Modeling Prep Endpoints

The FmpApiClient wraps the Financial Modeling Prep API, exposing every data set the report engine needs to grade an investment. A single symbol's report draws on roughly thirteen distinct endpoints, gathered in parallel.

Real-time quote
Historical price chart
Company profile
Balance sheets (yearly & quarterly)
Income statements
Cash-flow statements
Key metrics

Analyst recommendations
Price-target consensus
Earnings
ETF holdings (v4 fallback)
Mutual-fund holdings (v4 fallback)
Symbol search & validation

~933 lines The client lives in Gmi.RestClient.FinancialDataProvider/FmpApiClient.cs and is the single integration point for all third-party market data.

Token-Bucket Rate Limiter

The RateLimiter protects both Grade My Investments and the FMP service from runaway concurrency. It is tuned to the FMP enterprise tier of roughly 3,000 calls per minute.

Token-bucket algorithm guarded by a SemaphoreSlim
Partial token refill proportional to elapsed time
Tuned to ~3,000 calls/min (FMP enterprise tier)
Logs rate statistics every 50 requests for observability

Pre-Flight Token Acquisition

Before a symbol's parallel fan-out, the client acquires roughly thirteen tokens up front, one for each endpoint it is about to hit. All per-symbol data is then gathered via a single Task.WhenAll across those ~13 endpoints, maximizing throughput while staying inside the budget.

Resilience

GetWithRetryAsync wraps every outbound call with exponential backoff retry, absorbing transient failures from the upstream API.

Retries on HTTP 429 and 5xx
Exponential backoff: 1s, 2s, 4s
Delay computed via Math.Pow(2, attempt)

Caching Strategy

An FMP proxy hosted in Azure Functions sits in front of the upstream API and short-circuits repeat calls.

Response cache 5-minute TTL
Symbol-lookup cache 30-minute TTL

The response cache is an in-memory ConcurrentDictionary with periodic stale-key cleanup. The 30-minute symbol-lookup cache serves batch symbol validation, avoiding per-symbol round trips.

Integration Layer at a Glance

Concern	Mechanism	Key Detail
Data Provider	FMP	13+ endpoints via `FmpApiClient`
Rate Limiting	Token bucket	~3,000 calls/min, SemaphoreSlim-guarded, partial refill
Retry	Exponential backoff	1s / 2s / 4s on 429 and 5xx
Response Cache	Azure Functions proxy	5-min TTL, ConcurrentDictionary, stale-key cleanup
Symbol Cache	Lookup cache	30-min TTL for batch symbol validation
Throughput	Parallel fan-out	~13 pre-flight tokens, Task.WhenAll across endpoints
Observability	Rate stats	Logged every 50 requests